Mapping the external attacked surface of the organization: Passive DNS, CT-logs and hunt for Shadow IT

Depov

Activist
ULTIMATE
SUPREME
PREMIUM
MEMBER
Joined
Feb 18, 2025
Messages
126
Reaction score
115
Deposit
0$
At the pre-engagement phase of the pentest industrial company, I uploaded the root domain to cl.sh and received 47 subdomains. The IT department knew about 28. Among the remaining 19 - staging-stand of the corporate ERP, Jenkins without authorization and three dev-environment with expired certificates. None of these assets were in the CMDB. The situation is painfully typical: organizations do not control 30 to 40% of their external assets. For blue team, the output is simple - there is no full perimeter map yet, there is nothing to protect.
The place in the chain of attack: why blue team reconnaissance of its own perimeter
All techniques below are the Reconnaissance phase by MITRE ATT&CK. The attacker starts from here: Passive DNS (T1596.001), Digital Certificates (T1596.003), WHOIS (T1596.002), DNS enumeration (T1590.002). These methods do not generate traffic to the target infrastructure and leave no traces in SIEM - the blue team will not detect their after-fact fact.

The only working answer is to apply the same techniques to your own organization regularly, before the attacker does it. The result is an actual map of the external attacked surface, which shows:
• What assets are sticking out, including those that IT doesn’t know
• What DNS records indicate non-existent resources (dangling DNS - direct subdomain gainover vector)
• What certificates are issued for entity domains, including wildcard and staging environments
• where the shadow IT bypasses corporate change management processes
Without this card, the firewall rule, WAF-politics, vulnerability scanning - everything works blindly. The attacker sees the perimeter more fully than its owner.

NIST CSF 2.0, ID.AM-01 (Asset Management) function, directly requires: "Constantly monitor networks to detect new hardware and autoname updateIES""Constant network tracking to detect new equipment and automatic inventory update". Cutting the outer surface is basic hygiene, without which it is impossible to manage cyber risks. Not "bonus" activity before the pentest, but the foundation.
Passive DNS Exploration: How Historical Records Reveal Forgotten Infrastructure
Passive DNS is a database collected from real DNS answers that were observed by recursive rezollers. The active DNS request shows the current state. Passive DNS holds a story: which domain to which IP resolved, when fixed for the first time and when last time.

To map the external attacked surface of the organization, this means this: if the corporate subdomain dev.example.com Six months ago, he pointed to IP in AWS, and now the DNS record is deleted - in passive DNS this connection has been preserved. You see both a subdomain and a cloud resource that may still work and is available via direct IP. Dead entries in DNS are live servers in the cloud.
Sources and tools Passive DNS
The main public and semi-public sources:
• SecurityTrails - commercial API, there is no free range (available free for trial evaluation). Covers historical A, AAAA, MX, NS, CNAME, SOA and TXT records. For regular use, you need a paid subscription - but the coverage is worth it.
• CIRCL Passive DNS is a free service from Computer Indictability Center Luxembourg. API requests in JSON format. Coating uneven: European domains are well represented, Russian infrastructure is much worse.
• VirusTotal - passive DNS as a byproduct of mass scanning. For a quick check, it will come down, but the completeness depends on whether someone scanned the domain earlier.
From CLI-tools: dnsx (ProjectDiscovery, actively supported) for mass resheding, subfinder (ProjectDiscovery) for aggregation of passive DNS from dozens of sources into one stream, amass (OWASP project) to build a graph of connections between domains, IP and ASN.
What passive DNS will show - and what will not show
Showing:
• Historical A/CNAME subdomain records that have already been removed from the current DNS
• IP addresses of cloud providers on which assets are placed or placed (T1590.005)T1590.005
• Communication between domains via shared IP – two domains that seemed independent, resolved on one server
Will not show:
• Subdomains that never redeveloped publicly (internal split-horizon DNS)
• Domains for CDN/WAF, if there were no historical records before the proxy was included
• Assets without DNS name (IP only) - they need a separate approach through scanning IP blocks (T1595.001)
Certificate Transparency: crt.sh and search for subdomains through certificate logs
[Applicable: external pentest, bug bounty, blue team continuous]

Certificate Transparency (CT) - mechanism for RFC 6962 (CT v1, used in production). Browser-politics Chrome and Apple require that every publicly trusted certificate contains the SCT of at least two independent CT logs – without this, browsers do not trust the certificate. Each certificate contains Subject Alternative Names (SAN) - a list of domains and subdomains for which it is issued. This data is public and available for search.

Since April 2018, Chrome (Google CT Policy) and from October 2018 - Apple is demanding the inclusion of publicly trusted certificates in CT logs. This creates a complete, search record of each subdomain for which a certificate has ever been issued - from api.example.com to internal-tools.example.com. For OSINT analytics, CT-logs is the most reliable source of subdomains. DNS brute-force depends on the quality of the dictionary. CT covers every name for which a certificate has been issued at least once. The difference is fundamental.
Practice: query to crt.sh for search for subdomains
crt.sh is a free search engine from Sectigo, aggregating data from Google Argon, Cloudflare Nimbus, DigiCert Yeti and other CT Lengths. Supports JSON answers for automation.
Bash:
curl -s "https://crt.sh/?q=%.example.com&output=json" \
| jq -r '.[].name_value' \
| sort -u \
| grep -v '^\*\.'
The result is a degraded list of subdomains without wildcard records. For a corporate domain with a history of 5-7 years, it is usually 50-200 unique names. In addition to crt.sh, as a fallback source of CT data used by the CT scanter from SSLMate - it helps out when crt.sh responds slowly or spins in the rate limit.

What to look at when analyzing the results:
• Subdomains with patterns staging., dev., test., internal., vpn.*- potential shadow IT or forgotten stands
• Certificates issued by Let's Encrypt for subdomains that IT does not know - one of the employees raised the service arbitrarily
• Subdomains with date not_afterin the past and missing DNS-sautoning - candidates for dangling DNS
Restrictions of CT Lengths
CT does not see:
• HTTP-only services without TLS (dev-stands on the port 8080 without certificate)
• Subdomains under a wildcard certificate, if a separate certificate was not issued for them
• Infrastructure for private CA (internal PKI corporations)
That is why the certificate transparency pentest methodology is part of the combined approach, and not the only source.
Subdomain list: DNS brate-force, zone transfer and combined tools
[Applicable to: blue team without restrictions on its infrastructure; external pentest - gently with active techniques]

Passive DNS and CT blogs cover a passive part of the reconnaissance. For the completeness of the picture, active equipment is needed - but with an understanding of their cost.
DNS brete-force tools
DNS brute-force sends thousands of requests for an authoritative domain DNS server, checking the existence of subdomains from the dictionary. Tools: puredns (solving through public DNS servers with wildcard filtering), shuffledns (ProjectDiscovery, wrapper over massdns).

Works if: The organization uses a predictable naming scheme (srv-db-01, app-prod-02, mail-gw) Does not work if: Subdomains are randomized (UUID-based names in Kubernetes, Terraform-generated names). Limitation: generates noticeable traffic to target DNS servers – tens of thousands of queries, can trigger rate limiting and alertas in SOC. On its infrastructure - without restrictions, on someone else's - only with written permission.
Zone Transfer: AXFR vulnerability
Attempt to request a full copy of the DNS zone via the AXFR dig axfr example.com @ns1.example.com. If the DNS server allows the zone transfer for arbitrary IP, it is both a critical find (zone transfer vulnerability) and an instant full subdomain map.

Works if: DNS server with erroneous configuration allow-transfer { any; }. Does not work: BIND 9.x+ with correct allow-transfer, cloud DNS (Route53, Cloud DNS, Azure DNS) - all block AXFR by default. In 2025, this works on units of percent of the target. But the test costs 2 seconds - it's a sin not to try.
NSEC Walking
For zones with NSEC based on NSEC (not NSEC3), you can serially list all records of the zone through ldns-walk. Does not work: zones with NSEC3 (hcheded names), zones without DNSSEC at all. It is even less common than the open AXFR.
Trade-off table: choice of tool for listing subdomains

1782161202620.png

Optimal combination for blue team: subfinder (passive fee) -> crt.sh (CT-logs) -> puredns (active brute-force in its domains) -> dnsx (Resolving reshed). To install and update the entire ProjectDiscovery stack, I advise pdtm (ProjectDiscovery Tool Manager) - it manages versions subfinder, dnsx and httpx At a single point: pdtm -install-all.
Shadow IT detection: from dangling DNS to the forgotten Jenkins
Shadow IT in the context of the external perimeter - any assets that the IT department does not know about, but which are available from the Internet and are tied to the domains of the organization. According to MITRE ATT&CK is Domain Properties (T1590.001) and WHOIS (T1596.002)
Typical Finds on Real Projects
At dozens of pre-engagement phases, I encountered repetitive patterns:

Corporate Jira and Confluence, registered by employees on personal post offices and placed on subdomains of the form pm.company.com or wiki.team-company.com. WHOIS shows registration to an individual, DNS points to Atlasian Cloud. None of these services appeared in the CMDB.

Jenkins and GitLab CI without authorization or with default creeds on subdomins ci., build., deploy.*. Detected through CT blogs: Let's Encrypt certificate is issued - it means that the subdomain is fixed. On one project Jenkins on build.company.com There was an password for eight months. Eight.

Staging and enterprise application devestations with working data (sometimes with a copy of the product base). Patterns of names: staging., dev., uat., test., sandbox.*.

Cloud resources: S3-bouquets with predictable names (company-backups, company-uploads), Azure Blob Storage, Google Cloud Storage. Cloud storage configuration errors are one of the most common reasons for leaks, as the typical advisory patterns of AWS and GCP.
Subdomain Takeover via dangling DNS
Dangling DNS - DNS record (usually CNAME) indicating a resource that no longer exists. Classical script: CNAME blog.example.com -> example.herokuapp.com, the application on Heroku has been deleted, but the DNS record remains. The attacker registers a new application with the same name on Heroku - and controls blog.example.com.

How to find: After listing subdomains, check each CNAME for the status of a target resource. Tool subjack Automates verification for dozens of cloud providers. List of vulnerable services supported in the repository can-i-take-over-xyz(GitHub - EdOverflow/can-itake-wer-xyz: "Can I take over XYZ?" - a list of services how and to claim (sub)doamus with dangling DNS records.) on GitHub.

Typical goals of takeover: GitHub Pages, Heroku, AWS S3, Azure Blob Storage, Shopify, Fastly, Pantheon. Not all dangling CNAMEs are vulnerable - CloudFront with OAI, the Azure CDN with domain verification protects against unauthorized interception.
Step-by-step pipeline: from domain to map of the attacked surface
Adjustments to the environment
• OS: GNU/Linux (Kali, Ubuntu 22.04+), macOS; Windows - via WSL2
• RAM: minimum 4 GB for subfinder and dnsx, 8 GB is recommended when working with Amass or BBOT on large scumpes (100+ root domains)
• Dependence: Go 1.21+ (for installation of ProjectDiscovery tools), jq, curl, dig
• Installation: go install github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latestor through pdtm: go install github.com/projectdiscovery/pdtm/cmd/pdtm@latest && pdtm -install-all
• Network: Internet access is needed for requests to API sources and DNS servers. Offline mode is not possible
• API Keys (recommended): SecurityTrails, Censys, Shodan, VirusTotal, Chaos (ProjectDiscovery). Subfinder operates without keys, but the coating falls by 40-60%. Keys are configured in ~/.config/subfinder/provider-config.yaml
Step 1. Definition of seed domains
Start with the root domains of the organization. Sources:
• WHOIS requests for known domains: Revision Organization -> reverse organization search via SecurityTrails or DomainTools
• SPF/DKIM/DMARC Main Domain Recording through dig txt example.com- SPF often lists of third-party services, disclosing the SaaS used
• ASN analysis: define an autonomous system of the organization through whois -h whois.radb.net -- '-i origin AS12345'- get all IP prefixes registered by the company
Step 2. Passive transfer of subdomains
Run subfinder with all custom-made API keys: subfinder -d example.com -all -o subs_passive.txt. In parallel - a request to crt.sh (camp curl team from the section above). Combination of results:
Bash:
cat subs_passive.txt subs_ct.txt | sort -u > subs_combined.txt
echo $(wc -l < subs_combined.txt)"
dnsx -l subs_combined.txt -resp -a -cname -o resolved.txt
dnsx verifies which of the subdomains found will actually be resolved, and shows the current A-records and CNAME. Subdomains with CNAME to external services are candidates for subdomain takeover.
Step 3. Passive DNS analysis (historical records)
For each root domain, request a history through the SecurityTrails API or web interface. Look for: IP addresses that used to be in A-records, but now there are no - check if the resource is alive on this IP through curl -k https://<IP>. A change of hosting provider (IP from the corporate range -> IP cloud provider) may indicate a cloud resource left unattended.
Step 4. Building a map and classification
Identify the hosting provider by IP (ASN lookup through whois or API ipinfo.io). Check the HTTP responses on each subdomain:
Bash:
httpx -l resolved.txt -status-code -title -tech-detect -o http_results.txt
httpx will show the HTTP status, page title and the technology used – according to these data, Jenkins, GitLab, WordPress and other economy are quickly calculated. Classify every asset: production, staging, dev, unknown.
Step 5. Search for dangling DNS
From the resolved.txt allocate subdomains from CNAME to external services (Heroku, GitHub Pages, S3, Azure). For everyone, check: is there a target resource? If CNAME returns NXDOMAIN or a specific provider (404 GitHub, "No such app" Heroku) is a dangling DNS.
Step 6. Artifact for the team
The final result is the table of assets with speakers: subdomain, IP, ASN/provider, HTTP status, technology, availability in CMDB (yes/no), recommendation (deactivate / include in monitoring / transfer to management). This artifact is not a one-time document, but a living table updated monthly.
 
Top Bottom