Open Source Intelligence (OSINT) Lab: DNS Reconnaissance, RDAP/WHOIS Data, and Metadata Extraction
Conduct authorized technical reconnaissance against an exposed infrastructure target to uncover hidden subdomains, examine RDAP domain registration trails, and extract GPS coordinates and device serials from uploaded images.
1. Production Scenario & Incident
During a security assessment, you are provided with a corporate target domain. The organization claims that its internal administrative portals, staging environments, and database clusters are completely private. However, by executing systematic DNS record reconnaissance (querying historical TXT records, SPF rules, and DNSSEC keys), querying RDAP registrar endpoints, and extracting EXIF metadata from images uploaded to public asset servers, you uncover critical exposure of employee hardware, IP addresses, and private staging hostnames. You must document the intelligence collection steps and remediate the information leaks.
2. Architectural Background & RFC Standards
Open Source Intelligence (OSINT) is the collection, correlation, and analysis of publicly available data to discover technical infrastructure, relationships, and operational vulnerabilities without launching active network attacks.
- DNS Reconnaissance: SPF records (`v=spf1 include:...`) frequently reveal third-party SaaS vendors and internal mail servers; TXT records disclose domain verification tokens for cloud infrastructure.
- RDAP (Registration Data Access Protocol, RFC 7480-7484): The modern, RESTful JSON successor to legacy port-43 WHOIS, providing structured registrant, registrar, network ASN, and abuse contact data.
- EXIF Metadata (Exchangeable Image File Format): Embedded metadata in JPEG, PNG, and WebP files containing camera models, shutter speeds, timestamps, lens serial numbers, and precise GPS latitude/longitude coordinates.
- Certificate Transparency (CT) Logs: Public append-only cryptographic ledgers of all issued TLS certificates, allowing instant discovery of previously unpublished subdomains (e.g., `vpn.corp.com` or `dev-api.corp.com`).
3. Hands-On Lab Instructions
Inspect the simulated public DNS records for `target-corp.com` in the OSINT workbench.
Parse the SPF TXT record to identify internal mail relays and cloud hosting providers.
Query Certificate Transparency logs to discover exposed development and staging subdomains.
Upload a simulated sample photo to the EXIF Metadata Inspector and extract the embedded GPS coordinates and device model.
Apply metadata stripping and DNS record hygiene to eliminate operational security leaks.
4. Interactive Lab Workbench
Inspect live DNS records, parse RDAP network blocks, extract EXIF camera and GPS metadata, and uncover subdomain certificates.
Found 3 staging subdomains via Certificate Transparency. Extracted GPS Coordinates: 37.7749° N, 122.4194° W (San Francisco, CA).
5. Diagnostic Comprehension Questions
How do Certificate Transparency (CT) logs allow security researchers to discover hidden subdomains without brute-force scanning?
Reveal Answer & Explanation →
✓ Correct Answer: Certificate Authorities are required by browsers to publicly log every TLS certificate issued for any domain, creating a searchable public record of all hostnames.
Under RFC 6962, publicly trusted Certificate Authorities must publish all issued TLS certificates to public append-only logs. Because developers request certificates for new subdomains (e.g., `staging-api.example.com`), CT log monitoring instantly reveals these hostnames.
What sensitive privacy information can be extracted from uncleaned JPEG photos taken on modern smartphones?
Reveal Answer & Explanation →
✓ Correct Answer: Exact GPS latitude, longitude, altitude, device make, camera serial number, and timestamp of capture embedded in EXIF metadata.
Modern smartphones automatically embed EXIF metadata into photos, including high-precision GPS coordinates, capture date/time, camera focal length, and hardware serial numbers. If not scrubbed before publication, this exposes the exact physical location of users.
6. Progressive Diagnostic Hints
💡 Hint #1▼
💡 Hint #2▼
💡 Hint #3▼
7. Verified Production Solution
Automate image EXIF metadata stripping in media upload pipelines, audit public DNS TXT/SPF records to avoid leaking internal hostnames, and use wildcard TLS certificates to obscure staging environments from Certificate Transparency logs.
// Node.js Production EXIF Metadata Stripper using Sharp
import sharp from 'sharp';
export async function processAndSanitizeUpload(inputBuffer: Buffer): Promise<Buffer> {
// Strip all EXIF, GPS, and device metadata while preserving image quality
const sanitizedImageBuffer = await sharp(inputBuffer)
.rotate() // Auto-orient based on EXIF before stripping orientation tag
.withMetadata({
// Explicitly omit GPS, camera serials, and user comments
exif: {} // Clears EXIF header block entirely
})
.toBuffer();
return sanitizedImageBuffer;
}
// Recommended Wildcard TLS Configuration for Staging Privacy
// Instead of requesting cert for "dev-secret-api.example.com",
// request "*.staging.example.com" to prevent CT log subdomain leaks.- ✓1. Implement automated EXIF stripping in all user image upload endpoints using libraries like `sharp` or `exiftool`.
- ✓2. Cleanse DNS TXT records of outdated SaaS verification strings (e.g., stale Google/Atlassian/Facebook domain tokens).
- ✓3. Use wildcard TLS certificates (`*.example.com`) for internal or staging infrastructure to prevent unique subdomains from being permanently indexed in public Certificate Transparency logs.
- ✓4. Ensure RDAP/WHOIS privacy protection is enabled at the domain registrar to prevent exposure of employee names, phone numbers, and physical addresses.
8. In-Depth Technical Explanation
"If an organization requests individual TLS certificates for internal hostnames, Certificate Authorities deterministically publish the fully qualified domain names to public append-only CT logs; consequently, OSINT monitors instantly index the hidden infrastructure without sending a single network packet to the target domain."
Underlying Root Cause Analysis
Open Source Intelligence operates because modern internet protocols and services are designed to be discoverable and verifiable. Certificate Transparency logs enforce accountability against rogue CAs, but inadvertently act as an automated asset discovery directory for attackers. EXIF headers provide photography software with camera metrics, but inadvertently leak geographic coordinates. Understanding these data trails enables defensive engineers to sanitize public assets.
Production Engineering Best Practices
- →Always scrub EXIF metadata on server upload pipelines before serving images to public users.
- →Monitor Certificate Transparency logs for your domains using tools like crt.sh or Facebook CT alerts.
- →Conduct regular external perimeter scans to identify unintentionally exposed DNS records and API documentation.
Common Pitfalls to Avoid
- ✕Relying on "security through obscurity" by using obscure subdomain names like `admin-portal-xyz987.example.com` (which show up immediately in CT logs).
- ✕Uploading employee profile pictures to company portals containing the GPS coordinates of their private homes.
- ✕Leaving legacy SPF records with obsolete IP blocks that attackers can claim on cloud providers.
Écosystème Technique Associé et Prochaines Étapes
Articles Techniques Associés
What Is HTTP (Hypertext Transfer Protocol)?
Foundational stateless request-response protocol of the World Wide Web.
HTTP/1.1 vs HTTP/2: Multiplexing & Performance
Head-of-line blocking compared to binary framing and stream multiplexing.
Troubleshooting: HTTP 403 Forbidden Error
Diagnose server authorization denial, WAF IP blocks, and permission issues.
Questions Connexes
Why Is POST Non-Idempotent While PUT Is Idempotent?
Answer: PUT replaces or creates a specific resource deterministically, while POST processes input according to server logic, typically creating new items on repeated requests.
What Is the Difference Between ETag and Last-Modified?
Answer: Last-Modified relies on 1-second filesystem timestamps, while ETag provides an exact cryptographic digest of content, eliminating false positive cache misses.
Can HTTP/2 Multiplex Requests Over a Single TCP Connection?
Answer: Yes, HTTP/2 breaks requests into binary frames with stream IDs, allowing interleaved requests and responses concurrently over one TCP socket.
Spécifications et Références Développeur
Poursuivre l’Apprentissage
HTTP Protocol & Header Inspection Lab
Craft raw HTTP headers and observe conditional 304 responses in the interactive workbench.