We have been quite busy this year, but we would like to release a small project nonetheless: Curating a list of subdomains based on certificate transparency logs that we’ll happily share with the community and publish new results every hour.
Certificate Transparency is a project initiated by Google that tries to monitor all issued SSL/TLS certificates with the goal of identified and revoking mis-issued certificates. Certificate Authorities are requested to publish information about their issued certificates into the publicly accessible CT logs. That is why you will find all certificates for internetwache.org on crt.sh for example.
A log entry contains all information about a certificate, and therefore the fully qualified domain name (FQDN). Since the start of Let’s Encrypt, it has become really easy to obtain and use SSL-/TLS certificates free-of-cost. However, it does not support wildcard domains (yet), but it’s on the roadmap for January 2018. Until then, every subdomain has to be explicitly listed in the certificate (or multiple certificates used). This means, that subdomains whose existence was not known before (security by obscurity, e.g. xyz-asd.domain.tld…) are easy to discover now.
Undoubtedly this information can be interesting from a hacker’s perspective. Sebastian’s idea was to use the certstream python module to subscribe to Certificate Transparency log updates, parse the subdomain and save the result in a database. Each subdomain also comes with a counter to sort it by frequency. The resulting lists with the top 100/1000/10,000/100,000 subdomains is automatically exported to the “CT_Subdomains” GitHub repository on an hourly basis.
The files have the following scheme:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | |
Link to the GitHub “CT_subdomains” repository
One could use this list as the input for DNS reconnaissance tools like gobuster.
There has been previous research about security implications in regards to certificate transparency logs, for example by Hanno Boeck at the DEF CON 25.
The team of internetwache.org