Uncover Hidden Subdomains with Sublist3r and curl: Boost Your Bug Hunting Success

Hello Everyone, We know that bug bounty is growing popular day by day and many people are getting into this field which is great for the community but it also increases competition within the bug bounty hunters, and thus makes it harder to find bugs as they are already found by other hunters.

In this blog we will talk about some tips that can help you in findings bugs before others do !!

Let’s Begin !!!

Subdomain enumeration is an essential part of bug bounty hunting and web application security. Often, hidden or forgotten subdomains can house vulnerable applications, making them prime targets for hackers. In this guide, I’ll show you how to efficiently find subdomains using two powerful tools: Sublist3r and crt.sh with curl and jq.

Why Subdomain Enumeration Matters

Subdomains often host less secure services, test environments, or outdated versions of applications that attackers can exploit. By discovering these subdomains, bug hunters can uncover valuable targets for further testing, increasing the likelihood of finding vulnerabilities that might go unnoticed in a regular pentest.

Tools You’ll Need:

1. Sublist3r: A tool for listing subdomains of a website using various search engines.
2. crt.sh: A free certificate transparency log search engine that allows us to find subdomains associated with a given domain.
3. curl & jq: Simple command-line tools to query and process data from crt.sh.

Step 1: Using Sublist3r for Subdomain Enumeration

Sublist3r is a widely-used subdomain enumeration tool. It scrapes search engines like Google, Yahoo, Bing, and more to find subdomains related to the target domain.

Installation:

Sublist3r can be installed directly from its GitHub repository:

git clone https://github.com/aboul3la/Sublist3r.git
cd Sublist3r
pip install -r requirements.txt

Once installed, you can start enumerating subdomains with:

python sublist3r.py -d targetdomain.com -o subdomains.txt

• -d targetdomain.com: Specify the target domain you want to find subdomains for.
• -o subdomains.txt: Output the results to a file.

Why Sublist3r?

Sublist3r automates the process of querying various search engines for subdomains, making it a fast and reliable tool for reconnaissance. By pulling data from multiple sources, it often reveals subdomains that other tools might miss.

Step 2: Using crt.sh with curl to Find Subdomains

crt.sh is a certificate transparency log that tracks issued SSL certificates. It’s an excellent resource for finding subdomains since SSL certificates often list multiple domains and subdomains.

Here’s how you can leverage crt.sh with a simple curl command to enumerate subdomains:

Command Breakdown:

curl -s https://crt.sh/\?q\=domain.com\&output\=json | jq -r '.[].name_value' | grep -Po '(\w+\.\w+\.\w+)$' >> subdomains.txt

• curl -s https://crt.sh/\?q\=domain.com\&output\=json: This command queries crt.sh for certificates related to the domain (replace domain.com with your target domain).
• jq -r '.[].name_value': This parses the JSON response and extracts the subdomain values from the name_value field.
• grep -Po '(\w+\.\w+\.\w+)$': Filters the results to ensure we only get valid subdomains in the format subdomain.domain.tld.
• >> subdomains.txt: Appends the discovered subdomains to a file.

Why crt.sh?

Certificate transparency logs provide invaluable data for finding subdomains associated with SSL certificates. Often, you’ll find subdomains that aren’t publicly listed anywhere else because companies issue certificates for internal services or development environments.

Step 3: Combining the Results for Maximum Coverage

Once you’ve collected subdomains using both Sublist3r and crt.sh, you’ll want to combine them and remove duplicates to create a comprehensive list. You can do this easily with the sort and uniq commands:

sort subdomains.txt | uniq > final_subdomains.txt

This will sort and remove any duplicate entries from your list, leaving you with a clean file of unique subdomains to target.

Bonus: Automating Subdomain Discovery with a Simple Script

You can combine both methods into a single bash script to automate the process of subdomain enumeration:

#! /bin/bash
domain=$1
# Sublist3r command
python sublist3r.py -d $domain -o subdomains_sublister.txt
# crt.sh with curl and jq
curl -s https://crt.sh/\?q\=$domain\&output\=json | jq -r '.[].name_value' | grep -Po '(\w+\.\w+\.\w+)$' >> subdomains_crt.txt
# Combine and remove duplicates
cat subdomains_sublister.txt subdomains_crt.txt | sort | uniq > final_subdomains.txt
echo "Subdomain enumeration complete. Results saved in final_subdomains.txt"

Save this as subdomain_enum.sh, give it execute permissions, and run it as follows:

chmod +x subdomain_enum.sh
./subdomain_enum.sh domain.com

Now, you’ve got a powerful, automated solution to enumerate subdomains using both Sublist3r and crt.sh.

Conclusion

Subdomain enumeration is a crucial first step in any bug bounty or penetration testing engagement. By leveraging tools like Sublist3r and crt.sh, you can uncover hidden subdomains that might house valuable attack vectors. Combining these tools increases your coverage, making sure no stone is left unturned.

Thank You !

My X.com