Skip to content
HackIndex logo

HackIndex

Web Content Discovery

7 min read Apr 24, 2026

Content discovery finds endpoints the app doesn't link to: admin panels, backups, APIs, debug paths, and unlinked apps. Run it after tech fingerprinting so you know which extensions and wordlists match the stack. Discovery output feeds into parameter crawling and exposed git directory checks.

ffuf Baseline

ffuf is the default choice. Fast, flexible filtering, and handles custom 404s cleanly:

┌──(kali㉿kali)-[~]
└─$ ffuf -w /usr/share/seclists/Discovery/Web-Content/common.txt -u http://$TARGET_IP:$PORT/FUZZ -mc 200,204,301,302,307,401,403
Explain command
-w /usr/share/seclists/Discovery/Web-Content/common.txt Wordlist file path used for fuzzing input.
-u http://$TARGET_IP:$PORT/FUZZ Target URL with FUZZ keyword marking the injection point.
$TARGET_IP:$PORT Variable placeholders for the target host IP and port.
-mc 200,204,301,302,307,401,403 Match only responses with these specific HTTP status codes.

301/302 to a new path is real content. 401/403 is still interesting — it's a valid endpoint behind auth. If everything returns a hit, you're seeing a custom 404 and need size filtering.

Handle Custom 404s with Size Filtering

Measure the error response size, then exclude it:

┌──(kali㉿kali)-[~]
└─$ curl -sk http://$TARGET_IP:$PORT/thisshouldnotexist | wc -c 
4242
Explain command
-s Silent mode; suppresses progress meter and error messages.
-k Allow insecure TLS connections, skipping certificate verification.
http://$TARGET_IP:$PORT/thisshouldnotexist Target URL with variable host/port to a non-existent path.
┌──(kali㉿kali)-[~]
└─$ ffuf -w /usr/share/seclists/Discovery/Web-Content/common.txt -u http://$TARGET_IP:$PORT/FUZZ -mc 200,204,301,302,307,401,403 -fs 4242
Explain command
-w /usr/share/seclists/Discovery/Web-Content/common.txt Wordlist file path used for fuzzing input.
-u http://$TARGET_IP:$PORT/FUZZ Target URL with FUZZ keyword as the injection point.
$TARGET_IP:$PORT Variable placeholders for the target host IP and port.
-mc 200,204,301,302,307,401,403 Match only responses with these HTTP status codes.
-fs 4242 Filter out responses with a body size of 4242 bytes.

Replace 4242 with the actual size. For apps returning varying sizes on 404, filter by word count with -fw instead.

Stack-Matched Extensions

Match extensions to what fingerprinting revealed. Wrong extensions waste significant time on large wordlists:

┌──(kali㉿kali)-[~]
└─$ ffuf -w /usr/share/seclists/Discovery/Web-Content/common.txt -u http://$TARGET_IP:$PORT/FUZZ -e .php,.php5,.phtml,.html,.txt,.bak,.old,.zip -mc 200,204,301,302,307,401,403
Explain command
-w /usr/share/seclists/Discovery/Web-Content/common.txt Wordlist file used for fuzzing input.
-u http://$TARGET_IP:$PORT/FUZZ Target URL with FUZZ keyword marking the injection point.
$TARGET_IP:$PORT Variable placeholders for the target host IP and port.
-e .php,.php5,.phtml,.html,.txt,.bak,.old,.zip File extensions to append to each wordlist entry.
-mc 200,204,301,302,307,401,403 Match only responses with these HTTP status codes.
┌──(kali㉿kali)-[~]
└─$ ffuf -w /usr/share/seclists/Discovery/Web-Content/common.txt -u http://$TARGET_IP:$PORT/FUZZ -e .asp,.aspx,.ashx,.asmx,.config,.txt,.bak -mc 200,204,301,302,307,401,403
Explain command
-w /usr/share/seclists/Discovery/Web-Content/common.txt Wordlist file to use for fuzzing input.
-u http://$TARGET_IP:$PORT/FUZZ Target URL with FUZZ keyword as the injection point.
$TARGET_IP:$PORT Variable placeholders for target host IP and port.
-e .asp,.aspx,.ashx,.asmx,.config,.txt,.bak Append these extensions to each wordlist entry during fuzzing.
-mc 200,204,301,302,307,401,403 Match only responses with these HTTP status codes.
┌──(kali㉿kali)-[~]
└─$ ffuf -w /usr/share/seclists/Discovery/Web-Content/common.txt -u http://$TARGET_IP:$PORT/FUZZ -e .jsp,.jspx,.do,.action,.txt,.xml -mc 200,204,301,302,307,401,403
Explain command
-w /usr/share/seclists/Discovery/Web-Content/common.txt Wordlist file to use for fuzzing input.
-u http://$TARGET_IP:$PORT/FUZZ Target URL with FUZZ as the injection point placeholder.
$TARGET_IP:$PORT Variable placeholders for the target host IP and port.
-e .jsp,.jspx,.do,.action,.txt,.xml Append these extensions to each wordlist entry during fuzzing.
-mc 200,204,301,302,307,401,403 Match only responses with these specific HTTP status codes.

Recursive Discovery

ffuf handles recursion but feroxbuster is faster and more reliable for deep trees:

┌──(kali㉿kali)-[~]
└─$ ffuf -w /usr/share/seclists/Discovery/Web-Content/common.txt -u http://$TARGET_IP:$PORT/FUZZ -mc 200,204,301,302,307,401,403 -recursion -recursion-depth 2
Explain command
-w /usr/share/seclists/Discovery/Web-Content/common.txt Wordlist file to use for fuzzing input.
-u http://$TARGET_IP:$PORT/FUZZ Target URL with FUZZ keyword marking the injection point.
$TARGET_IP:$PORT Placeholder for target host IP address and port number.
-mc 200,204,301,302,307,401,403 Match only responses with these HTTP status codes.
-recursion Enable recursive fuzzing on discovered directories.
-recursion-depth 2 Limit recursive fuzzing to a maximum depth of 2 levels.
┌──(kali㉿kali)-[~]
└─$ feroxbuster -u http://$TARGET_IP:$PORT/ -w /usr/share/seclists/Discovery/Web-Content/common.txt -x php,html,txt,js,json -t 30 -k
Explain command
-u http://$TARGET_IP:$PORT/ Target URL to perform directory/file brute-forcing against.
-w /usr/share/seclists/Discovery/Web-Content/common.txt Wordlist file used for brute-forcing paths and filenames.
-x php,html,txt,js,json File extensions to append to each wordlist entry during fuzzing.
-t 30 Number of concurrent threads to use for scanning.
-k Disable TLS certificate validation for HTTPS targets.
┌──(kali㉿kali)-[~]
└─$ feroxbuster -u http://$TARGET_IP:$PORT/ -w /usr/share/seclists/Discovery/Web-Content/common.txt -x php,html,txt,js,json -C 404,302 -o ferox_results.txt
Explain command
-u Target URL to begin directory/file brute-forcing against.
http://$TARGET_IP:$PORT/ Base URL with variable host and port placeholders.
-w Wordlist file path to use for brute-forcing.
/usr/share/seclists/Discovery/Web-Content/common.txt Common web content wordlist from SecLists collection.
-x File extensions to append to each wordlist entry during fuzzing.
php,html,txt,js,json Extensions to test: PHP, HTML, plain text, JavaScript, and JSON.
-C HTTP status codes to filter out (exclude) from results.
404,302 Suppresses responses with Not Found and Redirect status codes.
-o Output file path to save scan results.
ferox_results.txt File where discovered results will be written.

Authenticated Discovery

Run a second pass authenticated to find endpoints only visible after login:

┌──(kali㉿kali)-[~]
└─$ ffuf -w /usr/share/seclists/Discovery/Web-Content/common.txt -u http://$TARGET_IP:$PORT/FUZZ -H "Cookie: session=REPLACE_ME" -mc 200,204,301,302,307,401,403
Explain command
-w /usr/share/seclists/Discovery/Web-Content/common.txt Wordlist file to use for fuzzing input.
-u http://$TARGET_IP:$PORT/FUZZ Target URL with FUZZ keyword marking the injection point.
$TARGET_IP:$PORT Placeholder for the target host IP and port number.
-H "Cookie: session=REPLACE_ME" Adds a custom Cookie header with a session token to each request.
-mc 200,204,301,302,307,401,403 Match only responses with these specific HTTP status codes.
┌──(kali㉿kali)-[~]
└─$ ffuf -w /usr/share/seclists/Discovery/Web-Content/common.txt -u http://$TARGET_IP:$PORT/FUZZ -u $USER:$PASSWORD -mc 200,204,301,302,307,401,403
Explain command
-w /usr/share/seclists/Discovery/Web-Content/common.txt Specifies the wordlist file to use for fuzzing.
-u http://$TARGET_IP:$PORT/FUZZ Target URL with FUZZ keyword as the injection point.
$TARGET_IP:$PORT Placeholder for the target host IP and port number.
-u $USER:$PASSWORD Specifies credentials (likely misused flag; intended for -u in some tools).
$USER:$PASSWORD Placeholder for authentication username and password credentials.
-mc 200,204,301,302,307,401,403 Match HTTP response codes to include in results.

Larger Wordlists for Deeper Coverage

When common.txt misses things, upgrade the wordlist. raft and directory-list are heavier but surface more results:

┌──(kali㉿kali)-[~]
└─$ ffuf -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt -u http://$TARGET_IP:$PORT/FUZZ -mc 200,204,301,302,307,401,403
Explain command
-w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt Wordlist file used for fuzzing directory names.
-u http://$TARGET_IP:$PORT/FUZZ Target URL with FUZZ keyword marking the injection point.
$TARGET_IP:$PORT Variable placeholders for the target host IP and port.
-mc 200,204,301,302,307,401,403 Match only responses with these specific HTTP status codes.
┌──(kali㉿kali)-[~]
└─$ ffuf -w /usr/share/seclists/Discovery/Web-Content/directory-list-2.3-medium.txt -u http://$TARGET_IP:$PORT/FUZZ -mc 200,204,301,302,307,401,403 -t 100
Explain command
-w /usr/share/seclists/Discovery/Web-Content/directory-list-2.3-medium.txt Wordlist file to use for fuzzing input.
-u http://$TARGET_IP:$PORT/FUZZ Target URL with FUZZ keyword marking the injection point.
$TARGET_IP:$PORT Placeholder for the target host IP and port number.
-mc 200,204,301,302,307,401,403 Match only responses with these HTTP status codes.
-t 100 Number of concurrent threads to use for requests.

Gobuster and dirsearch as Alternatives

┌──(kali㉿kali)-[~]
└─$ gobuster dir -u http://$TARGET_IP:$PORT/ -w /usr/share/seclists/Discovery/Web-Content/common.txt -x php,html,txt,js,json --random-agent -t 50 -o gobuster.txt
Explain command
dir Use directory/file brute-forcing mode.
-u Target URL to enumerate.
http://$TARGET_IP:$PORT/ Base URL with variable host and port placeholders.
-w Wordlist file path to use for brute-forcing.
/usr/share/seclists/Discovery/Web-Content/common.txt Common web content wordlist from SecLists.
-x File extensions to append to each wordlist entry.
php,html,txt,js,json Extensions to try: php, html, txt, js, and json.
--random-agent Use a random User-Agent string per request.
-t Number of concurrent threads to use.
50 Thread count value — 50 parallel requests.
-o Write output results to a specified file.
gobuster.txt Output file where results will be saved.
┌──(kali㉿kali)-[~]
└─$ dirsearch -u http://$TARGET_IP:$PORT/ -w /usr/share/seclists/Discovery/Web-Content/common.txt -e php,html,txt,js,json -i 200,204,301,302,307,401,403 -t 50 -o dirsearch.txt
Explain command
-u http://$TARGET_IP:$PORT/ Target URL to perform directory brute-forcing against.
-w /usr/share/seclists/Discovery/Web-Content/common.txt Wordlist file used for directory and file enumeration.
-e php,html,txt,js,json File extensions to append to each wordlist entry during scan.
-i 200,204,301,302,307,401,403 Only include responses matching these HTTP status codes.
-t 50 Number of concurrent threads to use during the scan.
-o dirsearch.txt Write scan output/results to the specified file.

References