What is Public Archive 

Public archives, like the Wayback Machine, OTX Alien vault, and URLscan.io are goldmines for security researchers and bug bounty hunters. These archives store historical snapshots of websites, providing a unique chance to analyze past versions of web applications and find overlooked vulnerabilities. By exploring these archives, researchers can identify security flaws that may still be exploitable and earn rewards through bug bounty programs.

This guide will show how to use public archives to uncover hidden bugs, sharpen hacking skills, and turn discoveries into profit.

Public Web Archive Services 

There are a lot of public archive services on the internet. Here are some prominent public archive services along with their features and API capabilities:

• Wayback Machine (web.archive.org)

• Common Crawl (index.commoncrawl.org)

• AlienVault OTX (otx.alienvault.com)

• URLScan (urlscan.io)

• VirusTotal (virustotal.com) 

Public Web Archive Scraper Tools 

However, scraping the data from the archive manually is time consuming, so let’s introduce various handy tools:

• Waybackurls written in Go lang by @tomnomnom

• Gau written in Go lang by @lc

• Gau Plus written in Go lang by @bp0lr

• Waymore written in Python by @xnl-h4ck3r

Uncovering Bugs with the Public Archives 

In this article, As we’ve described before. We will focus on How to find bugs by using public web archives & some useful tools rather than step-by-step guides on how to use tools.

Subdomain enumeration: 

Exploring public web archives is a fantastic method for passively and quickly identifying subdomains from the past that might still be operational or obsolete. Using tools such as Waymore and Unfurl, you can efficiently extract subdomains and apex domains from these archives.

Endpoints Discovery:

Often, web developers update a website’s front end but leave the back end unchanged, meaning old endpoints might still exist even if they are no longer visible. For example, we can find hidden endpoints in old versions of a site's robots.txt file using the Wayback Machine. Companies often list endpoints they don't want Google to index in this file, and over time they may add or remove endpoints. The Wayback Machine archives it all, so old endpoints that may have been removed could still work and contain interesting functionality to explore.

‍Tokens and sensitive information leak:

Using the Wayback Machine, we can find valuable information such as passwords, PII data, API keys, the Company's private invitation tokens, and other sensitive data.

‍Parameter Enumeration

Hidden or old parameter discovery is a crucial component of the reconnaissance phase in cybersecurity assessments. This technique involves retrieving parameters used in previous versions of an application but may no longer be visible or active in the current version. These parameters can often be leveraged to uncover potential security vulnerabilities despite being obsolete or hidden.

‍Js File Enumeration:

JavaScript files can be a goldmine for bug hunters, uncovering misconfigurations, exposed API keys, and secret endpoints. Using web archives, you can retrieve old JavaScript files and compare them to the current codebase to identify vulnerabilities

‍Recent findings in the Wild 

At this point, consider whether this type of bug is acceptable for a bounty. To clarify, we discovered a vulnerability that exposed customers’ transaction PDF files through Wayback Machine. By utilizing the above mentioned approach, we identified the issue, reported it, and were subsequently awarded $1,500 for our findings. 🤑.

‍Conclusion 

Public web archives are a crucial resource for bug hunters seeking to uncover hidden vulnerabilities in JavaScript files. Researchers can access historical data, identify security weaknesses, and track changes over time by utilizing tools such as Waybackurls, Waymore, Gau, Gau Plus, and Unfurl. Combined with other resources like Common Crawl, AlienVault OTX, URLScan, and VirusTotal, these methods provide a comprehensive approach to web security research.

Let’s start using this for fun & profit. 

‍