warcannon

WARCannon - Catastrophically powerful parallel WARC processing WARCannon was built to simplify and cheapify the process of 'grepping the internet'. With WARCannon, you can: Build and test regex patterns against real Common Crawl data Easily load Common Crawl datasets for parallel processing Scale...

paskto

Paskto - Passive Web Scanner Paskto will passively scan the web using the Common Crawl internet index either by downloading the indexes on request or parsing data from your local system. URLs are then processed through Nikto and known URL lists to identify interesting content. Hash signatures are...

xurlfind3r

xurlfind3r xurlfind3r is a command-line utility designed to discover URLs for a given domain in a simple, efficient way. It works by gathering information from a variety of passive sources, meaning it doesn't interact directly with the target but instead gathers data that is already publicly...

urx

Extracts URLs from OSINT Archives for Security Insights. Urx collects URLs for a domain from OSINT archives such as the Wayback Machine and Common Crawl. It queries every source in parallel, then filters, tests and formats the results for security work. Features Keyless sources: Wayback Machine,...

gau

getallurls gau getallurls gau obtiene URLs conocidas de AlienVault's Open Threat Exchange, Wayback Machine, Common Crawl y URLScan para cualquier dominio dado. Inspirado en waybackurls de Tomnomnom. Recursos Uso Instalación Nota sobre ohmyzsh Uso: Ejemplos: $ printf example.com | gau $ cat...

warcannon

WARCannon - Procesamiento paralelo de WARC catastróficamente potente WARCannon fue construido para simplificar y abaratar el proceso de 'grep en internet'. Con WARCannon, puedes: Construir y probar patrones de expresiones regulares contra datos reales de Common Crawl Cargar fácilmente conjuntos d...

paskto

Paskto - Escáner Web Pasivo Paskto escaneará pasivamente la web utilizando el índice de Common Crawl, ya sea descargando los índices a petición o analizando datos de su sistema local. Las URL se procesan luego a través de Nikto y listas de URL conocidas para identificar contenido interesante...

You Get What You Sample: Evaluating Sampling Strategies for Web Security Measurements

Web measurement studies rely on domain datasets such as Tranco to quantify the prevalence and impact of security issues at scale, but exhaustively analyzing these datasets is often infeasible because of the cost of advanced analysis techniques, requiring the use of sampling. Despite its widesprea...

Sigurlfind3R - A Reconnaissance Tool, It Fetches URLs From AlienVault's OTX, Common Crawl, URLScan, Github And The Wayback Machine

sigurlfind3r is a passive reconnaissance tool, it fetches known URLs from AlienVault's OTX , Common Crawl , URLScan , Github and the Wayback Machine. DiSCLAIMER: fetching urls from github is a bit slow. Usage sigurlfind3r -h This will display help for the tool. | |/ | | / / | |/ | | | | '| | || |...

Sigurls - A Reconnaissance Tool, It Fetches URLs From AlienVault's OTX, Common Crawl, URLScan, Github And The Wayback Machine

sigurls is a reconnaissance tool, it fetches URLs from AlienVault's OTX , Common Crawl , URLScan , Github and the Wayback Machine. Usage To display help message for sigurls use the -h flag: $ sigurls -h | | / | |/ | | | | '| / | \ \ | | | || | | | \ \ |/|, |,|| ||/ v1.3.1 |/ USAGE: sigurls...

CWFF - Create Your Custom Wordlist For Fuzzing

CWFF is a tool that creates a special High quality fuzzing/content discovery wordlist for you at the highest speed possible using concurrency and it's heavily inspired by @tomnomnom's Who, What, Where, When, Wordlist NahamCon2020. Usage CWFF -h --threads --github --subdomains --recursive...