katana

A next-generation crawling and spidering framework Features • Installation • Usage • Scope • Config • Filters • Join Discord Features Fast And fully configurable web crawling Standard and Headless mode JavaScript parsing / crawling Customizable automatic form filling Scope control - Preconfigured...

cariddi

Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more Coded with 💙 by edoardottt Share on Twitter! Install • Usage • Get Started • Changelog • Contributing • License Installation 📡 Homebrew brew install cariddi Snap sudo snap install caridd...

xurlfind3r

xurlfind3r xurlfind3r is a command-line utility designed to discover URLs for a given domain in a simple, efficient way. It works by gathering information from a variety of passive sources, meaning it doesn't interact directly with the target but instead gathers data that is already publicly...

urx

Extracts URLs from OSINT Archives for Security Insights. Urx is a command-line tool designed for collecting URLs from OSINT archives, such as the Wayback Machine and Common Crawl. Built with Rust for efficiency, it leverages asynchronous processing to rapidly query multiple data sources. This too...

jshole

jshole A simple JavaScript components vulnrability scanner, based on RetireJS. Why use JShole instead of RetireJS? By default, RetireJS only searches one page, but JShole tries to crawl all pages. Get Started Requirements requests Install git clone https://github.com/callforpapers-source/jshole.g...

galer

galer /' '/'' | | /'' ' | || | | | '\ ','\ | /' @dwisiswant0 A fast tool to fetch URLs from HTML attributes by crawl-in. Inspired by the @omespino Tweet, which is possible to extract src, href, url and action values by evaluating JavaScript through Chrome DevTools Protocol. Resources...

paskto

Paskto - Passive Web Scanner Paskto will passively scan the web using the Common Crawl internet index either by downloading the indexes on request or parsing data from your local system. URLs are then processed through Nikto and known URL lists to identify interesting content. Hash signatures are...

Crawl4AI < 0.8.0 - Unauthenticated Remote Code Execution via Hooks Parameter

Crawl4AI versions prior to 0.8.0 contain a remote code execution vulnerability in the Docker API deployment. The /crawl endpoint accepts a hooks parameter containing Python code that is executed using exec. The import builtin was included in the allowed builtins, allowing unauthenticated remote...

warcannon

WARCannon - Catastrophically powerful parallel WARC processing WARCannon was built to simplify and cheapify the process of 'grepping the internet'. With WARCannon, you can: Build and test regex patterns against real Common Crawl data Easily load Common Crawl datasets for parallel processing Scale...

CVE-2026-109238

Crawl4AI 0.7.0 before 0.9.4 contains a server-side request forgery vulnerability that allows API clients to reach internal addresses by enabling linkpreviewconfig on /crawl requests. Attackers can supply a controlled page with includeexternal and includepatterns set, causing AsyncUrlSeeder to fet...

CVE-2026-109238

The Crawl4AI library versions 0.7.0 before 0.9.4 is vulnerable to a Server-Side Request Forgery (SSRF) . This flaw exists in the AsyncUrlSeeder component when link_preview_config is enabled on /crawl requests. An attacker can provide a controlled page with specific include_external and include_pa...

domdig

DOMDig DOMDig es un escáner de DOM XSS que se ejecuta dentro del navegador web Chromium y puede escanear aplicaciones de página única SPA de forma recursiva. A diferencia de otros escáneres, DOMDig puede rastrear cualquier aplicación web incluyendo gmail manteniendo un registro de las...

gau

getallurls gau getallurls gau obtiene URLs conocidas de AlienVault's Open Threat Exchange, Wayback Machine, Common Crawl y URLScan para cualquier dominio dado. Inspirado en waybackurls de Tomnomnom. Recursos Uso Instalación Nota sobre ohmyzsh Uso: Ejemplos: $ printf example.com | gau $ cat...

warcannon

WARCannon - Procesamiento paralelo de WARC catastróficamente potente WARCannon fue construido para simplificar y abaratar el proceso de 'grep en internet'. Con WARCannon, puedes: Construir y probar patrones de expresiones regulares contra datos reales de Common Crawl Cargar fácilmente conjuntos d...

paskto

Paskto - Escáner Web Pasivo Paskto escaneará pasivamente la web utilizando el índice de Common Crawl, ya sea descargando los índices a petición o analizando datos de su sistema local. Las URL se procesan luego a través de Nikto y listas de URL conocidas para identificar contenido interesante...

skipfish

=========================================== skipfish - escáner de seguridad de aplicaciones web http://code.google.com/p/skipfish/ Escrito y mantenido por: Michal Zalewski [email protected] Niels Heinen [email protected] Sebastian Roschke [email protected] Copyright 2009 - 2012...

CVE-2026-82286-gpt-crawler-Arbitrary-File-Write

CVE-2026-82286 — gpt-crawler Unauthenticated Arbitrary File Write Proof-of-concept for an unauthenticated arbitrary file write in BuilderIO/gpt-crawler ≈22k ★ via the outputFileName parameter of the POST /crawl API endpoint. | ---|--- CVE| CVE-2026-82286 Product| gpt-crawler BuilderIO Affected|...

EUVD-2026-44639

PraisonAI: DNS rebinding bypass in webcrawl SSRF protection allows internal response disclosure...

PYSEC-2026-4032 Decepticon: Role-boundary forgery via ChatML special-token literals in web crawl output composed into LLM context

Summary Decepticon wraps web crawl results — the output of agent reconnaissance against target services — into LLM messages without neutralizing ChatML special-token literals. Under the BYOK Bring Your Own Key deployment model, users configure their own LLM credentials to any OpenAI-compatible...

PT-2026-105789

Summary Decepticon wraps web crawl results — the output of agent reconnaissance against target services — into LLM messages without neutralizing ChatML special-token literals. Under the BYOK Bring Your Own Key deployment model, users configure their own LLM credentials to any OpenAI-compatible...