Photon

Photon Incredibly fast crawler designed for OSINT. Photon Wiki • How To Use • Compatibility • Photon Library • Contribution • Roadmap Sponsored By Key Features Data Extraction Photon can extract the following data while crawling: URLs in-scope & out-of-scope URLs with parameters...

AIL-framework

AIL Framework Open-source framework for the collection, crawling, processing, and analysis of unstructured information. AIL framework is an open-source platform to collect, crawl, process and analyse unstructured data from the clear web, Tor, I2P, chats, files and external feeds. Originally...

robin

Robin: AI-Powered Dark Web OSINT Tool Robin is an AI-powered tool for conducting dark web OSINT investigations. It leverages LLMs to refine queries, filter search results from dark web search engines, and provide an investigation summary. Installation • Robin as MCP • Troubleshooting • Contributi...

HTTPLoot

HTTPLoot An automated tool which can simultaneously crawl, fill forms, trigger error/debug pages and "loot" secrets out of the client-facing code of sites. Usage To use the tool, you can grab any one of the pre-built binaries from the Releases section of the repository. If you want to build the...

gospider

GoSpider GoSpider - Fast web spider written in Go Painless integrate Gospider into your recon workflow? this project was part of Osmedeus Engine. Check out how it was integrated at @OsmedeusEngine Installation GO install GO111MODULE=on go install github.com/jaeles-project/gospider@latest Docker...

SourceWolf

SourceWolf ​ Tested environments: Windows, MAC, linux, and windows subsystem for linux WSL Sponsors Support this project by becoming a sponsor. Checkout these awesome sponsors: Oxylabs provides premium proxies with 100M+ IPs across 195 countries and ready-to-use Scraper APIs. You can use this...

domain_analyzer

Domain Analyzer v0.8.3 What Domain analyzer is a security analysis tool that automatically discovers and reports information about a given domain. Its main purpose is to analyze domains in an unattended way. It has many crazy features, such as getting more domains from DNS zones, automatic nmap,...

responsible-disclosure-email-gathering

Responsible Disclosure Email Gathering Purpose You may have found a vulnerability on a host but the said host/org does not have an active program on HackerOne or BugCrowd. If this is the case this workflow is for you! Broadly it will crawl the website and look for any security related emails from...

festin

FestIN el potente buscador de buckets S3 y descubridor de contenido ¿Qué es FestIn? ¿Por qué Festin? Instalación Usando Python Usando Docker Opciones completas Uso Configurar dominios de búsqueda Concurrencia Configuración del rastreo HTTP Gestión de resultados Uso de proxy Opciones de DNS Soport...

katana

Un framework de rastreo y spidering de nueva generación Características • Instalación • Uso • Alcance • Configuración • Filtros • Únete a Discord Características Rastreo web rápido y totalmente configurable Modo Standard y Headless Análisis / rastreo de JavaScript Relleno automático de formulario...

domain_analyzer

Domain Analyzer v0.8.3 Qué es Domain analyzer es una herramienta de análisis de seguridad que descubre e informa automáticamente información sobre un dominio dado. Su propósito principal es analizar dominios de manera desatendida. Tiene muchas características locas, como obtener más dominios de...

subcrawl

SubCrawl SubCrawl es un framework desarrollado por Patrick Schläpfer, Josh Stroschein y Alex Holland del equipo de Threat Research de HP Inc. SubCrawl está diseñado para encontrar, escanear y analizar directorios abiertos. El framework es modular, compuesto por cuatro componentes: módulos de...

jshole

jshole Un escáner simple de vulnerabilidades de componentes JavaScript, basado en RetireJS. ¿Por qué usar JShole en lugar de RetireJS? Por defecto, RetireJS solo busca en una página, pero JShole intenta rastrear todas las páginas. Comenzar Requisitos requests Instalación git clone...

Lurkstrain

Lurkstrain An all in one admin panel crawler for pentesters...

CVE-2026-84366

Scrapy is a high-level web crawling and scraping framework for Python. Prior to 2.17.0, in scrapy/core/downloader/handlers/s3.py, Scrapy's S3DownloadHandler converts an S3-scheme bucket and key request into a plaintext HTTP request to the corresponding S3 endpoint unless request.meta"issecure" is...

UBUNTU-CVE-2026-84366

Scrapy is a high-level web crawling and scraping framework for Python. Prior to 2.17.0, in scrapy/core/downloader/handlers/s3.py, Scrapy's S3DownloadHandler converts an S3-scheme bucket and key request into a plaintext HTTP request to the corresponding S3 endpoint unless request.meta"issecure" is...

CVE-2026-76993

A vulnerability was determined in GreyDGL PentestGPT up to 1.0.0. This vulnerability affects unknown code of the component Web-Page Crawling. Executing a manipulation of the argument Traceback can lead to injection. The attack can be executed remotely. A high complexity level is associated with...

EUVD-2026-63377

A vulnerability was determined in GreyDGL PentestGPT up to 1.0.0. This vulnerability affects unknown code of the component Web-Page Crawling. Executing a manipulation of the argument Traceback can lead to injection. The attack can be executed remotely. A high complexity level is associated with...

CVE-2026-76993 GreyDGL PentestGPT Web-Page Crawling injection

A vulnerability was determined in GreyDGL PentestGPT up to 1.0.0. This vulnerability affects unknown code of the component Web-Page Crawling. Executing a manipulation of the argument Traceback can lead to injection. The attack can be executed remotely. A high complexity level is associated with...

PT-2026-79029

A vulnerability was determined in GreyDGL PentestGPT up to 1.0.0. This vulnerability affects unknown code of the component Web-Page Crawling. Executing a manipulation of the argument Traceback can lead to injection. The attack can be executed remotely. A high complexity level is associated with...