AIL-framework

AIL Framework Open-source framework for the collection, crawling, processing, and analysis of unstructured information. AIL framework is an open-source platform to collect, crawl, process and analyse unstructured data from the clear web, Tor, I2P, chats, files and external feeds. Originally...

HTTPLoot

HTTPLoot An automated tool which can simultaneously crawl, fill forms, trigger error/debug pages and "loot" secrets out of the client-facing code of sites. Usage To use the tool, you can grab any one of the pre-built binaries from the Releases section of the repository. If you want to build the...

Photon

Photon Incredibly fast crawler designed for OSINT. Photon Wiki • How To Use • Compatibility • Photon Library • Contribution • Roadmap Key Features Data Extraction Photon can extract the following data while crawling: URLs in-scope & out-of-scope URLs with parameters example.com/gallery.php?id=2...

gospider

GoSpider GoSpider - Fast web spider written in Go Painless integrate Gospider into your recon workflow? this project was part of Osmedeus Engine. Check out how it was integrated at @OsmedeusEngine Installation GO install root@kitploit: GO111MODULE=on go install...

SourceWolf

SourceWolf ​ Tested environments: Windows, MAC, linux, and windows subsystem for linux WSL Sponsors Support this project by becoming a sponsor. Checkout these awesome sponsors: Oxylabs provides premium proxies with 100M+ IPs across 195 countries and ready-to-use Scraper APIs. You can use this...

festin

FestIN el potente buscador de buckets S3 y descubridor de contenido ¿Qué es FestIn? ¿Por qué Festin? Instalación Usando Python Usando Docker Opciones completas Uso Configurar dominios de búsqueda Concurrencia Configuración del rastreo HTTP Gestión de resultados Uso de proxy Opciones de DNS Soport...

responsible-disclosure-email-gathering

Responsible Disclosure Email Gathering Purpose You may have found a vulnerability on a host but the said host/org does not have an active program on HackerOne or BugCrowd. If this is the case this workflow is for you! Broadly it will crawl the website and look for any security related emails from...

domain_analyzer

Domain Analyzer v0.8.3 What Domain analyzer is a security analysis tool that automatically discovers and reports information about a given domain. Its main purpose is to analyze domains in an unattended way. It has many crazy features, such as getting more domains from DNS zones, automatic nmap,...

domain_analyzer

Domain Analyzer v0.8.3 Qué es Domain analyzer es una herramienta de análisis de seguridad que descubre e informa automáticamente información sobre un dominio dado. Su propósito principal es analizar dominios de manera desatendida. Tiene muchas características locas, como obtener más dominios de...

subcrawl

SubCrawl SubCrawl es un framework desarrollado por Patrick Schläpfer, Josh Stroschein y Alex Holland del equipo de Threat Research de HP Inc. SubCrawl está diseñado para encontrar, escanear y analizar directorios abiertos. El framework es modular, compuesto por cuatro componentes: módulos de...

katana

Un framework de rastreo y spidering de última generación Características • • • • • •...

jshole

jshole Un escáner simple de vulnerabilidades de componentes JavaScript, basado en RetireJS. ¿Por qué usar JShole en lugar de RetireJS? Por defecto, RetireJS solo busca en una página, pero JShole intenta rastrear todas las páginas. Comenzar Requisitos requests Instalación git clone...

dark-web-osint-tools

🕵️‍♂️ OSINT Tools for the Dark Web 🔎 Dark Web Search Engine Tools Katana - https://github.com/adnane-X-tebbaa/Katana OnionSearch - https://github.com/megadose/OnionSearch Darkdump - https://github.com/josh0xA/darkdump Ahmia Search Engine - ahmia.fi, https://github.com/ahmia/ahmia-site Darkus -...

Lurkstrain

Lurkstrain An all in one admin panel crawler for pentesters...

CVE-2026-84366

Scrapy is a high-level web crawling and scraping framework for Python...

UBUNTU-CVE-2026-84366

Scrapy is a high-level web crawling and scraping framework for Python...

CVE-2026-76993

A vulnerability was determined in GreyDGL PentestGPT up to 1.0.0. This vulnerability affects unknown code of the component Web-Page Crawling. Executing a manipulation of the argument Traceback can lead to injection. The attack can be executed remotely. A high complexity level is associated with...

EUVD-2026-63377

A vulnerability was determined in GreyDGL PentestGPT up to 1.0.0. This vulnerability affects unknown code of the component Web-Page Crawling. Executing a manipulation of the argument Traceback can lead to injection. The attack can be executed remotely. A high complexity level is associated with...

CVE-2026-76993 GreyDGL PentestGPT Web-Page Crawling injection

A vulnerability was determined in GreyDGL PentestGPT up to 1.0.0. This vulnerability affects unknown code of the component Web-Page Crawling. Executing a manipulation of the argument Traceback can lead to injection. The attack can be executed remotely. A high complexity level is associated with...

[SECURITY] Fedora 44 Update: python-scrapy-2.14.2-1.fc44

Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing...