Inventus

Inventus Inventus is a spider designed to find subdomains of a specific domain by crawling it and any subdomains it discovers. It's a Scrapy spider, meaning it's easily modified and extendable to your needs. Demo asciicast Requirements Linux -- I haven't tested this on Windows. Python 2.7 or Pyth...

SourceWolf

SourceWolf ​ Tested environments: Windows, MAC, linux, and windows subsystem for linux WSL Sponsors Support this project by becoming a sponsor. Checkout these awesome sponsors: Oxylabs provides premium proxies with 100M+ IPs across 195 countries and ready-to-use Scraper APIs. You can use this...

akca

AKCA Advanced Web Security Scanner Discover endpoints. Test web applications. Inspect the evidence. Installation · Usage · Workflow · Profiles · Coverage · Reports · Support · Features · Changelog AKCA is an open-source, evidence-oriented Dynamic Application Security Testing DAST scanner written ...

MANSPIDER

MANSPIDER Crawl SMB shares for juicy information. File content searching + regex is supported! What's New in v2.0 Manspider 2.0 is here! This brings significant improvements: New and improved text extraction powered by Kreuzberg - now supporting PDF, DOCX, XLSX, PPTX, images with OCR, and many...

gospider

GoSpider GoSpider - Fast web spider written in Go Painless integrate Gospider into your recon workflow? this project was part of Osmedeus Engine. Check out how it was integrated at @OsmedeusEngine Installation GO install GO111MODULE=on go install github.com/jaeles-project/gospider@latest Docker...

dark-web-osint-tools

🕵️‍♂️ OSINT Tools for the Dark Web 🔎 Dark Web Search Engine Tools Katana - https://github.com/adnane-X-tebbaa/Katana OnionSearch - https://github.com/megadose/OnionSearch Darkdump - https://github.com/josh0xA/darkdump Ahmia Search Engine - ahmia.fi, https://github.com/ahmia/ahmia-site Darkus -...

spider

Spider The fastest web crawler and scraper for Rust. spider.cloud · Guides · Docs · Examples · Discord Spider is a concurrency-first crawling engine written in Rust. It streams pages as they arrive, renders JavaScript only on pages that need it, and scales from a single script to a distributed...

Photon

Photon Incredibly fast crawler designed for OSINT. Photon Wiki • How To Use • Compatibility • Photon Library • Contribution • Roadmap Sponsored By Key Features Data Extraction Photon can extract the following data while crawling: URLs in-scope & out-of-scope URLs with parameters...

subcrawl

SubCrawl SubCrawl is a framework developed by Patrick Schläpfer, Josh Stroschein and Alex Holland of HP Inc’s Threat Research team. SubCrawl is designed to find, scan and analyze open directories. The framework is modular, consisting of four components: input modules, processing modules, output...

webvulscan

WebVulScan Synopsis WebVulScan is a web application vulnerability scanner. It is a web application itself written in PHP and can be used to test remote, or local, web applications for security vulnerabilities. As a scan is running, details of the scan are dynamically updated to the user. These...

Ashok

Ashok Incredible Fast Recon Swiss Army Knife 📌 Introduction Reconnaissance is the first phase of penetration testing which means gathering information before any real attacks are planned So Ashok is an Incredible fast recon tool for penetration tester which is specially designed for Reconnaissanc...

domain_analyzer

Domain Analyzer v0.8.3 What Domain analyzer is a security analysis tool that automatically discovers and reports information about a given domain. Its main purpose is to analyze domains in an unattended way. It has many crazy features, such as getting more domains from DNS zones, automatic nmap,...

protections-artifacts

Protections Artifacts Elastic Security prevents ransomware and malware, detects advanced threats, and arms responders with vital context. It’s free and open, ready for every endpoint. Protections-Artifacts is the home of our detection logic rules, yara, etc for Elastic Security for endpoint. At...

Scrapling

Effortless Web Scraping for the Modern Web -DFE0E5 !Static Badgehttps://img.shields.io/badge/Skill-black?style=flat&label=Agent&...

AIL-framework

AIL Framework Open-source framework for the collection, crawling, processing, and analysis of unstructured information. AIL framework is an open-source platform to collect, crawl, process and analyse unstructured data from the clear web, Tor, I2P, chats, files and external feeds. Originally...

katana

A next-generation crawling and spidering framework Features • Installation • Usage • Scope • Config • Filters • Join Discord Features Fast And fully configurable web crawling Standard and Headless mode JavaScript parsing / crawling Customizable automatic form filling Scope control - Preconfigured...

burp-dom-scanner

Burp DOM Scanner It's a Burp Suite's extension to allow for recursive crawling and scanning of Single Page Applications. It runs a Chromium browser to scan the webpage for DOM-based XSS. It can also collect all the requests XHR, fetch, websockets, etc issued during the crawling allowing them to b...

smbcrawler

SmbCrawler SmbCrawler is no-nonsense tool that takes credentials and a list of hosts and 'crawls' or 'spiders' through those shares. Features: takes host names, IP addresses, IP ranges, or an nmap xml file as input checks permissions check for 'write' permissions is opt-in, because it requires...

responsible-disclosure-email-gathering

Responsible Disclosure Email Gathering Purpose You may have found a vulnerability on a host but the said host/org does not have an active program on HackerOne or BugCrowd. If this is the case this workflow is for you! Broadly it will crawl the website and look for any security related emails from...

dcrawl

dcrawl dcrawl is a simple, but smart, multi-threaded web crawler for randomly gathering huge lists of unique domain names. How it works? dcrawl takes one site URL as input and detects all links in the site's body. Each found link is put into the queue. Successively, each queued link is crawled in...