PenCrawLer

PenCrawLer An Advanced Web Crawler and DirBuster PeNCrawLer is an advanced webcrawler and dirbuster designed to use in penetration testing based on Windows Os. To read more about features please read the wiki: https://github.com/TheM4hd1/PenCrawLer/wiki...

dirmap

Dirmap English Herramienta avanzada de escaneo de directorios web, con funciones más potentes que DirBuster, Dirsearch, cansina, Yujian Análisis de requisitos Después de una extensa investigación, se resume que una herramienta de escaneo de directorios web excelente debe tener al menos las...

hakrawler

Hakrawler Rastreador web rápido en golang para recolectar URLs y ubicaciones de archivos JavaScript. Básicamente es una implementación simple de la increíble librería Gocolly. Ejemplos de uso URL única: root@kitploit: echo https://google.com | hakrawler Múltiples URLs: root@kitploit: cat urls.txt...

hakrawler

Hakrawler Fast golang web crawler for gathering URLs and JavaScript file locations. This is basically a simple implementation of the awesome Gocolly library. Example usages Single URL: root@kitploit: echo https://google.com | hakrawler Multiple URLs: root@kitploit: cat urls.txt | hakrawler Timeou...

dcrawl

dcrawl dcrawl is a simple, but smart, multi-threaded web crawler for randomly gathering huge lists of unique domain names. How it works? dcrawl takes one site URL as input and detects all links in the site's body. Each found link is put into the queue. Successively, each queued link is crawled in...

Photon

Photon Incredibly fast crawler designed for OSINT. Photon Wiki • How To Use • Compatibility • Photon Library • Contribution • Roadmap Key Features Data Extraction Photon can extract the following data while crawling: URLs in-scope & out-of-scope URLs with parameters example.com/gallery.php?id=2...

wxpath

wxpath - declarative web graph traversal with XPath NEW: TUI - Interactive terminal interface powered by Textual for testing wxpath expressions and exporting data. Install Requires Python 3.10+. root@kitploit: pip install wxpath For TUI support: pip install "wxpathtui" Immediately launch the TUI...

webpwn3r

webpwn3r This project is not maintained for years, and was written when I started learning Python, but still people likes it :D. it only supports python2.7 WebPwn3r - Web Applications Security Scanner. By Ebrahim Hegazy - @Zigoo0 Thanks: @lnxg33k, @dia2diab @Aelhemily, @okamalo Please send all yo...

evine

Evine Interactive CLI Web Crawler. Evine is a simple, fast, and interactive web crawler and web scraper written in Golang. Evine is useful for a wide range of purposes such as metadata and data extraction, data mining, reconnaissance and testing. If you like the project, give it a star. It forces...

EasySpider

易采集/EasySpider: Visual Code-Free Web Crawler 一个完全免费 （包括商业使用和二次开发 ）的可视化浏览器自动化测试/数据采集/爬虫软件，可以使用图形化界面，无代码可视化的设计和执行任务。只需要在网页上选择自己想要操作的内容并根据提示框操作即可完成任务的设计和执行。同时软件还可以单独以命令行的方式进行执行，从而可以很方便的嵌入到其他系统中。 A completely free including for commercial use and secondary development visual browser automation...

vexhub-crawler

VEX Hub Crawler vexhub-crawler is a component of the VEX Hub that automatically retrieves VEX documents from source repositories. Overview The crawler identifies source repositories from registered PURLs Package URLs and copies VEX documents into VEX Hub. This process ensures that VEX Hub maintai...

dirhunt

.. image:: https://raw.githubusercontent.com/Nekmo/dirhunt/v0.2.0/images/dirhunt.png | .. image:: https://raw.githubusercontent.com/Nekmo/dirhunt/pip-rating-badge/pip-rating-badge.svg :target: https://github.com/Nekmo/dirhunt/actions/workflows/pip-rating.yml :alt: pip-rating badge .. image::...

secator

The pentester's swiss knife. Features • Supported commands • Installation • Usage • Documentation • Join us on Discord ! secator is a task and workflow runner used for security assessments. It supports dozens of well-known security tools and it is designed to improve productivity for pentesters a...

Pyison

Pyison Pyison is a tarpit for AI webcrawlers Introduction Like all web crawlers, AI bots request pages from webservers, and then follow links on the page to other pages. By doing this, they can build an index of an entire website. Unlike other web crawlers, though, AI bots present some unique...

second-order

Second Order Escanea aplicaciones web para la toma de control de subdominios de segundo orden rastreando la aplicación y recolectando URLs y otros datos que coincidan con ciertas reglas o respondan de cierta manera. Instalación Desde binario Descarga un binario precompilado desde la página de...

Instagram-Crawler

Instagram-Crawler Non API. Crawling post photo, likes, comments, date ... by username, hashtags Installation 1. Make sure you have Chrome browser installed. 2. Download chromedriver and put it into driver folder: ./driver/chromedriver 3. Install requirements pip install -r requirements.txt...

FinalRecon

Twitter - Telegram - thewhiteh4t's Blog FinalRecon is an all in one automatic web reconnaissance tool written in python. Goal of FinalRecon is to provide an overview of the target in a short amount of time while maintaining the accuracy of results. Instead of executing several tools one after...

Web-Cache-Vulnerability-Scanner

Web Cache Vulnerability Scanner WCVS is a fast and versatile CLI scanner for web cache poisoning and web cache deception developed by Hackmanit and Maximilian Hildebrand. The scanner supports many different web cache poisoning and web cache deception techniques, includes a crawler to identify...

festin

FestIN the powered S3 bucket finder and content discover What is FestIn Why Festin Install Using Python Using Docker Full options Usage Configure search domains Concurrency HTTP Crawling configuration Manage results Proxy usage DNS Options Full Text Support Running as a service or watching mode...

sql-global-injector

Ultimate-Dork web crawler And Scan SQL injection vulnerability Usage & Installation root@kitploit: - Installation apt-get install python python3 git git clone https://github.com/nu11secur1ty/sql-global-injector.git python3 -m pip install requests mechanicalsoup cd sql-global-injector - Usage...