479 matches found
trawl
TRAWL Self-hosted web scraping for applications and AI agents. Get page content through an API or MCP, with built-in handling for supported browser challenges and CAPTCHAs. TRAWL starts with HTTP, reuses verified clearance and opens Camoufox when a browser is needed. Repeat requests fetch fresh...
TorCrawl.py
TorCrawl.py is a Python script designed for anonymous web scraping via the Tor network. It combines ease of use with the robust privacy features of Tor, allowing for secure and untraceable data collection. Ideal for both novice and experienced programmers, this tool is essential for responsible...
crawlee
A web scraping and browser automation library Crawlee covers your crawling and scraping end-to-end and helps you build reliable scrapers. Fast. Your crawlers will appear human-like and fly under the radar of modern bot protections even with the default configuration. Crawlee gives you the tools t...
Jbin-website-secret-scraper
๐๏ธ Jbin will gather all the URLs from the website and then it will try to expose the secret data from them. It collects both URLs and JS links to scrape secrets out of it. Also if you are looking for a specific string in a page or want to run custom regex then you can do that too now with the new...
BridgeKeeper
BridgeKeeper Scrape employee names from search engine LinkedIn profiles Hunt for emails and username formats via Hunter.io Transform names to given username formats Usage usage: bridgekeeper.py flags BridgeKeeper - v1.0.0 options: -h, --help show this help message and exit Targets: -c COMPANY,...
requests-ip-rotator
Proudly sponsored by Swiftproxy High-performance residential proxies with an 80M+ IP pool across 195+ countries โ improve request success rates and reduce IP blocks during large-scale data collection and browser automation. Perfect for use with requests-ip-rotator, Playwright, Selenium, and other...
thermoptic
thermoptic "I don't believe it, thermoptic camouflage!" What is it? This is an HTTP proxy designed to bypass services that use fingerprinting such as JA4+ to block certain HTTP clients. Using this proxy, you can use your preferred HTTP clients like curl and still have magically indistinguishable...
internal-security-detect
Internal Security Detection via Safebase Exposed SafeBase portals serve two audiences at once: prospects who need proof of your security posture, and security researchers blue and red teamers. Each green tick reveals a control that supposedly exists today; every missing tick is an equally loud hi...
miasma
Miasma AI companies continually scrape the internet at an enormous scale, swallowing up all of its contents to use as training data for their next models. If you have a public website, they are already stealing your work. Many tools attempt to block scrapers, but what if you could punish them...
gorilla
gorilla gorilla is the ultimate wordlist tool packing a lot of amazing utilities like: building wordlists based on patterns like crunch building wordlists based on common password/username formats like cupp scrap a web page and build a wordlist from its words like cewl extending existing wordlist...
BinGoo
,--="=--, / ." .-. "./ / ,/ : : /\ \ | /o\ :: /o\ | / -'| :=" "=: | \ / .-"-. \ | / .-"-. .--- --| /,.-'-.,\ |-- ---. /-===- / | / | | | | | | ' | | / \ / | | | | | | | | | | | | | || |/| || | || / / '-----v1--------------By-Hood3dRob1n----' Welcome to the BinGoo README file! This should give you...
Rock-ON
Rock-ON A One-Shoot Killer Rock-On is a all in one recon tool that will help your Recon process give a boost. It is mainley aimed to automate the whole process of recon and save the time that is being wasted in doing all this stuffs manually. A thorough blog will be up in sometime. Stay tuned for...
DarkScrape
=====The Dark-Scrape===== OSINT Tool to find the Media Links in Tor Sites Available in Tested On Kali Linux 2019.2 Ubuntu 18.04 Nethunter Arc Linux Installation git clone https://github.com/itsmehacker/DarkScrape.git pip3 install -r requirements.txt Features Download Media Scrape From Single Url...
camoufox
Camoufox Camoufox is an open source anti-detect browser built for webscraping & AI agents. ๐ฆ โ ๏ธ This project is under development. It may not be suitable for stable production use. โ ๏ธ !NOTE All of the latest documentation is available atcamoufox.com. Sponsors View/Collapse All Premium | NodeMaven:...
firecrawl-mcp-server
Firecrawl MCP Server A Model Context Protocol MCP server that brings Firecrawl, the context API for AI agents, to MCP-compatible clients: search, scrape, and interact with the live web and get back clean Markdown or structured data. Big thanks to @vrknetha, @knacklabs for the initial...
user-scanner
User Scanner A powerful 2-in-1 OSINT suite engineered for deep Email and Username Intelligence. With 2720+ total scan vectors โincluding 210+ email-integrated sites and 2510+ username platforms โyou can map digital footprints, analyze target behavior, uncover interests, full metadata of usernames...
darkdump
darkdump About Darkdump is an open-source OSINT tool for deep web investigation. Given a search query it fetches results from multiple dark web search engines, optionally scrapes each result site for emails, metadata, keywords, documents, and images, and streams everything live to either the...
Scrapling
Effortless Web Scraping for the Modern Web -DFE0E5 !Static Badgehttps://img.shields.io/badge/Skill-black?style=flat&label=Agent&...
username-anarchy
Username Anarchy Version: 0.6 20 September 2024 Author: urbanadventurer Andrew Horton Homepage: https://www.morningstarsecurity.com/research/username-anarchy Source: https://github.com/urbanadventurer/username-anarchy/ License: MIT Description Tools for generating usernames when penetration...
API-dnsdumpster.com
DNSDumpster API UNOFFICIAL โ ๏ธ IMPORTANT DISCLAIMER : This is an UNOFFICIAL API wrapper for dnsdumpster.com that scrapes the public website. DNSDumpster.com offers an official API plan for commercial and heavy usage. If you need reliable, production-grade access, please consider their official API:...