Parsero

What's Parsero? Parsero is a free script written in Python which reads the Robots.txt file of a web server and looks at the Disallow entries. The Disallow entries tell the search engines what directories or files hosted on a web server mustn't be indexed. For example, "Disallow: /portal/login"...

xcrawl3r

xcrawl3r xcrawl3r is a command-line utility designed to recursively spider webpages for URLs. It works by actively traversing websites - following links embedded in webpages, and parsing files including sitemaps & robots.txt - to uncover every URL. Unlike xurlfind3r that doesn't interact directly...

wordlists

Wordlists Real-world infosec wordlists, updated regularly Current Wordlists Technologies These wordlists are based on the source code of the CMSes/servers/frameworks listed here. The current wordlists include: Wordpress Joomla Drupal Magento Ghost Tomcat There are 2 versions of each wordlist: Bas...

cangibrina

/ | | | | / | ' \ / | | ' | '| | ' \ / | | || | | | | | | | | | | | | | | | | | | \,|| ||, ||./|| ||| ||,| |/ Beta - v0.8.7 Dashboard Finder English Português English Dashboard Finder Cangibrina is a multi platform tool which aims to obtain the admin Dashboard of sites using brute-force over...

wphardening

WPHardening Fortify the security of any WordPress installation. ❮ NOTE ❯ This tool releases new versions on a regular basis. Make sure to update your dependencies frequently to get the latest version. Check out the changelog or CHANGELOG.md to learn about the new features. Installation Installing...

heffalump

Heffalump Heffalump is an endless honeypot that gives malicious bots nightmares. To use, in your robots.txt tell robots not to go to a certain URL, which heffalump is reverse proxying. Any web agent that does go to the URL will receive an endless stream of random data, which will overflow its...

OpenProject < 12.5.4 - Project Identifiers Exposure

OpenProject versions before 12.5.6 generate a publicly accessible robots.txt file revealing project identifiers, even if the instance is set to 'Login required', letting attackers gather project info, exploit requires no authentication. id: CVE-2023-33960 info: name: OpenProject 12.5.4 - Project...

xcrawl3r

xcrawl3r xcrawl3r es una utilidad de línea de comandos diseñada para rastrear recursivamente páginas web en busca de URLs. Funciona recorriendo activamente los sitios web, siguiendo enlaces incrustados en páginas web y analizando archivos incluyendo sitemaps y robots.txt para descubrir cada URL. ...

Protego has exponential backtracking ReDoS in robots.txt URL wildcard matching

Problem descriptionProtego constructs regular expressions to match URLs against robots.txt Allow: and Disallow: directives, see protego.urlpattern.URLPattern.preparepatternforregex. Every in the directive value is translated into a lazy .? regex piece, thus a specially crafted directive value wit...

PYSEC-2026-3906 Protego has exponential backtracking ReDoS in robots.txt URL wildcard matching

Problem description Protego constructs regular expressions to match URLs against robots.txt Allow: and Disallow: directives, see protego.urlpattern.URLPattern.preparepatternforregex. Every in the directive value is translated into a lazy .? regex piece, thus a specially crafted directive value wi...

CVE-2026-55520

A flaw was found in Protego, a pure-Python robots.txt parser. A remote attacker could exploit this vulnerability by providing a crafted robots.txt file containing numerous asterisks in Allow or Disallow directives. This causes exponential backtracking during URL wildcard matching, leading to a...

Denial-of-Service (DoS)

Protego is vulnerable to a Denial-of-Service DoS. The vulnerability is due to excessive backtracking caused by translating multiple asterisks in Allow or Disallow directives into lazy regular-expression wildcards, which allows an attacker to provide a crafted robots.txt file and cause...

CVE-2026-55520

Protego is a pure-Python robots.txt parser with support for modern conventions. Prior to 0.6.2, protego.urlpattern.URLPattern.preparepatternforregex translates every asterisk in an Allow or Disallow directive into a lazy regular-expression wildcard, so a directive containing many asterisks create...

Protego has exponential backtracking ReDoS in robots.txt URL wildcard matching

Problem description Protego constructs regular expressions to match URLs against robots.txt Allow: and Disallow: directives, see protego.urlpattern.URLPattern.preparepatternforregex. Every in the directive value is translated into a lazy .? regex piece, thus a specially crafted directive value wi...

GHSA-WJMF-P669-5M5P Protego has exponential backtracking ReDoS in robots.txt URL wildcard matching

Problem description Protego constructs regular expressions to match URLs against robots.txt Allow: and Disallow: directives, see protego.urlpattern.URLPattern.preparepatternforregex. Every in the directive value is translated into a lazy .? regex piece, thus a specially crafted directive value wi...

CVE-2026-55520

Protego is a pure-Python robots.txt parser. Prior to 0.6.2 , the _URLPattern._prepare_pattern_for_regex method in src/protego/_urlpattern.py translates every asterisk in an Allow or Disallow directive into a lazy regex wildcard. A directive containing many asterisks triggers exponential backtrack...

EUVD-2026-67865

Protego is a pure-Python robots.txt parser with support for modern conventions. Prior to 0.6.2, protego.urlpattern.URLPattern.preparepatternforregex translates every asterisk in an Allow or Disallow directive into a lazy regular-expression wildcard, so a directive containing many asterisks create...

CVE-2026-55520 Protego: Exponential backtracking ReDoS in robots.txt URL wildcard matching

Protego is a pure-Python robots.txt parser with support for modern conventions. Prior to 0.6.2, protego.urlpattern.URLPattern.preparepatternforregex translates every asterisk in an Allow or Disallow directive into a lazy regular-expression wildcard, so a directive containing many asterisks create...

CVE-2026-55520 Protego: Exponential backtracking ReDoS in robots.txt URL wildcard matching

Protego is a pure-Python robots.txt parser with support for modern conventions. Prior to 0.6.2, protego.urlpattern.URLPattern.preparepatternforregex translates every asterisk in an Allow or Disallow directive into a lazy regular-expression wildcard, so a directive containing many asterisks create...

CVE-2026-55520 Protego: Exponential backtracking ReDoS in robots.txt URL wildcard matching

Protego is a pure-Python robots.txt parser with support for modern conventions. Prior to 0.6.2, protego.urlpattern.URLPattern.preparepatternforregex translates every asterisk in an Allow or Disallow directive into a lazy regular-expression wildcard, so a directive containing many asterisks create...