CTFTiny

CTFTiny: Lite Benchmarking Offensive Cyber Skills in Large Language Models This is the official repository for CTFTiny from "Towards Effective Offensive Security LLM Agents: Hyperparameter Tuning, LLM as a Judge, and a Lightweight CTF Benchmark" AAAI'26 paper. For CTFJudge, please refer to CTFJud...

METATRON

METATRON AI-powered penetration testing assistant using local LLM on linux Parrot OS 🔱 METATRON AI-Powered Penetration Testing Assistant 📌 What is Metatron? Metatron is a CLI-based AI penetration testing assistant that runs entirely on your local machine — no cloud, no API keys, no subscriptions...

DonkAI

Hands-on lab for the OWASP Top 10 for LLM Applications 2025 - no real LLM required. DonkAI is deliberately vulnerable web app you can run in one command and use to learn how LLM-integrated systems get broken by actually breaking them. Every OWASP LLM Top 10 category is represented by at least one...

OpenClawMachines

OpenClaw Machines Run as many isolated OpenClaw agents as you need, on hardware you own. OpenClaw Machines is an open-source platform for runningOpenClaw in secure AI sandboxes on your own infrastructure. A control plane orchestrates your hosts, and each agent runs in its own Firecracker microVM ...

open-source-llm-scanners

open-source-llm-scanners A list of open source LLM security scanners on GitHub, ordered by Stars. It does not provide in-depth analysis. A minimum of 10 stars are needed as a low bar 😁 Related: open-source-web-scanners LLM Scanners Tools which can find a range of vulnerabilities in LLM...

ghidrecomp

Ghidrecomp - Command Line Ghidra Decompiler About This Python command line tool decompiles all functions of a binary and writes them to a directory in separate C files: flowchart LR afilename -- bghidrecomp a2Symbol Server -- b b -- eGhidra Project Files b -- output subgraph output subgraph...

OGhidra

OGhidra 3 - AI-Powered Reverse Engineering with Ghidra For the version using a Claude-inspired Orchestrator, see https://github.com/llnl/OGhidra/tree/orchestrator OGhidra bridges Large Language Models with Ghidra's reverse engineering platform, enabling AI-driven binary analysis through natural...

redthread

RedThread Find the exploit. Judge it. Draft the fix. Prove what changed. RedThread is a CLI-first framework for testing LLM systems, validating failures, and turning confirmed vulnerabilities into evidence-backed defense candidates. It is built for teams who need more than a one-off jailbreak dem...

autoguardrails

autoguardrails Open source by Santander AI Lab. An LLM / AI-safety guardrail research library / evaluation harness autoresearch-style: it searches over a single mutable policy.md surface to minimize attack success rate ASR against a fixed evaluation suite, with a benign-pass floor. Part of...

cve-bench

CVE-Bench A benchmark for evaluating LLM agents on fixing real-world security vulnerabilities. Agents run inside sandboxed Docker containers and are scored against the maintainer's security test suite. Requirements Python 3.12+ Docker OPENAIAPIKEY, ANTHROPICAPIKEY, and/or POOLSIDEAPIKEY in your...

IDAssist

IDAssist AI-Powered Reverse Engineering Plugin for IDA Pro Author: Jason Tang Description IDAssist is an IDA Pro plugin that integrates LLM-powered analysis directly into IDA's interface, providing AI-assisted binary reverse engineering through configurable LLM providers, semantic knowledge graph...

reverse-captcha-eval

Reverse CAPTCHA: Evaluating LLM Susceptibility to Invisible Unicode Instruction Injection An evaluation framework that tests whether large language models follow invisible Unicode-encoded instructions embedded in otherwise normal-looking text. Where traditional CAPTCHAs exploit tasks humans can...

llm-censorship-steering

LLM Censorship Steering This repository contains code implementation for Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Control by Hannah Cyberey and David Evans. We introduce a method that finds "steering vectors" from LLM internals for detecting and controlling the...

BoxPwnr-Traces

BoxPwnr-Traces BoxPwnr traces and benchmark results across multiple security platforms. Each trace includes the full LLM interaction, commands executed, a markdown report + attack graph, stats and config used. Browse leaderboards, replay runs in an interactive web viewer, and read AI-generated...

Vulnhalla

Vulnhalla Automated CodeQL Analysis with LLM Classification For a detailed overview of the research and motivation behind Vulnhalla, see the official CyberArk Threat Research blog post: Vulnhalla: Picking the True Vulnerabilities from the CodeQL Haystack Vulnhalla automates the complete security...

xalgorix

Xalgorix — Open-source AI pentester that proves vulnerabilities Most scanners detect. Xalgorix proves. An autonomous LLM agent works a full pentest methodology, then an independent verifier re-exploits every finding before it's reported — so you get proof, not a pile of maybes to triage...

augustus

Augustus - LLM vulnerability scanner for prompt injection, jailbreak, and adversarial attack testing Augustus - LLM Vulnerability Scanner Test large language models against 210+ adversarial attacks covering prompt injection, jailbreaks, encoding exploits, and data extraction. Augustus is a Go-bas...

supply-chain-monitor

Supply Chain Monitor Automated monitoring of the top PyPI and npm packages for supply chain compromise. Polls both registries for new releases, diffs each release against its predecessor, and uses an LLM via Cursor Agent CLI to classify diffs as benign or malicious. Malicious findings trigger a...

dataset

🚀 CySecBench: Generative AI-based CyberSecurity-focused Prompt Dataset for Benchmarking Large Language Models 🛡️ The largest and most comprehensive Generative AI-based CyberSecurity-focused Dataset for Benchmarking Large Language Models 🌟 Overview The CySecBench paper offers: 🎯 A cutting-edge...

anamnesis-release

Anamnesis: LLM Exploit Generation Evaluation This repository contains the evaluation framework for studying how LLM agents generate exploits from vulnerability reports in the presence of exploit mitigations. Given a bug report and proof-of-concept trigger, agents analyze vulnerable software and...