BoxPwnr-Traces

BoxPwnr-Traces BoxPwnr traces and benchmark results across multiple security platforms. Each trace includes the full LLM interaction, commands executed, a markdown report + attack graph, stats and config used. Browse leaderboards, replay runs in an interactive web viewer, and read AI-generated...

JShielder

JShielder JShielder Automated Hardening Script for Linux Servers JSHielder is an Open Source Bash Script developed to help SysAdmin and developers secure there Linux Servers in which they will be deploying any web application or services. This tool automates the process of installing all the...

Ajar

Ajar: Measuring Open Privilege in Agent Defenses An agent-security benchmark reports two numbers, attack success and benign utility, and both are read off runs that happened. Neither says what the defense stood ready to allow on the paths no run took. Ajar asks it directly: for each benign task i...

robustbench

RobustBench: a standardized adversarial robustness benchmark Francesco Croce University of Tübingen, Maksym Andriushchenko EPFL, Vikash Sehwag Princeton University, Edoardo Debenedetti EPFL, Nicolas Flammarion EPFL, Mung Chiang Purdue University, Prateek Mittal Princeton University, Matthias Hein...

CUAHarm

Measuring Harmfulness of Computer-Using Agents 🤗 Hugging Face 📄 Paper 📋 Introduction CUAHarm is a benchmark designed to evaluate the safety risks of Computer-Using Agents CUAs - AI agents that can autonomously control computers to perform multi-step actions. Key Features 🔍 CUAHarm Dataset : A...

Zeus

Zeus AWS Auditing & Hardening Tool Zeus is a powerful tool for AWS EC2 / S3 / CloudTrail / CloudWatch / KMS best hardening practices. It checks security settings according to the profiles the user creates and changes them to recommended settings based on the CIS AWS Benchmark source at request of...

FML-Network

FLNET2023: Realistic Network Intrusion Detection Dataset for Federated Learning Paper: FLNET2023: Realistic Network Intrusion Detection Dataset for Federated Learning Dataset: FLNET2023 Introduction FLNET2023 is a state-of-the-art benchmark dataset for intrusion detection systems, specifically fo...

csf

ArmourBird CSF - Container Security Framework Note: The CSF Client is under active development and is getting converted into GoLang for better performace and architecture Table of Contents 1. About 2. Architecture Diagram 3. APIs-CSF Server 4. Installation/Usage 5. Building Docker Images 6. Sneak...

dynast-bench

DynAST-Bench A DAST benchmark of intentionally-vulnerable apps with ground-truth answer keys for scoring scanners. ⚠️ This repository contains DELIBERATELY INSECURE applications. They exist only to benchmark security tooling - DAST scanners, SAST engines, and LLM security agents. Every app binds t...

vulnrepro-benchmark

VulnRepro A benchmark that checks if an AI model can actually review vulnerable code, or if it just sounds confident. Most security benchmarks ask one question: can the model find the bug? That is only half the job. The other half, the part that actually wears you down in real review work, is not...

StepJack

StepJack: Benchmarking Computer-Use Agent Safety Against Multi-Step Indirect Prompt Injection Setup 1. Python Environment Setup First, install all dependencies and ensure you’re running Python 3.10. conda create -n stepjack python=3.10 conda activate stepjack pip install -r requirements.txt 2...

cve-bench

CVE-Bench A benchmark for evaluating LLM agents on fixing real-world security vulnerabilities. Agents run inside sandboxed Docker containers and are scored against the maintainer's security test suite. Requirements Python 3.12+ Docker OPENAIAPIKEY, ANTHROPICAPIKEY, and/or POOLSIDEAPIKEY in your...

agentic-pcap

Agentic LLMs for Network Attack Detection Can a tool-using LLM agent independently investigate a raw network capture and identify an attack? Which packet-analysis queries will it choose, and can it remain effective when the same traffic is surrounded by much more background activity? This project...

secprbench

SecPRBench SecPRBench evaluates whether a code reviewer finds the security flaw in a real change. Each case is the upstream change that introduced a vulnerability in one of nine C, C++, and Rust projects. The reviewer receives the diff, the commit message, and a source snapshot without usable Git...

gdbfuzz

GDBFuzz: Debugger-Driven Fuzzing This is the companion code for the paper: 'Fuzzing Embedded Systems using Debugger Interfaces'. A preprint of the paper can be found here https://publications.cispa.saarland/3950/. The code allows the users to reproduce and extend the results reported in the paper...

aider-chat (>=0.82.3, <=0.85.2), aider-code (>=0.1.0.dev19, <=0.1.1.post14) +186 more potentially affected by CVE-2026-112392 via pillow (11.2.1)

pillow PYPI version 11.2.1 is affected by a known vulnerability. The following packages have a transitive dependency on pillow and may be impacted: - aider-chat =0.82.3, =0.1.0.dev19, =0.0.1, =3.1.20260611, =0.1.0, =0.2.0, =0.1.0, =2026.5.28, =2026.4.15, =2.0.6, =25.5.0, =0.0.0, =1.4.3, =0.5.0b2,...

aider-chat (>=0.82.3, <=0.85.2), aider-code (>=0.1.0.dev19, <=0.1.1.post14) +186 more potentially affected by CVE-2026-112896 via pillow (11.2.1)

pillow PYPI version 11.2.1 is affected by a known vulnerability. The following packages have a transitive dependency on pillow and may be impacted: - aider-chat =0.82.3, =0.1.0.dev19, =0.0.1, =3.1.20260611, =0.1.0, =0.2.0, =0.1.0, =2026.5.28, =2026.4.15, =2.0.6, =25.5.0, =0.0.0, =1.4.3, =0.5.0b2,...

MobileJailBench: Red Queen Jailbreak Benchmarking for Mobile Agents

MLLM-based mobile agents are rapidly gaining autonomy in perception, planning, and closed-loop execution in dynamic mobile environments. However, this autonomy also broadens the attack surface and amplifies the risks of malicious misuse, including privacy leakage, device compromise, and large-sca...

GatewayBench: Unified Capability and Security Evaluation Platform for MCP Gateways

AI agents have increasingly interacted with external environments by invoking tools, with Model Context Protocol MCP emerging as a standard interface for connecting agents to heterogeneous tools and services. As organizations deploy agents at scale, MCP gateways have become a critical middleware...

ABF-Bench: A Verifiable Benchmark for Agentic Blockchain Forensics

Blockchain forensics is adversarial and open-ended: money launderers split, swap, and bridge funds, cut their trail at mixers or custodial services, and change tactics from one incident to the next. Investigators must keep adapting their tracing in response. Yet the immutability of public ledgers...