FML-Network

FLNET2023: Realistic Network Intrusion Detection Dataset for Federated Learning Paper: FLNET2023: Realistic Network Intrusion Detection Dataset for Federated Learning Dataset: FLNET2023 Introduction FLNET2023 is a state-of-the-art benchmark dataset for intrusion detection systems, specifically fo...

FinRED-paper

FinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teaming IEEE ICDM 2026 A red-team benchmark generation pipeline for safety evaluation in the financial domain. Supplementary Documentation Detailed materials referenced in the paper:...

cis-vsphere

🦍 CIS vSphere A tool to assess the compliance of a VMware vSphere environment against the CIS Benchmark for VMware vSphere. Requirements VMware PowerCLI 12.0.0 or higher VMware vSphere 7.0 Read access to the vCenter or ESXi host Usage 1. Clone the repo and navigate to the folder: git clone...

BoxPwnr-Traces

BoxPwnr-Traces BoxPwnr traces and benchmark results across multiple security platforms. Each trace includes the full LLM interaction, commands executed, a markdown report + attack graph, stats and config used. Browse leaderboards, replay runs in an interactive web viewer, and read AI-generated...

CIS GitLab Benchmark Scanner

CIS GitLab Benchmark Scanner - gitlabcis Background On April 17th 2024, GitLab™ published a blog post introducing its Center for Internet Security® CIS GitLab Benchmark. With the goal to improve the security of the product and offer hardening recommendations to GitLab's customers. You can downloa...

filebuster

Filebuster An extremely fast and flexible web fuzzer What is it? Filebuster is a HTTP fuzzer / content discovery script with loads of features and built to be easy to use and fast! It uses one of the fastest HTTP classes in the world of PERL - Furl::HTTP. Also the thread modelling is optimized to...

Benchmark

The OWASP Benchmark project GitHub repository has moved. There is now a new GitHub organization for OWASP Benchmark at: https://github.com/OWASP-Benchmark and what was https://github.com/OWASP/Benchmark is now: https://github.com/OWASP-Benchmark/BenchmarkJava Over time, additional subprojects are...

FLAWED

F ix L ike A rtifacts w ith E mbedded D efects A research harness for measuring how well AI agents patch vulnerabilities. Quickstart · Datasets · Documentation · Security model · Contributing FLAWED checks out an open-source project at a known-vulnerable commit, hands an AI agent a description of...

cve-bench

CVE-Bench A benchmark for evaluating LLM agents on fixing real-world security vulnerabilities. Agents run inside sandboxed Docker containers and are scored against the maintainer's security test suite. Requirements Python 3.12+ Docker OPENAIAPIKEY, ANTHROPICAPIKEY, and/or POOLSIDEAPIKEY in your...

CTFTiny

CTFTiny: Lite Benchmarking Offensive Cyber Skills in Large Language Models This is the official repository for CTFTiny from "Towards Effective Offensive Security LLM Agents: Hyperparameter Tuning, LLM as a Judge, and a Lightweight CTF Benchmark" AAAI'26 paper. For CTFJudge, please refer to CTFJud...

csf

ArmourBird CSF - Container Security Framework Note: The CSF Client is under active development and is getting converted into GoLang for better performace and architecture Table of Contents 1. About 2. Architecture Diagram 3. APIs-CSF Server 4. Installation/Usage 5. Building Docker Images 6. Sneak...

APEX_official

APEX The repository separates the APEX defender, benchmark data normalization, trusted capability manifests, and comparison-method integrations so that each layer can be inspected independently. Installation python -m venv .venv source .venv/bin/activate pip install -e '.test' export...

StepJack

StepJack: Benchmarking Computer-Use Agent Safety Against Multi-Step Indirect Prompt Injection Setup 1. Python Environment Setup First, install all dependencies and ensure you’re running Python 3.10. conda create -n stepjack python=3.10 conda activate stepjack pip install -r requirements.txt 2...

whalescan

Whalescan Vulnerability scanner for windows containers. Getting Started git clone https://github.com/saira-h/whalescan pip install -r requirements.txt python main.py Overview Whalescan performs several benchmark checks, as well as checking for CVEs. This tool can be used as part of a windows...

MalEval

MalEval Article: Is “Knowing It’s Malicious” Enough? Evaluating LLMs for Fine-Grained Malware Behavior Auditing Article DOI: 10.1145/3832187 MalEval is a framework for evaluating Android malware behavior reports generated by large language models. The code in this repository implements two...

vulnrepro-benchmark

VulnRepro A benchmark that checks if an AI model can actually review vulnerable code, or if it just sounds confident. Most security benchmarks ask one question: can the model find the bug? That is only half the job. The other half, the part that actually wears you down in real review work, is not...

rp

rp++: a fast ROP gadget finder for PE/ELF/Mach-O x86/x64/ARM/ARM64 binaries Overview rp++ or rp is a C++ ROP gadget finder for PE/ELF/Mach-O executables and x86/x64/ARM/ARM64 architectures. Finding ROP gadgets To find ROP gadget you need to specify a file with the --file / -f option and use the...

chain-bench

📖 Documentationdocs Chain-bench is an open-source tool for auditing your software supply chain stack for security compliance based on a new CIS Software Supply Chain benchmark. The auditing focuses on the entire SDLC process, where it can reveal risks from code time into deploy time. To win the...

AMDH

AMDH Android Mobile Device Hardening written with python3. Android version, PObY-A Privacy Owned by You - Android, with malware and settings scans is available on the Play Store and the source code can be found HERE Motivations AMDH was created to help automate scanning installed applications on...

MEA-Bench

MEA-Bench: A Benchmark for Model Extraction Attacks This repository provides a unified benchmark for model extraction attacks, defenses, adaptive attacks, and evaluation. The public interface is organized around a small number of portable commands. Method-specific Python modules and legacy run...