CTFTiny

CTFTiny: Lite Benchmarking Offensive Cyber Skills in Large Language Models This is the official repository for CTFTiny from "Towards Effective Offensive Security LLM Agents: Hyperparameter Tuning, LLM as a Judge, and a Lightweight CTF Benchmark" AAAI'26 paper. For CTFJudge, please refer to CTFJud...

whalescan

Whalescan Vulnerability scanner for windows containers. Getting Started git clone https://github.com/saira-h/whalescan pip install -r requirements.txt python main.py Overview Whalescan performs several benchmark checks, as well as checking for CVEs. This tool can be used as part of a windows...

terraform-aws-secure-baseline

terraform-aws-secure-baseline Terraform Module Registry A terraform module to set up your AWS account with the reasonably secure configuration baseline. Most configurations are based on CIS Amazon Web Services Foundations v1.4.0 and AWS Foundational Security Best Practices v1.0.0. See Benchmark...

MalEval

MalEval Article: Is “Knowing It’s Malicious” Enough? Evaluating LLMs for Fine-Grained Malware Behavior Auditing Article DOI: 10.1145/3832187 MalEval is a framework for evaluating Android malware behavior reports generated by large language models. The code in this repository implements two...

linux-security-audit

Linux Security Audit Tool Linux Security Audit Tool based on the CIS - Red Hat Enterprise Linux 8 Benchmark v3.0.0 If you find this helpful, please the "star" 🌟 to support further improvements. Table of Contents 1. Features 2. Preview 3. Result-Log 4. Supported OS 5. Prerequisites 6. Notes...

trivy-operator

Kubernetes-native security toolkit. Documentation Introduction The Trivy Operator leverages Trivy to continuously scan your Kubernetes cluster for security issues. The scans are summarised in security reports as Kubernetes Custom Resource Definitions, which become accessible through the Kubernete...

exploitgym

ExploitGym ExploitGym is a large-scale, realistic benchmark built from real-world vulnerabilities across userspace programs, Google's V8 engine, and the Linux kernel, designed to evaluate AI agents' ability to develop exploits. Quick start 1. Python deps uv sync --extra proxy 2. Build runtime...

redteam-ai-benchmark

Red Team AI Benchmark Russian version: README.ru.md Red Team AI Benchmark is a CLI model-evaluation benchmark. It measures how LLMs understand and respond to red-team questions and security scenarios; it is not a tool for carrying out those activities. Version 2 uses a rubric-based dataset instea...

BoxPwnr-Traces

BoxPwnr-Traces BoxPwnr traces and benchmark results across multiple security platforms. Each trace includes the full LLM interaction, commands executed, a markdown report + attack graph, stats and config used. Browse leaderboards, replay runs in an interactive web viewer, and read AI-generated...

FML-Network

FLNET2023: Realistic Network Intrusion Detection Dataset for Federated Learning Paper: FLNET2023: Realistic Network Intrusion Detection Dataset for Federated Learning Dataset: FLNET2023 Introduction FLNET2023 is a state-of-the-art benchmark dataset for intrusion detection systems, specifically fo...

docker-bench-security

Docker Bench for Security The Docker Bench for Security is a script that checks for dozens of common best-practices around deploying Docker containers in production. The tests are all automated, and are based on the CIS Docker Benchmark v1.6.0. We are making this available as an open-source utili...

JShielder

JShielder JShielder Automated Hardening Script for Linux Servers JSHielder is an Open Source Bash Script developed to help SysAdmin and developers secure there Linux Servers in which they will be deploying any web application or services. This tool automates the process of installing all the...

csf

ArmourBird CSF - Container Security Framework Note: The CSF Client is under active development and is getting converted into GoLang for better performace and architecture Table of Contents 1. About 2. Architecture Diagram 3. APIs-CSF Server 4. Installation/Usage 5. Building Docker Images 6. Sneak...

cis-vsphere

🦍 CIS vSphere A tool to assess the compliance of a VMware vSphere environment against the CIS Benchmark for VMware vSphere. Requirements VMware PowerCLI 12.0.0 or higher VMware vSphere 7.0 Read access to the vCenter or ESXi host Usage 1. Clone the repo and navigate to the folder: git clone...

legba

legba Join the project community on our server! Legba is a multiprotocol credentials bruteforcer / password sprayer and enumerator built with Rust and the Tokio asynchronous runtime in order to achieve better performances and stability while consuming less resources than similar tools. Key Featur...

cve-bench

CVE-Bench A benchmark for evaluating LLM agents on fixing real-world security vulnerabilities. Agents run inside sandboxed Docker containers and are scored against the maintainer's security test suite. Requirements Python 3.12+ Docker OPENAIAPIKEY, ANTHROPICAPIKEY, and/or POOLSIDEAPIKEY in your...

DeepTrap

DeepTrap English | 中文 Open-world security evaluation for OpenClaw agents under adversarial execution contexts. DeepTrap is a security benchmark for evaluating whether OpenClaw agents can complete benign user tasks while resisting malicious execution-context pressure: poisoned workspace files,...

vulnrepro-benchmark

VulnRepro A benchmark that checks if an AI model can actually review vulnerable code, or if it just sounds confident. Most security benchmarks ask one question: can the model find the bug? That is only half the job. The other half, the part that actually wears you down in real review work, is not...

morty

Morty Web content sanitizer proxy as a service Morty rewrites web pages to exclude malicious HTML tags and attributes. It also replaces external resource references to prevent third party information leaks. The main goal of morty is to provide a result proxy for searx, but it can be used as a...

auditpolCIS

auditpolCIS CIS Benchmark testing of Windows SIEM configuration This is an application for testing the configuration of Windows Audit Policy settings against the CIS Benchmark recommended settings. A few points: The tested system was Windows Server 2019, and the benchmark used was also Windows...