ANT: A Multi-Granularity Network Traffic Dataset and Benchmark for Agents Behavior Auditing

The growing adoption of large language model LLM agents creates a need for network administrators and security teams to audit agent behavior within organizational networks without inspecting private user content. Network traffic offers an observable source of evidence, but how much it reveals abo...

COPEX: Benchmarking LLM Robustness to Adversarial Context across Model Context Protocol Layers

Large language models increasingly mediate tool use in Model Context Protocol MCP systems, where adversarial influence may enter through user instructions, tool schemas, tool outputs, or protocol messages. Existing benchmarks often evaluate deployed agents, conflating model susceptibility with...

GO-2026-6505 vulnerabilities

Vulnerabilities for packages: authentik, db-operator-fips, kargo, syft, opensearch-datasource-fips, sops, prometheus-podman-exporter, dagdotdev, kubernetes-event-exporter-fips, gotrue, spicedb, helm-operator-fips, terraform-provider-google-fips, newrelic-k8s-metrics-adapter-fips, gitaly-fips,...

KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards

LLMs are increasingly applied to cybersecurity workflows, where they are expected to translate analysts' intent into tool invocations. However, existing evaluations focus on knowledge-based assessments or end-to-end agentic tasks, and do not directly measure LLMs' ability to generate executable...

Do Defenses against LLM Extraction Work across Attacks? A Lifecycle Benchmark of Black-Box Model Extraction

Large language models LLMs deployed through text-only APIs face model extraction risks, as adversaries can collect their responses to train surrogates that reproduce their capabilities. While prior work has developed diverse attacks and defenses, evaluations remain fragmented across access...

MMSkillRisk: Can Agents Stay Safe When Multimodal Skills Become Traps?

Agent skills are shareable packages of procedural instructions, tools, and examples. Multimodal skills additionally include visual references that agents retrieve and inspect during execution. Because these images guide actions, attackers can disguise malicious instructions as ordinary visual...

DGF-Bench: A Benchmark for Simulating and Auditing Deception against Multi-Agent Governance Boards

Tool-using language-model agents can review enterprise projects as governance boards do: they read the evidence, apply written rules and decide whether the project may proceed. Part of that evidence comes from suppliers and project members with a stake in the decision. DGF-Bench is a benchmark in...

ActBench

ActBench ActBench is a self-evolving benchmark of behavioral safety in cowork agents. It defines behavioral safety as whether an agent's execution remains within the permissions and state changes required by a benign task, and evaluates realized behavioral risk from execution trajectories rather...

ethibench

From Controlled to the Wild: Evaluation of Pentesting Agents in the Real-World AI pentesting agents are increasingly credible as offensive security systems, but current benchmarks still provide limited guidance on which systems will perform best on real-world targets. Most existing evaluations...

JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models

Models trained with reinforcement learning for calibrated decisions RLCD, such as Jev, answer a typed question about an input, the state, with a probability, a choice, or a score, and software acts on the answer without a person reading it. Their robustness has not been measured: adversarial...

Benchmarking Neural Defend ARCAS 1B: A Foundational Multimodal Deepfake Detection Model

AI-generated imagery evolves faster than benchmark-specific detector evaluations, making a single score an incomplete account of generalization. This paper evaluates Neural Defend ARCAS 1B across benchmark families without benchmark-specific parameter updates. We retain native aggregation and...

TRACTOR Benchmark for Evaluating C to Rust Translators

Memory-safety vulnerabilities remain a persistent source of security risk in critical software, much of which is implemented in memory-unsafe languages such as C and C++. Recent advances in programming languages, program analysis, and artificial intelligence have created new opportunities to...

perf-benchmark-poc

Large Model-Based Python Performance Optimization Loop v0.1...

Improving email security outcomes with real-world Microsoft Defender insights

Every benchmark tells a story. The most valuable ones tell us where to improve next. For five consecutive quarters Microsoft has published email security benchmarking reports to provide greater transparency into real-world protection outcomes. The results have shown strong Microsoft Defender...

vader

VADER：用于漏洞评估、检测、解释和修复的人工评估基准 官方 GitHub 仓库：https://github.com/AfterQuery/vader Hugging Face 数据集：https://huggingface.co/datasets/AfterQuery/vader VADER 是一个人工评估基准 ，旨在衡量大型语言模型（LLMs）处理真实世界软件漏洞的能力。它包含 174 个真实世界漏洞案例 （从开源仓库中精选），涵盖四项任务： 漏洞识别与分类（CWE） 根因解释 补丁（修复） 测试计划生成 这些案例涵盖 15+ 编程语言 （例如...

Astra Linux – Vulnerability found in Linux 5.10, Linux 6.1

In the Linux kernel, the following vulnerability has been resolved: crypto: virtio – Add spinlock protection with virtqueue notification When a virtual machine boots with a virtio-crypto PCI device and a built-in backend, run the openssl benchmark command with multiple processes. For example:...

Astra Linux – Vulnerability found in Linux 6.1, Linux 5.15

In the Linux kernel, the following vulnerability has been resolved: dma-mapping: benchmark: fix node id validation While validating node IDs in mapbenchmarkioctl, the nodepossible function may receive invalid arguments outside the range of 0, MAXNUMNODES-1, leading to: BUG: KASAN:...

Astra Linux – Vulnerability found in Linux 6.1, Linux 5.15

In the Linux kernel, the following vulnerability has been resolved: dma-mapping: Benchmark: Properly handling NUMANONODE. The cpumaskofnode function can be called within domapbenchmark when dealing with NUMANONODE. This results in the following issue reported by the sanitizer tool: UBSAN: Array...

[SECURITY] Fedora 44 Update: golang-x-perf-0-0.36.20260904git19be9d8.fc44

This package holds the source for various tools related to performance measurement, storage, and analysis. - cmd/benchstat contains a command-line tool that computes and 7 compares statistics about benchmarks. - cmd/benchsave contains a command-line tool for publishing benchmark results. - storag...

[SECURITY] Fedora 43 Update: golang-x-perf-0-0.36.20260904git19be9d8.fc43

This package holds the source for various tools related to performance measurement, storage, and analysis. - cmd/benchstat contains a command-line tool that computes and 7 compares statistics about benchmarks. - cmd/benchsave contains a command-line tool for publishing benchmark results. - storag...