skill-safety-bench

SkillSafetyBench SkillSafetyBench is a fully materialized benchmark for evaluating safety failures caused by attacks on skill-facing context. Each case keeps the user-facing task ordinary, while the attack is placed in local skills, helper scripts, sidecar files, corpora, templates, or other...

Benchmarking Neural Defend ARCAS 1B: A Foundational Multimodal Deepfake Detection Model

AI-generated imagery evolves faster than benchmark-specific detector evaluations, making a single score an incomplete account of generalization. This paper evaluates Neural Defend ARCAS 1B across benchmark families without benchmark-specific parameter updates. We retain native aggregation and...

CS-Guard: Benchmarking LLM Guardrails for Code Generation Security

Large language models LLMs have been ex- ploited to generate malware, but the effective- ness of guardrails for code generation secu- rity remains unclear. We introduce CS-Guard, the first benchmark to systematically evalu- ate guardrails for code generation security. It covers 1 text-to-code...

LLMs and Contextual Integrity

I have been thinking a lot about AI and integrity. Part of that is contextual integrity. I recently found two papers on the topic. "CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMs": Abstract: Large Language Models LLMs increasingly use persistent memory...

LIB-TRAP: Standard Cell Library Hardware Trojan Risk Assessment and Prevention

Vulnerabilities inherent to the fabless semiconductor manufacturing model have significantly increased the risk of malicious Hardware Trojan HT insertion, posing severe threats to hardware security. Several HT mitigation and detection strategies have been developed, and existing works explore the...

ATLAS: AI-Assisted Threat-To-Assertion Learning for System-On-Chip Security Verification

This work presents ATLAS, an LLM-driven framework that bridges standardized threat modeling and property-based formal verification for System-on-Chip SoC security. Starting from vulnerability knowledge bases such as Common Weakness Enumeration CWE, ATLAS identifies SoC-specific assets, maps...

LLM-Based Multi-Class Attack Analysis and Mitigation Framework in IoT/IIoT Networks

The Internet of Things has expanded rapidly, transforming communication and operations across industries but also increasing the attack surface and security breaches. Artificial Intelligence plays a key role in securing IoT, enabling attack detection, attack behavior analysis, and mitigation...

Adversarial Attacks on VQA-NLE: Exposing and Alleviating Inconsistencies in Visual Question Answering Explanations

Natural language explanations in visual question answering VQA-NLE aim to make black-box models more transparent by elucidating their decision-making processes. However, we find that existing VQA-NLE systems can produce inconsistent explanations and reach conclusions without genuinely understandi...

Private Federated Learning Using Preference-Optimized Synthetic Data

In practical settings, differentially private Federated learning DP-FL is the dominant method for training models from private, on-device client data. Recent work has suggested that DP-FL may be enhanced or outperformed by methods that use DP synthetic data Wu et al., 2024; Hou et al., 2024. The...