23 matches found
roninforge-hono
roninforge-hono Cursor plugin for Hono v4 TypeScript edge web framework + TypeScript. Pinned to hono ^4.12.19, @hono/zod-validator ^0.8.0, @hono/zod-openapi ^1.4.0 peer zod ^4.x, @hono/node-server ^2.0.3 Node 20+. Teaches the v4 APIs that LLMs trained on pre-2024 data do not know c.json always...
RLCDAlignBench
Just Ask Jev: Reinforcement Learning for Calibrated Decisions as a Zero-Shot Detector of AI Alignment Failures This repository holds RLCDAlignBench and the code behind the paper. The benchmark measures whether a detector can tell when a language model's output is an alignment failure. It has 44...
Invisible Ink, Visible Lies: How Production Watermarking Causes LLMs to Hallucinate
Text watermarking helps identify AI-generated content, but its effect on factual reliability remains underexplored. In this paper, we study watermarking hallucination: factual errors induced or amplified by watermarking even when the required evidence is present in the context and the unwatermark...
Bridging the First-Hour Gap: Evaluating AI Reliability and Benchmarking Deficiencies in Cyber Incident Response for Law Enforcement
The actions of frontline law enforcement officers in the initial hour of a cyber incident play a vital role in determining the ultimate success of an investigation. The minor mistakes they commit might result in irreversible critical impacts. The integrity of the investigation can be compromised,...
garak v0.17.0
garak, LLM vulnerability scanner Generative AI Red-teaming & Assessment Kit garak checks if an LLM can be made to fail in a way we don't want. garak probes for hallucination, data leakage, prompt injection, misinformation, toxicity generation, jailbreaks, and many other weaknesses. If you know nm...
Ensembling LLMs for AI-Augmented Cybersecurity Software Requirements Generation
Translating high-level controls from security standards into concrete, system-specific requirements is central to cybersecurity requirements engineering. Large language models LLMs can accelerate this labor-intensive, recall-sensitive task, but any single run is unreliable: it misses valid...
AutoTrans: AI-Assisted Automatic Translation of Security Assertions for RISC-V Processors
Reusing a set of verified security assertions across RISC-V processor targets remains one of the most expensive bottlenecks in hardware security verification. Manual translation takes hours per assertion. Raw LLM translation is fast but unreliable, introducing signal hallucination, where the mode...
Names Can Hurt: Spotting Slopsquatting Risks Caused by Package Name Hallucinations in Local Coding LLMs
When a code generating language model fabricates a Python package name, an adversary who has pre-registered that name on PyPI can convert that hallucination into a supply chain compromise. This event has been termed as 'slopsquatting'. We propose a two layer detector to counter this issue. The...
Beyond the Hype: Evaluating LLM Integration and Practical Limitations in Security Operation Centers
Large Language Models LLMs are increasingly being explored within Security Operation Centers SOCs to support text-heavy analytical work such as alert contextualization, incident summarization, and drafting investigative artifacts. Despite this interest, practitioners describe critical operational...
How Agents Ask for Permission: User Permissions for AI Agents, from Interfaces to Enforcement
As AI agents gain prevalance, users are increasingly exposed to the risks such systems entail. Prompt injection attacks, as well as hallucination, can cause agents to leak private information to third parties. As autonomous systems, agents also present the more active danger of performing sensiti...
New HalluSquatting Attack Could Trick AI Coding Assistants Into Installing Botnet Malware
AI coding assistants have a habit of making things up. Ask one to fetch a popular tool, and it will sometimes hand back a real-sounding name for a project that does not exist. New research, which its authors call HalluSquatting , turns that habit into an attack: work out the fake names an AI...
Beware of Agentic Botnets: Scalable Untargeted Promptware Attacks Via Universal and Transferable Adversarial HalluSquatting
The growing adoption of agentic LLM applications has introduced a new threat previously named as promptware. While prior work has established that adversaries can exploit direct channels to LLM applications to apply promptware under weak threat models, many applications do not provide any direct...
Agentic and Generative AI for Open-Source Intelligence and Cyber Investigations: Taxonomy, Evaluation, Challenges, and Future Directions
The rapid growth of publicly available digital information has rendered manual open-source intelligence OSINT analysis insufficient for modern intelligence, cybersecurity, and cyber investigation. Large language models LLMs and agentic AI systems, capable of tool use, multi-step reasoning, and...
Giving AI a Headache: Acoustic Adversarial Attacks to Computer Vision Applications
Artificial Intelligence AI is increasingly used to automate a variety of real-world computer vision CV applications, such as autonomous vehicle control, facial recognition, and security cameras. Recent research has shown that acoustic vibration can induce real physical motion in cameras,...
Hallucination As Exploit: Evidence-Carrying Multimodal Agents
Multimodal agents use screenshots, documents, and webpages to choose tool calls. When a false visual claim triggers a click, email, extraction, or transfer, hallucination becomes an authorization failure rather than an answer-quality error. We formalize this failure mode as hallucination-to-actio...
Hallucination-Resistant Security Planning with a Large Language Model
Large language models LLMs are promising tools for supporting security management tasks, such as incident response planning. However, their unreliability and tendency to hallucinate remain significant challenges. In this paper, we address these challenges by introducing a principled framework for...
A Prompt-Based Framework for Loop Vulnerability Detection Using Local LLMs
Loop vulnerabilities are one major risky construct in software development. They can easily lead to infinite loops or executions, exhaust resources, or introduce logical errors that degrade performance and compromise security. The problem are often undetected by traditional static analyzers becau...
Measuring the Vulnerability Disclosure Policies of AI Vendors
As AI is increasingly integrated into products and critical systems, researchers are paying greater attention to identifying related vulnerabilities. Effective remediation depends on whether vendors are willing to accept and respond to AI vulnerability reports. In this paper, we examine the...
Incident Response Planning Using a Lightweight Large Language Model with Reduced Hallucination
Timely and effective incident response is key to managing the growing frequency of cyberattacks. However, identifying the right response actions for complex systems is a major technical challenge. A promising approach to mitigate this challenge is to use the security knowledge embedded in large...
SVAgent: AI Agent for Hardware Security Verification Assertion
Verification using SystemVerilog assertions SVA is one of the most popular methods for detecting circuit design vulnerabilities. However, with the globalization of integrated circuit design and the continuous upgrading of security requirements, the SVA development model has exposed major...