CVE-2024-56516

free-one-api allows users to access large language model reverse engineering libraries through the standard OpenAI API format. In versions up to and including 1.0.1, MD5 is used to hash passwords before sending them to the backend. MD5 is a cryptographically broken hashing algorithm and is no...

A Critical Evaluation of Defenses against Prompt Injection Attacks

Large Language Models LLMs are vulnerable to prompt injection attacks, and several defenses have recently been proposed, often claiming to mitigate these attacks successfully. However, we argue that existing studies lack a principled approach to evaluating these defenses. In this paper, we argue...

An Attack to Break Permutation-Based Private Third-Party Inference Schemes for LLMs

Recent advances in Large Language Models LLMs have led to the widespread adoption of third-party inference services, raising critical privacy concerns. Existing methods of performing private third-party inference, such as Secure Multiparty Computation SMPC, often rely on cryptographic methods...

An End-To-End Model for Logits Based Large Language Models Watermarking

The rise of LLMs has increased concerns over source tracing and copyright protection for AIGC, highlighting the need for advanced detection technologies. Passive detection methods usually face high false positives, while active watermarking techniques using logits or sampling manipulation offer...

Alignment under Pressure: the Case for Informed Adversaries When Evaluating LLM Defenses

Large language models LLMs are rapidly deployed in real-world applications ranging from chatbots to agentic systems. Alignment is one of the main approaches used to defend against attacks such as prompt injection and jailbreaks. Recent defenses report near-zero Attack Success Rates ASR even again...

Securing RAG: a Risk Assessment and Mitigation Framework

Retrieval Augmented Generation RAG has emerged as the de facto industry standard for user-facing NLP applications, offering the ability to integrate data without re-training or fine-tuning Large Language Models LLMs. This capability enhances the quality and accuracy of responses but also introduc...

From Nuclear Safety to LLM Security: Applying Non-Probabilistic Risk Management Strategies to Build Safe and Secure LLM-Powered Systems

Large language models LLMs offer unprecedented and growing capabilities, but also introduce complex safety and security challenges that resist conventional risk management. While conventional probabilistic risk analysis PRA requires exhaustive risk enumeration and quantification, the novelty and...

vLLM 代码问题漏洞

vLLM is a vLLM open source high throughput and memory efficient inference and service engine for LLM. A code issue vulnerability exists in vLLM versions 0.6.5 through 0.8.4, which stems from PyNcclPipe KV cache transfers not properly limiting the scope of TCPStore interface access...

BeamClean: Language Aware Embedding Reconstruction

In this work, we consider an inversion attack on the obfuscated input embeddings sent to a language model on a server, where the adversary has no access to the language model or the obfuscation mechanism and sees only the obfuscated embeddings along with the model's embedding table. We propose...

The Impact of Emerging Phishing Threats: Assessing Quishing and LLM-Generated Phishing Emails against Organizations

Modern organizations are persistently targeted by phishing emails. Despite advances in detection systems and widespread employee training, attackers continue to innovate, posing ongoing threats. Two emerging vectors stand out in the current landscape: QR-code baits and LLM-enabled pretexting. Yet...

GuardReasoner-VL: Safeguarding VLMs Via Reinforced Reasoning

To enhance the safety of VLMs, this paper introduces a novel reasoning-based VLM guard model dubbed GuardReasoner-VL. The core idea is to incentivize the guard model to deliberatively reason before making moderation decisions via online RL. First, we construct GuardReasoner-VLTrain, a reasoning...

Automating Security Audit Using Large Language Model Based Agent: an Exploration Experiment

In the current rapidly changing digital environment, businesses are under constant stress to ensure that their systems are secured. Security audits help to maintain a strong security posture by ensuring that policies are in place, controls are implemented, gaps are identified for cybersecurity...

Removing Watermarks with Partial Regeneration Using Semantic Information

As AI-generated imagery becomes ubiquitous, invisible watermarks have emerged as a primary line of defense for copyright and provenance. The newest watermarking schemes embed semantic signals - content-aware patterns that are designed to survive common image manipulations - yet their true...

Optimized Couplings for Watermarking Large Language Models

Large-language models LLMs are now able to produce text that is, in many cases, seemingly indistinguishable from human-generated content. This has fueled the development of watermarks that imprint a signal'' in LLM-generated text with minimal perturbation of an LLM's output. This paper provides a...

Improved Algorithms for Differentially Private Language Model Alignment

Language model alignment is crucial for ensuring that large language models LLMs align with human preferences, yet it often involves sensitive user data, raising significant privacy concerns. While prior work has integrated differential privacy DP with alignment techniques, their performance...

RuleGenie: SIEM Detection Rule Set Optimization

SIEM systems serve as a critical hub, employing rule-based logic to detect and respond to threats. Redundant or overlapping rules in SIEM systems lead to excessive false alerts, degrading analyst performance due to alert fatigue, and increase computational overhead and response latency for actual...

Security Steerability Is All You Need

The adoption of Generative AI GenAI in various applications inevitably comes with expanding the attack surface, combining new security threats along with the traditional ones. Consequently, numerous research and industrial initiatives aim to mitigate these security threats in GenAI by developing...

LATENT: LLM-Augmented Trojan Insertion and Evaluation Framework for Analog Netlist Topologies

Analog and mixed-signal A/MS integrated circuits ICs are integral to safety-critical applications. However, the globalization and outsourcing of A/MS ICs to untrusted third-party foundries expose them to security threats, particularly analog Trojans. Unlike digital Trojans which have been...

Large Language Model-Driven Security Assistant for Internet of Things Via Chain-Of-Thought

The rapid development of Internet of Things IoT technology has transformed people's way of life and has a profound impact on both production and daily activities. However, with the rapid advancement of IoT technology, the security of IoT devices has become an unavoidable issue in both research an...

An LLM-Based Self-Evolving Security Framework for 6G Space-Air-Ground Integrated Networks

Recently emerged 6G space-air-ground integrated networks SAGINs, which integrate satellites, aerial networks, and terrestrial communications, offer ubiquitous coverage for various mobile applications. However, the highly dynamic, open, and heterogeneous nature of SAGINs poses severe security...