Amazon Nova AI Challenge -- Trusted AI: Advancing Secure, AI-Assisted Software Development

AI systems for software development are rapidly gaining prominence, yet significant challenges remain in ensuring their safety. To address this, Amazon launched the Trusted AI track of the Amazon Nova AI Challenge, a global competition among 10 university teams to drive advances in secure AI. In...

PRvL: Quantifying the Capabilities and Risks of Large Language Models for PII Redaction

Redacting Personally Identifiable Information PII from unstructured text is critical for ensuring data privacy in regulated domains. While earlier approaches have relied on rule-based systems and domain-specific Named Entity Recognition NER models, these methods fail to generalize across formats...

Prompt Injection Vulnerability of Consensus Generating Applications in Digital Democracy

Large Language Models LLMs are gaining traction as a method to generate consensus statements and aggregate preferences in digital democracy experiments. Yet, LLMs may introduce critical vulnerabilities in these systems. Here, we explore the impact of prompt-injection attacks targeting consensus...

AWS VDP: AWS | Self Registration Internal LibreChat : Access to internal/proprietary LLMs

Issue Summary A LibreChat endpoint/UI is found to be accessible to the public Internet, with self registration for any non AWS/Amazon Corporate domains enabled, allowing an attacker to use a ChatGPT like UI to access multiple public models Example : Claude with the API access it has enabled, as...

When Good Sounds Go Adversarial: Jailbreaking Audio-Language Models with Benign Inputs

As large language models become increasingly integrated into daily life, audio has emerged as a key interface for human-AI interaction. However, this convenience also introduces new vulnerabilities, making audio a potential attack surface for adversaries. Our research introduces WhisperInject, a...

From Legacy to Standard: LLM-Assisted Transformation of Cybersecurity Playbooks into CACAO Format

Existing cybersecurity playbooks are often written in heterogeneous, non-machine-readable formats, which limits their automation and interoperability across Security Orchestration, Automation, and Response platforms. This paper explores the suitability of Large Language Models, combined with Prom...

Can LLMs Effectively Provide Game-Theoretic-Based Scenarios for Cybersecurity?

Game theory has long served as a foundational tool in cybersecurity to test, predict, and design strategic interactions between attackers and defenders. The recent advent of Large Language Models LLMs offers new tools and challenges for the security of computer systems; In this work, we investiga...

A Survey on Data Security in Large Language Models

Large Language Models LLMs, now a foundation in advancing natural language processing, power applications such as text generation, machine translation, and conversational systems. Despite their transformative potential, these models inherently rely on massive amounts of training data, often...

LLM-Assisted Model-Based Fuzzing of Protocol Implementations

Testing network protocol implementations is critical for ensuring the reliability, security, and interoperability of distributed systems. Faults in protocol behavior can lead to vulnerabilities and system failures, especially in real-time and mission-critical applications. A common approach to...

Proactive Disentangled Modeling of Trigger-Object Pairings for Backdoor Defense

Deep neural networks DNNs and generative AI GenAI are increasingly vulnerable to backdoor attacks, where adversaries embed triggers into inputs to cause models to misclassify or misinterpret target labels. Beyond traditional single-trigger scenarios, attackers may inject multiple triggers across...

Semantic Encryption: Secure and Effective Interaction with Cloud-Based Large Language Models Via Semantic Transformation

The increasing adoption of Cloud-based Large Language Models CLLMs has raised significant concerns regarding data privacy during user interactions. While existing approaches primarily focus on encrypting sensitive information, they often overlook the logical structure of user inputs. This oversig...

DUP: Detection-Guided Unlearning for Backdoor Purification in Language Models

As backdoor attacks become more stealthy and robust, they reveal critical weaknesses in current defense strategies: detection methods often rely on coarse-grained feature statistics, and purification methods typically require full retraining or additional clean models. To address these challenges...

Using LLMs as a reverse engineering sidekick

This research explores how large language models LLMs can complement, rather than replace, the efforts of malware analysts in the complex field of reverse engineering. LLMs may serve as powerful assistants to streamline workflows, enhance efficiency, and provide actionable insights during malware...

SAEL: Leveraging Large Language Models with Adaptive Mixture-Of-Experts for Smart Contract Vulnerability Detection

With the increasing security issues in blockchain, smart contract vulnerability detection has become a research focus. Existing vulnerability detection methods have their limitations: 1 Static analysis methods struggle with complex scenarios. 2 Methods based on specialized pre-trained models...

How Microsoft defends against indirect prompt injection attacks

Summary The growing adoption of large language models LLMs in enterprise workflows has introduced a new class of adversarial techniques: indirect prompt injection. Indirect prompt injection can be used against systems that leverage large language models LLMs to process untrusted data...

Can We End the Cat-And-Mouse Game? Simulating Self-Evolving Phishing Attacks with LLMs and Genetic Algorithms

Anticipating emerging attack methodologies is crucial for proactive cybersecurity. Recent advances in Large Language Models LLMs have enabled the automated generation of phishing messages and accelerated research into potential attack techniques. However, predicting future threats remains...

Invisible Injections: Exploiting Vision-Language Models through Steganographic Prompt Embedding

Vision-language models VLMs have revolutionized multimodal AI applications but introduce novel security vulnerabilities that remain largely unexplored. We present the first comprehensive study of steganographic prompt injection attacks against VLMs, where malicious instructions are invisibly...

Secure Coding for Web Applications: Frameworks, Challenges, and the Role of LLMs

Secure coding is a critical yet often overlooked practice in software development. Despite extensive awareness efforts, real-world adoption remains inconsistent due to organizational, educational, and technical barriers. This paper provides a comprehensive review of secure coding practices across...

Securing Cloud AI and LLMs with TotalAI for Visibility, Risk Context and Control

As enterprises accelerate AI adoption, large language models LLMs hosted on public cloud platforms are quickly becoming the norm due to their simplified access and pricing model. Cloud-native services like AWS Bedrock, Azure AI Foundry, and Google Vertex AI offer powerful, pay-as-you-go access to...

The vulnerability of Ollama’s system for running and managing large language models lies in its lack of proper input data validation, allowing attackers to execute arbitrary code.

The vulnerability of Ollama’s system for running and managing large language models is related to insufficient validation of input data. Exploiting this vulnerability could allow a remote attacker to execute arbitrary code...