Differential Privacy in Machine Learning: from Symbolic AI to LLMs

Machine learning models should not reveal particular information that is not otherwise accessible. Differential privacy provides a formal framework to mitigate privacy risks by ensuring that the inclusion or exclusion of any single data point does not significantly alter the output of an algorith...

QGuard:Question-Based Zero-Shot Guard for Multi-Modal LLM Safety

The recent advancements in Large Language ModelsLLMs have had a significant impact on a wide range of fields, from general domains to specialized areas. However, these advancements have also significantly increased the potential for malicious users to exploit harmful and jailbreak prompts for...

MEraser: an Effective Fingerprint Erasure Approach for Large Language Models

Large Language Models LLMs have become increasingly prevalent across various sectors, raising critical concerns about model ownership and intellectual property protection. Although backdoor-based fingerprinting has emerged as a promising solution for model authentication, effective attacks for...

Universal Jailbreak Suffixes Are Strong Attention Hijackers

We study suffix-based jailbreaks$\unicodex2013$a powerful family of attacks against large language models LLMs that optimize adversarial suffixes to circumvent safety alignment. Focusing on the widely used foundational GCG attack Zou et al., 2023, we observe that suffixes vary in efficacy: some...

LLMs on Support of Privacy and Security of Mobile Apps: State of the Art and Research Directions

Modern life has witnessed the explosion of mobile devices. However, besides the valuable features that bring convenience to end users, security and privacy risks still threaten users of mobile apps. The increasing sophistication of these threats in recent years has underscored the need for more...

CipherMind: the Longest Codebook in the World

In recent years, the widespread application of large language models has inspired us to consider using inference for communication encryption. We therefore propose CipherMind, which utilizes intermediate results from deterministic fine-tuning of large model inferences as transmission content. The...

From LLMs to MLLMs to Agents: a Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem

Large language models LLMs are rapidly evolving from single-modal systems to multimodal LLMs and intelligent agents, significantly expanding their capabilities while introducing increasingly severe security risks. This paper presents a systematic survey of the growing complexity of jailbreak...

LASA: Enhancing SoC Security Verification with LLM-Aided Property Generation

Ensuring the security of modern System-on-Chip SoC designs poses significant challenges due to increasing complexity and distributed assets across the intellectual property IP blocks. Formal property verification FPV provides the capability to model and validate design behaviors through security...

Detecting Hard-Coded Credentials in Software Repositories Via LLMs

Software developers frequently hard-code credentials such as passwords, generic secrets, private keys, and generic tokens in software repositories, even though it is strictly advised against due to the severe threat to the security of the software. These credentials create attack surfaces...

LLM-Powered Intent-Based Categorization of Phishing Emails

Phishing attacks remain a significant threat to modern cybersecurity, as they successfully deceive both humans and the defense mechanisms intended to protect them. Traditional detection systems primarily focus on email metadata that users cannot see in their inboxes. Additionally, these systems...

Towards Effective Complementary Security Analysis Using Large Language Models

A key challenge in security analysis is the manual evaluation of potential security weaknesses generated by static application security testing SAST tools. Numerous false positives FPs in these reports reduce the effectiveness of security analysis. We propose using Large Language Models LLMs to...

MM-AttacKG: a Multimodal Approach to Attack Graph Construction with Large Language Models

Cyber Threat Intelligence CTI parsing aims to extract key threat information from massive data, transform it into actionable intelligence, enhance threat detection and defense efficiency, including attack graph construction, intelligence fusion and indicator extraction. Among these research topic...

Five Uncomfortable Truths About LLMs in Production

Many tech professionals see integrating large language models LLMs as a simple process -just connect an API and let it run. At Wallarm, our experience has proved otherwise. Through rigorous testing and iteration, our engineering team uncovered several critical insights about deploying LLMs secure...

Exploring Traffic Simulation and Cybersecurity Strategies Using Large Language Models

Intelligent Transportation Systems ITS are increasingly vulnerable to sophisticated cyberattacks due to their complex, interconnected nature. Ensuring the cybersecurity of these systems is paramount to maintaining road safety and minimizing traffic disruptions. This study presents a novel...

Specification and Evaluation of Multi-Agent LLM Systems -- Prototype and Cybersecurity Applications

Recent advancements in LLMs indicate potential for novel applications, e.g., through reasoning capabilities in the latest OpenAI and DeepSeek models. For applying these models in specific domains beyond text generation, LLM-based multi-agent approaches can be utilized that solve complex tasks by...

Step-By-Step Reasoning Attack: Revealing 'Erased' Knowledge in Large Language Models

Whitepaper called Step-By-Step Reasoning Attack: Revealing 'Erased' Knowledge In Large Language Models...

SoK: Evaluating Jailbreak Guardrails for Large Language Models

Large Language Models LLMs have achieved remarkable progress, but their deployment has exposed critical vulnerabilities, particularly to jailbreak attacks that circumvent safety mechanisms. Guardrails--external defense mechanisms that monitor and control LLM interaction--have emerged as a promisi...

ELFuzz: Efficient Input Generation Via LLM-Driven Synthesis over Fuzzer Space

Generation-based fuzzing produces appropriate testing cases according to specifications of input grammars and semantic constraints to test systems and software. However, these specifications require significant manual efforts to construct. This paper proposes a new approach, ELFuzz Evolution...

Design Patterns for Securing LLM Agents against Prompt Injections

As AI agents powered by Large Language Models LLMs become increasingly versatile and capable of addressing a broad spectrum of tasks, ensuring their security has become a critical challenge. Among the most pressing threats are prompt injection attacks, which exploit the agent's resilience on...

On the Ethics of Using LLMs for Offensive Security

Large Language Models LLMs have rapidly evolved over the past few years and are currently evaluated for their efficacy within the domain of offensive cyber-security. While initial forays showcase the potential of LLMs to enhance security research, they also raise critical ethical concerns regardi...