7579 matches found
Sentinel: SOTA Model to Protect against Prompt Injections
Large Language Models LLMs are increasingly powerful but remain vulnerable to prompt injection attacks, where malicious inputs cause the model to deviate from its intended instructions. This paper introduces Sentinel, a novel detection model, qualifire/prompt-injection-sentinel, based on the...
Deconstructing Obfuscation: a Four-Dimensional Framework for Evaluating Large Language Models Assembly Code Deobfuscation Capabilities
Large language models LLMs have shown promise in software engineering, yet their effectiveness for binary analysis remains unexplored. We present the first comprehensive evaluation of commercial LLMs for assembly code deobfuscation. Testing seven state-of-the-art models against four obfuscation...
Toward a Human-Centered Evaluation Framework for Trustworthy LLM-Powered GUI Agents
The rise of Large Language Models LLMs has revolutionized Graphical User Interface GUI automation through LLM-powered GUI agents, yet their ability to process sensitive data with limited human oversight raises significant privacy and security risks. This position paper identifies three key risks ...
FedShield-LLM: a Secure and Scalable Federated Fine-Tuned Large Language Model
Federated Learning FL offers a decentralized framework for training and fine-tuning Large Language Models LLMs by leveraging computational resources across organizations while keeping sensitive data on local devices. It addresses privacy and security concerns while navigating challenges associate...
Identifying and Understanding Cross-Class Features in Adversarial Training
Adversarial training AT has been considered one of the most effective methods for making deep neural networks robust against adversarial attacks, while the training mechanisms and dynamics of AT remain open research problems. In this paper, we present a novel perspective on studying AT through th...
Urania: Differentially Private Insights into AI Use
We introduce $Urania$, a novel framework for generating insights about LLM chatbot interactions with rigorous differential privacy DP guarantees. The framework employs a private clustering mechanism and innovative keyword extraction methods, including frequency-based, TF-IDF-based, and LLM-guided...
Membership Inference Attacks on Sequence Models
Sequence models, such as Large Language Models LLMs and autoregressive image generators, have a tendency to memorize and inadvertently leak sensitive information. While this tendency has critical legal implications, existing tools are insufficient to audit the resulting risks. We hypothesize that...
Privacy Amplification through Synthetic Data: Insights from Linear Regression
Synthetic data inherits the differential privacy guarantees of the model used to generate it. Additionally, synthetic data may benefit from privacy amplification when the generative model is kept hidden. While empirical studies suggest this phenomenon, a rigorous theoretical understanding is stil...
Big Bird: Privacy Budget Management for W3C'S Privacy-Preserving Attribution API
Privacy-preserving advertising APIs like Privacy-Preserving Attribution PPA are designed to enhance web privacy while enabling effective ad measurement. PPA offers an alternative to cross-site tracking with encrypted reports governed by differential privacy DP, but current designs lack a principl...
Seven Security Challenges That Must Be Solved in Cross-Domain Multi-Agent LLM Systems
Large language models LLMs are rapidly evolving into autonomous agents that cooperate across organizational boundaries, enabling joint disaster response, supply-chain optimization, and other tasks that demand decentralized expertise without surrendering data ownership. Yet, cross-domain...
Hermes: High-Performance Homomorphically Encrypted Vector Databases
Whitepaper called Hermes: High-Performance Homomorphically Encrypted Vector Databases...
Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol Ecosystem
The Model Context Protocol MCP is an emerging standard designed to enable seamless interaction between Large Language Model LLM applications and external tools or resources. Within a short period, thousands of MCP services have already been developed and deployed. However, the client-server...
STOPA: a Database of Systematic VariaTion of DeePfake Audio for Open-Set Source Tracing and Attribution
A key research area in deepfake speech detection is source tracing - determining the origin of synthesised utterances. The approaches may involve identifying the acoustic model AM, vocoder model VM, or other generation-specific parameters. However, progress is limited by the lack of a dedicated,...
Comprehensive Vulnerability Analysis Is Necessary for Trustworthy LLM-MAS
This paper argues that a comprehensive vulnerability analysis is essential for building trustworthy Large Language Model-based Multi-Agent Systems LLM-MAS. These systems, which consist of multiple LLM-powered agents working collaboratively, are increasingly deployed in high-stakes applications bu...
Explainer-Guided Targeted Adversarial Attacks against Binary Code Similarity Detection Models
Binary code similarity detection BCSD serves as a fundamental technique for various software engineering tasks, e.g., vulnerability detection and classification. Attacks against such models have therefore drawn extensive attention, aiming at misleading the models to generate erroneous predictions...
QA-HFL: Quality-Aware Hierarchical Federated Learning for Resource-Constrained Mobile Devices with Heterogeneous Image Quality
This paper introduces QA-HFL, a quality-aware hierarchical federated learning framework that efficiently handles heterogeneous image quality across resource-constrained mobile devices. Our approach trains specialized local models for different image quality levels and aggregates their features...
Quantum Secure Key Exchange with Position-Based Credentials
Quantum key distribution QKD provides an information-theoretic way of securely exchanging secret keys, and typically relies on pre-shared keys or public keys for message authentication. To lift the requirement of pre-shared or public keys, Buhrman et. al. SIAM J. Comput. 43, 150 2014 proposed...
Children's Voice Privacy: First Steps and Emerging Challenges
Children are one of the most under-represented groups in speech technologies, as well as one of the most vulnerable in terms of privacy. Despite this, anonymization techniques targeting this population have received little attention. In this study, we seek to bridge this gap, and establish a...
A Threat Intelligence Event Extraction Conceptual Model for Cyber Threat Intelligence Feeds
In response to the escalating cyber threats, the efficiency of Cyber Threat Intelligence CTI data collection has become paramount in ensuring robust cybersecurity. However, existing works encounter significant challenges in preprocessing large volumes of multilingual threat data, leading to...
Hello, Won'T You Tell Me Your Name?: Investigating Anonymity Abuse in IPFS
The InterPlanetary File SystemIPFS offers a decentralized approach to file storage and sharing, promising resilience and efficiency while also realizing the Web3 paradigm. Simultaneously, the offered anonymity raises significant questions about potential misuse. In this study, we explore methods...
Learning to Diagnose Privately: DP-Powered LLMs for Radiology Report Classification
Purpose: This study proposes a framework for fine-tuning large language models LLMs with differential privacy DP to perform multi-abnormality classification on radiology report text. By injecting calibrated noise during fine-tuning, the framework seeks to mitigate the privacy risks associated wit...
VLMs Can Aggregate Scattered Training Patches
Whitepaper called VLMs Can Aggregate Scattered Training Patches...
BESA: Boosting Encoder Stealing Attack with Perturbation Recovery
To boost the encoder stealing attack under the perturbation-based defense that hinders the attack performance, we propose a boosting encoder stealing attack with perturbation recovery named BESA. It aims to overcome perturbation-based defenses. The core of BESA consists of two modules: perturbati...
Poisoning Behavioral-Based Worker Selection in Mobile Crowdsensing Using Generative Adversarial Networks
With the widespread adoption of Artificial intelligence AI, AI-based tools and components are becoming omnipresent in today's solutions. However, these components and tools are posing a significant threat when it comes to adversarial attacks. Mobile Crowdsensing MCS is a sensing paradigm that...
PCEvolve: Private Contrastive Evolution for Synthetic Dataset Generation Via Few-Shot Private Data and Generative APIs
The rise of generative APIs has fueled interest in privacy-preserving synthetic data generation. While the Private Evolution PE algorithm generates Differential Privacy DP synthetic images using diffusion model APIs, it struggles with few-shot private data due to the limitations of its DP-protect...
Clustering and Median Aggregation Improve Differentially Private Inference
Differentially private DP language model inference is an approach for generating private synthetic text. A sensitive input example is used to prompt an off-the-shelf large language model LLM to produce a similar example. Multiple examples can be aggregated together to formally satisfy the DP...
Through the Stealth Lens: Rethinking Attacks and Defenses in RAG
Retrieval-augmented generation RAG systems are vulnerable to attacks that inject poisoned passages into the retrieved set, even at low corruption rates. We show that existing attacks are not designed to be stealthy, allowing reliable detection and mitigation. We formalize stealth using a...
Design, Implementation, and Analysis of Fair Faucets for Blockchain Ecosystems
The present dissertation addresses the problem of fairly distributing shared resources in non-commercial blockchain networks. Blockchains are distributed systems that order and timestamp records of a given network of users, in a public, cryptographically secure, and consensual way. The records,...
Prediction Inconsistency Helps Achieve Generalizable Detection of Adversarial Examples
Adversarial detection protects models from adversarial attacks by refusing suspicious test samples. However, current detection methods often suffer from weak generalization: their effectiveness tends to degrade significantly when applied to adversarially trained models rather than naturally train...
Towards Trustworthy Federated Learning with Untrusted Participants
Resilience against malicious participants and data privacy are essential for trustworthy federated learning, yet achieving both with good utility typically requires the strong assumption of a trusted central server. This paper shows that a significantly weaker assumption suffices: each pair of...
Evaluating Apple Intelligence'S Writing Tools for Privacy against Large Language Model-Based Inference Attacks: Insights from Early Datasets
The misuse of Large Language Models LLMs to infer emotions from text for malicious purposes, known as emotion inference attacks, poses a significant threat to user privacy. In this paper, we investigate the potential of Apple Intelligence's writing tools, integrated across iPhone, iPad, and...
FERRET: Private Deep Learning Faster and Better Than DPSGD
We revisit 1-bit gradient compression through the lens of mutual-information differential privacy MI-DP. Building on signSGD, we propose FERRET--Fast and Effective Restricted Release for Ethical Training--which transmits at most one sign bit per parameter group with Bernoulli masking. Theory: We...
Gradient Inversion Attacks on Parameter-Efficient Fine-Tuning
Federated learning FL allows multiple data-owners to collaboratively train machine learning models by exchanging local gradients, while keeping their private data on-device. To simultaneously enhance privacy and training efficiency, recently parameter-efficient fine-tuning PEFT of large-scale...
The Hashed Fractal Key Recovery (HFKR) Problem: from Symbolic Path Inversion to Post-Quantum Cryptographic Keys
Classical cryptographic systems rely heavily on structured algebraic problems, such as factorization, discrete logarithms, or lattice-based assumptions, which are increasingly vulnerable to quantum attacks and structural cryptanalysis. In response, this work introduces the Hashed Fractal Key...
Privacy and Security Threat for OpenAI GPTs
Large language models LLMs demonstrate powerful information handling capabilities and are widely integrated into chatbot applications. OpenAI provides a platform for developers to construct custom GPTs, extending ChatGPT's functions and integrating external services. Since its release in November...
Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation
Achieving differentially private computations in decentralized settings poses significant challenges, particularly regarding accuracy, communication cost, and robustness against information leakage. While cryptographic solutions offer promise, they often suffer from high communication overhead or...
Client-Side Zero-Shot LLM Inference for Comprehensive In-Browser URL Analysis
Malicious websites and phishing URLs pose an ever-increasing cybersecurity risk, with phishing attacks growing by 40% in a single year. Traditional detection approaches rely on machine learning classifiers or rule-based scanners operating in the cloud, but these face significant challenges in...
Watermarking Degrades Alignment in Language Models: Analysis and Mitigation
Watermarking techniques for large language models LLMs can significantly impact output quality, yet their effects on truthfulness, safety, and helpfulness remain critically underexamined. This paper presents a systematic analysis of how two popular watermarking approaches-Gumbel and KGW-affect...
Spanning-Tree-Packing Protocol for Conference Key Propagation in Quantum Networks
We consider a network of users connected by pairwise quantum key distribution QKD links. Using these pairwise secret keys and public classical communication, the users want to generate a common conference secret key at the maximal rate. We propose an algorithm based on spanning tree packing a kno...
ChainMarks: Securing DNN Watermark with Cryptographic Chain
With the widespread deployment of deep neural network DNN models, dynamic watermarking techniques are being used to protect the intellectual property of model owners. However, recent studies have shown that existing watermarking schemes are vulnerable to watermark removal and ambiguity attacks...
Attention Knows Whom to Trust: Attention-Based Trust Management for LLM Multi-Agent Systems
Large Language Model-based Multi-Agent Systems LLM-MAS have demonstrated strong capabilities in solving complex tasks but remain vulnerable when agents receive unreliable messages. This vulnerability stems from a fundamental gap: LLM agents treat all incoming messages equally without evaluating...
Poster: Libdebug, Build Your Own Debugger for a Better (Hello) World
Automated debugging, long pursued in a variety of fields from software engineering to cybersecurity, requires a framework that offers the building blocks for a programmable debugging workflow. However, existing debuggers are primarily tailored for human interaction, and those designed for...
A Review of Various Datasets for Machine Learning Algorithm-Based Intrusion Detection System: Advances and Challenges
IDS aims to protect computer networks from security threats by detecting, notifying, and taking appropriate action to prevent illegal access and protect confidential information. As the globe becomes increasingly dependent on technology and automated processes, ensuring secured systems,...
I2P 2.9.0
I2P is an anonymizing network, offering a simple layer that identity-sensitive applications can use to securely communicate. All data is wrapped with several layers of encryption, and the network is both distributed and dynamic, with no trusted parties. This is the source code release version...
Vulnerability Management Chaining: an Integrated Framework for Efficient Cybersecurity Risk Prioritization
Cybersecurity teams face an overwhelming vulnerability crisis: with 25,000+ new CVEs disclosed annually, traditional CVSS-based prioritization requires addressing 60% of all vulnerabilities while correctly identifying only 20% of those actually exploited. We propose Vulnerability Management...
ATAG: AI-Agent Application Threat Assessment with Attack Graphs
Evaluating the security of multi-agent systems MASs powered by large language models LLMs is challenging, primarily because of the systems' complex internal dynamics and the evolving nature of LLM vulnerabilities. Traditional attack graph AG methods often lack the specific capabilities to model...
Samsung S24 VC1 Decoder Out-Of-Bounds Memset
There are several calls to memset in the vc1 decoder on the Samsung S24, which can write out of bounds of a heap buffer. The length of the memsets in svc1expandrightapfrm are calculated based on length values that don't always correspond to the heap buffer length...
An Algorithmic Pipeline for GDPR-Compliant Healthcare Data Anonymisation: Moving toward Standardisation
High-quality real-world data RWD is essential for healthcare but must be transformed to comply with the General Data Protection Regulation GDPR. GDPRs broad definitions of quasi-identifiers QIDs and sensitive attributes SAs complicate implementation. We aim to standardise RWD anonymisation for GD...
TherMod Communication: Low Power or Hot Air?
The Kirchhoff-Law-Johnson-Noise KLJN secure key exchange scheme leverages statistical physics to enable secure communication with zero average power flow in a wired channel. While the original KLJN scheme requires significant power for operation, a recent wireless modification, TherMod, proposed ...