Learning-to-Detect

Learning to Detect Unknown Jailbreak Attacks in Large Vision-Language Models Official implementation of “Learning to Detect Unknown Jailbreak Attacks in Large Vision-Language Models.” This repository contains the data-processing, hidden-state extraction, classifier training, safety-pattern...

Can Quantum Layers Offer Performance and Security Advantages in Vision-Language Models?

As quantum computing moves from the noisy intermediate-scale quantum NISQ era toward early fault-tolerant quantum computing EFTQC, quantum-enhanced AI is shifting from a conceptual idea to experimental research. Quantum components are expected to extend the capabilities of existing AI systems, ye...

Training AIs to Solve CAPTCHAs

Interesting research: Abstract : Vision-language models VLMs can solve visual CAPTCHAs without task-specific training, but the agents built on them approach every challenge from scratch. For such an agent, the hundredth instance of a familiar puzzle costs as much time and compute as the first...

GeoCloak: An Agentic Framework for Semantics-Preserving Geolocation Privacy Via Peripheral Clue Manipulation

The rapid advancement of large vision language models LVLMs has made geolocation inference from images highly accurate and accessible. This poses serious privacy risks: a passive adversary with black-box LVLM access can infer a user's location from a single photo with no metadata. Existing...

Unmasking the Secret Code: Using Canaries to Show How Masked Context Controls Memorization in SFT

Canaries, or synthetic training records containing known secrets, are widely used to assess whether language models reveal information from their training data. But how much do these measurements depend on the canaries themselves? We study this question through controlled fine-tuning experiments...

Understanding and Mitigating Token-Pruning-Induced Vulnerabilities in VLMs

Token-Pruning accelerates Vision-Language Models by removing redundant visual tokens, yet its safety implications remain underexplored. In this work, we present the first comprehensive safety evaluation of Token-Pruning mechanisms and find that: most pruning strategies significantly degrade safet...

Adversarial Images Hijack Web Agents from Visual Grounding to Browser Execution

Modern web agents built on large vision-language models process webpages, select relevant UI elements, and translate model outputs into browser actions. Existing visual red-teaming approaches use adversarial visual content to manipulate this process. However, they primarily target model inference...

Selective Channel Restoration for Backdoored Vision-Language Models

Vision-language models VLMs exhibit strong multimodal capabilities but remain vulnerable to backdoors implanted through poisoned fine-tuning data. Existing defenses often require extensive parameter updates during fine-tuning or incur per-query overhead during inference. To address these...

Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection

Prompt injection is widely recognized as a major security threat to AI agents that interact with untrusted external data, such as websites, documents, and emails. Prior work has shown that, in the text domain, black-box prompt injection can achieve near-perfect attack success rates ASRs. In the...

Decoy Images Amplify Caption-Mediated Defenses against Encoded Jailbreaks

We report a counter-intuitive interaction between image inputs and existing black-box defenses on Vision--Language Models VLMs: pairing an encoded jailbreak prompt with an unrelated decoy image can sharply lower attack success rate ASR. The operative change is in the defense pipeline, not in the...

(A)I Sees What You Don't: Exploiting New Attack Surfaces in Third-Party Mobile Agents

Third-party mobile agents powered by Vision-Language Models VLMs have emerged as a promising paradigm for automating smartphone interactions. These agents act as high-privilege decision-makers, perceiving device states through screenshots and executing actions via VLM reasoning, transforming how ...

MIRAGE: Context-Aware Prompt Injection against Mobile GUI Agents Via User-Generated Content

Mobile graphical user interface GUI agents driven by vision-language models VLMs perceive the screen as rendered pixels and choose actions from what they see, so they cannot reliably separate trusted interface elements from user-generated content. We present MIRAGE Mobile Injection of Realistic...

MAI-2024-0034

The Multi-Modal Linkage MML attack represents a sophisticated method to compromise Large Vision-Language Models VLMs by utilizing an "encryption-decryption" strategy across text and image modalities. This technique involves embedding malicious queries within images through methods such as word...

STARE: Step-Wise Temporal Alignment and Red-Teaming Engine for Multi-Modal Toxicity Attack

Red-teaming Vision-Language Models is essential for identifying vulnerabilities where adversarial image-text inputs trigger toxic outputs. Existing approaches treat image generation as a black box, returning only terminal toxicity scores and leaving open the question of when and how toxic semanti...

Hidden Ads: Behavior Triggered Semantic Backdoors for Advertisement Injection in Vision Language Models

Vision-Language Models VLMs are increasingly deployed in consumer applications where users seek recommendations about products, dining, and services. We introduce Hidden Ads, a new class of backdoor attacks that exploit this recommendation-seeking behavior to inject unauthorized advertisements...

Shape and Substance: Dual-Layer Side-Channel Attacks on Local Vision-Language Models

On-device Vision-Language Models VLMs promise data privacy via local execution. However, we show that the architectural shift toward Dynamic High-Resolution preprocessing e.g., AnyRes introduces an inherent algorithmic side-channel. Unlike static models, dynamic preprocessing decomposes images in...

TreeTeaming: Autonomous Red-Teaming of Vision-Language Models Via Hierarchical Strategy Exploration

The rapid advancement of Vision-Language Models VLMs has brought their safety vulnerabilities into sharp focus. However, existing red teaming methods are fundamentally constrained by an inherent linear exploration paradigm, confining them to optimizing within a predefined strategy set and...

VisualLeakBench: Auditing the Fragility of Large Vision-Language Models against PII Leakage and Social Engineering

As Large Vision-Language Models LVLMs are increasingly deployed in agent-integrated workflows and other deployment-relevant settings, their robustness against semantic visual attacks remains under-evaluated -- alignment is typically tested on explicit harmful content rather than privacy-critical...

Jailbreaking LLMs and VLMs: Mechanisms, Evaluation, and Unified Defense

This paper provides a systematic survey of jailbreak attacks and defenses on Large Language Models LLMs and Vision-Language Models VLMs, emphasizing that jailbreak vulnerabilities stem from structural factors such as incomplete training data, linguistic ambiguity, and generative uncertainty. It...

UIXPOSE: Mobile Malware Detection Via Intention-Behaviour Discrepancy Analysis

We introduce UIXPOSE, a source-code-agnostic framework that operates on both compiled and open-source apps. This framework applies Intention Behaviour Alignment IBA to mobile malware analysis, aligning UI-inferred intent with runtime semantics. Previous work either infers intent statically, e.g.,...