The New Toolkit: LLMs, Prompts, and Basic Tool Interaction

...

The Rise of Agentic AI: Uncovering Security Risks in AI Web Agents

In our first post, we introduced the world of AI web agents - defining what they are, outlining their core capabilities, and surveying the leading frameworks that make them possible. Now, we’re shifting gears to look at the other side of the coin: the vulnerabilities and attack surfaces that aris...

Jailbroken AIs are helping cybercriminals to hone their craft

Cybercriminals are bypassing the guardrails that are supposed to keep AI models from carrying out criminal activities, according to researchers. We've seen the misuse of AI models by cybercriminals growing rapidly over the past several years, shaping a new era of digital threats. Early on,...

The Paradigm Shift: From Traditional APIs to Language-Driven Integration

Large language models LLMs are changing the game, offering a new way for systems to interact based on understanding language, not just strict formats...

On the Feasibility of Poisoning Text-To-Image AI Models Via Adversarial Mislabeling

Today's text-to-image generative models are trained on millions of images sourced from the Internet, each paired with a detailed caption produced by Vision-Language Models VLMs. This part of the training pipeline is critical for supplying the models with large volumes of high-quality image-captio...

Cybercriminal abuse of large language models

Cybercriminals are continuing to explore artificial intelligence AI technologies such as large language models LLMs to aid in their criminal hacking activities. Some cybercriminals have resorted to using uncensored LLMs or even custom-built criminal LLMs for illicit purposes. Advertised features ...

JsDeObsBench: Measuring and Benchmarking LLMs for JavaScript Deobfuscation

Deobfuscating JavaScript JS code poses a significant challenge in web security, particularly as obfuscation techniques are frequently used to conceal malicious activities within scripts. While Large Language Models LLMs have recently shown promise in automating the deobfuscation process,...

E-FreeM2: Efficient Training-Free Multi-Scale and Cross-Modal News Verification Via MLLMs

The rapid spread of misinformation in mobile and wireless networks presents critical security challenges. This study introduces a training-free, retrieval-based multimodal fact verification system that leverages pretrained vision-language models and large language models for credibility assessmen...

KnowML: Improving Generalization of ML-NIDS with Attack Knowledge Graphs

Despite extensive research on Machine Learning-based Network Intrusion Detection Systems ML-NIDS, their capability to detect diverse attack variants remains uncertain. Prior studies have largely relied on homogeneous datasets, which artificially inflate performance scores and offer a false sense ...

Echo Chamber Jailbreak Tricks LLMs Like OpenAI and Google into Generating Harmful Content

Cybersecurity researchers are calling attention to a new jailbreaking method called Echo Chamber that could be leveraged to trick popular large language models LLMs into generating undesirable responses, irrespective of the safeguards put in place. "Unlike traditional jailbreaks that rely on...

UCD: Unlearning in LLMs Via Contrastive Decoding

Machine unlearning aims to remove specific information, e.g. sensitive or undesirable content, from large language models LLMs while preserving overall performance. We propose an inference-time unlearning algorithm that uses contrastive decoding, leveraging two auxiliary smaller models, one train...

Alphabet Index Mapping: Jailbreaking LLMs through Semantic Dissimilarity

Large Language Models LLMs have demonstrated remarkable capabilities, yet their susceptibility to adversarial attacks, particularly jailbreaking, poses significant safety and ethical concerns. While numerous jailbreak methods exist, many suffer from computational expense, high token usage, or...

Risks and Benefits of LLMs and GenAI for Platform Integrity, Healthcare Diagnostics, Cybersecurity, Privacy and AI Safety: a Comprehensive Survey, Roadmap and Implementation Blueprint

Large Language Models LLMs and generative AI GenAI systems such as ChatGPT, Claude, Gemini, LLaMA, and Copilot, developed by OpenAI, Anthropic, Google, Meta, and Microsoft are reshaping digital platforms and app ecosystems while introducing key challenges in cybersecurity, privacy, and platform...

VulStamp: Vulnerability Assessment Using Large Language Model

Although modern vulnerability detection tools enable developers to efficiently identify numerous security flaws, indiscriminate remediation efforts often lead to superfluous development expenses. This is particularly true given that a substantial portion of detected vulnerabilities either possess...

Image Corruption-Inspired Membership Inference Attacks against Large Vision-Language Models

Large vision-language models LVLMs have demonstrated outstanding performance in many downstream tasks. However, LVLMs are trained on large-scale datasets, which can pose privacy risks if training images contain sensitive information. Therefore, it is important to detect whether an image is used t...

NAP-Tuning: Neural Augmented Prompt Tuning for Adversarially Robust Vision-Language Models

Vision-Language Models VLMs such as CLIP have demonstrated remarkable capabilities in understanding relationships between visual and textual data through joint embedding spaces. Despite their effectiveness, these models remain vulnerable to adversarial attacks, particularly in the image modality,...

MEraser: an Effective Fingerprint Erasure Approach for Large Language Models

Large Language Models LLMs have become increasingly prevalent across various sectors, raising critical concerns about model ownership and intellectual property protection. Although backdoor-based fingerprinting has emerged as a promising solution for model authentication, effective attacks for...

Universal Jailbreak Suffixes Are Strong Attention Hijackers

We study suffix-based jailbreaks$\unicodex2013$a powerful family of attacks against large language models LLMs that optimize adversarial suffixes to circumvent safety alignment. Focusing on the widely used foundational GCG attack Zou et al., 2023, we observe that suffixes vary in efficacy: some...

QGuard:Question-Based Zero-Shot Guard for Multi-Modal LLM Safety

The recent advancements in Large Language ModelsLLMs have had a significant impact on a wide range of fields, from general domains to specialized areas. However, these advancements have also significantly increased the potential for malicious users to exploit harmful and jailbreak prompts for...

FAA Framework: a Large Language Model-Based Approach for Credit Card Fraud Investigations

The continuous growth of the e-commerce industry attracts fraudsters who exploit stolen credit card details. Companies often investigate suspicious transactions in order to retain customer trust and address gaps in their fraud detection systems. However, analysts are overwhelmed with an enormous...