ctf-skills

ctf-skills Agent Skills for solving CTF challenges — web exploitation, binary pwn, crypto, reverse engineering, forensics, OSINT, and more. Works with any tool that supports the Agent Skills spec, including Claude Code. Installation root@kitploit: npx skills add ljagiello/ctf-skills Run with Frid...

machine_learning_security

Machine Learning and Security Source codes about machine learning and security. Line up. Cyber security and Machine Learning course The elementary training course of Machine learning for security engineer. Vulnerabilities of Machine Learning Summary of Machine Learning vulnerability. Analytics...

WAF-A-MoLE

WAF-A-MoLE A guided mutation-based fuzzer for ML-based Web Application Firewalls, inspired by AFL and based on the FuzzingBook by Andreas Zeller et al. Given an input SQL injection query, it tries to produce a semantic invariant query that is able to bypass the target WAF. You can use this tool f...

deep-pwning

Deep-pwningは、機械学習モデルを対象に、意欲的な攻撃者に対する堅牢性を評価するための軽量フレームワークです。 現在の状態のdeep-pwningは、 成熟や完成には程遠い ことに注意してください。これは、あなたが実験し、拡張し、発展させていくことを意図しています。そうすることで初めて、真に 統計的機械学習モデルのための侵入テストツールキット となるのです。 背景 研究者たちは、機械学習モデル（分類器、クラスタリング器、回帰器など）をだまして客観的に誤った決定をさせることが驚くほど簡単であることを発見しました。この研究分野は 敵対的機械学習...

ai-security-battle

🤖 AI Security Battle System Red Team vs Blue Team: Adversarial ML Combat Simulation A comprehensive adversarial machine learning security simulation where Red Team attackers battle against Blue Team defenders in a 30-minute AI/ML security warfare scenario. 🎯 Project Overview This system simulates...

Agent Security Is a Systems Problem

We take the position that agent security must be approached as a systems problem: the AI model powering the agent must be treated as an untrusted component, and security invariants must be enforced at the system level. Through this lens, efforts to increase model robustness the dominant viewpoint...

A Systematic Literature Review on LLM Defenses against Prompt Injection and Jailbreaking: Expanding NIST Taxonomy

The rapid advancement and widespread adoption of generative artificial intelligence GenAI and large language models LLMs has been accompanied by the emergence of new security vulnerabilities and challenges, such as jailbreaking and other prompt injection attacks. These maliciously crafted inputs...

Leveraging Trustworthy AI for Automotive Security in Multi-Domain Operations: Towards a Responsive Human-AI Multi-Domain Task Force for Cyber Social Security

Multi-Domain Operations MDOs emphasize cross-domain defense against complex and synergistic threats, with civilian infrastructures like smart cities and Connected Autonomous Vehicles CAVs emerging as primary targets. As dual-use assets, CAVs are vulnerable to Multi-Surface Threats MSTs,...

Quantum Support Vector Regression for Robust Anomaly Detection

Anomaly Detection AD is critical in data analysis, particularly within the domain of IT security. In recent years, Machine Learning ML algorithms have emerged as a powerful tool for AD in large-scale data. In this study, we explore the potential of quantum ML approaches, specifically quantum kern...

Bypassing Prompt Injection and Jailbreak Detection in LLM Guardrails

Large Language Models LLMs guardrail systems are designed to protect against prompt injection and jailbreak attacks. However, they remain vulnerable to evasion techniques. We demonstrate two approaches for bypassing LLM prompt injection and jailbreak detection systems via traditional character...

A Taxonomy of Adversarial Machine Learning Attacks and Mitigations

NIST just released a comprehensive taxonomy of adversarial machine learning attacks and countermeasures...

Microsoft AI Red Team building future of safer AI

An essential part of shipping software securely is red teaming. It broadly refers to the practice of emulating real-world adversaries and their tools, tactics, and procedures to identify risks, uncover blind spots, validate assumptions, and improve the overall security posture of systems. Microso...

Security Risks of AI

Stanford and Georgetown have a new report on the security risks of AI--particularly adversarial machine learning--based on a workshop they held on the topic. Jim Dempsey, one of the workshop organizers, wrote a blog post on the report: As a first step, our report recommends the inclusion of AI...

WAF-A-MoLE - A Guided Mutation-Based Fuzzer For ML-based Web Application Firewalls

A guided mutation-based fuzzer for ML-based Web Application Firewalls, inspired by AFL and based on the FuzzingBook by Andreas Zeller et al. Given an input SQL injection query, it tries to produce a semantic invariant query that is able to bypass the target WAF. You can use this tool for assessin...

The Supreme Court Narrowed the CFAA

In a 6-3 ruling, the Supreme Court just narrowed the scope of the Computer Fraud and Abuse Act: In a ruling delivered today, the court sided with Van Buren and overturned his 18-month conviction. In a 37-page opinion written and delivered by Justice Amy Coney Barrett, the court explained that the...

New Framework Released to Protect Machine Learning Systems From Adversarial Attacks

Microsoft, in collaboration with MITRE, IBM, NVIDIA, and Bosch, has released a new open framework that aims to help security analysts detect, respond to, and remediate adversarial attacks against machine learning ML systems. Called the Adversarial ML Threat Matrix, the initiative is an attempt to...

Adversarial Machine Learning and the CFAA

I just co-authored a paper on the legal risks of doing machine learning research, given the current state of the Computer Fraud and Abuse Act: Abstract: Adversarial Machine Learning is booming with ML researchers increasingly targeting commercial ML systems such as those used in Facebook, Tesla,...

Fooling NLP Systems Through Word Swapping

MIT researchers have built a system that fools natural-language processing systems by swapping words with synonyms: The software, developed by a team at MIT, looks for the words in a sentence that are most important to an NLP classifier and replaces them with a synonym that a human would find...

Introduction and Application of Model Hacking

ARCHIVED STORY Introduction and Application of Model Hacking By Steve Povolny · Febraury 19, 2020 Catherine Huang, Ph.D., and Shivangee Trivedi contributed to this blog. The term “Adversarial Machine Learning” AML is a mouthful! The term describes a research field regarding the study and design o...

Introduction and Application of Model Hacking

ARCHIVED STORY Introduction and Application of Model Hacking By Steve Povolny · Febraury 19, 2020 Catherine Huang, Ph.D., and Shivangee Trivedi contributed to this blog. The term “Adversarial Machine Learning” AML is a mouthful! The term describes a research field regarding the study and design o...