Lessons from Defending Gemini against Indirect Prompt Injections

Gemini is increasingly used to perform tasks on behalf of users, where function-calling and tool-use capabilities enable the model to access user data. Some tools, however, require access to untrusted data introducing risk. Adversaries can embed malicious instructions in untrusted data which caus...

Improving LLM Outputs against Jailbreak Attacks with Expert Model Integration

Using LLMs in a production environment presents security challenges that include vulnerabilities to jailbreaks and prompt injections, which can result in harmful outputs for humans or the enterprise. The challenge is amplified when working within a specific domain, as topics generally accepted fo...

Three Tips to Protect Your Secrets from AI Accidents

Last year, the Open Worldwide Application Security Project OWASP published multiple versions of the "OWASP Top 10 For Large Language Models," reaching a 1.0 document in August and a 1.1 document in October. These documents not only demonstrate the rapidly evolving nature of Large Language Models,...

Google Expands Its Bug Bounty Program to Tackle Artificial Intelligence Threats

Google has announced that it's expanding its Vulnerability Rewards Program VRP to compensate researchers for finding attack scenarios tailored to generative artificial intelligence AI systems in an effort to bolster AI safety and security. "Generative AI raises new and different concerns than...

PYSEC-2023-18

In LangChain through 0.0.131, the LLMMathChain chain allows prompt injection attacks that can execute arbitrary code via the Python exec method...