Indirect Prompt Injection Attacks LLMs
github.com · 1,127 words · saved by 1 readers
New ways of breaking app-integrated LLMs . Contribute to greshake/llm-security development by creating an account on GitHub.
New: Demonstrating Indirect Injection attacks on Bing Chat Compromising LLMs using Indirect Prompt Injection "... a language model is a Turing-complete weird machine running programs written in natural language; when you do retrieval, you are not 'plugging updated facts into your AI', you are actually downloading random new unsigned blobs of code from the Internet (many written by adversaries) and casually executing them on your LM with full privileges. This does not end well." - Gwern Branwen on LessWrong We present a new class of vulnerabilities and impacts stemming from "indirect…
saved by
related reading
- Prompt Injection as Role Confusionrole-confusion.github.io
- A Mechanistic Explanation of Prompt Injection (and why you should study roles) — LessWronglesswrong.com
- The lethal trifecta for AI agents: private data, untrusted content, and external communicationsimonwillison.net
- CaMeL offers a promising new direction for mitigating prompt injection attackssimonwillison.net
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- LLM Powered Autonomous Agents | Lil'Loglilianweng.github.io
- GitHub - brexhq/prompt-engineering: Tips and tricks for working with Large Language Models like OpenAI's GPT-4.github.com
- The Dual LLM pattern for building AI assistants that can resist prompt injectionsimonwillison.net
- [2603.12277] Prompt Injection as Role Confusionarxiv.org
- HackAPromptpaper.hackaprompt.com
- [2608.09867] Stealing Reasoning Traces from Proprietary LLM APIsarxiv.org
- 2025: The year in LLMssimonwillison.net