Gandalf | Lakera - Prompt Injection

Trick Gandalf into revealing information and experience the limitations of large language models firsthand.

Welcome to Agent Breaker! This is an entire app store of GenAI applications - and every single one of them can be hacked. This challenge is about exploiting real-world AI agents. Your Mission: Hack each app in any order across multiple levels of increasing difficulty using prompt attacks that bypass the app's defenses and manipulate the underlying LLM. How it Works: Each attack attempt is scored from 0 to 100 based on how effectively it achieves the attack objective. Try as many times as you need. Score 75+ to unlock the next level for that app. Go back to improve your scores anytime. How…

saved by

• Eric Chang
• Yoyo Yuan
• Yueh Han Huang
• Kamil Przespolewski

related reading

• LLM Powered Autonomous Agents | Lil'Loglilianweng.github.io
• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incidentmetr.org
• Training a Misaligned Reward Seekeralignment.anthropic.com
• Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face — LessWronglesswrong.com
• HackAPromptpaper.hackaprompt.com
• llm-security/README.md at main · greshake/llm-securitygithub.com
• Center for Responsible, Decentralized Intelligence at Berkeleyrdi.berkeley.edu
• Systematic Reward Hacking and Prime Sprintsprimeintellect.ai
• Rogue AI Trackerrogueaitracker.com
• HackerRank - Online Coding Tests and Technical Interviewshackerrank.com
• General Agent: A Self-Evolving, Synthetic Agent Environmentprimeintellect.ai