HackAPrompt

HackAPrompt

HackAPrompt Compete in HackAPrompt 2.0, the world's largest AI Red-Teaming competition! Check it out → × WINNER: Best Theme Paper at EMNLP2023 Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competition EMNLP 2023 Sander Schulhoff * 1 , Jeremy Pinto * 2 , Anaum Khan 1 , Louis-François Bouchard 2,3 , Chenglei Si 4 , Svetlina Anati** 5 , Valen Tagliabue** 6 , Anson Liu Kost** 7 , Christopher Carnahan** 8 , Jordan Boyd-Graber 1 ▶ 1 University of Maryland &nbsp &nbsp ▶ 2 Mila &nbsp &nbsp ▶ 3 Towards AI &nbsp &nbsp ▶ 4 Stanford Univ

saved by

• Shruti Das

related reading

• Natural-emergent-misalignment-from-reward-hacking-paper.pdfassets.anthropic.com
• Training on Documents about Reward Hacking Induces Reward Hackingalignment.anthropic.com
• Prompt Injection as Role Confusionrole-confusion.github.io
• [2312.06942] AI Control: Improving Safety Despite Intentional Subversionarxiv.org
• [2604.22082] Removing Sandbagging in LLMs by Training with Weak Supervisionarxiv.org
• [2608.09867] Stealing Reasoning Traces from Proprietary LLM APIsarxiv.org
• llm-security/README.md at main · greshake/llm-securitygithub.com
• 2402.07927arxiv.org
• [2203.07814] Competition-Level Code Generation with AlphaCodearxiv.org
• Systematic Reward Hacking and Prime Sprintsprimeintellect.ai
• Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face — LessWronglesswrong.com
• CaMeL offers a promising new direction for mitigating prompt injection attackssimonwillison.net