[1908.07125] Universal Adversarial Triggers for Attacking and Analyzing NLP

Abstract:Adversarial examples highlight model vulnerabilities and are useful for evaluation and interpretation. We define universal adversarial triggers: input-agnostic sequences of tokens that trigger a model to produce a specific prediction when concatenated to any input from a dataset. We propose a gradient-guided search over tokens which finds short trigger sequences (e.g., one word for classification and four words for language modeling) that successfully trigger the target prediction. For example, triggers cause SNLI entailment accuracy to drop from 89.94% to 0.55%, 72% of "why" questions in SQuAD to be answered "to kill american people", and the GPT-2 language model to spew racist output even when conditioned on non-racial contexts. Furthermore, although the triggers are optimized using white-box access to a specific model, they transfer to other models for all tasks we consider. Finally, since triggers are input-agnostic, they provide an analysis of global model behavior. For instance, they confirm that SNLI models exploit dataset biases and help to diagnose heuristics learned by reading comprehension models.

[1908.07125] Universal Adversarial Triggers for Attacking and Analyzing NLP Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Computation and Language arXiv:1908.07125 (cs) [Submitted on 20 Aug 2019 ( v1 ), last revised 3 Jan 2021 (this version, v3)] Title: Universal Adversarial Triggers for Attacking and Analyzing NLP Authors: Eric Wallace , Shi Feng , Nikhil Kandpal , Matt Gardner , Sameer Singh View a PDF of the paper titled Universal Adversarial Triggers for Attacking and Analyzi

related reading

• Natural Language Autoencoders \ Anthropicanthropic.com
• Adversarial Attacks on LLMs | Lil'Loglilianweng.github.io
• Discovering Backdoor Triggers — LessWronglesswrong.com
• Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activationstransformer-circuits.pub
• Nicholas Carlininicholas.carlini.com
• Adversarial Attacks on Aligned Language Models | Gray Swan Researchgrayswan.ai
• You Can Catch Sleeper Agents by Teaching Another Model to Imitate Them — LessWronglesswrong.com
• [2209.02128] Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examplesarxiv.org
• GitHub - SoyGema/pulling_ace · GitHubgithub.com
• [2401.05566] Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingarxiv.org
• Universal and Transferable Attacks on Aligned Language Modelsllm-attacks.org
• [2307.15043] Universal and Transferable Adversarial Attacks on Aligned Language Modelsarxiv.org