Adversarial Examples Are Not Bugs, They Are Features – gradient science

Research highlights and perspectives on machine learning and optimization from MadryLab.

Read the paper Download the datasets Over the past few years, adversarial examples – or inputs that have been slightly perturbed by an adversary to cause unintended behavior in machine learning systems – have received significant attention in the machine learning community (for more background, read our introduction to adversarial examples here ). There has been much work on training models that are not vulnerable to adversarial examples (in previous posts, we discussed methods for training robust models: part 1 , part 2 , but all this research does not really confront the fundamental question

related reading

• The Dimpled Manifold Model of Adversarial Examples in Machine Learningarxiv.org
• Some Lessons from Adversarial Machine Learning | FAR.AIfar.ai
• Toy Models of Superpositiontransformer-circuits.pub
• A Recipe for Training Neural Networkskarpathy.github.io
• Deep Learning is Not So Mysterious or Differentarxiv.org
• Solving adversarial attacks in computer vision as a baby version of general AI alignment | Stanislav Fortstanislavfort.com
• [1610.00768] Technical Report on the CleverHans v2.1.0 Adversarial Examples Libraryarxiv.org
• Feature Visualizationdistill.pub
• [2003.01690] Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacksarxiv.org
• Mediumai-alignment.com
• [2503.02113] Deep Learning is Not So Mysterious or Differentarxiv.org
• A Recipe for Training Neural Networkskarpathy.github.io