Feature-wise transformations

A simple and surprisingly effective family of conditioning mechanisms.

Feature-wise transformations Distill Feature-wise transformations A simple and surprisingly effective family of conditioning mechanisms. Authors Affiliations Vincent Dumoulin Google Brain Ethan Perez Rice University , MILA Nathan Schucher Element AI Florian Strub Univ. of Lille, Inria Harm de Vries MILA Aaron Courville MILA Yoshua Bengio MILA Published July 9, 2018 DOI 10.23915/distill.00011 Many real-world problems require integrating multiple sources of information. Sometimes these problems involve multiple, distinct modalities of information — vision, language, audio, etc. — as is required

saved by

• Winnie Xu
• Noah T
• Ali Ahmad
• Davis Brown

related reading

• Feature Visualizationdistill.pub
• Naturally Occurring Equivariance in Neural Networksdistill.pub
• Toy Models of Superpositiontransformer-circuits.pub
• Sparse Attention Post-Training for Mechanistic Interpretabilityarxiv.org
• The Building Blocks of Interpretabilitydistill.pub
• Fourier Feature Networksbmild.github.io
• transformer_attention.pdfarxiv.org
• Do Machine Learning Models Memorize or Generalize?pair.withgoogle.com
• Recent Advances in Language Model Fine-tuningruder.io
• Feature Visualizationdistill.pub
• [1810.00825] Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networksarxiv.org
• Uncovering Neural Geometry in Vision Models With Block-Sparse Featurizersgoodfire.ai