✳flâneur — a map of the web's best reading

Modeling Symmetries | Mark Neumann

markneumann.xyz · 4,128 words · saved by 1 readers

Chaitanya K. Joshi's excellent blog post on his PhD work inspired me to write down some thoughts on symmetries and invariances in neural network models—particularly in the context of molecular data. Discussions around removing architectural biases can lean a little absolutist ("Bitter Lesson, GPUs go brrr!"), whereas the reality is more nuanced. I am actually not so opposed to the idea of encoding physical priors, but I am interested in when such symmetries should be embedded directly into the model architecture, and when they might be better handled implicitly: through data augmentation, regularization or inference-time tricks. This post represents some of my current thinking on the topic, through the lens of two modeling domains in which the approaches to encoding symmetries are very different: neural network potentials and diffusion models. When modeling image data, there are many desirable invariances one can imagine (translation being the predominant one baked into the most common

Chaitanya K. Joshi's excellent blog post on his PhD work inspired me to write down some thoughts on symmetries and invariances in neural network models—particularly in the context of molecular data. Discussions around removing architectural biases can lean a little absolutist (" Bitter Lesson , GPUs go brrr!"), whereas the reality is more nuanced. I am actually not so opposed to the idea of encoding physical priors, but I am interested in when such symmetries should be embedded directly into the model architecture, and when they might be better handled implicitly: through data augmentation, re

Explore this link on the map →

saved by

related reading