geometricdeeplearning.com/book/graphs.html
146 Chapter 5 The principle of equivariance requires that changing the order of m subgraphs in the multiset (permutation group S m ) and the order of the n nodes in the sub- graphs (permutation group S n ) yields an equivalent representation, or in other words, the architecture is equivariant w.r.t. the product group 20 G = S m ×S n . This example serves as a relevant instance of combining symmetries within the same architecture; we will explore more involved instances of this when we discuss geometric graphs in latter Chapters. 5.2.5.4 Synchronisation and “clocks” Thus far, we have discussed sev- eral ways in which the message passing equation may be edited: its parametric functions (ψ, ϕ), aggregation function ( L ), features (x u ), and computation graph (N u ). It may seem as if we’ve covered all the angles possible by doing so, but in fact, there remains one hidden but important aspect – the synchroni- sation between the different steps of the mechanism. In reality, Equation 5.56
5 Graphs • In multiple branches of science, from sociology to particle physics, graphs are a fundamental model of systems of relations and interactions. • Graphs giv e rise to a basic type of symmetry modelled by the group of permutations. This is why we start our in vestigation by studying them. • Most other objects of interest to us, such as grids and sets, can be obtained as a particular case of graphs, and many architectures we will discuss here will be expressible in the language of graph neural networks. • This also includes the modern large language model (LLM) stack, which we present a
Explore this link on the map →saved by
related reading
- Beyond Message Passing: a Physics-Inspired Paradigm for Graph Neural Networksthegradient.pub
- A Gentle Introduction to Graph Neural Networksdistill.pub
- Understanding Convolutions on Graphsdistill.pub
- Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
- Graph Convolutional Networks | Thomas Kipf | Google DeepMindtkipf.github.io
- Verifying your browser | OpenReviewopenreview.net
- Naturally Occurring Equivariance in Neural Networksdistill.pub
- 10. Equivariant Neural Networks — deep learning for molecules & materialsdmol.pub
- Distill — Latest articles about machine learningdistill.pub
- Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
- [2402.17805] Graph Neural Networks and Arithmetic Circuitsarxiv.org
- GraphGPS: Navigating Graph Transformers | Towards Data Sciencetowardsdatascience.com