Tversky Neural Networks - by Grigory Sapunov - Gonzo ML

Authors: Moussa Koulako Bala Doumbouya, Dan Jurafsky, Christopher D. Manning Paper: https://arxiv.org/abs/2506.11035 Once a year, some interesting architecture inevitably appears where they change some fundamental building block. This happened with KAN last year, where they changed the parameterization of the neuron activation function (though it's unclear what the outcome is after a year — many follow-up works seem to have appeared, but KANs haven't displaced anyone anywhere yet). The same is true in this current work, where they change the proximity function from the classic scalar product as in transformers (or cosine similarity, which is roughly the same) to a more sophisticated asymmetric function named after Amos Tversky. Jurafsky and Manning are co-authors (and in KANs, Tegmark was a co-author), so these aren't exactly random people. Gonzo ML is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Modern deep lear

Authors: Moussa Koulako Bala Doumbouya, Dan Jurafsky, Christopher D. Manning Paper: https://arxiv.org/abs/2506.11035 Once a year, some interesting architecture inevitably appears where they change some fundamental building block. This happened with KAN last year, where they changed the parameterization of the neuron activation function (though it's unclear what the outcome is after a year — many follow-up works seem to have appeared, but KANs haven't displaced anyone anywhere yet). The same is true in this current work, where they change the proximity function from the classic scalar…

related reading

• [2506.11035] Tversky Neural Networks: Psychologically Plausible Deep Learning with Differentiable Tversky Similarityarxiv.org
• Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
• Zoom In: An Introduction to Circuitsdistill.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Distill — Latest articles about machine learningdistill.pub
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• Neural Networks, Types, and Functional Programming -- colah's blogcolah.github.io
• Neural Networks, Manifolds, and Topology -- colah's blogcolah.github.io
• The Decade of Deep Learning | Leo Gaobmk.sh
• Feature Visualizationdistill.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• NL.pdfabehrouz.github.io