Tversky Neural Networks - by Grigory Sapunov - Gonzo ML

Authors: Moussa Koulako Bala Doumbouya, Dan Jurafsky, Christopher D. Manning Paper: https://arxiv.org/abs/2506.11035 Once a year, some interesting architecture inevitably appears where they change some fundamental building block. This happened with KAN last year, where they changed the parameterization of the neuron activation function (though it's unclear what the outcome is after a year — many follow-up works seem to have appeared, but KANs haven't displaced anyone anywhere yet). The same is true in this current work, where they change the proximity function from the classic scalar product as in transformers (or cosine similarity, which is roughly the same) to a more sophisticated asymmetric function named after Amos Tversky. Jurafsky and Manning are co-authors (and in KANs, Tegmark was a co-author), so these aren't exactly random people. Gonzo ML is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Modern deep lear

Authors: Moussa Koulako Bala Doumbouya, Dan Jurafsky, Christopher D. Manning Paper: https://arxiv.org/abs/2506.11035 Once a year, some interesting architecture inevitably appears where they change some fundamental building block. This happened with KAN last year, where they changed the parameterization of the neuron activation function (though it's unclear what the outcome is after a year — many follow-up works seem to have appeared, but KANs haven't displaced anyone anywhere yet). The same is true in this current work, where they change the proximity function from the classic scalar product a