Einsum is easy and useful | Erik Jenner

`einsum` is one of the most useful functions in Numpy/Pytorch/Tensorflow and yet many people don't use it. It seems to have a reputation as being difficult to understand and use, which is completely backwards in my view: the reason `einsum` is great is precisely because it is *easier* to use and reason about than the alternatives. So this post tries to set the record straight and show how simple `einsum` really is.

einsum is one of the most useful functions in Numpy/Pytorch/Tensorflow and yet many people don’t use it. It seems to have a reputation as being difficult to understand and use, which is completely backwards in my view: the reason einsum is great is precisely because it is easier to use and reason about than the alternatives. So this post tries to set the record straight and show how simple einsum really is. The general syntax for einsum is einsum ( "some string describing an operation" , tensor_1 , tensor_2 , . . . ) with an arbitrary number of tensors after the string. (I’ll be saying “tensor

related reading

• Tim Rocktäschelrockt.ai
• Tim Rocktäschelrockt.github.io
• Computing sharding with einsum : ezyang's blogblog.ezyang.com
• Einops tutorial, part 1: basics - Einopseinops.rocks
• Einops: Powerful library for tensor operations in deep learning | Oxford Protein Informatics Groupblopig.com
• Chapter 0: Fundamentals - ARENAlearn.arena.education
• Tensor Considered Harmfulnlp.seas.harvard.edu
• Making Deep Learning go Brrrr From First Principleshorace.io
• How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
• Tensors — PyTorch Tutorials 2.12.0+cu130 documentationpytorch.org
• PyTorch internals : ezyang's blogblog.ezyang.com
• A Recipe for Training Neural Networkskarpathy.github.io