Alazar Manakelew

0 followers · 1 following · 267 views

on the atlas — 6

The No Free Lunch Theorem tells us that no method can outperform others across all problems. Every method has its own domain of applicability. Constructing toy models is a way to explore this scope of applicability.
How to ground your ideas? | Ziming Liu
When learning existing methods, we often encounter various arguments explaining why a method works or does not work. Although these arguments may sound intuitively reasonable, I rarely feel fully convinced, nor do I easily incorporate them into my own knowledge graph. Instead, I tend to construct a toy model to test these arguments. I also try to modify the toy model to break these arguments, in order to understand the conditions and boundaries under which they hold.
How to ground your ideas? | Ziming Liu
Key takeaway: TPUs are very simple. They load weights from HBM into VMEM, then from VMEM into a systolic array which can perform around 200 trillion multiply-adds per second. The HBM VMEM and VMEM systolic array bandwidths set fundamental limits on what computations TPUs can do efficiently.
How to Think About TPUs | How To Scale Your Model
What he’s really done is he’s created this immense vertical filing cabinet in his brain of layers and layers and layers of files of information that he can draw back on now for more than 70 years worth of data.
Curius / Onboarding