flâneur

Introducing tinygrad

simonwenkel.com · 779 words · saved by 1 readers

Update: Initial version was based on a tinygrad version released in February 2021. Updated to recent version on 2022-06-26 Apparently, it seems to be just another deep learning framework. Well, not exactly. It is a few orders of magnitude smaller and simpler than e.g. PyTorch. Everyone who ever tried to read the low-level source code of PyTorch or TensorFlow would appreciate that. It certainly promises a low barrier of adding more accelerators. Due to its extreme simplicity, it aims to be the easiest framework to add new accelerators to, with support for both inference and training. Where does the name come from? tinygrad will always be below 1000 lines. If it isn’t, we will revert commits until tinygrad becomes smaller. I think that this is a bit too ambitious unless you split it into various packages to add e.g. support for various accelerators and outsource e.g. model structures to config files and load it from there or so. But yes, it certainly sounds interesting. Update: seems lik

Contents Introduction Structure Accelerator usage GPU ANE First Steps Conclusions Introduction Update: Initial version was based on a tinygrad version released in February 2021. Updated to recent version on 2022-06-26 Apparently, it seems to be just another deep learning framework . Well, not exactly. It is a few orders of magnitude smaller and simpler than e.g. PyTorch. Everyone who ever tried to read the low-level source code of PyTorch or TensorFlow would appreciate that. It certainly promises a low barrier of adding more accelerators. Due to its extreme simplicity, it aims to be the easies

saved by

related reading