flâneur

Distillation Walkthrough

vladfeinberg.com · 1,446 words · saved by 2 readers

Vlad's Blog

Distillation Walkthrough Distillation is a critical technique towards improving a network’s quality while keeping its serving latency constant. This is becoming crucial as people focus on serving larger and larger models. Image found on LinkedIn . Distillation is a powerful technique, but a couple things about it are quite mystical. The purpose of this post is to: Provide a very high level explainer (but mostly refer to source papers) of distillation. Show that you can create a simple, linear example where distillation works. How is distillation implemented? At an algorithmic level, distillati

saved by

related reading