‘You Could Have Invented Transformers’ tutorial proposal, by Gwern · Gwern.net
gwern.net · saved by 7 readers
Proposal for a ‘You Could Have Invented Transformers’ tutorial; someone should write a series showing a logical recreation of Transformers from primitive 𝑛-gram language models to full-strength Transformers.