Rohan Tibrewal
4 followers · 4 following · 730 views
on the atlas · 11
- Touching the Elephant - TPUs | Consider the Bulldog1 savers
- Introducing Husky, Datadog's Third-Generation Event Store | Datadog2 savers
- Defeating Nondeterminism in LLM Inference - Thinking Machines Lab40 savers
- H-Nets - the Past | Goomba Lab6 savers
- On the Tradeoffs of SSMs and Transformers | Goomba Lab8 savers
- Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention) – Jay Alammar – Visualizing machine learning one concept at a time.9 savers
- The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.37 savers
- Transformers Inference Optimization Toolset | AstraBlog3 savers
- A basic systems architecture for AI agents that do autonomous research — LessWrong4 savers
- 1910.02054v31 savers
- Curius / Onboarding2704 savers
highlights · 12
The trade off is between compute bandwidth and array utilization
Touching the Elephant - TPUs | Consider the BulldogMatrix multiply, some activation looking functions on that result, feed the results to storage, repeat
Touching the Elephant - TPUs | Consider the BulldogSpecialization asks what the optimal way to spend a fixed transistor and energy budget is to squeeze out performance.
Touching the Elephant - TPUs | Consider the BulldogThese nodes are completely stateless and can autoscale. Critically, we never communicate with the Writer nodes at query time. This dramatically reduces the ability of queries to impair ingestion, or vice versa.
Introducing Husky, Datadog's Third-Generation Event Store | DatadogThe easiest solution is to simply ignore these cases altogether.
Defeating Nondeterminism in LLM Inference - Thinking Machines Labthe primary reason nearly all LLM inference endpoints are nondeterministic is that the load (and thus batch-size) nondeterministically varies!
Defeating Nondeterminism in LLM Inference - Thinking Machines LabThe load determines the batch size that the kernels are run under
Defeating Nondeterminism in LLM Inference - Thinking Machines Labin the typical forward pass of an LLM, there is usually not a single atomic add present.
Defeating Nondeterminism in LLM Inference - Thinking Machines Labthe black box that performs this conversion in the best way possible
On the Tradeoffs of SSMs and Transformers | Goomba Labself-attention could have been derived from first principles as the canonical model that stores a cache (“KV” or not) of its context.
On the Tradeoffs of SSMs and Transformers | Goomba Labtransition matrices vary through time and depend on the data itself
On the Tradeoffs of SSMs and Transformers | Goomba LabWhat he’s really done is he’s created this immense vertical filing cabinet in his brain of layers and layers and layers of files of information that he can draw back on now for more than 70 years worth of data.
Curius / Onboarding