Production Twitter on One Machine? 100Gbps NICs and NVMe are fast - Tristan Hume
In this post I’ll attempt the fun stunt of designing a system that could serve the full production load of Twitter with most of the features intact on a single (very powerful) machine. I’ll start by showing off a Rust prototype of the core tweet distribution data structure handling 35x full load by fitting the hot set in RAM and parallelizing with atomics, and then do math around how modern high-performance storage and networking might let you serve a close-to-fully-featured Twitter on one machine.
Production Twitter on One Machine? 100Gbps NICs and NVMe are fast - Tristan Hume Tristan Hume Github Resume + Project List Blog Production Twitter on One Machine? 100Gbps NICs and NVMe are fast --> In this post I’ll attempt the fun stunt of designing a system that could serve the full production load of Twitter with most of the features intact on a single (very powerful) machine. I’ll start by showing off a Rust prototype of the core tweet distribution data structure handling 35x full load by fitting the hot set in RAM and parallelizing with atomics, and then do math around how modern high-per
Explore this link on the map →saved by
related reading
- How we reduced the cost of building Twitter at Twitter-scale by 100x – Blogblog.redplanetlabs.com
- How to Twitter Successfully | near.blognear.blog
- Remains of the Dayeugenewei.com
- How to Blow Up a Timeline - Remains of the Dayeugenewei.com
- Building a web search engine from scratch in two months with 3 billion neural embeddingsblog.wilsonl.in
- The Serendipity Machine (Notes on Using Twitter) — Nabeel S. Qureshinabeelqu.co
- Reiner Pope – The math behind how LLMs are trained and serveddwarkesh.com
- Threads and the Social/Communications Map – Stratechery by Ben Thompsonstratechery.com
- Crawling a billion web pages in just over 24 hoursandrewkchan.dev
- GitHub - igorbrigadir/awesome-twitter-algo: The release of the Twitter algorithm, annotated for recsys · GitHubgithub.com
- The Serendipity Machine - Nabeel S. Qureshinabeelqu.substack.com
- abseil / Performance Hintsabseil.io