Why Twitter Didn’t Go Down: From a Real Twitter SRE

Twitter supposedly lost around 80% of its work force. What ever the real number is, there are whole teams with out engineers on it now. Yet, the website goes on and the tweets keep coming. This left a lot wondering what exactly was going on with all those engineers and made it seem like it was all just bloat. I’d like to explain my little corner of Twitter (though it wasn’t

Twitter supposedly lost around 80% of its work force. What ever the real number is, there are whole teams with out engineers on it now. Yet, the website goes on and the tweets keep coming. This left a lot wondering what exactly was going on with all those engineers and made it seem like it was all just bloat. I’d like to explain my little corner of Twitter (though it wasn’t so little) and some of the work that went on that kept this thing running. For five years I was a Site Reliability Engineer(SRE) at Twitter. For four of those years I was the sole SRE for the Cache team. There was a few…

related reading

• Remains of the Dayeugenewei.com
• Production Twitter on One Machine? 100Gbps NICs and NVMe are fast - Tristan Humethume.ca
• How to Blow Up a Timeline - Remains of the Dayeugenewei.com
• Being oncall taught me everything - Yao Yueyaoyue.org
• Threads and the Social/Communications Map – Stratechery by Ben Thompsonstratechery.com
• How to Twitter Successfully | near.blognear.blog
• How we reduced the cost of building Twitter at Twitter-scale by 100x – Blogblog.redplanetlabs.com
• More Than DNS: The 14 hour AWS us-east-1 outage – Jonathon Belotti [thundergolfer]thundergolfer.com
• Extremely softcore: the old Twitter was an idealist’s workplace and a naive businesstheverge.com
• The Serendipity Machine (Notes on Using Twitter) — Nabeel S. Qureshinabeelqu.co
• Building and operating a pretty big storage system called S3 | All Things Distributedallthingsdistributed.com
• Google SRE - IT Service Management: Automate Operationssre.google