Machine learning is going real-time

[Twitter thread]

[ Twitter thread ] For a follow up of this post, see Real-time machine learning: challenges and solutions (2022). After talking to machine learning and infrastructure engineers at major Internet companies across the US, Europe, and China, I noticed two groups of companies. One group has made significant investments (hundreds of millions of dollars) into infrastructure to allow real-time machine learning and has already seen returns on their investments. Another group still wonders if there’s value in real-time ML. There seems to be little consensus on what real-time ML means, and there hasn’t

related reading

• Real-time machine learning: challenges and solutionshuyenchip.com
• Introduction to streaming for data scientistshuyenchip.com
• Scaling is subtler than it seemsberen.io
• The Continual Learning Problemjessylin.com
• A friendly introduction to machine learning compilers and optimizershuyenchip.com
• You can’t imitation-learn how to continual-learn — LessWronglesswrong.com
• What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
• Why We Need Continual Learning | Andreessen Horowitza16z.com
• NL.pdfabehrouz.github.io
• Spending Inference Time - Kevin Lukevinlu.ai
• Q-learning is not yet scalableseohong.me
• Is Frontier Asynchronous RL Solved? — Luke J. Huangluk-huang.github.io