Applied LLMs - What We’ve Learned From A Year of Building with LLMs
It’s an exciting time to build with large language models (LLMs). Over the past year, LLMs have become “good enough” for real-world applications. And they’re getting better and cheaper every year. Coupled with a parade of demos on social media, there will be an estimated $200B investment in AI by 2025. Furthermore, provider APIs have made LLMs more accessible, allowing everyone, not just ML engineers and scientists, to build intelligence into their products. Nonetheless, while the barrier to entry for building with AI has been lowered, creating products and systems that are effective—beyond a demo—remains deceptively difficult. We’ve spent the past year building, and have discovered many sharp edges along the way. While we don’t claim to speak for the entire industry, we’d like to share what we’ve learned to help you avoid our mistakes and iterate faster. These are organized into three sections: Our intent is to make this a practical guide to building successful products with LLMs, dra
What We’ve Learned From A Year of Building with LLMs – Applied LLMs New: AI Evals course by the authors → It’s an exciting time to build with large language models (LLMs). Over the past year, LLMs have become “good enough” for real-world applications. And they’re getting better and cheaper every year. Coupled with a parade of demos on social media, there will be an estimated $200B investment in AI by 2025 . Furthermore, provider APIs have made LLMs more accessible, allowing everyone, not just ML engineers and scientists, to build intelligence into their products. Nonetheless, while the barrier
saved by
related reading
- Patterns for Building LLM-based Systems & Productseugeneyan.com
- What We Learned from a Year of Building with LLMs (Part I) – O’Reillyoreilly.com
- The bitter lesson of LLM evalsparsed.com
- Your AI Product Needs Evals – Hamel's Blog - Hamel Husainhamel.dev
- 2025 LLM Year in Review – karpathykarpathy.bearblog.dev
- Things we learned about LLMs in 2024simonwillison.net
- AI Revolution - Transformers and Large Language Models (LLMs)blog.eladgil.com
- 2025: The year in LLMssimonwillison.net
- Building LLM applications for productionhuyenchip.com
- Productizing Large Language Modelsblog.replit.com
- Malleable software in the age of LLMsgeoffreylitt.com
- Stuff we figured out about AI in 2023simonwillison.net