Applied LLMs - What We’ve Learned From A Year of Building with LLMs
It’s an exciting time to build with large language models (LLMs). Over the past year, LLMs have become “good enough” for real-world applications. And they’re getting better and cheaper every year. Coupled with a parade of demos on social media, there will be an estimated $200B investment in AI by 2025. Furthermore, provider APIs have made LLMs more accessible, allowing everyone, not just ML engineers and scientists, to build intelligence into their products. Nonetheless, while the barrier to entry for building with AI has been lowered, creating products and systems that are effective—beyond a demo—remains deceptively difficult. We’ve spent the past year building, and have discovered many sharp edges along the way. While we don’t claim to speak for the entire industry, we’d like to share what we’ve learned to help you avoid our mistakes and iterate faster. These are organized into three sections: Our intent is to make this a practical guide to building successful products with LLMs, dra
What We’ve Learned From A Year of Building with LLMs – Applied LLMs New: AI Evals course by the authors → It’s an exciting time to build with large language models (LLMs). Over the past year, LLMs have become “good enough” for real-world applications. And they’re getting better and cheaper every year. Coupled with a parade of demos on social media, there will be an estimated $200B investment in AI by 2025 . Furthermore, provider APIs have made LLMs more accessible, allowing everyone, not just ML engineers and scientists, to build intelligence into their products. Nonetheless, while the barrier
Explore this link on the map →related reading
- Patterns for Building LLM-based Systems & Productseugeneyan.com
- What We Learned from a Year of Building with LLMs (Part I) – O’Reillyoreilly.com
- 2025 LLM Year in Review – karpathykarpathy.bearblog.dev
- What We Learned from a Year of Building with LLMs (Part II) – O’Reillyoreilly.com
- Building LLM applications for productionhuyenchip.com
- Evaluating LLM Applicationshumanloop.com
- The bitter lesson of LLM evalsparsed.com
- LLM evaluation: a beginner's guideevidentlyai.com
- LLM Evaluation doesn't need to be complicatedphilschmid.de
- 2025: The year in LLMssimonwillison.net
- Your AI Product Needs Evals – Hamel's Blog - Hamel Husainhamel.dev
- Things we learned about LLMs in 2024simonwillison.net