Hamel’s Blog
I am a machine learning engineer with over 20 years of experience. I have worked with innovative companies such as Airbnb and GitHub, which included early LLM research used by OpenAI, for code understanding. I have also led and contributed to numerous popular open-source machine-learning tools. I am currently an independent consultant helping companies build AI products. I am currently teaching a course on AI Evals For Engineers & PMs from July 21 - August 22, 2025. If you need more hands-on help, I’m open to new consulting work and other forms of advisory. If you need help with your project, you can find more information here. I often share my experience building AI products. Below is a selected assortment of my longer-form writing. The best way to follow my activity is Twitter.
🎓 Learn With Me I co-teach AI Evals for Engineers and PMs, with over 5,000 students from 500+ companies (including OpenAI, Anthropic, and Google). Check the page for upcoming cohorts. I often share my experience building AI products. Below is a selection of my long-form writing. Date Title 8/12/26 AI Product Engineering Notes 7/11/26 Do Automated Evals Work? 6/29/26 “It’s Hard to Eval” Is a Product Smell 3/26/26 The Revenge of the Data Scientist 3/2/26 Evals Skills for Coding Agents 1/18/26 Why I Stopped Using nbdev 10/1/25 Selecting The Right AI Evals Tool 7/11/25 Stop…
saved by
related reading
- Your AI Product Needs Evals – Hamel's Blog - Hamel Husainhamel.dev
- A pragmatic guide to LLM evals for devsnewsletter.pragmaticengineer.com
- Demystifying evals for AI agents \ Anthropicanthropic.com
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- AI Design Field Guideaidesignfieldguide.com
- What We Learned from a Year of Building with LLMs (Part II) – O’Reillyoreilly.com
- The 2025 AI Engineering Reading List - Latent.Spacelatent.space
- The bitter lesson of LLM evalsparsed.com
- LLM Evals: Everything You Need to Know – Hamel’s Bloghamel.dev
- Successful language model evals - Jason Weijasonwei.net
- LLM evaluation: a beginner's guideevidentlyai.com
- The Rise of the AI Engineer - Latent.Spacelatent.space