How to Use LLM as a Judge (Without Getting Burned) - Manthan

TL;DR: LLM as a judge works well when you (1) use reference based evaluation over pointwise scoring, (2) debias pairwise comparisons by swapping positions, (3) ensemble across multiple models, (4) require reasoning before scores, and (5) calibrate against human judgments. Skip it when ground truth exists or for safety critical decisions. Table of Contents Why Traditional Evaluation Methods Fail Evaluation Modes Crafting Effective Judge Prompts Limitations and Biases When NOT to Use LLM as a judge Optimizations Conclusion Last month, I ran thousands of evals on an agentic system I was building. The responses were open-ended, context dependent, and impossible to validate with simple string matching. I tried exact match comparisons, they failed miserably. I tried BLEU scores, they penalized semantically correct but differently worded responses. I was stuck! Traditional metrics could not handle the nuance I needed, and manual evaluation was too slow and expensive.

TL;DR: LLM as a judge works well when you (1) use reference based evaluation over pointwise scoring, (2) debias pairwise comparisons by swapping positions, (3) ensemble across multiple models, (4) require reasoning before scores, and (5) calibrate against human judgments. Skip it when ground truth exists or for safety critical decisions. Table of Contents Why Traditional Evaluation Methods Fail Evaluation Modes Crafting Effective Judge Prompts Limitations and Biases When NOT to Use LLM as a judge Optimizations Conclusion Last month, I ran thousands of evals on an agentic system I…

saved by

• Turja Chowdhury

related reading

• LLM-as-a-judge: a complete guide to using LLMs for evaluationsevidentlyai.com
• Patterns for Building LLM-based Systems & Productseugeneyan.com
• LLM evaluation: a beginner's guideevidentlyai.com
• LLM Evaluation doesn't need to be complicatedphilschmid.de
• Successful language model evals - Jason Weijasonwei.net
• Demystifying evals for AI agents \ Anthropicanthropic.com
• Your AI Product Needs Evals – Hamel's Blog - Hamel Husainhamel.dev
• What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
• Agent Evaluation: A Detailed Guidecameronrwolfe.substack.com
• The bitter lesson of LLM evalsparsed.com
• Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
• LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com