✳flâneur — a map of the web's best reading

Chapter 4: Alignment Science - ARENA

learn.arena.education · 1,855 words · saved by 1 readers

Now that you've had a look at some basic keyword-based scoring for misalignment, you should be able to see some flaws in this method. Simple heuristics like these can be cheap and fast, but they miss a lot of subtlety, for example: To get around this, lots of alignment research involves using autoraters (also called LLM-as-judge). We write a prompt that asks a language model to evaluate other models' outputs, looking for particular concepts or behaviours, and ask for output often in a structured format (e.g. a score from 1-5, or a classification label). First, let's get a few helper functions to call external APIs. We'll use the concurrent.futures library to run multiple API calls in parallel, which speeds up our work a lot (although you should watch out for rate limits and high API costs!). Now you'll implement an autorater to score model responses more accurately than keyword matching. In this exercise you should fill in the judge prompt and the score_with_autorater function (which i

2️⃣ Quantifying Misalignment Learning Objectives Understand limitations of heuristic scoring methods Design and implement LLM-as-judge autoraters for measuring misalignment Compare different scoring approaches and understand their tradeoffs Run systematic evaluations across multiple behavioral categories Now that you've had a look at some basic keyword-based scoring for misalignment, you should be able to see some flaws in this method. Simple heuristics like these can be cheap and fast, but they miss a lot of subtlety, for example: The model might use different words (no regex-based classifier

Explore this link on the map →

related reading