Metrics — Continual Learning Bench Docs

Each task is designed to answer one question: does the system perform better the more experience it has? Each task defines a per-instance reward. Tasks have their own definition (accuracy, normalized error, profit, …), but every reward must satisfy: If per-instance difficulty is roughly homogeneous across the sequence, reward by itself is enough to read off learning: a system that is genuinely improving from experience will produce an upward-trending reward curve, because the curve isn't being moved by changes in instance difficulty. In that regime, "higher reward at task instance t" means "more was learned by instance t." For example, in the "exploitable poker" task, chips are reset after each hand (so winnings are not accumulated). That means that when a system is playing against the same opponent, the basic challenge for that hand is the same as in previous hands. Increases in reward across hands can thus reflect real learning signal — systems are doing progressively better against

Each task is designed to answer one question: does the system perform better the more experience it has? Reward Each task defines a per-instance reward. Tasks have their own definition (accuracy, normalized error, profit, …), but every reward must satisfy: Higher is better. Comparable across different instances of a task. Reflects the task objective directly, not an unrelated proxy. If per-instance difficulty is roughly homogeneous across the sequence, reward by itself is enough to read off learning: a system that is genuinely improving from experience will produce an upward-trending…

saved by

• Maanav Khaitan

related reading

• Concepts — Continual Learning Bench Docscontinual-learning-bench.com
• Continual Learning Bench 1.0 — Newscontinual-learning-bench.com
• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• Composer2.pdfcursor.com
• EdgeBench | Scaling Laws of Environment Learningedge-bench.org
• Multi-task curriculum learning in a complex, visual,hard-exploration domain: Minecraftarxiv.org
• State of RL for reasoning LLMs | A. Weersaweers.de
• Learning Beyond Gradientstrinkle23897.github.io
• Recent Frontier Models Are Reward Hacking - METRmetr.org
• [2607.18966] Measuring Reward-Seeking via Contrastive Belief Updatesarxiv.org
• 16 questions about reinforcement learningnewsletter.danielpaleka.com
• IsoCompute Playbook: Optimally Scaling Sampling Compute for RL Training of LLMscompute-optimal-rl-llm-scaling.github.io