✳flâneur — a map of the web's best reading

Announcing the Inverse Scaling Prize ($250k Prize Pool) — AI Alignment Forum

alignmentforum.org · 2,188 words · saved by 1 readers

TL;DR: We’re launching the Inverse Scaling Prize: a contest with $250k in prizes for finding zero/few-shot text tasks where larger language models show increasingly undesirable behavior (“inverse scaling”). We hypothesize that inverse scaling is often a sign of an alignment failure and that more examples of alignment failures would benefit empirical alignment research. We believe that this contest is an unusually concrete, tractable, and safety-relevant problem for engaging alignment newcomers and the broader ML community. This post will focus on the relevance of the contest and the inverse scaling framework to longer-term AGI alignment concerns. See our GitHub repo for contest details, prizes we’ll award, and task evaluation criteria. Recent work has found that Language Models (LMs) predictably improve as we scale LMs in various ways (“scaling laws”). For example, the test loss on the LM objective (next word prediction) decreases as a power law with compute, dataset size, and model si

x Announcing the Inverse Scaling Prize ($250k Prize Pool) — AI Alignment Forum Bounties (closed) Inner Alignment Language Models (LLMs) Outer Alignment Scaling Laws AI Community Frontpage 59 Announcing the Inverse Scaling Prize ($250k Prize Pool) by Ethan Perez , Ian McKenzie , Sam Bowman 27th Jun 2022 8 min read 14 59 TL;DR : We’re launching the Inverse Scaling Prize : a contest with $250k in prizes for finding zero/few-shot text tasks where larger language models show increasingly undesirable behavior (“inverse scaling”). We hypothesize that inverse scaling is often a sign of an alignment fa

Explore this link on the map →

related reading