16 questions about reinforcement learning

AlphaProof constructed a large curriculum of tasks by randomly misformalizing math olympiad problems; creating many problems of varying difficulty, from trivial to full math olympiad difficulty. This is a beautiful idea for building a curriculum; I dub it “task mutations”. Do task mutations work on other domains? Getting RL to scale from 0% to 1% is a big deal. But what about getting from 99% to 100%? These problems seem symmetrical to me; whether you are almost always wrong or almost always correct, you’re basically not getting any reward either way. Are the RL updates on correct samples reinforcing correct reasoning in a way that helps even when the model is getting everything correct, or is naive RL just limited as a method for getting to 100% reliability, in the same way it doesn’t work to get from 0% to 1%? Is there a canonical best method for getting out of the zero-reward regime, or is it always ad-hoc? The amount of information in a typical RLVR episode is at most 1 bit (we lea

AlphaProof constructed a large curriculum of tasks by randomly misformalizing math olympiad problems; creating many problems of varying difficulty, from trivial to full math olympiad difficulty. This is a beautiful idea for building a curriculum; I dub it “task mutations”. Do task mutations work on other domains? Getting RL to scale from 0% to 1% is a big deal. But what about getting from 99% to 100%? These problems seem symmetrical to me; whether you are almost always wrong or almost always correct, you’re basically not getting any reward either way. Are the RL updates on correct samples rein