Stress-testing Alignment Midtraining
Sid Baines*1, Jonathan Bostock*1, Maria Angelica Martinez*1, Andrew Draganov1, David Africa2 & Daniel Tan1 1Arcadia Impact · 2Resolution · *Equal contribution Existing work [1, 2, 3] on alignment midtraining (AMT) suggests that it can instill general, aligned motivations into AI models. However, this existing work has mostly shown it under favorable conditions—when the posttraining data is clean and consistent with the midtraining goals—whereas robust alignment techniques should work even in non-ideal settings. We stress-test AMT across model and token budget scales. We confirm that AMT is effective in instilling motivations in these idealized post-training conditions. However, we also find simple, realistic conditions under which it loses effectiveness: i.) if the post-training data is incomplete, and ii) if the post-training data conflicts with the target motivation. Given our analysis, we are unable to find evidence that midtraining can tackle the hard problems of AI alignment—namel