[2602.04735] From Data to Behavior: Predicting Unintended Model Behaviors Before Training

Abstract:Large Language Models (LLMs) can acquire unintended biases from seemingly benign training data even without explicit cues or malicious content. Existing methods struggle to detect such risks before fine-tuning, making post hoc evaluation costly and inefficient. To address this challenge, we introduce Data2Behavior, a new task for predicting unintended model behaviors prior to training. We also propose Manipulating Data Features (MDF), a lightweight approach that summarizes candidate data through their mean representations and injects them into the forward pass of a base model, allowing latent statistical signals in the data to shape model activations and reveal potential biases and safety risks without updating any parameters. MDF achieves reliable prediction while consuming only about 20% of the GPU resources required for fine-tuning. Experiments on Qwen3-14B, Qwen2.5-32B-Instruct, and Gemma-3-12b-it confirm that MDF can anticipate unintended behaviors and provide insight into pre-training vulnerabilities.

From Data to Behavior: Predicting Unintended Model Behaviors Before Training Mengru Wang1,2 , Zhenqian Xu1 , Junfeng Fang2 , Yunzhi Yao1 , Shumin Deng2 , Huajun Chen1 , Ningyu Zhang1 * 1 Zhejiang University, 2 National University of Singapore…

related reading

• Subliminal Learning: Language Models Transmit Behavioral Traits via Hidden Signals in Dataalignment.anthropic.com
• Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• Probe-Based Data Attribution: Surfacing and Mitigating Undesirable Behaviors in LLM Post-Traininggoodfire.ai
• Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
• [2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• [2602.05910] Chunky Post-Training: Data Driven Failures of Generalizationarxiv.org
• Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
• Mechanistically Eliciting Latent Behaviors in Language Models — AI Alignment Forumalignmentforum.org
• Emergent Misalignment is Easy, Narrow Misalignment is Hardarxiv.org
• [2501.11120] Tell me about yourself: LLMs are aware of their learned behaviorsarxiv.org
• Discovering Undesired Rare Behaviors via Model Diff Amplification - Goodfiregoodfire.com