Narrow finetuning is different — LessWrong

It is common to use finetuning on a narrow data distribution, or narrow finetuning (NFT), to study AI safety. In these experiments, a model is traine…

x Narrow finetuning is different — LessWrong AI Frontpage 70 Narrow finetuning is different by cloud , Stewy Slocum 5th Aug 2025 4 min read 3 70 It is common to use finetuning on a narrow data distribution, or narrow finetuning (NFT), to study AI safety. In these experiments, a model is trained on a very specific type of data, then evaluated for broader properties, such as a capability or general disposition. Ways that narrow finetuning is different Narrow finetuning is different than the training procedures that frontier AI companies use, like pretraining on the internet , or posttraining on

saved by

• Julian H
• Arjun Khandelwal
• Will Anderson

related reading

• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
• Conditioning, Prompts, and Fine-Tuning — LessWronglesswrong.com
• Narrow Misalignment is Hard, Emergent Misalignment is Easy — LessWronglesswrong.com
• LoRA Without Regret - Thinking Machines Labthinkingmachines.ai
• Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment — LessWronglesswrong.com
• Narrow Finetuning Leaves Clearly Readable Traces in Activation Differences — LessWronglesswrong.com
• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
• Fine-tuning (deep learning) - Wikipediaen.wikipedia.org
• frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
• Recent Advances in Language Model Fine-tuningruder.io
• Fine-tuning LLMs Guide | Unsloth Documentationdocs.unsloth.ai