✳flâneur — a map of the web's best reading

Introspection Adapters: Training LLMs to Report Their Learned Behaviors

arxiv.org · 12,076 words · saved by 2 readers

When model developers or users fine-tune an LLM, this can induce behaviors that are unexpected, deliberately harmful, or hard to detect. It would be far easier to audit LLMs if they could simply describe their behaviors in natural language. Here, we study a scalable approach to rapidly identify learned behaviors of many LLMs derived from a shared base LLM. Given a model 𝑀 , our method works by finetuning models 𝑀 𝑖 from 𝑀 with implanted behaviors 𝑏 𝑖 ; the ( 𝑀 𝑖 , 𝑏 𝑖 ) pairs serve as labeled training data. We then train an introspection adapter (IA): a single LoRA adapter jointly trained across the finetunes 𝑀 𝑖 to cause them to verbalize their implanted behaviors. We find that this IA induces self-description of learned behaviors even in finetunes of 𝑀 that were trained in very different ways from the 𝑀 𝑖 . For example, IAs generalize to AuditBench, achieving state-of-the-art at identifying explicitly hidden concerning behaviors. IAs can also be used to de

Introspection Adapters: Training LLMs to Report Their Learned Behaviors Keshav Shenoy Li Yang Abhay Sheshadri Sören Mindermann Jack Lindsey Sam Marks Rowan Wang Abstract When model developers or users fine-tune an LLM, this can induce behaviors that are unexpected, deliberately harmful, or hard to detect. It would be far easier to audit LLMs if they could simply describe their behaviors in natural language. Here, we study a scalable approach to rapidly identify learned behaviors of many LLMs derived from a shared base LLM. Given a model M M , our method works by finetuning models M i M_{i} fro

Explore this link on the map →

Introspection Adapters: Training LLMs to Report Their Learned Behaviors

saved by

related reading