[2310.18512] Preventing Language Models From Hiding Their Reasoning

ar5iv.labs.arxiv.org · 10,956 words · saved by 1 readers

Large language models (LLMs) often benefit from intermediate steps of reasoning to generate answers to complex problems. When these intermediate steps of reasoning are used to monitor the activity of the model, it is e…

Preventing Language Models From Hiding Their Reasoning Fabien Roger ∗ Ryan Greenblatt Redwood Research Abstract Large language models (LLMs) often benefit from intermediate steps of reasoning to generate answers to complex problems. When these intermediate steps of reasoning are used to monitor the activity of the model, it is essential that this explicit reasoning is faithful, i.e. that it reflects what the model is actually reasoning about. In this work, we focus on one potential way intermediate steps of reasoning could be unfaithful: encoded reasoning, where an LLM could encode intermediat

[2310.18512] Preventing Language Models From Hiding Their Reasoning

related reading