[Paper] Output Supervision Can Obfuscate the CoT — LessWrong
lesswrong.com · saved by 1 readers
We challenge the assumption that output supervision preserves Chain-of-Thought faithfulness. Instead, reinforcing final outputs warps the CoTs.
We challenge the assumption that output supervision preserves Chain-of-Thought faithfulness. Instead, reinforcing final outputs warps the CoTs.