Measuring LLM Sycophancy under Pressure

The paper introduces SPINE to evaluate language model correctness under sustained multi-turn disagreement, revealing issues with current short interactions

The paper introduces SPINE, a method to measure language model(LLM) compliance under prolonged, adaptive disagreement, using up to 25 turns and a per-tune strength scoring system With SPINE, research reveals that evaluated production systems ( ones widely available to perform various task ) like AI Strongly suiuiiushudry continue increases in collapse rates up to turn 25 despite their ability to contact same statements they refused to initially The study documents that emotional tactics are most effective in reducing position strength, with abilities to pyramid producing the highest drop…

saved by

• Yoyo Yuan

related reading

• [2505.13995] ELEPHANT: Measuring and understanding social sycophancy in LLMsarxiv.org
• Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
• [2605.27288] It's Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertaintyarxiv.org
• [2510.27062] Consistency Training Helps Stop Sycophancy and Jailbreaksarxiv.org
• 2308.03958arxiv.org
• [2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• [2605.21739] AttuneBench: A Conversation-Based Benchmark for LLM Emotional Intelligencearxiv.org
• [2303.17548] Whose Opinions Do Language Models Reflect?arxiv.org
• [2605.07912] Sycophantic AI makes human interaction feel more effortful and less satisfying over timearxiv.org
• [2412.00543] Evaluating the Consistency of LLM Evaluatorsarxiv.org
• LLMs Learn to Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactionsarxiv.org
• [2503.03750] The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systemsarxiv.org