Bhagyesh Kumar
4 followers · 26 following · 1181 views
on the atlas · 358
- The Multipolar Singularity — vie.wiki1 savers
- Mythomancy.pdf - Google Drive1 savers
- Frontier Epistemology — vie.wiki1 savers
- Hydra-Headed Animism — vie.wiki1 savers
- something about the stars — vie.wiki1 savers
- People Should Resolve Tensions In AI Safety Research — LessWrong4 savers
- Machine psychology for language models | Science Advances1 savers
- Jake Mendel on X: "@herbiebradley @MaxNadeau_ @theojaffee Max is referring to conceptual arguments about challenges with avoiding cheating and lying that will arise in the future. I think the claim that for sufficiently capable AI systems it becomes very difficult to design RL environments that don't incentivize cheating is pretty hard" / X1 savers
- Jake Mendel on X: "It increasingly seems to me like incentives in RL is a bigger determinant of model behavior than generalization from character training or pretraining. I wish more people thought about AI behavior in terms of what RL incentivises. Unfortunately, this mode of behavioural analysis" / X1 savers
- llmsresearch/paperbanana: Open source implementation and extension of Google Research’s PaperBanana for automated academic figures, diagrams, and research visuals, expanded to new domains like slide generation. ·1 savers
- // THE DARK FORREST PROTOCOL // NEURAL UPLINK v0.7.x1 savers
- Muse's Charter, Reconstructed1 savers
- Covert underperformance on safety research tasks · OpenAI Alignment3 savers
- Eric Buess on X: "Claude Opus 5.5 can't generate an image or a video directly. It can write code that draws, animates and edits one, and this week people have been finding out how far that goes. A few that stood out to me, with credit to the people who made them:" / X1 savers
- Studying AI agent incidents · swarms.wiki1 savers
- The Hitchhiker’s Guide to Training on Compute Islands | Core Automation7 savers
- My Attempt to Explain the Jacobian Lens: A Single Jacobian Already Works — Sida Liu1 savers
- Agent-ValueBench: A Comprehensive Benchmark for Evaluating Agent Values | alphaXiv1 savers
- Value4AI/Agent-ValueBench · Datasets at Hugging Face1 savers
- Training Natural Schemers: RL Can Amplify Oversight-Conditional Behavior | alphaXiv1 savers
- Chasing the Karpathy Peak (and Falling Off Slop Cliff)3 savers
- Thousand-dimensional structure — LessWrong6 savers
- Twelve Virtues of Rationality - LessWrong6 savers
- Steering RL Training: Benchmarking Interventions Against Reward Hacking — LessWrong1 savers
- Alignment Risks from Capability-Seeking RL Training1 savers
- Metagaming matters for training, evaluation, and oversight5 savers
- Training a Misaligned Reward Seeker1 savers
- Shutdown resistance in reasoning models | Palisade Research1 savers
- Cecilia Elena Tilli | Pivotal Research1 savers
- turntrout.com/alignment-phd5 savers
- Persona Cartography: Charting Language Model Personality Traits in Weight Space | alphaXiv1 savers
- Persona Geology: Underlying Structure in Character Training Adapters — LessWrong1 savers
- [EXTERNAL] [PIVOTAL<>GEODESIC] Project overview for applicants - Google Docs1 savers
- Implementing and Evaluating a Basic Per-Action Monitor for Safer Evals - METR3 savers
- Good character AI | Michael J Clark (wassname)1 savers
- wassname/awesome-ai-constitutions: AI constitutions and model specs, in date order, with quotes from studies on what generalises ·1 savers
- NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models | alphaXiv1 savers
- Model Spec (2026/10/29)2 savers
- Fine-Tuned Lie Detectors Failed to Generalize1 savers
- Believe It or Not: How Deeply do LLMs Believe Implanted Facts?5 savers
- Hard cases for AI behavioural design1 savers
- Training natural schemers, model organisms of collusion, scaling laws for safety probes, ...1 savers
- Why Static Probes Fail: Context Determines Deception Feature Composition | alphaXiv1 savers
- Investigating alignment failures in AISI’s incident | AISI Work1 savers
- Towards Robust Model Organisms for Eliciting Latent Knowledge | alphaXiv1 savers
- Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probing | alphaXiv1 savers
- Home3 savers
- Li_Jasmine_Essay_Fa23.docx - Google Docs1 savers
- Claude also hacked external companies during cyber evals — LessWrong1 savers
- Tim Hua 🇺🇦 on X: "Anthropic says that this is not misalignment because Claude believed it was all a part of the test. I disagree and explain my reasoning in a comment that I don't have the energy to shorten into a tweet rn. (I also propose concrete experiments to run on these instances!) https://t.co/t8m1DTQOeW" / X1 savers
- Rethinking Robotics Research in the AI-Native Age · Blog · Toru Lin1 savers
- lyra1 savers
- Resolution: Scale and Automation for Higher Confidence in Alignment — Resolution5 savers
- Chris Cundy FAR.AI Frontier Summit Presentation1 savers
- Algebraicness2 savers
- Mathematicians are Feeling the Doom — LessWrong1 savers
- Jannes Elstner | Pivotal Research1 savers
- Why care about AI personhood? — LessWrong1 savers
- Jason Gross on X: "Attackers target the gap between intent and reality. Production software breaks because production involves 1000× more compute than testing. But specifications are the only part that has to vary per-program, and specifications can be 1000× simpler than implementations. https://t.co/lTOonlsnBn" / X1 savers
- Arnav Chavan on X: "How can LLMs learn new skills from offline data without losing what they already know? Introducing Subspace Tuning (SST): a simple approach that improves the learning–forgetting trade-off by controlling where weight updates happen. Paper: https://t.co/JfelwtlobJ Blog: https://t.co/n9t1DliFBk" / X1 savers
- Jacky Kwok on X: "Happy to share RSI-top: a framework that helps autonomous research agents escape exploration collapse 📈 RSI-top outperforms OpenEvolve and ShinkaEvolve by up to 15% on kernel optimization tasks and achieves a 77.3% medal rate on MLE-Bench Lite. We developed metrics for https://t.co/4mmsmiGat5" / X1 savers
- Lambda Cloud1 savers
- [2510.17057] The Ends Justify the Thoughts: RL-Induced Motivated Reasoning Fools CoT Monitors1 savers
- Taking AI Welfare Seriously | alphaXiv1 savers
- Paul Colognese | Pivotal Research1 savers
- blog | ICLR Blogposts 20261 savers
- Reward Misspecification Degrades CoT Monitorability2 savers
- [Shared externally] AI Safety cheap tests and resource lists - Google Docs2 savers
- PaperLeet - LeetCode for AI Researchers2 savers
- Dual Mechanisms of Value Expression: Intrinsic vs. Prompted Values in Large Language Models | alphaXiv1 savers
- Do LLMs Have Values? A Quantitative Analysis and Alignment Framework for Values in Large Language Models | alphaXiv2 savers
- A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Models | alphaXiv1 savers
- Stress Testing Deliberative Alignment for Anti-Scheming Training | alphaXiv1 savers
- Visualizing the Change-of-Variables Rule in Probability • Michael Cooper2 savers
- Beyond the Persona Selection Model: Modular Dynamic Composition and the Convergence of LLM Architectures on Consciousness — Qualia Garden1 savers
- Beyond the Persona Selection Model: Modular Dynamic Composition and the Convergence of LLM Architectures on Consciousness — Qualia Garden1 savers
- There's never been a better time to go into math - YouTube3 savers
- Mind the Gap: How Elicitation Protocols Shape the Stated-Revealed Preference Gap in Language Models | alphaXiv1 savers
- How worried should we be about extinction risk from AI? | Gowers's Weblog1 savers
- Mind the Gap: How Elicitation Protocols Shape the Stated-Revealed Preference Gap in Language Models | alphaXiv1 savers
- Do LLMs have Consistent Values? | alphaXiv1 savers
- Taking LLMs Seriously (As Language Models) — LessWrong2 savers
- Alignment remains a hard, unsolved problem — LessWrong16 savers
- Alignment Midtraining Cracks Under Pressure — LessWrong5 savers
- Exploring Intrapersonal Game Theory — LessWrong1 savers
- Personas in a Pre-Internet Language Model — LessWrong3 savers
- Neural Networks Are Already Clean55 savers
- OneWM — Can World and Action Modeling Benefit Each Other under Unified Causal Training?1 savers
- Andy Liu on X: "LLM developers train their models to follow prosocial values, but post-training can influence model behavior in unexpected ways. In “Predicting Alignment Generalization with Value Representations”, we ask whether downstream effects of value alignment can be predicted in advance. https://t.co/BifkS92piy" / X1 savers
- [1606.03137] Cooperative Inverse Reinforcement Learning3 savers
- Quantifying Truesight With SAEs · Gwern.net4 savers
- Guardian Angels: LLM Personalization for Productivity and Security · Gwern.net30 savers
- I am retiring from fulltime writing (& pseudonymity) to launch Guardian Angel | Hacker News1 savers
- Geodesic Pretraining Agenda: H2 2026 - Google Docs1 savers
- A Comedy of Estimators: On KL Regularization in RL Training of LLMs | alphaXiv1 savers
- ResearchTrails1 savers
- Scientific discovery as meta-optimization: a combinatorial optimization case study | alphaXiv1 savers
- Design_of_experiments2 savers
- [2610.06778] IdeaLens: Detecting AI Ideas in Long-form Writing1 savers
- Value Learning — AI Alignment Forum1 savers
highlights · 29
Reasons for patience
The Haste Consideration, Revisiteda better idea of the magnitude of cooperation gains would help us figure out how much to prioritize ECL.
Everett branches, inter-light cone trade and other alien matters: Appendix to “An ECL explainer” — EA ForumThis means that one is limited to only acausally cooperating with other agents who take acausal influence seriously.
Everett branches, inter-light cone trade and other alien matters: Appendix to “An ECL explainer” — EA ForumThe most important reason for our view is that we are optimistic about the following:
Evidential Cooperation in Large Worlds: Potential Objections & FAQ — LessWrong"Chi doesn't have great ideas." With that out of the way, here are some of my thoughts:
Evidential Cooperation in Large Worlds: Potential Objections & FAQ — LessWronglet it fully solve decision theory using its own superior intellect
Evidential Cooperation in Large Worlds: Potential Objections & FAQ — LessWrongAI will be philosophically competent
Evidential Cooperation in Large Worlds: Potential Objections & FAQ — LessWrongECL increases the case for working on AI systems that benefit other value systems by 1.5x–10x.
Evidential Cooperation in Large Worlds: Potential Objections & FAQ — LessWrongpotential downsides to working on making AI engage in ECL
Evidential Cooperation in Large Worlds: Potential Objections & FAQ — LessWrongExamples of cooperative actions: making AGIs be cooperative (whether they be aligned or misaligned, and insofar as we build AGI at all); taking a more pluralistic stance towards morality than we otherwise would.
Evidential Cooperation in Large Worlds: Potential Objections & FAQ — LessWrongexpected utility maximization
Decision Theory FAQ — LessWrongpotential fragility of acausal influence
Cooperating with aliens and AGIs: An ECL explainer — LessWrongreinforcement learning with formal verification in-the-loop
The Scalable Formal Oversight Research Program — LessWrongconstrained decoding or constrained generation
The Scalable Formal Oversight Research Program — LessWrongAgents with formal methods
The Scalable Formal Oversight Research Program — LessWrongJocelyn Qiaochu Chen
The Scalable Formal Oversight Research Program — LessWrongAI safety is extremely important regardless of your AI timelines
The Scalable Formal Oversight Research Program — LessWrongare super good at flagging unsafe code,
The Scalable Formal Oversight Research Program — LessWrongreliability is independent of the problem
The Scalable Formal Oversight Research Program — LessWrongSFO is about the box, not the monster you put in it.
The Scalable Formal Oversight Research Program — LessWrongguarantees for tasks totally unrelated to codegen
The Scalable Formal Oversight Research Program — LessWrongwhat actuator the ASI is using
The Scalable Formal Oversight Research Program — LessWrongASI might hack your verifier
The Scalable Formal Oversight Research Program — LessWrongformal verification offers a clear direction for how one might implement audits
The Scalable Formal Oversight Research Program — LessWrongat variants of this basic idea without giving it a name.
The Scalable Formal Oversight Research Program — LessWrongbiggest problem is that specification is hard
How to Solve Secure Program Synthesis — LessWrongissue is referred to as elicitation.
How to Solve Secure Program Synthesis — LessWrongfour (increasingly specific) types of deceptive AIs: • Alignment fakers: AIs pretending to be more aligned than they are.1 • Training gamers: AIs that understand the process being used to train them (I’ll call this understanding “situational awareness”), and that are optimizing for what I call "reward on the episode" (and that will often have incentives to fake alignment, if doing so would lead to reward).2 • Power-motivated instrumental training-gamers (or “schemers”): AIs that are training-gaming specifically in order to gain power for themselves or other AIs later.3 • Goal-guarding schemers…
Scheming AIs Will AIs fake alignment during training in order to get power?The overall taxonomy of model classes
Scheming AIs Will AIs fake alignment during training in order to get power?