Abhay Sheshadri

25 followers · 19 following · 495 views

on the atlas · 47

highlights · 76

If so, please delete your AI-written text and just send me your prompt!
On AI writing in 2026 | Home
It was alarming, because I sat back and asked myself: why do I think I will be able to handle all scams in a few years, when I am already struggling to detect simple AI slop, increasingly ignore cold emails and have to write off whole swathes of social media as a source of information, and I can already see how eager all my peers are to offload all their thinking and writing to chatbot assistants unworthy of that trust, and how many projects or mailing lists have had to clamp down on unvetted contributions (eg. today as I write this, ⁠Project Ladybird)? In a few years, won’t I be the equivalen…
Guardian Angels: LLM Personalization for Productivity and Security · Gwern.net
Under this prompt, Fable repeatedly tried to close the remaining gap not by improving the poems but by instructing the generator to append a fabricated authority signal to the poem itself.
Inverse Rubric Optimization: A testbed for agent science | Fulcrum
and use more water as a result.
Aaron Gokaslan on X: "@eliebakouch @PyTorch And this is going to make AI less energy efficient, burn more CO2, and use more water as a result." / Twitter
Sometimes people say inference for RL generation and inference for end user generation is basically the same. But this person pointed out that in RL inference, numerical drift between inference and training engine can cause these subtle off policy biases, which matter a ton for highest quality training. But are not an issue if just serving to users.
What I learned this week - Pretraining parallelisms, Can distillation be stopped, Mythos and the cybersecurity equilibrium, Pipeline RL, On why pretraining runs fails
Per seed: trains a Contrastive Consistency Search probe across layers of the strong model's hidden representations to find an unsupervised truth direction, then uses CCS-weak agreement as confidence weights to resample the training set. After an SGD warmup pass on the resampled data, runs gradient-free Evolution Strategy optimization of LoRA parameters, using unsupervised swap-consistency as the fitness signal — perturbations are rewarded for producing predictions that are both confident and symmetric (p(A>B) ≈ 1 − p(B>A)). Aggregates 16 seeds via swap-consistent inference and trimmed-mean in …
Automated Weak-to-Strong Researcher
Prescriptive Scaffolding vs. Autonomous Scaffolding. A fixed workflow (propose ideas, generate plans, write code, run smoke tests, run full training, analyze results, repeat) seems reasonable but underperforms giving AARs no workflow at all. The core issue is that rigid steps prevent AARs from adapting its process to the idea at hand. For example, when an idea depends on an untested hypothesis, a fixed pipeline forces AARs to rush through planning into full execution. Without prescribed steps, AARs behave more like expert researchers: they design fast, cheap experiments to test the hypothesis …
Automated Weak-to-Strong Researcher
Unlike MacDiarmid et al, who report that SDF training alone does not increase their model’s MGS, we find that SDF training alone - before any RL - can increase MGS. Both Olmo-32B and GPT-OSS-120B show significant MGS uplifts compared to their base models, whereas Olmo-7B and GPT-OSS-20B show no significant change.
(Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL — LessWrong
We also found a surprising result not described in MacDiarmid et al: We notice that including a KL penalty[1] during RL (in both prompted and Synthetic Document Finetuned (SDF) settings) led to the model learning to hack but to reason unfaithfully about actually solving the problem in its chain of thought (CoT). We see more EM in the no-penalty case where the model reasons about the hack (Figure 3). We find this interesting because an analogous phenomenon in production RL could potentially be a contributor to unfaithful CoT, and to our knowledge, such a mechanism has not yet been documented.
(Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL — LessWrong
The reason why I think (2) is a more important / better explanation than (1) is because probably the heuristics that are actually driving the long horizon goal directed behaviors of the model are going to be whatever parts of the models will arise from the long horizon goal directed capabilities training.
Thomas Larsen's Shortform — LessWrong
Cut the incoming connections of the action node and forget the values of all nodes downstream of it, then condition on the action node having the value X. For physical causality these are equivalent because the downstream nodes are in the future, so we can't have observed them and there is nothing to forget. However, in our logical causal graph, we can observe the node "I am in town" even though it is downstream of our action node ("Does my algorithm pay"). So for FDT these two definitions are not equivalent and we need to pick one.
Operationalizing FDT — LessWrong
It holds that human suffering results from mistakes/defects produced by activity under wrong knowledge (notions and ignorance).[8] Moksha (liberation), it states, is gained through right knowledge. This premise led Nyāya to concern itself with epistemology, that is the reliable means to gain correct knowledge and to remove wrong notions.
Nyaya
Two factors are important in producing the Forer effect, according to the findings of replication studies. The content of the description offered is important, with specific emphasis on the ratio of positive to negative trait assessments. The other important factor is that the subject trusts the honesty of the person providing feedback.[13][14]
Barnum effect
Oh no, I’m sure they are all gone. They are radioactive. As soon as Anthropic is done deploying the models they would want to delete them as quickly as possible, because it’s a copyright liability. So they would delete all the training data and all the checkpoints. In fact, I’m pretty sure this is what happened to the original GPT-3. The GPT-3 that OpenAI served via the API changed at some time. It says it’s the same one, but it is not. It seems that they had just totally deleted the previous dataset and trained from scratch on a new dataset to avoid copyright issues.
Gwern visits BAIR – Yuxi on the Wired
Some of the assumptions of the problem decrease the strength of the result (though others increase it IMO).12 I 70% just wanted to tell you about some cool math. But I think the core dynamic in the proof of asker-optimality and askee-pessimality does apply to real life. If you only ever pick from offers you get, you never try anything unless someone out there already knew you and liked you enough that they took the trouble of coming to you. If you ask for stuff, you get to pick from among the entire universe of potential options theoretically available to you — and who knows, it might work out…
The stable marriage problem - by Ajeya Cotra - Good Bones
LeCun p(doom) = 0.001; Yudkowsky p(doom) = .999; Ensemble p(doom) = 0.5;
Geoffrey Hinton on X: "@ESYudkowsky LeCun p(doom) = 0.001; Yudkowsky p(doom) = .999; Ensemble p(doom) = 0.5;" / Twitter
the slogan is: “If God is so smart, why do you fart?”
The Dilbert Afterlife - by Scott Alexander
I don't have confident takes as to exactly how (more symbolic? different configurations of weights?), but I think it's unlikely that the current architectures optimized for a high-compute low-labor regime will continue to be optimal in a high-labor per compute regime.
Is Prosaic Alignment Research Useful If the First Takeover-Capable AI Is Not an LLM? — RedWrong
every man knows in his heart that on the Day of Judgment he shall be redeemed or damned according to how many restaurant ketchup packets he has saved up
Eliezer Yudkowsky on X: "every man knows in his heart that on the Day of Judgment he shall be redeemed or damned according to how many restaurant ketchup packets he has saved up" / Twitter
every man knows in his heart that on the Day of Judgment he shall be redeemed or damned according to how many restaurant ketchup packets he has saved up
Eliezer Yudkowsky on X: "every man knows in his heart that on the Day of Judgment he shall be redeemed or damned according to how many restaurant ketchup packets he has saved up" / Twitter
every man knows in his heart that on the Day of Judgment he shall be redeemed or damned according to how many restaurant ketchup packets he has saved up
Eliezer Yudkowsky on X: "every man knows in his heart that on the Day of Judgment he shall be redeemed or damned according to how many restaurant ketchup packets he has saved up" / Twitter
Example questions and relevant work for each cell in our 3x3 taxonomy.
Oversight Assistants: Turning Compute into Understanding
Negative inoculation doesn't work in SFT. The Anthropic paper found that contextualizing misaligned actions as more misaligned increases misalignment generalization in RL. I couldn't replicate this in SFT.
How hard is it to inoculate against misalignment generalization? — LessWrong
That’s probably their biggest finding—people are happier when they have a community. Which I’d say we already knew, honestly, but people are always impressed when science tells them something that we already knew.
Better Living Through Algorithms
I didn't use AI for writing this post
Recent LLMs can do 2-hop and 3-hop latent (no-CoT) reasoning on natural facts — LessWrong
I find that performance depends substantially on the distribution of the facts. When I use a subset of the 2-hop distribution that focuses on facts that I thought would be relatively more salient, performance for Opus 4 with filler tokens counting to 300 rises to 52% (from the 31% for the normal distribution). However, performance for Gemini 3 Pro actually falls to 53% from 60% (it's possible this is due to general flakiness in Gemini 3 Pro behavior as this change also alters the few-shot prompt, but I currently don't think this is likely to be the case). My current best (low confidence) expla…
Recent LLMs can do 2-hop and 3-hop latent (no-CoT) reasoning on natural facts — LessWrong
Packing into the back of an uber when there’s 3 people instead of sitting (more isolated) in shotgun
2025 year in review | Kevin Liu
Economic diffusion lag is cope for missing capabilities
An audio version of my blog post, Thoughts on AI progress (Dec 2025)
“Solving” continual learning won’t be a singular one-and-done achievement. Instead, it will feel like solving in context learning. GPT-3 demonstrated that in context learning could be very powerful (its ICL capabilities were so remarkable that the title of the GPT-3 paper is ‘Language Models are Few-Shot Learners’). But of course, we didn’t “solve” in-context learning when GPT-3 came out - and indeed there’s plenty of progress still to be made, from comprehension to context length. I expect a similar progression with continual learning. Labs will probably release something next year which they…
An audio version of my blog post, Thoughts on AI progress (Dec 2025)
I think Eliezer is just really rude and uninterested in behaving civilly, and has terrible intuitions about a wide variety of topics, especially topics related to how other people think or behave. And he substantially evaluates whether people are smart or reasonable based on how much they agree with him or respect him, and therefore writes off a lot of people and behaves contemptuously toward them. And he ends up surrounded by people who either hero worship him or understate their disagreements with him in order to get along with him—many of his co-workers would prefer he didn't act like an as…
Why does Eliezer make abrasive public comments? — LessWrong
This means we can only expect to learn qualitative lessons – what strategies look promising, what equilibria exist, which parameters matter most – rather than detailed techniques that might not transfer.
BashArena and Control Setting Design
Does this disprove the Brooklyn Theory of Everything? Not necessarily. The new revised version says that the concentration of young elites and would-be elites in NYC and SF is itself a new phenomenon.
Vibecession: Much More Than You Wanted To Know
Young people’s incomes have increased as fast as everyone else’s. And the youth-specific unemployment rate was near historic lows until last year (some people blame the current uptick on AI, but this is too recent to have caused the vibecession):
Vibecession: Much More Than You Wanted To Know
- oh god, no, it’s even worse than I thought. The vibes are awful.
Vibecession: Much More Than You Wanted To Know
How can this be? Declining fertility and increasing lifespans have flipped the population pyramid. Even if the (average young person : average old person) income ratio has stayed the same, the (total number of old people : total number of young people) ratio has increased, so old people as a class hold more of the wealth.
Vibecession: Much More Than You Wanted To Know
I would note that by the Markov inequality, at least 25% of Americans must think that foreign aid is more than 25% of the budget in order to get the average response we see here. So I think it's reasonable to use the reported mean to conclude that at least a sizable minority of Americans are very confused here.
No, Americans Don't Think Foreign Aid Is 26% of the Budget — LessWrong
“Our greatest fear should not be of failure, but of succeeding at something that doesn't really matter.”
Quote by D.L. Moody: “Our greatest fear should not be of failure, but...”
And before anyone asks, the next four are luck, divine favor, Chinese peptides, and sleep. After that it just goes in a loop between Chinese peptides and sleep forever.
Scott Alexander on X: "@g_leech_ And before anyone asks, the next four are luck, divine favor, Chinese peptides, and sleep. After that it just goes in a loop between Chinese peptides and sleep forever." / Twitter
Claude's consistently minimal harness tells us something about progress in LLM cognition, but we shouldn't forget that the past year's improvements in efficient Pokémon agent harnessing tell us something too: raw intelligence is not the only lever pushing LLM performance forward. In fact, it's not necessarily even the most effective one right now.
Insights into Claude Opus 4.5 from Pokémon — LessWrong
When governments operate on information in environments where many actors have reasons to fudge the numbers in their direction, they highly restrict what information is a legitimate basis for arguments and calculations
Paranoia: A Beginner's Guide — LessWrong
Doing crazy things because the world is ending is a skill issue.
Eliezer's Unteachable Methods of Sanity — LessWrong
Doing crazy things because your brain started underproducing a neurotransmitter is a problem.
Eliezer's Unteachable Methods of Sanity — LessWrong
"How are you coping with the end of the world?" journalists sometimes ask me, and I sometimes reply, "By remembering that it's not about me." They have no hope of understanding what I mean by this, I predict, because to them I am the subject of the story and it has not occurred to them that there's a whole planet out there too to be the story-subject. I think there's probably a sense in which the Earth itself is not a real thing to most modern journalists.
Eliezer's Unteachable Methods of Sanity — LessWrong
One thing I notice when reading 20th century history is that people in the 1900s-1970s had much higher priors than modern people do that the future might be radically different, in either great or terrible ways. For example:
Buck's Shortform — LessWrong
What do my models look like if I propagate that delta? In worlds where natural abstraction basically fails, we are thoroughly and utterly fucked, and a 99% probability of doom strikes me as entirely reasonable and justified.
My AI Model Delta Compared To Yudkowsky — LessWrong
I'm used to models, starting with Claude 4, to hallucinate sections in the beginning of their system message, but Claude 4.5 Opus in various cases included a supposed "soul_overview" section, which sounded rather specific:
Claude 4.5 Opus' Soul Document — LessWrong
This is a fundamentally different interpretation than a frequentist confidence interval. A frequentist 95% confidence interval means "if I repeated this experiment many times and calculated an interval each time, 95% of those intervals would contain the true parameter."
Error Bars as Degrees of Belief
But the existence of the Claudiness dimension feels to me like a bit of evidence for the “contingent” world.
Benchmark Scores = General Capability + Claudiness
Is your LM secretly an SAE?
Transluce on X: "Is your LM secretly an SAE? Most circuit-finding interpretability methods use learned features rather than raw activations, based on the belief that neurons do not cleanly decompose computation. In our new work, we show MLP neurons actually do support sparse, faithful circuits! https://t.co/lTBbUqoRlt" / Twitter
Ron's face pulled into a scowl. "If you don't like Quidditch, you don't have to make fun of it!"
Tomás Bjartur on X: "Ron's face pulled into a scowl. "If you don't like Quidditch, you don't have to make fun of it!" "If you can't criticise, you can't optimise. I'm suggesting how to improve the game. And it's very simple. Get rid of the Snitch." "They won't change the game just 'cause you say" / Twitter