Dhruv Gautam
26 followers · 23 following · 1140 views
on the atlas · 38
- Approximating KL Divergence11 savers
- [2602.05970] Inverse Depth Scaling From Most Layers Being Similar2 savers
- ~180 lines of code to win the in silico portion of the Adaptyv Nipah binding competition1 savers
- pubs.acs.org/doi/pdf/10.1021/acs.jmedchem.5c03222?ref=article_openPDF1 savers
- Economics of Orbital vs Terrestrial Data Centers3 savers
- PyTorch Profiling 101 with Modded-NanoGPT2 savers
- The Dilbert Afterlife - by Scott Alexander1 savers
- The Final Offshoring14 savers
- 2025 letter | Dan Wang62 savers
- How We Build Trillion Parameter Reasoning RL with 10% GPUs1 savers
- arxiv.org/pdf/2512.24880#page=3.562 savers
- Computers can be understood14 savers
- Mesa — JURA Bio, Inc.1 savers
- Training Math Reasoning Model with Reinforcement Learning - NVIDIA ADLR1 savers
- Principles of Technology Leadership | Bryan Cantrill | Monktoberfest 2017 - YouTube1 savers
- CS294 - Final Project - Google Slides1 savers
- arxiv.org/pdf/1704.014441 savers
- On-Policy Distillation - Thinking Machines Lab30 savers
- UNDERSTANDING IS A POOR SUBSTITUTE FOR CONVEXITY (ANTIFRAGILITY) | Edge.org2 savers
- LoRA Without Regret - Thinking Machines Lab37 savers
- Thinking about High-Quality Human Data | Lil'Log5 savers
- State of torch.compile for training (August 2025) : ezyang’s blog1 savers
- Extending AFM-4.5B to 64k Context Length1 savers
- Nucleotide context models outperform protein language models for predicting antibody affinity maturation | bioRxiv1 savers
- Dear Graduates.pdf - Google Drive10 savers
- cs.pomona.edu/~jcoa2018/f20/cs190/papers/moral.pdf1 savers
- Treescope — treescope1 savers
- arxiv.org/pdf/2505.248322 savers
- Stream of Search (SoS): Learning to Search in Language4 savers
- [2505.08243] Training Strategies for Efficient Embodied Reasoning1 savers
- IIIa. Racing to the Trillion-Dollar Cluster - SITUATIONAL AWARENESS9 savers
- Curius / Onboarding2704 savers
- I should have loved biology36 savers
- Towards Monosemanticity: Decomposing Language Models With Dictionary Learning23 savers
- Some Math behind Neural Tangent Kernel | Lil'Log6 savers
- [2404.01476] TraveLER: A Multi-LMM Agent Framework for Video Question-Answering5 savers
- [2405.07987] The Platonic Representation Hypothesis3 savers
- Why Success Won't Make You Happy - The Atlantic3 savers
highlights · 30
Obviously, it destroyed him.
The Dilbert Afterlife - by Scott AlexanderOn the other hand, this book
The Dilbert Afterlife - by Scott AlexanderAlthough it is generally difficult to train reinforcement learning models with process supervision, these results indicate that as a broad direction, process supervision and dense rewards have the potential to improve learning efficiency by an order of magnitude. This matches earlier results in RL research from Lightman et al.
On-Policy Distillation - Thinking Machines LabAs another point of comparison, DeepSeek-R1-Zero was trained on 5.3M episodesTraining took place for 10,400 steps, each step consisting of 32 unique questions, each question sampled 16 times., corresponding to 5.3M bits of information.
LoRA Without Regret - Thinking Machines LabFortunately there's a handy heuristic we can use to pick one: which does DeepSeek use?
Extending AFM-4.5B to 64k Context Length“Very nice, Michelle, very nice,” he said under his breath at the end of a piece by former student Michelle Pan, who came from Cupertino to play. “A little sassy at the end.
Retirement has a nice ring to UC Berkeley’s Campanile carillonist14 percent of elite performers would accept a fatal cardiovascular condition in exchange for an Olympic gold medal
Why Success Won't Make You Happy - The AtlanticBut success also resembles addiction in its effect on human relationships.
Why Success Won't Make You Happy - The AtlanticThe shadow workspace was a 1-week, 1-person project to create an implementation to solve the immediate need that we had of showing lints to the AI.
Shadow Workspace: Iterating on Code in the BackgroundA key milestone for AI revenue that I like to think about is: when will a big tech company (Google, Microsoft, Meta, etc.) hit a $100B revenue run rate from AI
IIIa. Racing to the Trillion-Dollar Cluster - SITUATIONAL AWARENESSWe hypothesize that this convergence is driving toward a shared statistical model of reality, akin to Plato's concept of an ideal reality.
[2405.07987] The Platonic Representation HypothesisWe tentatively find that the frequency at which these quanta are used in the training distribution roughly follows a power law corresponding with the empirical scaling exponent for language models, a prediction of our theory.
[2303.13506] The Quantization Model of Neural ScalingIt was hard to get through a sentence without having to consult Wikipedia.
I should have loved biologyDespite the simplicity of our method, complex tasks with large action spaces require more demonstrations to learn well, which unfortunately can easily go beyond the input length limit of in-context learning.
2210.03629.pdfQualitatively, we observed that ReAct-IM often made mistakes in identifying when subgoals were finished, or what the next subgoal should be, due to a lack of high- level goal decomposition
2210.03629.pdfI got ~75% on a subset of MATH so it's basically as good as me at math.
Dan Hendrycks on X: "I got ~75% on a subset of MATH so it's basically as good as me at math." / XWhile it is unlikely that certain underlying philosophical tensions will ever be resolved, especially when inextricably intertwined with the incentives of different actors in the AI space, we encourage future work to address today’s deficits in empirical evidence.
On the Societal Impact of Open Foundation ModelsWe are unaware of existing evidence that malicious users have suc- cessfully used open foundation models to automate vulnerability detection.
On the Societal Impact of Open Foundation ModelsHowever, model weights alone are insufficient for several forms of scientific research. Other assets, especially the data used to build the model, are necessary.
On the Societal Impact of Open Foundation Modelsn all cases, foundation model develop- ers do not observe inference by default, making monitoring or moderation challenging, especially for local inference
On the Societal Impact of Open Foundation ModelsThe Supreme Court held that Ivey had cheated, and was thus not entitled to the payment sought from Genting Casinos.
Ivey v Genting Casinos - Wikipediait increasingly seems like a large chunk of the mechanistic interpretability agenda will now turn on succeeding at a difficult engineering and scaling problem, which frontier AI labs have significant expertise in.
Towards Monosemanticity: Decomposing Language Models With Dictionary Learningit suggests that our basic theory of superposition in toy models is missing an important dimension of the problem by not adequately studying highly correlated and "action sharing" features.
Towards Monosemanticity: Decomposing Language Models With Dictionary LearningOn its surface, this problem may seem impossible: we're asking to determine a high-dimensional vector from a low-dimensional projection. Put another way, we're trying to invert a very rectangular matrix. The only thing which makes it possible is that we are looking for a high-dimensional vector that is sparse! This is the famous and well-studied problem of compressed sensing, which is NP-hard in its exact form. It is possible to store high-dimensional sparse structure in lower-dimensional spaces, but recovering it is hard.
Towards Monosemanticity: Decomposing Language Models With Dictionary LearningThis is mathematically identical to the classic problem of dictionary learning.
Towards Monosemanticity: Decomposing Language Models With Dictionary LearningConjecture: An improved NLP optimization attack may be able to induce harmful output in an otherwise aligned language model.
2306.15447.pdfOur results indicate that LMs encode conceptual information structurally similarly to vision-based models, even those that are solely trained on images
[2209.15162] Linearly Mapping from Image to Text Spacewhere we find that training on adversarial examples teaches our models to improve the accuracy of their backdoored policies rather than removing the backdoor
2401.05566.pdfThe Cayley graph of a finite Coxeter group is Hamiltonian (For more information on Hamiltonian paths in Cayley graphs, see the Lovász conjecture.)
Hamiltonian pathWhat he’s really done is he’s created this immense vertical filing cabinet in his brain of layers and layers and layers of files of information
Curius / Onboarding