Clementine Yang
0 followers · 291 views
on the atlas — 41
- Torchvista: Building an Interactive Pytorch Visualization Package for Notebooks | Towards Data Science1 savers
- I failed my Anthropic interview and came to tell you all about it so you don't have to | Fail. Learn. Repeat.2 savers
- Alignment Research Center1 savers
- A Pragmatic Vision for Interpretability — LessWrong3 savers
- On the Biology of a Large Language Model32 savers
- Mechanistic Interpretability: Circuits, Induction Heads - Interactive | Michael Brenndoerfer1 savers
- A Comprehensive Mechanistic Interpretability Explainer & Glossary — Neel Nanda2 savers
- Language models can explain neurons in language models | OpenAI2 savers
- Hilbert’s 23 Problems at ICM 2026: Where Are We Now?1 savers
- Do Machine Learning Models Memorize or Generalize?3 savers
- Compression is Intelligence: Two Counter-Intuitive Hypotheses | by Zicheng Xu | Medium1 savers
- Embedding Layer | TensorTonic1 savers
- All About Rooflines | How To Scale Your Model10 savers
- Toy Models of Superposition30 savers
- Modern Pretraining Strategies: A Hands-On Guide1 savers
- AI for Science: Scaling in AI for Scientific Discovery | AI for Science1 savers
- AI for Science: Scaling in AI for Scientific Discovery | AI for Science1 savers
- 9. Classes — Python 3.11.3 documentation4 savers
- RL Explainer - Interactive Visualization of RL Algorithms for LLM Training1 savers
- CS231n Convolutional Neural Networks for Visual Recognition2 savers
- Observations Upon the Prophecies of Daniel, and the Apocalypse of St. John - Sir Isaac Newton Online1 savers
- Dreams, the Unconscious, and Spiritual Maturity1 savers
- The History of Spiritual Direction1 savers
- How AI-generated prose diverges from human writing and why it matters | Reuters Institute for the Study of Journalism1 savers
- Large-Scale Online Deanonymization with LLMs1 savers
- VRPRM: Process Reward Modeling via Visual Reasoning1 savers
- What are skills? - Agent Skills1 savers
- A Death in South Sudan – The Pennsylvania Gazette1 savers
- o1 and Reasoning | AndoLogs1 savers
- The Era of Experience Paper.pdf15 savers
- Gradient Ascent. You find yourself lost in the woods in… | by Brendan Coady | Common Notes | Medium1 savers
- Lightweight Guide to understanding GRPO and RL principles - Musings of Murali1 savers
- MolmoAct Action Reasoning Models that can Reason in Space1 savers
- 50+ graph interview questions and cheat sheet - IGotAnOffer1 savers
- Kiteworks1 savers
- Efficient World Models with Context-Aware Tokenization1 savers
- Parallelizing Linear Transformers with the Delta Rule over Sequence Length1 savers
- Key-value memory in the brain1 savers
- Descartes' Error: Emotion, Reason, and the Human Brain2 savers
- MambaByte: Token-free Selective State Space Model1 savers
- Byte Latent Transformer: Patches Scale Better Than Tokens1 savers
highlights — 167
He was certain that with the right axioms, all of mathematics would be knowable.
Hilbert’s 23 Problems at ICM 2026: Where Are We Now?Hilbert’s list could almost seem amateurish in comparison. That word, amateur, comes from the Latin for “love.” An amateur pursues a hobby or activity for enjoyment, not for financial gain, and perhaps Hilbert’s problems are amateurish in that sense as well. His love for mathematics radiates from the page.
Hilbert’s 23 Problems at ICM 2026: Where Are We Now?ost of them smoothly interpolate between the two solutions. The rapid generalization occurs when the last weights connected to the distracting digits are pruned by weight decay.
Do Machine Learning Models Memorize or Generalize?in loss actually slightly increases before the model generalizes as it exchanges loss related to outputting the correct label for having lower w
Do Machine Learning Models Memorize or Generalize?e’re pushing our model to do two things during training — output a high probability for the correct label (called minimizing loss 6) and have weights with low magnitudes (known as weight decay 7
Do Machine Learning Models Memorize or Generalize?To "embed" a token with ID tt, you simply retrieve row tt from the table:
Embedding Layer | TensorTonicTo "embed" a token with ID tt, you simply retrieve row tt from the table:
Embedding Layer | TensorTonico "embed" a token with ID tt, you simply retrieve row tt from the table:
Embedding Layer | TensorTonicow fast our computer can do math (OPs/second), the bandwidth available for moving data around (bytes/second), and the total memory available to store data (bytes)
All About Rooflines | How To Scale Your ModelBut everything on this plot between 0 (not learning a feature) and 1 (dedicating a dimension to a feature) is superposition. Superposition is what happens when features have fractional dimensionality.
Toy Models of SuperpositionDuring training, you don't just store the weights; you store the gradients, the optimizer states (which can be 3-4x the size of the weights!), and the activations for every layer.
Modern Pretraining Strategies: A Hands-On Guideou don't want to be the person who hits "Run" only to see an Out of Memory (OOM) error five seconds later because you forgot to account for the optimizer states.
Modern Pretraining Strategies: A Hands-On Guideuse the ICML LaTeX template and change the title to under review at ICML 2024 AI for Science workshop.
AI for Science: Scaling in AI for Scientific Discovery | AI for Science4-8 page paper (with unlimited references and appendices) of
AI for Science: Scaling in AI for Scientific Discovery | AI for ScienceMethods may call other methods by using method attributes of the self argument:
9. Classes — Python 3.11.3 documentationNow f, g and h are all attributes of class C
9. Classes — Python 3.11.3 documentationassigning a function object to a local variable in the class is also ok
9. Classes — Python 3.11.3 documentationcritic-free, sequence-level optimization is winnin
RL Explainer - Interactive Visualization of RL Algorithms for LLM TrainingBut how do we efficiently determine the parameters that give the best (lowest) loss? This process is optimization,
CS231n Convolutional Neural Networks for Visual Recognitionhe believed the world was to be replaced with a new one based upon a transition to an era of divinely inspired peace
Observations Upon the Prophecies of Daniel, and the Apocalypse of St. John - Sir Isaac Newton OnlineLargely unknown and unpublished documents, evidently written by Isaac Newton, indicate that he believed the world could end in 2060 AD.
Observations Upon the Prophecies of Daniel, and the Apocalypse of St. John - Sir Isaac Newton OnlineThey ask us to grow beyond a faith that needs constant reassurance, toward one that can trust the slow work of truth unfolding within us. They teach us to listen — not only to God as we imagine God should speak, but to the deeper movements of soul through which grace so often works.
Dreams, the Unconscious, and Spiritual MaturityDreams invite us into a spirituality that is less defended, less idealised, and more whole.
Dreams, the Unconscious, and Spiritual MaturityIn this sense, some dreams are not asking to be understood so much as companioned.
Dreams, the Unconscious, and Spiritual MaturityWhen dreams are explored alongside another, what felt overwhelming begins to find language. What felt isolating becomes shared. What felt frightening may reveal itself as a threshold rather than a threat.
Dreams, the Unconscious, and Spiritual MaturityGood accompaniment does not rush to interpret, explain away, spiritualise, or diagnose. Instead, it provides a steady presence capable of holding psychological depth and spiritual sensitivity together.
Dreams, the Unconscious, and Spiritual MaturityIt cultivates humility, reverence, and patience — qualities essential to a mature spiritual life.
Dreams, the Unconscious, and Spiritual MaturitySymbolic listening involves staying close to the image itself: noticing how it feels, what it evokes, what memories or emotions it stirs, and how it resonates with our waking life — without deciding too soon what it “means.”
Dreams, the Unconscious, and Spiritual MaturityIn this sense, dreams are not obstacles to spiritual maturity; they are often agents of it. They are less about instruction and more about integration — inviting a movement toward wholeness that can only unfold over time.
Dreams, the Unconscious, and Spiritual MaturityOften what appears in dreams belongs to what has not yet been welcomed into consciousness — aspects of the self that carry fear, longing, anger, or unclaimed vitality — what depth psychology has long named the shadow.
Dreams, the Unconscious, and Spiritual MaturityThey tend to surface not when life feels settled and clear, but when we are unsettled
Dreams, the Unconscious, and Spiritual MaturityDreams often arrive at moments when something within us is shifting. We wake with an image that lingers, a feeling that does not easily fade, or a story that seems to hover at the edge of understanding, asking not to be solved but to be noticed.
Dreams, the Unconscious, and Spiritual Maturityanam cara, the Buddhist 'Kalyana-mitra' or 'noble friend', is one who gently and firmly confronts you with your own blindness, to see what you cannot yet see, about yourself, your longings and the world in which you create around you in your mind, to help you grow in true clarity.
The History of Spiritual Direction“If you put AI-generated text in front of most editors, they will shake their heads, because it is empty. The dead giveaway for me is just that it's a lot of words that don't say anything. In journalism, we always say, ‘Show, don't tell
How AI-generated prose diverges from human writing and why it matters | Reuters Institute for the Study of JournalismAs candidate pools grow to tens of thousands, LLM-based attacks degrade gracefully at high precision.
Large-Scale Online Deanonymization with LLMsAt the same time, if the final judgment is wrong, the process evaluation is considered to have failed and the process reward is set to zero
VRPRM: Process Reward Modeling via Visual Reasoningtwo reward rules, namely format compliance check and process accuracy check
VRPRM: Process Reward Modeling via Visual ReasoningDiscovery: At startup, agents load only the name and description of each available skill, just enough to know when it might be relevant. Activation: When a task matches a skill’s description, the agent reads the full SKILL.md instructions into context. Execution: The agent follows the instructions, optionally loading referenced files or executing bundled code as needed.
What are skills? - Agent Skillsn education would not serve our kids unless they could emerge as independent thinkers and quality writers
A Death in South Sudan – The Pennsylvania GazetteThe generators are trained on their own solutions that have been verified as correct by the verifier
o1 and Reasoning | AndoLogshey propose utilizing both the correct and incorrect solutions generated during the self-improvement process to train a learned verifier using DPO that judges correctness of model-generated solutions. This verifier is used at inference time to select one solution among many candidate solutions.
o1 and Reasoning | AndoLogsectly ground thinking in the external world is to build a world model [37] that predicts the consequences of the agent’s actions upon the world, including predicting rewa
The Era of Experience Paper.pdfrove the way in which it selects or combines signa
The Era of Experience Paper.pdfth the user and the environment as inp
The Era of Experience Paper.pdfautonomous optimisation of grounded, non-human reward signals
The Era of Experience Paper.pdfis is because the achievement of a simple goal in a complex environment may often require a wide variety of skills to be master
The Era of Experience Paper.pdfne could in principle create a variety of distinct agents, each optimising for one grounded signal as its rewar
The Era of Experience Paper.pdfrounded rewards: signals that arise from the environment itsel
The Era of Experience Paper.pdfe agent cannot discover better strate- gies that are underappreciated by the human rate
The Era of Experience Paper.pdffrom exclusively human-privileged communication, to much more autonomous interactions where the agent is able to act independently in the world. Such
The Era of Experience Paper.pdf