flâneur

Sophia Zhang

4 followers · 12 following · 229 views

on the atlas — 13

highlights — 23

  • We also do not think the system achieved third-order generalization, what we call the ignition condition, where the improved inner-loop agent becomes a better outer-loop agent. Ignition is a necessary condition for an intelligence explosion, not a sufficient one. So we believe we are not near an intelligence explosion with the current system.
    AIDE²: First Evidence of Recursive Self-Improvement | Weco AI
  • We built a recursive self-improvement (RSI) system by running autoresearch on autoresearch. The system, AIDE2, took eight days to discover a better autoresearch harness than the one we built over the last two years. Fully autonomously, AIDE2 designed a novel search algorithm, reduced the prompt size by 16×, and built a layered system against reward hacking.
    AIDE²: First Evidence of Recursive Self-Improvement | Weco AI
  • But achieving recursive improvement alone does not suggest an immediate change in how industrial production occurs, societies organize, or markets function. More intelligence can’t learn what a drug does over decades of use, can’t hold elections sooner than a constitution dictates, and can’t turn a stranger into an old friend in a weekend. For most people, the felt pace of this future will still be set by the bottlenecks, even if the laboratory upstream runs at the speed of compute. That collision, where recursive intelligence building itself ever faster meets the world of humans, relationship…
    When AI builds itself \ Anthropic
  • Anthropic employees working with highly capable models—far more than we have the capacity to pursue. The rate at which organizations can spot and fix these bottlenecks may be a skill that improves over time, and it may become the most important skill for any organization.
    When AI builds itself \ Anthropic
  • Alternately, the binding constraint to AI progress could be in the supply chain, not the model: advancing and diffusing the frontier may require more energy and compute than presently exists.
    When AI builds itself \ Anthropic
  • But these trajectories may actually turn out to be S-curves. We may be approaching the bend in the curve, where returns to scale diminish and the line straightens, then flattens.
    When AI builds itself \ Anthropic
  • Edison said that genius is 1% inspiration and 99% perspiration. But we see perspiration becoming increasingly automated.
    When AI builds itself \ Anthropic
  • We view this result as an early signal that AI systems are getting better at making the kinds of judgement calls that AI research depends on.
    When AI builds itself \ Anthropic
  • But within those bounds, the agents designed every experiment themselves. Direction-setting was the only meaningful role a human played.
    When AI builds itself \ Anthropic
  • We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for.
    When AI builds itself \ Anthropic
  • Invention, not automation, will be the greatest contribution of superintelligence.
    The AI Future Is for Everyone - WSJ
  • But I don’t agree with the letter’s assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers. It seems at least as likely to me that the opposite will be true.
    Our position on open-weights models \ Anthropic
  • Companies died anyway. “The existential fear that there are people trying to come kill our companies is part of the way we do venture,” she said. “There’s commitment and loyalty to the team. It’s us against the world.”
    Sarah Guo's Wager - Colossus
  • In the eight weeks between my two interviews with Guo, she had her fourth child with Grady, walked the Met Gala carpet in 45 pounds of custom chainmail with her mother, recorded multiple podcasts, and was still answering her founders’ texts within minutes.
    Sarah Guo's Wager - Colossus
  • they outwork everyone. “There’s just always more to be done.”
    Sarah Guo's Wager - Colossus
  • “If I texted her right now and said, ‘Can you talk to a candidate today?’ she’d be like, ‘What’s their number?’ And she’ll do it within an hour.”
    Sarah Guo's Wager - Colossus
  • I felt that they were just fundamentally correct.
    Sarah Guo's Wager - Colossus
  • the work of helping a human get value out of them keeps getting bigger
    Sarah Guo's Wager - Colossus
  • Better to get your dopamine from improving your ideas than from having them validated
    Nat Friedman
  • The cultural prohibition on micromanagement is harmful
    Nat Friedman
  • Most people are other people
    Nat Friedman
  • the best way to achieve a growth mindset might just be not to mention growth mindset at all
    Why You Cannot Believe Your Way Into Learning
  • world’s largest dataset of neurovascular ultrasound
    Ultrasound imaging of the brain — Aleph