flâneur

Carolanne Jiang

10 followers · 5 following · 428 views

on the atlas — 101

highlights — 39

  • This suggests that a critical part of being effective at staring into the abyss is timing. If you do it too little, you’ll end up taking too long to make important life improvements; but if you do it too often, you might end up not investing enough in being great at your current job or relationship because you’re too focused on the prospect of next one.
    Staring into the abyss as a core life skill
  • When I first graduated, I was afraid that if I worked with boring technology, I’d get bored. Instead, I learned it was possible—and fun—to optimize on other dimensions, or play a different game. Rather than competing for an A+ on a hard problem, I could try to solve an easy problem as quickly as possible (like Wave’s accounting), or find the easiest problem whose solution would be useful (like identifying Kenyan names), or hire a team to solve easy problems faster than I ever could myself.
    You don’t need to work on hard problems
  • Q: but I want to major in math/philosophy— Don’t. Math and philosophy are too fun. Take math up to real analysis and algebra. Be careful around superstimuli that remove you from the real world.
    College Q&A - Alexey Guzey
  • If I loved music for its own sake and wanted to be a talented musician so I could express the melodies dancing within my heart, then none of this matters. But insofar as I want to be good at music because I feel bad that other people are better than me at music, that’s a road without an end.
    The Parable Of The Talents — LessWrong
  • Specifically, it’s whether I can say “No, I’m really not cut out to be Elon Musk” and go do something else I’m better at without worrying that I’m killing everyone in Canada.
    The Parable Of The Talents — LessWrong
  • Insofar as there’s no such thing as innate aptitude, I have no excuse for not being Aubrey de Grey. Or if Aubrey de Grey doesn’t impress you much, Norman Borlaug. Or if you don’t know who either of those two people are, Elon Musk.
    The Parable Of The Talents — LessWrong
  • But insofar as there’s no such thing as innate aptitude, just hard work and grit – then by not being gritty enough, I’m a monster who’s complicit in the death of a population greater than that of Canada.
    The Parable Of The Talents — LessWrong
  • It’s why there’s so much emphasis on “heroic responsibility” and how you, yes you, have to solve all the world’s problems personally.
    The Parable Of The Talents — LessWrong
  • Just transcribe your thoughts onto paper exactly like they sound in your head
    The Parable Of The Talents — LessWrong
  • if we do well about nipping small failures in the bud, we may not get any medium-sized warning shots at all.
    What failure looks like - LessWrong
  • Attempts to suppress influence-seeking behavior (call them “immune systems”) rest on the suppressor having some kind of epistemic advantage over the influence-seeker. Once the influence-seekers can outthink an immune system, they can avoid detection and potentially even compromise the immune system to further expand their influence.
    What failure looks like - LessWrong
  • Once we start searching over policies that understand the world well enough, we run into a problem: any influence-seeking policies we stumble across would also score well according to our training objective, because performing well on the training objective is a good strategy for obtaining influence.
    What failure looks like - LessWrong
  • We might describe the result as “going out with a whimper.” Human reasoning gradually stops being able to compete with sophisticated, systematized manipulation and deception which is continuously improving by trial and error; human control over levers of power gradually becomes less and less effective; we ultimately lose any real ability to influence our society’s trajectory. By the time we spread through the stars our current values are just one of many forces in the world, not even a particularly strong one.
    What failure looks like - LessWrong
  • Right now humans thinking and talking about the future they want to create are a powerful force that is able to steer our trajectory. But over time human reasoning will become weaker and weaker compared to new forms of reasoning honed by trial-and-error. Eventually our society’s trajectory will be determined by powerful optimization with easily-measurable goals rather than by human intentions about the future.
    What failure looks like - LessWrong
  • There’s an awful lot of persuasive but low-quality AI content around, some of it generated with malicious intent. In response to this, people withdraw into their own AI-mediated epistemic bubbles — and unlike today’s filter bubbles, these can be comprehensive enough that people rarely encounter friction with outside perspectives at all.
    AI and Epistemics: The Good, Bad and Ugly
  • Given the real chance that we end up stuck in an extremely positive or negative epistemic equilibrium, our initial trajectory seems very important.
    AI and Epistemics: The Good, Bad and Ugly
  • Distorted epistemic environments often also have self-perpetuating properties. Cults often require members to move into communal housing and cut contact with family and friends who question the group. Scientology frames psychiatry’s rejection of its claims as evidence of a conspiracy against it.
    AI and Epistemics: The Good, Bad and Ugly
  • AI slop. In hard-to-verify domains, AI could massively increase the quantity of plausible-looking but wrong information, without also being able to help us to verify which bits are right.
    AI and Epistemics: The Good, Bad and Ugly
  • One way to measure efficiency improvements is to look at the amount of computing power needed for an AI system to exhibit a particular level of performance, and consider how much more computing power was previously needed for AI systems to reach the same level of performance.
    Will AI R&D Automation Cause a Software Intelligence Explosion?
  • AI software is improving at a rate that likely outpaces the growth rate of research effort needed to achieve these software improvements. In our model, this finding implies that the positive feedback loop of AI improving AI software is powerful enough to overcome diminishing returns to research effort, causing AI progress to accelerate further and resulting in an SIE.
    Will AI R&D Automation Cause a Software Intelligence Explosion?
  • However, there’s another possibility: AI systems could become dramatically more capable just by finding software improvements that significantly boost performance on existing hardware.
    Will AI R&D Automation Cause a Software Intelligence Explosion?
  • Skeptics of an intelligence explosion often focus on hardware limitations –
    Will AI R&D Automation Cause a Software Intelligence Explosion?
  • (1) the fixed amount of computing power limits how many AI experiments can be run in parallel, and (2) training each new generation of AI system could take months.
    Will AI R&D Automation Cause a Software Intelligence Explosion?
  • Pushing towards an SIE is the positive feedback loop from increasingly powerful AI systems performing AI R&D. On the other hand, improvements to AI software face diminishing returns from lower hanging fruit being picked first – a force that pushes against an SIE.
    Will AI R&D Automation Cause a Software Intelligence Explosion?
  • AI Systems for AI R&D Automation (ASARA)
    Will AI R&D Automation Cause a Software Intelligence Explosion?
  • We call this scenario a software intelligence explosion (SIE)
    Will AI R&D Automation Cause a Software Intelligence Explosion?
  • neuron j ​ =relu(w j ​ ⋅x+b j ​ )
    AI Safety Crash Course
  • Machine learning refers to automatically searching across all the functions and finding ones that seem to do what we want.
    AI Safety Crash Course
  • ("forward pass" is the fancy machine learning name for running our function) t
    AI Safety Crash Course
  • Squaring punishes big misses quadratically: a miss twice as large hurts four times as much, so the search will happily accept lots of small errors to avoid one huge one. (Both losses are zero for perfect predictions, and neither can go negative.)
    AI Safety Crash Course
  • For each datapoint you simply take the difference between the real answer and your function's predicted answer, and square it, then average this result across all the datapoints.
    AI Safety Crash Course
  • That's the L1 loss (the average size of the miss); you've reinvented a classic. Here's its personality: a model forced to quote one price for every house would minimize your loss by quoting $159,500. That's the median sale price ($159,500); L1 always elects the median.
    AI Safety Crash Course
  • In a certain mood, one can feel like: “No! There are real stakes here!”, especially when the relevant form of relaxation/acceptance/peace is presented as a generalized ideal, and the stakes are moral. Here I think of someone I knew in undergrad, who told a group, proudly, “I am not chill.” That is, it can feel as though one is being encouraged to care less about things that matter, and it can seem like: screw that.
    On clinging - Joe Carlsmith
  • Instead of “don’t micromanage,” the advice I wish I’d gotten is: Manage projects according to the owner’s level of task-relevant maturity.✻✻ i.e. how experienced and autonomous they are at doing that particular task. Even people at a similar level of experience can have different task-relevant maturities for different skills: one senior engineer might be able to take a new system from design to production on their own but struggle to write understandable documentation, while another might flail around if given a project with ambiguous scope, but be unstoppable at chasing down tricky bugs. Peop…
    Some mistakes I made as a new manager | benkuhn.net
  • Bertrand Russell wrote: “Before the end of the present century, unless something quite unforeseeable occurs, one of three possibilities will have been realized. These three are: — 1. The end of human life, perhaps of all life on our planet. 2. A reversion to barbarism after a catastrophic diminution of the population of the globe. 3. A unification of the world under a single government, possessing a monopoly of all the major weapons of war.”
    Do short AI timelines make other cause areas useless? — EA Forum Bots
  • is a common psychological phenomenon whereby individuals give high accuracy ratings to descriptions of their personality that supposedly are tailored specifically to them, yet which are in fact vague and general enough to apply to a broad range of people
    Barnum effect
  • I got so good at coming up with plausible arguments for my beliefs that I often didn’t even realize I was full of shit.
    Rightness is a prison - by Cate Hall - Useful Fictions
  • Girard claimed that human desire functions imitatively, or mimetically, rather than arising as the spontaneous byproduct of human individuality, as much of theoretical psychology had assumed.
    René Girard - Wikipedia
  • Contextualization Don’t force every speaker to issue caveats. That’s bureaucratic honesty theater. The responsibility lies with the listener. Before letting a single-factor claim rewrite your worldview, ask: What else co-varies with this? Where does this explanation stop working? What happens when two factors change at once? Zealotry spreads through listeners who mistake partial truth for total truth. Contextualization is inoculation.
    Don't Get One-Shotted — LessWrong