Brady Gho

11 followers · 42 following · 355 views

on the atlas · 78

highlights · 45

Bro thought bro was cooking
Everyone Loves Hitler - by Damage Magazine and Sam Kriss
These algorithms then work well at the small scale, where bias is not bad, and can perhaps be beneficial, but all end up failing miserably at the larger scale where the bias becomes harmful. It would seem, then, that the fundamental trick is to find an unbiased algorithm which fulfills the biological constraints, even if it has vastly greater variance than SGD, and to hope that the scale of the brain is sufficient to make this workable even if it fails miserably at small task
How can LLM RL Work Despite Information-Theoretic Inefficiency
The bitter lesson essentially says that variance can be handled with more compute or data, while bias cannot. Thus, methods that scale are those which are as unbiased as possible even if they have high variance.
How can LLM RL Work Despite Information-Theoretic Inefficiency
he NTP loss provides an absolute firehose of bits from every sentence, and thus requires vastly larger models to properly saturate that flow.
How can LLM RL Work Despite Information-Theoretic Inefficiency
If we pretrain for a long time at some learning rate, and then suddenly drop the learning rate, we see a sudden and dramatic decrease in loss and improvement in capabilities5 as the model suddenly gains the ‘resolution’ to be able to descend into a deeper valley which previously it was skating around the top of
How can LLM RL Work Despite Information-Theoretic Inefficiency
model descends into a valley of some height determined by its SNR but cannot descend further into the sub-valleys within the valley due to the noise so it just effectively bounces around skittering over the entrances to the deeper valleys without ever entering
How can LLM RL Work Despite Information-Theoretic Inefficiency
An RL rollout only gives a single bit of information, but that bit is all signal
How can LLM RL Work Despite Information-Theoretic Inefficiency
It can successfully solve tasks where the required change is relatively few bits away from the base. It cannot solve tasks, such as learning language from scratch, which require many many many bits
How can LLM RL Work Despite Information-Theoretic Inefficiency
since RL only has the capacity to move the parameters a short distance6, but in a very, very precise direction.
How can LLM RL Work Despite Information-Theoretic Inefficiency
It is perhaps unsurprising that an interp researcher has reached the conclusion that only interp can save the day.
The fragile foundations of CoT monitoring | Christopher Potts
even though all exponentials eventually become sigmoids, this doesn’t necessarily happen at the exact moment you’re doing your analysis
The Sigmoids Won't Save You - by Scott Alexander
two tanks with one shell
what makes for a good bit? - by Kenneth Sun
“i go to the meetings”: when someone questions how you know something, the more obscure the better
what makes for a good bit? - by Kenneth Sun
“when i was your age…”: and proceed to tell someone slightly younger than you exactly what you were doing, like, yesterday
what makes for a good bit? - by Kenneth Sun
“that depends on your karma”
what makes for a good bit? - by Kenneth Sun
alternatively, for something harder, *parallel parks*, “learned that in the army”
what makes for a good bit? - by Kenneth Sun
(Incidentally, the treatment did work, although its fatality rate was high. The extreme fever killed or damaged the bacteria responsible for neurosyphilis. The malaria could then be treated with quinine.)
Your Book Review: Great And Desperate Cures
Lobotomies stopped not because there was some huge reckoning within the medical community, but because they were effectively outcompeted by a simpler, cheaper, less permanent treatment option.
Your Book Review: Great And Desperate Cures
you can only be alive if you do at least one thing seriously (competitive sport/starting a company/collecting stamps), you can do more, but definetly not less
Hi, I'm Yoni :)
consciousness is definitly spread over a non zero amount of time, leakage of info from what we call the future is plausible
Hi, I'm Yoni :)
the quick dopamine of writing software has obstructed many potentially great people from doing things that matter
Hi, I'm Yoni :)
almost no one is as impressive from up close as they are from afar
Hi, I'm Yoni :)
if you want to catch a wave you need to get in the water
Hi, I'm Yoni :)
The longer a historical perspective you take, the more obvious this point will be. We should take the plunge. We already have taken the plunge. We designed/tolerated our decentralized society so we could take the plunge.
Existential risk, AI, and the inevitable turn in human history - Marginal REVOLUTION
The case for destruction is so much more readily articulable — “boom!”
Existential risk, AI, and the inevitable turn in human history - Marginal REVOLUTION
Radical agnosticism is the correct response, where all specific scenarios are pretty unlikely. Nonetheless I am still for people doing constructive work on the problem of alignment, just as we do with all other technologies, to improve them.
Existential risk, AI, and the inevitable turn in human history - Marginal REVOLUTION
just about all of the responses I am seeing I interpret as “copes,” whether from the optimists, the pessimists, or the extreme pessimists (e.g., Eliezer)
Existential risk, AI, and the inevitable turn in human history - Marginal REVOLUTION
In other words, virtually all of us have been living in a bubble “outside of history.”
Existential risk, AI, and the inevitable turn in human history - Marginal REVOLUTION
Being a successful FDE required an unusual sensitivity to social context – what you really had to do was partner with your corporate (or government) counterparts at the highest level and gain their trust, which often required playing political games.
Reflections on Palantir - Nabeel S. Qureshi
But often what really gets in the way is organizational politics: a team, or group, controls a key data source, the reason for their existence is that they are the gatekeepers to that data source, and they typically justify their existence in a corporation by being the gatekeepers of that data source (and, often, providing analyses of that data)
Reflections on Palantir - Nabeel S. Qureshi
As with Palantir, working on AI probably isn’t 100% morally good, nor is it 100% evil. Not engaging with it – or calling for a pause/stop, which is a fantasy – is unlikely to be the best stance.
Reflections on Palantir - Nabeel S. Qureshi
The danger of this stance, of course, is that it becomes a fully general argument for doing whatever the power structure wants.
Reflections on Palantir - Nabeel S. Qureshi
I am inclined to think that the answer is yes – Scenario 1 and Scenario 2 are different in a way that is more than just a shift in perspective. The reason for this is that I don’t think two conscious beings can be treated as the same being, even if their thoughts and experiences are identical. If two identical people are suffering, that’s two people suffering.
The Periodic Box Problem | The Excluded Riddle
You wake up one day and find yourself in what appears to be an infinite metal hallway with an infinite row of your clones. Which of these two scenarios are you in? Is the hallway actually infinite, or are you in a box with a portal? Are those people actually clones, or are they just images of yourself? The big claim here is that it is impossible for you to distinguish between the two scenarios.
The Periodic Box Problem | The Excluded Riddle
And so it happened. By exatick , the Simulation reached complete periodicity. A single machine, feeling the same sensation over and over. Slowly accumulating moral value, according to the utilitarian philosophy of Prom.
The Prometheus Simulation | The Excluded Riddle
The lesson is that society cannot rely on ethics-motivated people standing firm
Why I Left Google DeepMind
Someone will say “Democrats want open borders.” Perhaps you ask “Oh, what fraction?” They say “I haven’t seen a survey, but look at this crazy Democrat who wants open borders. What a wackadoodle!”
The biggest bug in the English language | Ted Sanders
The next time someone says “Group X thinks Y,” I encourage you to wonder “Hmm, but what fraction of Group X thinks Y?” Often this technique helps me realize that nothing is being said at all.
The biggest bug in the English language | Ted Sanders
The obvious benefit to working quickly is that you'll finish more stuff per unit time. But there's more to it than that. If you work quickly, the cost of doing something new will seem lower in your mind. So you'll be inclined to do more.
Speed matters: Why working quickly is more important than it seems « the jsomers.net blog
You're welcome to say "I'm sorry, I made a bet and I lost. I'd make the bet again, though, knowing what I did then." Then you're still owning the choice. You're owning the failure, which is the important part.
Have no excuses
Try to quickly map out the spheres of influence within the org because they're frequently not related to the title of a person.
If you’re starting at a new job at a healthtech company | Out-Of-Pocket
What is every single step of the current process your customers do currently? What is every step of your company's process? How does your company get paid and where is most of the revenue coming from? Who are your biggest customers and exactly what are they buying? Why do customers NOT buy from your company?
If you’re starting at a new job at a healthtech company | Out-Of-Pocket
It is even better to ask: what experience must not happen to you? Do you believe that Élan vital explains the mysterious aliveness of living beings? Then what does this belief not allow to happen—what would definitely falsify this belief? A null answer means that your belief does not constrain experience; it permits anything to happen to you. It floats.
Making Beliefs Pay Rent (in Anticipated Experiences) — LessWrong
So for instance he'd have an idea of something he wanted to say: I grew up in canoes on northern lakes. Thirty years later, I was trying to choose a word or words that would explain why anyone in a modern nation would choose to go a long distance by canoe. I was damned if I was going to call it a sport, but nothing else occurred. And he'd go, Well, "sport" is kind of clunky, it's kind of humdrum. Maybe I can do better. And he'd look up "sport," and instead of the even more hopelessly banal "an activity involving physical exertion and skill" that I'd get out of my dictionary, he'd discover this…
You’re probably using the wrong dictionary « the jsomers.net blog
I should remind you of the rule that anyone writing a blog post advising against X is himself the worst Xer there is
Speed matters: Why working quickly is more important than it seems « the jsomers.net blog