Aleksi Maunu

4 followers · 4 following · 861 views

on the atlas — 31

highlights — 98

It puzzles me that Pause folks aren't more eager to engage with informed skeptics like Nora Belrose, Rohin Shah, Alex Turner, Katja Grace, Matthew Barnett, etc.
In DC, a new wave of AI lobbyists gains the upper hand — EA Forum
hypothesize a "midwit curve" for AI risk concern: At a low level of AI knowledge, members of the general public are apt to anthropomorphize AI models and fear them. As a person acquires AI expertise, they anthropomorphize AI models less, and become less afraid. Past that point, some folks become persuaded by specific technical arguments for AI risk.
In DC, a new wave of AI lobbyists gains the upper hand — EA Forum
Bans which are temporary are sometimes enacted to give the relevant community time to figure out what good policies are. The ban ends when these policies are enacted, but the new policies often maintain some restrictions from the previous ban. The Asilomar Conference on Recombinant DNA follows this pattern. In 1974, the National Academy of Sciences (NAS) enacted a temporary ban23 on all experiments involving recombinant DNA, in response to concerns about biohazardous experiments. The Asilomar Conference in 1975 prohibited certain kinds of experiments, and provided recommendations for other exp…
Are There Examples of Overhang for Other Technologies?
We think that, in addition to encoding human values into AI systems, a very complementary way to dramatically reduce AI risk is to create external safeguards that limit AI outputs.
Announcing Atlas Computing — LessWrong
Macross Plus (マクロスプラス)
マクロスプラス - Google Search
We applied standard methods such as activation patching, path patching, and attribution patching to identify model components (e.g. individual neurons or attention heads) that contribute significantly to refusal
Refusal in LLMs is mediated by a single direction — LessWrong
Dairy, for instance, causes over 800 times less suffering than chicken and over 1000 times less than eggs. Drinking a gallon of milk a day for a year is then about as bad as having a chicken sandwich once every four months.
If You're Going To Eat Animals, Eat Beef and Dairy — EA Forum
This idea of mistakes as feedback is a crucial lesson in creating habits. Visualize yourself crossing a shallow river, stepping across a path of large stones that someone has strung across the river. You have to step on one stone after another to get across. Now picture closing your eyes and trying to get across. You take a step into the water by accident. At this point, you could beat yourself up about stepping in the water, and then keep going in the same direction until you’ve fallen into the water completely and are totally off the path. That wouldn’t make any sense.
Don't miss 2 days in a row - Intend
In our Goal-Crafting Intensive workshops, we encourage people, once they've set some new goals, to come up with a thing to do towards each goal immediately (today or tomorrow) that they wouldn't have thought to do at all prior to having the goal. This is another way to point at proactive vs reactive approaches.
Intend philosophy & paradigm
International Day for Failure is held on October 13th. It all began in 2010 when students at Aalto University in Finland figured that Finland needed a rise in small startup businesses. The problem is that the fear of failing discourages many Finnish entrepreneurs from even trying.
International Day for Failure - October 13, 2024 | internationaldays.co
Michael then articulates how formalism allows us to tackle Chalmer’s Hard Problem of Consciousness – which is insoluble as formulated – by taking it apart into conceptually crisp modular subcomponents that are solvable (cf. “breaking down the problem of consciousness”). In particular, he modularizes the problem of consciousness into 8 subproblems: the Reality Mapping, Substrate, Boundary/Binding, Scale, Topology of Information, State Space, Vocabulary, and Translation problems (PQ, pg 21-22),
Research Lineages - Qualia Research Institute
realizes that IIT’s goal of taking a physical system and then generating a mathematical object out of it, such that the mathematical features of that object are isomorphic to the phenomenology of the experience the system produces, is not only a sensible goal, but a key goal any good theory of consciousness must have
Research Lineages - Qualia Research Institute
Likewise, the meta-heuristic of “how would Buddha research consciousness and suffering if he were alive today?" may be highly generative.
Research Lineages - Qualia Research Institute
jhanas as resonant modes of the brain
Research Lineages - Qualia Research Institute
the ‘self' as a leaky reification formed from self-reinforcing algorithmic processes
Research Lineages - Qualia Research Institute
Further, by clarifying our criteria for a “high-quality" phenomenological report, we can gather orders of magnitude more information than from hundreds of mediocre reports. QRI is careful to note the distinction between the “intentional content" (what happened) and the “phenomenal content" (how it felt) of such first-person accounts. For example, noting that one's “tracers followed a control-interrupt frequency of 15Hz" is very different from noting that: "the trees spoke to me". Generally, we value the phenomenal content over the intentional content of such experiences. For specific examples …
Research Lineages - Qualia Research Institute
Psychoactive substances provide an invaluable tool for studying consciousness. Such substances enable researchers to induce altered states with large effect sizes in a reliable manner. These exotic states are crucial for reverse-engineering the underlying formalism for consciousness. Ignoring them, in our view, is analogous to physicists ignoring extreme states such as black holes, plasma, or supercritical fluids when aiming to understand the nature of energy, matter, and the physical world.
Research Lineages - Qualia Research Institute
Most powerful stimuli for biology and central circadian clock: (1) light; (2) exercise; (3) feeding; (4) social cues – interact with people early in the day
Andrew Huberman's Sleep Cocktail / Routine • Podcast Notes
“the most powerful limitations, the most appealing ones, are those that have a conspicuousness and simplicity, that are qualitative and not a matter of degree, that provide recognizable boundaries.” Schelling Observed: “Some gas” raises complicated questions of how much, where, under what circumstances: “no gas” is simple and unambiguous. Gas only on military personnel; gas used only by defending forces; gas only when carried by vehicle or projectile; no gas without warning—a variety of limits is conceivable; some may make sense, and many might have been more impartial to the outcome of the wa…
Historical case studies of technology governance and international agreements – BlueDot Impact
I practiced rehearsing the words “I have never donated to charity, and if I did, I certainly wouldn’t care whether it was effective or not”.
My Left Kidney - by Scott Alexander - Astral Codex Ten
I make fun of Vox journalists a lot, but I want to give them credit where credit is due: they contain valuable organs, which can be harvested and given to others.
My Left Kidney - by Scott Alexander - Astral Codex Ten
Turn the skill on itself. Reinforce cognitive strategies that will help you with reinforcing cognitive strategies, and finding better ways to reinforce cognitive strategies. The skill will then quickly bootstrap itself into your most powerful and general thinking tool.
Tuning your Cognitive Strategies — LessWrong
How well have these particular deltas performed in the past? This amounts to maintaining a rough "track record" for all of them.
Tuning your Cognitive Strategies — LessWrong
For example, you can notice thought chains when you: choose the next task to do, do better or worse than expected, plan your day or week, process emotions, change the topic in conversations, accept or reject offers.
Tuning your Cognitive Strategies — LessWrong
If you think the same thought again, change the topic.
Tuning your Cognitive Strategies — LessWrong
More examples of useful cognitive strategies, and common low hanging fruit: If you hit an impasse (no new useful thoughts), relax and let your mind wander to related but different topics.
Tuning your Cognitive Strategies — LessWrong
perceive your cognitive strategies as they happen
Tuning your Cognitive Strategies — LessWrong
When you don't like whatever has risen up to the top of the cauldron, the last thing you want is to try to "fix it". You only have You only have access to the topmost layer, so it would be hopelessly ineffective anyway. But it's much worse than that - by attempting to "fix" your cognition, you stop being able to see how it works.
Tuning your Cognitive Strategies — LessWrong
I think AI safety research is a bit unusual in this respect: most fields of research aren't explicitly about "solving problems that don't exist yet." (Though a lot of research ends up useful for more important problems than the original ones it's studying.) As a result, doing AI safety research today is a bit like trying to study medicine in humans by experimenting only on lab mice (no human subjects available).
AI Safety Seems Hard to Measure
Finnish Standards Association
Cristina Andersson | LinkedIn
Although none of the examples above show AIs doing novel research, DeepMind’s AlphaTensor model discovered a new algorithm for matrix multiplication which was faster than any designed by humans. While AlphaTensor was specifically developed for this purpose (as opposed to the more general systems discussed above), the result is notable because matrix multiplication is the key step in training neural networks.
Visualizing the deep learning revolution | by Richard Ngo | Jan, 2023 | Medium
Human values won’t be the only thing shaping the future. Today humans trying to influence the future are the only goal-oriented process shaping the trajectory of society. Automating decision-making provides the most serious opportunity yet for that to change. It may be the case that machines make decisions in service of human interests, that machines share human values, or that machines have other worthwhile values. But it may also be that machines use their influence to push society in directions we find uninteresting or less valuable
Three impacts of machine intelligence | Rational Altruist
I was convinced by Gerald Monroe that getting a full moratorium was harder than I have previously argued based on an analogy to nuclear weapons. (I was not convinced that it “isn't going to happen without a series of extremely improbable events happening simultaneously” - largely because I think that countries will be motivated to preserve the status quo.)
How would a moratorium fail? — EA Forum
I don't think the future looks any brighter if the most responsible orgs develop AGI first than if the least responsible ones do.
Comments on Manheim's "What's in a Pause?" — EA Forum
I think we should be placing more focus on the human extinction and disempowerment risks posed by AGI
Comments on Manheim's "What's in a Pause?" — EA Forum
ll leading labs All leading labs coordinate to slow now: bad. This delays dangerous AI. But it burns leading labs' lead time, making them less able to slow progress later (because further slowing would cause them to fall behind, such that other labs would drive AI progress and the slowed labs' safety practices would be irrelevant).
How to think about slowing AI — EA Forum
Variables Many variables affect whether an intervention improves AI safety.[2] Here are four crucial variables at stake when slowing AI progress:[3] Time until critical Time until critical systems are deployed.[4] More time seems good for alignment research, governance, and demonstrating risks of powerful AI. Length of crunch time. In this post, "crunch time" means the time near critical systems before they are deployed.[5] More time until critical systems are deployed is good; more such time near critical systems is especially good. A lab is more likely to (be able to) pay an alignment tax fo…
How to think about slowing AI — EA Forum
The following are some very short stories about some of the ways that I expect AI regulation to make x-risk worse.
Ways I Expect AI Regulation To Increase X-Risk
"But," Alice continued, hesitantly, "I'm really concerned about some of the capabilities of the model, which I don't think we've explored completely. I know that it passed the standard safety checks, and I know I've explored some of its capabilities, but I think this is much much more important than -- " "Alice," interrupted Alex. "We've gone over this before. You're on the Alignment and Safety Team. That means part of your job is to make sure the model passes AMSA standards. That's what the Alignment and Safety team does." "But the AMSA regulations went through Congress before Ilya's Abductor…
Ways I Expect AI Regulation To Increase X-Risk
(a) Misdirected Regulations Reduce Effective Safety Effort; Regulations Will Almost Certainly Be Misdirected
Ways I Expect AI Regulation To Increase X-Risk
(b) Regulations Generally Favor The Legible-To-The-State
Ways I Expect AI Regulation To Increase X-Risk
(d) Regulations Are Likely To Maximize The Power of Companies Pushing Forward Capabilities the Most
Ways I Expect AI Regulation To Increase X-Risk
(c) Heavy Regulations Can Simply Disempower the Regulator
Ways I Expect AI Regulation To Increase X-Risk
a short pause (e.g. 6 months) probably just delays scheduled training runs, and during this pause, companies would stockpile compute and continue work on improving their algorithms. A medium length pause (e.g. 1 year) probably delays AI capabilities a little bit, but results in a significant jump in capability once the pause is lifted. A longer pause that lasts until we are confident that we have robust AI safety measures in place that allow for safe deployment would be helpful. I’m currently in favor of building the capacity of the world to create a long pause on AI.
Policy ideas for mitigating AI risk — EA Forum
Pausing just the training or deployment of the largest models is a shallow intervention that doesn’t affect the main drivers of capabilities progress. After the pause is lifted, we’ll likely see AI progress spike as actors immediately kick off much more advanced training runs than they had done in the past.
Policy ideas for mitigating AI risk — EA Forum
I would love feedback on our work. If you want to learn more about it, share considerations I did not address, or have ideas for improvements, please contact me at thomas@aipolicy.us.
Policy ideas for mitigating AI risk — EA Forum
Solutions: The regulatory agency will have robust conflict-of-interest provisions, “sunshine” laws that require the administrator to publicly report on their disagreements with the independent licensing judges, and a special “public interest” section that is charged with monitoring the administration’s actions and calling out any dangers that the administration has left unchecked.
Policy ideas for mitigating AI risk — EA Forum
Some ideas for increasing the government’s visibility into AI development are: A regulatory body that keeps track of AI development, makes predictions about model capabilities, and assesses risks. Required watermarking and traceability on advanced models, so that we can match AI outputs to specific AI models and developers. Whistleblower protections to incentivize researchers to report any dangerous AI development. Some ideas for giving the government brakes on dangerous AI development are: Strengthening the hardware export controls to include chips like A800s and H800s. Giving the regulator e…
Policy ideas for mitigating AI risk — EA Forum
Some others think it’s better to try to build aligned AIs that defend against AI catastrophes. For example, you can imagine building defensive AIs that identify and stop emerging rogue AIs. To me, the main problem with this plan is that it assumes we will have the ability to align the defensive AI systems.
Policy ideas for mitigating AI risk — EA Forum
Importantly, even though I focus on catastrophic risk here, there are many other reasons to ensure responsible AI development.
Policy ideas for mitigating AI risk — EA Forum