✳flâneur — a map of the web's best reading

Pivotal act — LessWrong

lesswrong.com · 2,605 words · saved by 1 readers

The term 'pivotal act' in the context of AI alignment theory is a guarded term to refer to actions that will make a large positive difference a billion years later. Synonyms include 'pivotal achievement' and 'astronomical achievement'. We can contrast this with existential catastrophes (or 'x-catastrophes'), events that will make a large negative difference a billion years later. Collectively, this page will refer to pivotal acts and existential catastrophes as astronomically significant events (or 'a-events'). 'Pivotal event' is a deprecated term for referring to astronomically significant events, and 'pivotal catastrophe' is a deprecated term for existential catastrophes. 'Pivotal' was originally used to refer to the superset (a-events), but AI alignment researchers kept running into the problem of lacking a crisp way to talk about 'winning' actions in particular, and their distinctive features. Usage has therefore shifted such that (as of late 2021) researchers use 'pivotal' and 'pivotal act' to refer to good events that upset the current gameboard - events that decisively settle a win, or drastically increase the probability of a win. Reason for guardedness Guarded definitions are deployed where there is reason to suspect that a concept will otherwise be over-extended. The case for having a guarded definition of 'pivotal act' (and another for 'existential catastrophe') is that, after it's been shown that event X is maybe not as important as originally thought, one side of that debate may be strongly tempted to go on arguing that, wait, really it could be "relevant" (by some strained line of possibility). Example 1: In the Zermelo-Fraenkel provability oracle dialogue, Alice and Bob consider a series of possible ways that an untrusted oracle could break an attempt to box it. We end with an extremely boxed oracle that can only output machine-checkable proofs of predefined theorems in Zermelo-Fraenkel set theory, with the proofs themselves being thrown away on

x Pivotal act — LessWrong Pivotal act Edited by Eliezer Yudkowsky , Rob Bensinger , et al. last updated 16th Nov 2021 The term 'pivotal act' in the context of AI alignment theory is a guarded term to refer to actions that will make a large positive difference a billion years later. Synonyms include 'pivotal achievement' and 'astronomical achievement'. We can contrast this with existential catastrophes (or 'x-catastrophes'), events that will make a large negative difference a billion years later. Collectively, this page will refer to pivotal acts and existential catastrophes as astronomically s

Explore this link on the map →

related reading