Simulators :: — Moire

generative.ink · 1,696 words · saved by 1 readers

Bot credits Model GPT-3 Tokens 1,455 Gain (bits) ρ (bits/token) λselection (tokens/bit) Selection 196.82 0.14 7.39 bold is prompt Outline: Recently, a relatively unprecedented type of AI has emerged, achieving (and totally transforming) SOTA on diverse tasks, and dominating researcher attention Massive (transformer) models trained on massive, self-supervised datasets Exemplified by GPT/codex “Foundation models” Referring to the tendency for these pretrained models to be used as the “foundation” for further fine tuning/RL But that is a prescriptive, not descriptive category Motivation: GPT is the most powerful AI we’ve created and it works very differently than others (real and hypothetical) How does it interact with preexisting and hypothetical types of AI?

Simulators Bot credits Model GPT-3 Tokens 1,455 Gain (bits) ρ (bits/token) λ selection (tokens/bit) Selection 196.82 0.14 7.39 bold is prompt Outline: Recently, a relatively unprecedented type of AI has emerged, achieving (and totally transforming) SOTA on diverse tasks, and dominating researcher attention Massive (transformer) models trained on massive, self-supervised datasets Exemplified by GPT/codex “Foundation models” Referring to the tendency for these pretrained models to be used as the “foundation” for further fine tuning/RL But that is a prescriptive, not descriptive category Motivati

Simulators :: — Moire

related reading