LLMs are (still) mostly powered by imitative learning, not RL — LessWrong
lesswrong.com · saved by 5 readers
LLMs get their impressive capabilities from a combination of: (1) Imitative learning from pretraining and SFT data (see my earlier discussion of “LLM pretraining magically transmutes observations into behavior, in a way that is profoundly disanalogous to how brains work”), and …