flâneur

The Near Future of AI is Action-Driven - by John McDonnell

jmcdonnell.substack.com · 1,794 words · saved by 2 readers

…and it will look a lot like AGI. LLMs can become extremely powerful by leveraging external cognitive assets and choosing actions.

In 2022, large language models (LLMs) finally got good. Specifically, Google and OpenAI have led the way in creating foundation models that respond to instructions more usefully. For OpenAI, this came in the form of Instruct-GPT (OpenAI blogpost), while for Google this was reflected in their FLAN training method (Wei et al. 2022, arxiv). Flan’s which beat the Hypermind forecast for MMLU performance two years early: Hyung Won Chung@hwchung27 New paper + models! We extend instruction finetuning by 1. scaling to 540B model 2. scaling to 1.8K finetuning tasks 3. finetuning on chain-of-thought…

saved by

related reading