Reading list to catch up to the state of the art in LLM? : r/LanguageTechnology

Are there any books or other resources that have been released since the introduction of the GPT models that introduce the technologies and terminology used in current state of the art LLMs for someone entering the field? I'll start with the following - Word Embeddings (word2vec/GloVe), Encoder-decoder, Attention is all you need white paper. Reinforcement learning human feedback (RLHF). These should generally inform you on how LLM and GPT works. The exact architecture for models like GPT4 is not released, but these are the basic building blocks they use. word2vec/GloVe Now watching Lectures 1(Intro),2(word2vec),3(GloVe) of Chris Manning's Natural Language Processing with Deep Learning. The lectures are from 2017 - anything likely to have changed in embedding or are they a stable foundation? Not for small models, but for large models they don't use word2vec/GloVe. On a related note can we ballpark a scale for small and large models? I'm restricted for the time being to 8GB of VRAM on cu

saved by

• 3017