Understanding Large Language Models -- A Transformative Reading List

Since transformers have such a big impact on everyone's research agenda, I wanted to flesh out a short reading list for machine learning researchers and ...

Large language models have taken the public attention by storm – no pun intended. In just half a decade large language models – transformers – have almost completely changed the field of natural language processing. Moreover, they have also begun to revolutionize fields such as computer vision and computational biology. Since transformers have such a big impact on everyone’s research agenda, I wanted to flesh out a short reading list (an extended version of my comment yesterday ) for machine learning researchers and practitioners getting started. The following list below is meant to be read mo

saved by

• Sahil Patel
• Shrey Joshi

related reading

• Understanding Large Language Modelsmagazine.sebastianraschka.com
• Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
• GenAI Handbookgenai-handbook.github.io
• LLM Resourcesforrestbicker.com
• Transformer Circuits Threadtransformer-circuits.pub
• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrouai.googleblog.com
• Generalized Language Models | Lil'Loglilianweng.github.io
• "Attention", "Transformers", in Neural Network "Large Language Models"bactra.org
• Recent Advances in Language Model Fine-tuningruder.io
• The Llama Hitchiking Guide to Local LLMs – hackerllamaosanseviero.github.io
• [1909.08053] Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismarxiv.org