Yury Polyanskiy
Yury Polyanskiy Professor (EECS) Curriculum Vitae Publications & Preprints Research Group Other Quantization and tokenization in AI (mini course) Class notes and materials (2025) This page contains some supplementary materials for a 4-lecture mini course taught by Y. Polyanskiy in Princeton ML School in 2025. Warning: The notes are handwritten and were not meant to be published, so they are very unpolished. Apologies for the rough presentation, missing references, and any errors. Lecture 1: Scalar Quantization. Quantization Motivation: Classical A/D conversion. Quantization in LLMs. Tokenization of non-text modalities (VQGAN, SoundStream) Scalar quantization. Uniform quantization: 6dB/bit Dithering Non-uniform quantization. FP4 Panter-Dite approximation Uniform quantization of weights in LLMs. Non-MSE metric GPTQ and LDLQ Lecture 2: Vector Quantization and Tokenization. GPTQ and LDLQ Hammin