Philip Kiely on X: "Scaling past a trillion parameters requires inference engineers to solve 3 issues: - Width: Routing between hundreds of sparse experts - Depth: Mixing global and local linear attention - Length: Processing 1M-token sequences I asked Ali and Bryce to draw it up. They delivered. https://t.co/ZOo4yxCCR2" / X
x.com · saved by 1 readers
Scaling past a trillion parameters requires inference engineers to solve 3 issues: - Width: Routing between hundreds of sparse experts - Depth: Mixing global and local linear attention - Length: Processing 1M-token sequences I asked Ali and Bryce to draw it up. They delivered.