Aaron Leslie — Software Engineer & Co-founder

Aaron Leslie — Software Engineer & Co-founder of Glyph

1. Challenge and result comma.ai's challenge: shrink a 37 MB dashcam video, with the constraint that two frozen neural networks (a SegNet and a PoseNet) must produce the same outputs on the reconstructed video as they would on the original. The score: score=100⋅Dseg+10⋅Dpose+25⋅∣archive∣∣source∣\text{score} = 100 \cdot D_\text{seg} + \sqrt{10 \cdot D_\text{pose}} + 25 \cdot \frac{|\text{archive}|}{|\text{source}|} Lowest wins. Our final number is 0.1988, against a field clustered between 0.32 (top neural) and 2.0 (most codec submissions). At the time of writing, we're #1 by 38%. 2. The…

saved by

• Aidan Smith

related reading

• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• The Annotated Transformernlp.seas.harvard.edu
• [2605.05331] ViTok-v2: Scaling Native Resolution Auto-Encoders to 5 Billion Parametersarxiv.org
• Compression and Intelligencegreene.sh
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• LLMs can invent their own compression - Rajan Agarwalrajan.sh
• [2309.10668] Language Modeling Is Compressionarxiv.org
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• The Annotated Transformernlp.seas.harvard.edu
• https://arxiv.org/pdf/2006.11239arxiv.org
• Speculative KV coding: losslessly compressing KV cache by up to ~4× using a predictor modelfergusfinn.com