Alexiy Buynitsky

6 followers · 3 following · 832 views

on the atlas · 17

highlights · 17

ptimal sleep time for me is around 7.5 hours, with an absolute minimum of around 4hrs.
Andrej's advice for success
The root cause of this issue was a latent race condition in the DynamoDB DNS management system that resulted in an incorrect empty DNS record for the service’s regional endpoint (dynamodb.us-east-1.amazonaws.com) that the automation failed to repair
More Than DNS: The 14 hour AWS us-east-1 outage
The concept of a warp is relevant for this second kernel, as sequential memory accesses by threads that are part of the same warp can be grouped and executed as one. This is referred to as global memory coalescing. It’s the most important thing to keep in mind when optimizing a kernel’s GMEM memory accesses toward achieving the peak bandwidth.
How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklog
More resources have led to less creativity. He described researchers constantly checking whether they've been "scooped" by competitors working on identical ideas, and academics choosing safe, publishable projects over risky, potentially transformative ones.
Sakana AI's CTO says he's 'absolutely sick' of transformers, the tech that powers every major AI model | VentureBeat
A single instruction can perform a full outer product: multiply every element of an X register with every element of a Y register, and accumulate with the Z element in the corresponding position.
Explore AMX instructions: Unlock the performance of Apple Silicon | Zheng's Notes
A good one-image summary of AMX is the following figure from abandoned patent US20180074824A1. Consider a 32x32 grid of compute units, where each unit can perform 16-bit multiply-accumulate,
Explore AMX instructions: Unlock the performance of Apple Silicon | Zheng's Notes
Here’s a neat programming trick to swap two values without needing a temp: x = x xor y y = x xor y x = x xor y
Swap two variables using XOR – BetterExplained
Perplexity is not a good measurement of how well a model actually performs
Perplexity is not a good measurement of how well a model actually performs : r/LocalLLaMA
Forward KL divergence corresponds to D K L ( p | q ) 𝐷 𝐾 𝐿 ( 𝑝 | 𝑞 ) , and reverse corresponds to D K L ( q | p ) 𝐷 𝐾 𝐿 ( 𝑞 | 𝑝 ) . The reverse KL divergence is said to be “mode-seeking”. This means that the divergence will be low when q 𝑞 places density only where p 𝑝 places density, and the divergence will be high when q 𝑞 places density where p 𝑝 does not.
Andy Jones
D K L ( P | | Q ) = how much more surprised you expect to be when observing data with distribution P , if you falsely believe the distribution is Q vs if you know the true distribution
Six (and a half) intuitions for KL divergence - LessWrong
technique for compactly representing and visualizing a probability distribution that is based on the language of undirected graphs. This class of models (known as Markov Random Fields or MRFs)
Markov random fields
a manifold is a curved surface that looks flat when you zoom in close enough. The locally flat approximation at a point on the manifold is called the tangent space to the manifold,
Modular Manifolds - Thinking Machines Lab
r supervised fine-tuning on small-to-medium-sized instruction-tuning and reasoning datasets, LoRA performs the same as full fine-tuning. For datasets that exceed LoRA capacity, LoRA underperforms FullFT.
LoRA Without Regret - Thinking Machines Lab
finetuned performance smoothly improves with increasing pretraining data. At the data scales we examined, we saw no evidence of performance discontinuities or sharp inflection points; AI scaling appears alive and well in robotics.
A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation
on-policy RL is simple: increase the likelihood of high-reward actions. Meanwhile, flow matching naturally increases likelihoods by redirecting probability flow toward training samples. This makes our objective clear—redirect the flow toward high reward actions.
Flow Matching Policy Gradients
demonstrations are most useful for identifying which actions should be explored, rather than forcing the policy to take certain actions.
Reinforcement Learning via Implicit Imitation Guidance
diffusion is really just approximate autoregression in frequency space, why not just do exact autoregression in frequency space instead, and maybe that will work just as well?
Diffusion is spectral autoregression – Sander Dieleman