Varun Goyal

1 followers · 3 following · 478 views

on the atlas · 68

highlights · 114

Deployment viability: compute reduction and wall-clock speed Retrieval capability: RULER, and MRCR v2
Subquadratic — Efficiency is Intelligence
DeepSeek Sparse Attention is a newer sparse approach
Subquadratic — Efficiency is Intelligence
State space models and recurrent alternatives take a different approach.
Subquadratic — Efficiency is Intelligence
Sliding windows, strided patterns, and dilated masks shrink the search space enough to achieve subquadratic scaling
Subquadratic — Efficiency is Intelligence
Fixed-pattern sparse attention reduces compute by limiting which positions a token can attend to
Subquadratic — Efficiency is Intelligence
it simply performs that work more efficiently.
Subquadratic — Efficiency is Intelligence
Each token acts as a query, comparing itself against every other token, scoring their relevance, and aggregating their information into its next representation.
Subquadratic — Efficiency is Intelligence
. A chunk may contain the right text while losing why that text matters. Agentic workflows decompose large tasks into smaller model calls, but errors compound across steps, orchestration logic becomes hand-authored policy, and context is repeatedly compressed between calls.
Subquadratic — Efficiency is Intelligence
A RAG system preserves semantic similarity, but loses position, hierarchy, neighboring context, and reference structure.
Subquadratic — Efficiency is Intelligence
Those techniques are useful, but they introduce their own failure modes.
Subquadratic — Efficiency is Intelligence
A functional context window tells you how many tokens a model can reliably reason over. SSA is designed for the second problem.
Subquadratic — Efficiency is Intelligence
SSA changes that scaling behavior. Instead of computing every pairwise interaction, SSA uses content-dependent selection to route attention toward the positions that matter, regardless of where those positions appear in the sequence.
Subquadratic — Efficiency is Intelligence
compute requirements scale quadratically with context length.
Subquadratic — Introducing SubQ: The First Fully Subquadratic LLM
A program takes inputs, manipulates state, and produces outputs.
Prompts are code, .json/.md files are state
think "inputs, state, outputs" instead of "chat with the AI" and suddenly you're closer to engineering solutions
Prompts are code, .json/.md files are state
What we need when using coding agents on bigger codebases is a structured way to engineer context.
Prompts are code, .json/.md files are state
While senior engineers strive for elegant, minimal solutions that reduce bugs and complexity, LLMs reach for 'best practices' and spit out over-engineered garbage.
Prompts are code, .json/.md files are state
there are only so many answers you can say before revealing it all.
i wish you were more curious. - a.r.
i’ve been conditioned to keep quiet
i wish you were more curious. - a.r.
what can you do for me that i cannot do on my own?
i wish you were more curious. - a.r.
but then someone smiles at you in a grocery store or asks about your favorite book, and you realize your heart still works
this is what being in your twenties feels like
there’s a loneliness that comes with it too, one that sits beside you in shared apartments and crowded metros. it’s not the kind of loneliness that begs to be fixed, it’s quieter, steadier. it teaches you how to sit with yourself, how to find comfort in small rituals. making coffee in the morning, cleaning the same kitchen over and over, lighting a candle after another long day. the small acts become anchors.
this is what being in your twenties feels like
you look at your parents differently now, not as invincible, but as people who also had dreams that never quite became what they wanted.
this is what being in your twenties feels like
some days it feels like you’re holding your whole life together with one trembling hand and a half-empty cup of coffee
this is what being in your twenties feels like
It feels like I have to drop some dramatic work or life story over a glass of wine every time we get together because the time feels more emotionally weighted when it’s scarcer
Are we caught in a culture of never-ending catch-ups? | Dazed
but platonic relationships don’t always have the same in-built expectations.
Are we caught in a culture of never-ending catch-ups? | Dazed
This can include, according to Bronstein, watching thought-provoking movies, participating in workshops or card games, or even attending a music festival.
Are we caught in a culture of never-ending catch-ups? | Dazed
By assuming that people will post updates or share them at the next catch-up session, we can lose small details that actually make many feel cared for, like how a meeting went and who you saw on a walk (or if you saw a bird).
Are we caught in a culture of never-ending catch-ups? | Dazed
You go on these dinner dates, where you catch up with friends, but you are not experiencing life together
Are we caught in a culture of never-ending catch-ups? | Dazed
After entering the workforce, it can feel like everyone you care about is living their lives alongside their co-workers and partners, and then working hard to find a time to simply keep you in the loop. This trap is what people are calling “catch-up culture”.
Are we caught in a culture of never-ending catch-ups? | Dazed
our now-adult schedules get in the way of that level of spontaneity and togetherness.
Are we caught in a culture of never-ending catch-ups? | Dazed
If you work with sensitive data, please take appropriate precautions. Agents are random and could act maliciously. Caveat emptor.
Agentic Coding For Scientists
Through encountering others, we become aware of the disintegrating effect that they have on us and that we do not have full control over our lives.
Heaven is other people — Dappled Things
Cartesian philosophy proposes that all we can know is ourselves and our own thinking (i.e. “cogito ergo sum”).
Heaven is other people — Dappled Things
However, it is really only in the present, under the gaze of those who desire our good, that we can understand who we are now and discover who we are meant to be.
Heaven is other people — Dappled Things
inferno is not characterized by fire and brimstone, but rather, by being stuck with his two annoying acquaintances for the rest of eternity.
Heaven is other people — Dappled Things
You are the gold standard of software engineering
The software engineer’s guide to asserting dominance in the workplace | by Daniel Wu | Feature Creep | Medium
Being at higher ground is a position of power. This advantage is especially important in open-office environments.
The software engineer’s guide to asserting dominance in the workplace | by Daniel Wu | Feature Creep | Medium
because in a world that keeps asking us to explain ourselves, there is something sacred about being understood without translation
my love language is not having to ask. - mixtapes by gor
the kind that builds a quiet home inside you without asking for space first.
my love language is not having to ask. - mixtapes by gor
Maybe that’s what October is really about: not ghosts or reunions, but the gentle resurfacing of what once mattered.”
Autumn is for the romantics - by Chris - ~interrupted~
An ex, an old friend, someone who you once held incredible affection for, it’s all the same under the gaze of the universe.
Autumn is for the romantics - by Chris - ~interrupted~
Romanticization is a powerful thing because it borders on delusion, but I firmly believe that a little insanity, a little delusion, a little romanticization can make you infinitely happier, though maybe happy is the wrong word.
Romanticize everything, it'll keep you sane - by Chris
I eventually realized that no one was going to give my life meaning. No one was going to “fix” me.
Romanticize everything, it'll keep you sane - by Chris
Music helps me romanticize my hobbies and the things I don’t really want to do, like clean or work.
Romanticize everything, it'll keep you sane - by Chris
“It is a truth universally acknowledged that when one part of your life starts going okay, another falls spectacularly to pieces.”
Romanticize everything, it'll keep you sane - by Chris
It’s a door to another world, escapism at its finest, and to escape to another place requires a desire to be there, or to view the world through someone else’s eyes.
Romanticize everything, it'll keep you sane - by Chris
More than the social and cozy aspects, reading is, I believe, a romantic act.
Romanticize everything, it'll keep you sane - by Chris
What you should do, however, is put on a cute outfit, make yourself feel handsome and pretty and all of the above, head to your local café (or one that you’ve been meaning to try), and journal like you’re the only person in the world with anything worthwhile to say. Writing is inherently romantic.
Romanticize everything, it'll keep you sane - by Chris
Making an aesthetic out of not doing well
Romanticize everything, it'll keep you sane - by Chris