flâneur

Kelvin Idanwekhai

0 followers · 356 views

on the atlas — 63

highlights — 90

  • Use platforms to find your readers, but bring them back to your house. It’s time to stop digital sharecropping on rented land.
    Substack writers, you need a website! – Elizabeth Tai
  • If you’re new to this I think it makes most sense to start by considering what kind of paper is going to work best in your life. Are you writing looseleaf, or in notebooks? Legal pads? Blue books? Remember, it’s okay to use lots of paper, so pick something that isn’t too precious and that is easy to replenish. I use a lot of moleskine notebooks and Mead comp books, which I can buy in bulk online. For composing fiction I use fancy looseleaf paper.
    Writing by Hand is Good for your Brain - Here's how to do it
  • The work of organizing is underappreciated by many people. But the other organizers are very attuned to this, and absolutely will notice who else is picking up the burdens.
    If You Build It, They Will Come - Ben Landau-Taylor
  • The kids coming up will not feel this as a loss, and they are not wrong not to. You cannot miss a relationship you never had. They will have a tool that does everything and asks for nothing, and they will be as easy with it as you are with the light switch you never once thought about. There is nothing sad in their ease. The grief is entirely ours. We happen to be the last people who knew a machine the only way a machine was ever really known: by having it be difficult. And we are also the last people who will know that anything is gone.
    The Last People Who Know How It Works · unix.foo
  • The original 2017 transformer applied normalization AFTER each sub-block (post-norm). This worked for shallow models but became harder to train reliably as depth increased. Modern transformers (GPT-2 onward, LLaMA, Mistral) commonly apply normalization BEFORE each sub-block (pre-norm). That’s one of the changes that made very deep transformers easier to train.
    How LLMs Actually Work | 0xkato
  • Grouped-Query Attention lets multiple query heads share fewer key/value heads. That cuts KV-cache memory while keeping many query views.
    How LLMs Actually Work | 0xkato
  • AI can support learning, but it can also produce an answer without teaching you how to arrive at it. The process of solving a physics problem or writing an essay may be considered by many students to be tedious (Which equations? Which sources? Which arguments?). But then, what is the point of being in school or of learning?
    Are we offloading too much of our thinking to AI?
  • I know you want to use AI as an escape. It is so tempting. You get to sit in your bubble and imagine everything and see it come to life and your agent buddy will keep cheering you on while you get nothing done with your life. And I think that’s the biggest danger of AI. You convince yourself that you are doing something useful when you are not.
    Punch Yourself in the Face with Reality | Adi
  • All memory must be statically allocated at startup. No memory may be dynamically allocated (or freed and reallocated) after initialization. This avoids unpredictable behavior that can significantly affect performance, and avoids use-after-free. As a second-order effect, it is our experience that this also makes for more efficient, simpler designs that are more performant and easier to maintain and reason about, compared to designs that do not consider all possible memory usage patterns upfront as part of the design.
    Zig Creator Calls Spade a Spade, Anthropic Blows Smoke
  • You're no longer a court jester performing for the algorithm hoping to go viral and appease the masses. You're no longer expected to have media training levels of competence around fame and fan engagement for blowing up overnight.
    some things to (un)learn – ava's blog
  • Email is an oft-cited example for demonstrating Lindy’s law: a theory positing that something’s future life expectancy is in proportion to its current age. The longer something lasts, the longer it’ll continue.
    Why I email complete strangers
  • The difficulty was the knowledge. You came to know that machine the way you come to know anything that pushes back. The resistance was the whole medium. You only ever know the things that you can lose to.
    The Last People Who Know How It Works · unix.foo
  • The stuff around us isn’t divine - these chairs we’re sitting on, the TV at the front of the classroom, the pots for the plants - all this stuff was invented and figured out and made by people.
    Factories are just rooms (Interconnected)
  • Or as the Bulgarian cultural anthropologist Ivaylo Dichev memorably put it: “Who better than comrade Stalin knows that love always involves a threesome [a man, a woman, and the state].”
    Man and Woman, Intimately
  • When people say “taste,” what they actually mean is experience. Pattern recognition built up over years of doing the work. But calling it “taste” instead of “experience” does something subtle and harmful: it makes a learnable skill sound like a gift.
    “Good Taste” Is Just Experience – Terrible Software
  • Here’s what I think the right aspiration looks like for computational researchers: becoming some sort of mini-PI. Not in terms of seniority or rank but how we orient ourselves. A PI’s job (to me) is not to write the most code. It’s to exercise good taste. It’s to know which problems are worth working on, how to allocate resources, and recognise good and bad results when they appear. That ability and capacity to assign value to ideas, to prioritise them, and to know what’s more important and what could be noise, is what I mean by research taste.
    Bravely wielding "Shirt Pocket Power" - by Rishabh Anand
  • The threat to musicians was also glaring, with complaints that pressing buttons and turning knobs was not real music, and that synths and electronic music devalued the craft.
    Bravely wielding "Shirt Pocket Power" - by Rishabh Anand
  • The resistance was also instantaneous: teachers and educators banned these calculators from classrooms to preserve the “thinking” needed to operate a slide rule. People who worried about that calculators would make for lazy engineers weren’t very wrong about the risk, but it certaintly didn’t make good engineers any worse.
    Bravely wielding "Shirt Pocket Power" - by Rishabh Anand
  • DMSD belongs to a family of compounds called siloxanes, molecules that contain a silicon-carbon-oxygen bond and occupy a kind of middle ground between organic chemistry and beach sand. Siloxanes (also called silicones) are common ingredients in cosmetics1, contact lenses, fake boobs, caulks, packaging, and all kinds of personal hygiene products, where they’re used to make things feel smooth and slippery. It’s siloxanes that give deodorant and hair conditioner their slick texture, and the same property makes them a popular industrial lubricant.
    L'Affaire Siloxane - by Maciej Cegłowski
  • In June 2010, thirteen months after the Water Processing Assembly went online, excessive levels of total organic carbon began to show up in the astronauts’ drinking water. Total organic carbon is a non-specific measurement that warns the crew about a contaminant being present, but gives them no clue to its identity.
    L'Affaire Siloxane - by Maciej Cegłowski
  • Processing Assembly would help take the station from a 45% to 80% water reuse rate. For the first time in the history of space flight, astronauts would be substantially recycling their water in an orbiting habitat.
    L'Affaire Siloxane - by Maciej Cegłowski
  • It’s Kinzie Street railroad bridge, and it was used by Chicago Sun-Times for their paper deliveries. However, following the downturn and moving the printing facility to the suburbs, it was left permanently open in 2000.
    Seesaws for giants – Aresluna
  • Or neural network, whatever you want to call it. We want the input to be a number, and the output to be the correct "fizzbuzz" representation of that number. In particular, we need to turn each input into a vector of "activations". One simple way would be to convert it to binary.
    - Fizz Buzz in Tensorflow
  • TLDR: Talent isn’t all that it takes to become a successful researcher. Temperament is greatly underrated. Stay curious and persistent, remain thoughtful and meticulous, and the ideas will come.
    Zen and the Art of Machine Learning Research
  • If you look at the career of great researchers, they likely spent lots of time working in obscurity before finding success. Get used to this. The more ambitious and forward-thinking an idea, the more work it may be to thoroughly implement and evaluate. This difficulty is a feature, not a bug.
    Zen and the Art of Machine Learning Research
  • before enlightenment, chop wood, carry water. after enlightenment, chop wood, carry water.
    Zen and the Art of Machine Learning Research
  • Mediocre hires hurt you twice: they get less done, but they also make you big, because you need more of them to solve a given problem.
    You Weren't Meant to Have a Boss
  • If you are requesting human attention, demonstrate human effort.
    If You are Asking for Human Attention, Demonstrate Human Effort | Tom Bedor's Blog
  • Failing to explain why forward KL is mean-covering and reverse KL is mode-seeking in an interview does not make you a bad ML researcher.
    ML Job Interviews: The Ultimate Guide – Silvia Sapora
  • Diffusion / Flow Matching: the MIT and Stanford courses are both good but quite math-heavy. If you're not actively researching in this area, questions will likely be superficial, so just get a high-level intuition and memorise the basics (e.g. diffusion SDEs and flow matching ODEs).
    ML Job Interviews: The Ultimate Guide – Silvia Sapora
  • Designing Machine Learning Systems by Chip Huyen: covers a lot of the fundamentals and applied ML questions you'll encounter. Highlight and take notes.
    ML Job Interviews: The Ultimate Guide – Silvia Sapora
  • Do at least Blind 75 and optionally NeetCode 150, focusing on Mediums.
    ML Job Interviews: The Ultimate Guide – Silvia Sapora
  • fairly modern architectures (Griffin, TransformerXL, S4). The list reflects that range.
    ML Job Interviews: The Ultimate Guide – Silvia Sapora
  • "tell me a time you received feedback"...), research-style interviews ("what topics are you interested in?", "where do you see the field going?"). These are more casual compared to the technical interviews,
    ML Job Interviews: The Ultimate Guide – Silvia Sapora
  • ML coding and debugging. Implementing attention, writing backward passes, spotting bugs in training loops
    ML Job Interviews: The Ultimate Guide – Silvia Sapora
  • Follow-up on brief mentions and rhetorical questions. Essays and articles sometimes have a sentence or two that is incredibly intriguing, but that the author never flushed out because it was only tangentially related to their argument. Follow-up and pursue these ideas! Similarly, taking rhetorical questions literally might be a fruitful way to find ideas.
    How to Find Writing Ideas — Niko McCarty
  • During the Salem witch trials a total of 20 people, accused by dozens of victims of witchcraft, were tried and executed. Previous theories on a medical explanation of witchcraft have attempted to account for the epidemic phenomenon amongst the townsfolk of Salem.
    The witchcraft of encephalitis in Salem | Journal of Neurology | Springer Nature Link
  • When you type a few short sentences into an AI image generator and it makes you an image for your blog post, you are likely generating slop because you are injecting relatively little information yoursel
    What Makes Art Great? - Nabeel S. Qureshi
  • I come by that desire honestly, I think. I’m not desperate for eyeballs on the page or followers. I don’t dumb things down, I don’t go for that cheap form of academic popularization that you see all over the internet, I try to stick to my own beliefs in a debate, even if they are unpopular (see fascism debate).
    The privilege of bad writers – Corey Robin
  • They are people who still suffer and savor their memories of that first year of graduate school, when everyone is completely intimidated by the knowledge of their professors or peers, not their knowledge of the subject matter, but their knowledge of The Literature. That is, what people have said about the subject matter (sort of).
    The privilege of bad writers – Corey Robin
  • Whatever the source of his monologue, he simply does not care about whether his audience is with him or not.
    The privilege of bad writers – Corey Robin
  • The telltale sign of the droner is that there is nothing in the movement of his sentences that suggests he cares or is aware of his audience.
    The privilege of bad writers – Corey Robin
  • To build intuition: suppose you walk past a geyser, and see a sign saying “This geyser last erupted 100,000 years ago”. You know nothing else about geysers. What’s the chance it will erupt in the next hour? It must be very low, right? If it erupted in the next hour, you would have walked past it 99.99999% of the way through its eruption cycle - in other words, your random sample had a higher value than 99.99999% of points. That’s not how random samples usually work! On the other hand, suppose you walk past another geyser, and see a sign saying “This geyser last erupted 10 minutes ago”. What is…
    The Sigmoids Won't Save You - by Scott Alexander
  • Every phase of life can be shopped for at Costco. Where else can you purchase a wedding ring, a baby carrier, and a casket? It follows your own life story—one does not experience Costco the same way before and after owning a home, or before and after having a child. My wife and I recently purchased our first house, and never could I have imagined feeling so drawn toward the allure of outdoor furniture: palapas apropos, gazebos akimbo.
    I Want to Live Like Costco People | TASTE
  • with an estimated 30% of Americans over the age of 18 holding membership cards.
    I Want to Live Like Costco People | TASTE
  • This is the bit that gets lost. The website isn't for the founder, the marketing manager, or the board. It's for the person you've never met - the customer weighing up a purchase, the lead chasing a phone number, the visitor sizing up your credibility or the member signing up to access gated content.
    Your website is not for you — Websmith Studio
  • Stripped of easy moralising, literature has the power to make us relish the search for truth, in the age when it is widely believed to be dead.
    The role of literature as the key to personal freedom | Aeon Essays
  • We are not all victims of abuse. But the currently predominant economic model that relies on the harvesting of personal data and relentless advertising makes us all targets of predation.
    The role of literature as the key to personal freedom | Aeon Essays
  • Today, people seem to turn much more willingly to therapists, experts or influencers to make sense of their lives. Yet I would argue that now may be a good moment for reconsidering the role of literature as the key to personal freedom – precisely because it is no longer the most common or most prestigious medium for interpreting experience
    The role of literature as the key to personal freedom | Aeon Essays
  • This encapsulates what literature can do today. In a society largely dominated by visual entertainment, communication and performance, when fewer and fewer people read for pleasure, the idea of the writer as a spiritual beacon may seem largely outmoded.
    The role of literature as the key to personal freedom | Aeon Essays