Bonnie Li

on the atlas · 49

• The Bitter Lesson82 savers
• Andrej Karpathy blog4 savers
• All Roads Lead to Rome: The Machine Learning Job Market in 2022 | Eric Jang18 savers
• Marginal Revolution15 savers
• Nintil12 savers
• Faiss: A library for efficient similarity search4 savers
• Sam Altman’s Manifest Destiny11 savers
• The myth of the myth of the well read person20 savers
• Lessons from Undergrad13 savers
• Play the Infinite Game — The Mindful Student3 savers
• Trickle-down workaholism in startups - Signal v. Noise3 savers
• Equilibre Technologies Founders1 savers
• A Slice of My Mind8 savers
• Home of Feross Aboukhadijeh » Feross.org2 savers
• On not overthinking — Ada Nguyen7 savers
• 28: Out of the valley - by Nadia - Monomythical7 savers
• Engineer by Day, Venture Capitalist by Night | by Casey Caruso | Medium11 savers
• How to commit to the few right things | thesephist.com9 savers
• My Principles for Exploration10 savers
• Algorithmic Game Theory (CS364A), Fall 20131 savers
• uniswap_analysis.pdf1 savers
• SWIG Git1 savers
• Proofs involving ordinary least squares1 savers
• [2109.08603] Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration1 savers
• Geometric Deep Learning2 savers
• HOWTO: Be more productive (Aaron Swartz's Raw Thought)34 savers
• Pre-launch Competition | Daniel Gross4 savers
• How to Work Hard29 savers
• Fierce Nerds6 savers
• Multi-task curriculum learning in a complex, visual,hard-exploration domain: Minecraft1 savers
• DDSP: Differentiable Digital Signal Processing2 savers
• The Ones Who Walk Away From Omelas2 savers
• Stanford CS330: Deep Multi-Task and Meta Learning - YouTube1 savers
• Anthropic11 savers
• Mimetic - Brian Timar44 savers
• There’s no speed limit57 savers
• The Perception of Your Current vs Future Self - Cole Killian Adventure2 savers
• Mutual Information State Intrinsic Control1 savers
• [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks4 savers
• [2103.06326] S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learning2 savers
• You and Your Research82 savers
• Advice · Patrick Collison69 savers
• An Opinionated Guide to ML Research59 savers
• Advice for ambitious teenagers -32 savers
• Peter Thiel's Religion - David Perell28 savers
• The Lesson to Unlearn9 savers
• it’s such a beautiful day5 savers
• [1807.03819] Universal Transformers4 savers
• “When are you going to start your own company?”4 savers

highlights · 23

• Intensity is the pursuit of aliveness. […] there are certain forms of knowledge that will be denied to you if you stay within the bounds. You could think of intensity as the pursuit of gnosis, of “transcendence arrived at through intuitive, interior means." It’s hard to deny that extreme results (whether it be athletic feats or works of art) generally require extreme focus, extreme commitment, and extreme personalities. Decide to do something, and when you do it, pursue it with the max speed and intensity that you have. It will get you to the next decision point more quickly, and you will be m…

  On not overthinking — Ada Nguyen

• San Francisco's version of New York-style philanthropy has long been the angel investing ecosystem, a way to pay it forward to future founders.

  28: Out of the valley - by Nadia - Monomythical

• flurry of startup exits and IPOs

  28: Out of the valley - by Nadia - Monomythical

• The failure mode of today's ascendant wealth class would be a backslide into aristocracy, perpetuating the bloat and disquiet of generational wealth, instead of finding ways to discharge it back into society

  28: Out of the valley - by Nadia - Monomythical

• ut to praise despair is to condemn delight, to embrace violence is to lose hold of everything else.

  The Ones Who Walk Away From Omelas

• The trouble is that we have a bad habit, encouraged by pedants and sophisticates, of considering happiness as something rather stupid. Only pain is intellectual, only evil interesting.

  The Ones Who Walk Away From Omelas

• Hold yourself to ambitious absolute standards in morals and productivity — write them down on post-it notes.

  Mimetic - Brian Timar

• how well does the algorithm handle harder problems? At what point does it break down? Finally, rather than thinking about incremental improvements to existing algorithms and benchmarks, constantly be thinking of harder benchmarks and new capabilities.

  Eric Jang

• As Ilya Sutskever says, most incremental algorithm improvements are just data in disguise.

  Eric Jang

• Neural Network Quine

  [1803.05859] Neural Network Quine

• BART waspre-trained using a denoising objective and a variety of different noising functions.

  [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

• Finding generalizable evidence by learning to convince q&a models.

  [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

• Evidence aggregation for answer re-ranking in open-domain question answering

  [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

• Reinforced ranker-reader for open-domainquestion answering

  [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

• Coarse-to-fine question answering for long documents

  [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

• During inference time, we apply thepassage encoderEPto all the passages and indexthem using FAISS (Johnson et al., 2017) offline.FAISS is an extremely efficient, open-source li-brary for similarity search and clustering of densevectors, which can easily be applied to billions ofvectors. Given a questionqat run-time, we deriveits embeddingvq=EQ(q)and retrieve the topkpassages with embeddings closest tovq.

  [2004.04906] Dense Passage Retrieval for Open-Domain Question Answering

• We definethe similarity between the question and the passageusing the dot product of their vectors

  [2004.04906] Dense Passage Retrieval for Open-Domain Question Answering

• We considerthree different types of negatives: (1) Random: anyrandom passage from the corpus; (2) BM25: toppassages returned by BM25 which don’t containthe answer but match most question tokens; (3)Gold: positive passages paired with other questionswhich appear in the training set.

  [2004.04906] Dense Passage Retrieval for Open-Domain Question Answering

• Given a query vector, return the list of database objects that are nearest to this vector in terms of Euclidean distance. Given a query vector, return the list of database objects that have the highest dot product with this vector.

  Faiss: A library for efficient similarity search

• x2=h(y2)x1=h(y1)fX(x) dx=∫y2y1fX(h(y)) h′(y) dy.

  Change of variables: Apply $\tanh$ to the Gaussian samples

• Machine Learning PhD students are the new Investment Banking analysts, both seek optionality in their career choices but differ in superficial ways like preferring Meditation over Parties and Marijuana & Adderall over Alcohol and Cocaine. A Machine Learning PhD is now just an extended interview for FAANG.

  Machine Learning: The Great Stagnation

• The bureaucrats are running the asylum

  Machine Learning: The Great Stagnation

• Please also note that DrQ was designed for improving sample complexity rather than generalization.

  Automatic Data Augmentation for Generalization in Reinforcement...