Karan Dalal

9 followers · 2 following · 1295 views

on the atlas · 33

highlights · 66

Verification: asking the LLM to explain (retrieve) the sources where it gets the answer from.
RLHF: Reinforcement Learning from Human Feedback
The second hypothesis is that hallucination is caused by the mismatch between the LLM’s internal knowledge and the labeler’s internal knowledge. In his UC Berkeley talk (April 2023), John Schulman, OpenAI co-founder and PPO author, suggested that behavior cloning causes hallucination. During SFT, LLMs are trained to mimic responses written by humans. If we give a response using the knowledge that we have but the LLM doesn’t have, we’re teaching the LLM to hallucinate.
RLHF: Reinforcement Learning from Human Feedback
The challenge with training a reward model is with obtaining trustworthy data. Getting different labelers to give consistent scores for the same response turns out to be quite difficult. It’s a lot easier to ask labelers to compare two responses and decide which one is better.
RLHF: Reinforcement Learning from Human Feedback
Once the publicly available data is exhausted, the most feasible path for more training data is with proprietary data. I suspect that any company that somehow gets its hand on a massive amount of proprietary data – copyrighted books, translations, contracts, medical records, genome sequences, user data, etc. – will have a competitive advantage.
RLHF: Reinforcement Learning from Human Feedback
But if some of the things you’ve committed yourself to wouldn’t almost kill you with their absence, can you even say you’re alive?
If I Asked You About Love, Which Sonnet Would You Quote?
bout 14% of Berkeley CS undergrads go to graduate school (counting master’s programs), and I’d say this is actually the top 14% of the major. Maybe top 20%, if you want to be safe.
"How Do You Feel About Grad School?"
Once I see papers as promising ideas instead of ground truth, prior work doesn’t intimidate me as much
"How Do You Feel About Grad School?"
But it might be worth taking a contrarian bet that training on the world of bits is not enough, and that Moravec’s Paradox is not a paradox at all, but rather a consequence of us not having solved the “bulk of intelligence”.
All Roads Lead to Rome: The Machine Learning Job Market in 2022 | Eric Jang
“All Roads Lead to Rome - Every Successful (Tech) Company will be an AGI company”.
All Roads Lead to Rome: The Machine Learning Job Market in 2022 | Eric Jang
Having futuristic technology is important for recruiting engineers because many of them don’t want to waste years of their life building a capability that someone else already has.
All Roads Lead to Rome: The Machine Learning Job Market in 2022 | Eric Jang
The most important deciding factor for me was whether the company has some kind of technological edge years ahead of its competitors
All Roads Lead to Rome: The Machine Learning Job Market in 2022 | Eric Jang
Neuroscience is bottlenecked by neurotechnology, where “neurotechnology” is defined as tools that directly, exogenously observe and manipulate the state of biological nervous systems.
Neurotechnology is Critical for AI Alignment
But if we want to assess AI systems for their (degree of) presence or absence, we need to operationalize them somehow.
Neurotechnology is Critical for AI Alignment
A detailed mechanistic understanding of moral reasoning is key to identifying them and characterizing when and how they occur.
Neurotechnology is Critical for AI Alignment
Currently our mechanistic understanding of human cognition is pretty poor. We have words for different emotions and can read them decently off faces, we can measure your IQ and guess what kind of job you might have, and we know object permanence develops before language ability. But we can’t reliably detect when someone is lying, or identify which type of treatment will help with someone’s depression, or agree on whether humans have subconscious minds or what those subconscious minds might be doing.
Neurotechnology is Critical for AI Alignment
If a startup can quickly, cheaply, and uniquely generate the data needed in a domain to train and fine-tune AI models, they can unlock incredible value.
Defensibility in the Age of AI
Because robotics hasn’t kept pace with AI, companies that build physically aren’t at risk of being disrupted by AI in the near future. Companies like SpaceX, Cover, or Solugen will gain efficiencies by implementing AI but have little to fear from the latest LLM.
Defensibility in the Age of AI
Designing from first principles allows Anduril to create novel defense systems.
Anduril: The Business of Defense - by Mario Gabriele
Trae Stephens described Anduril’s product strategy as a “Trojan horse.” Buyers used to purchasing hardware are sated by the company’s physical goods, while software system Lattice is slipped in relatively sub-rosa.
Anduril: The Business of Defense - by Mario Gabriele
In this approach, contractors are not paid a fixed, pre-agreed fee. Instead, they bill the government for their work, receiving a percentage on top of the total cost, usually between 6-8%.
Anduril: The Business of Defense - by Mario Gabriele
With the threat of war with Russia waning, fewer and smaller contracts would be available. Contractors were left with a choice: consolidate or die.
Anduril: The Business of Defense - by Mario Gabriele
It can take years for a defense contractor to ship a new product; Anduril had its sentry tower in the field within six months.
Anduril: The Business of Defense - by Mario Gabriele
“There is a category of things that are a moral good but feel bad,” Trae Stephens said to me. Selling weapons to the United States and its allies may be one of them.
Anduril: The Business of Defense - by Mario Gabriele
They have a laser focus on the next step in front of them combined with long-term vision. Most people only have one or the other.
Sam Altman
Our goal is essentially to develop: better techniques for making AI systems safer, better ways of identifying how safe or unsafe AI systems are.
Anthropic | Core Views on AI Safety: When, Why, What, and How
If we build an AI system that’s significantly more competent than human experts but it pursues goals that conflict with our best interests, the consequences could be dire.
Anthropic | Core Views on AI Safety: When, Why, What, and How
So far, no one knows how to train very powerful AI systems to be robustly helpful, honest, and harmless.
Anthropic | Core Views on AI Safety: When, Why, What, and How
The state space model is defined by this simple equation. It maps a 1-D input signal � ( � ) u(t) to an � N-D latent state � ( � ) x(t) before projecting to a 1-D output signal � ( � ) y(t).
The Annotated S4
There are two capabilities that we need to be able to do associative recall: memorize tokens over the entire sequence, and compare the current token to previous tokens.
H3: Language Modeling with State Space Models and (Almost) No Attention · Hazy Research
However, in our recent paper, we find that even when temporal compositionality is not a primary concern, offline RL does provide benefits over imitation learning.
Should I Use Offline RL or Imitation Learning? – The Berkeley Artificial Intelligence Research Blog
The problem I have with this definition is that while there is automation of some activity, we’ve done so by highly constraining the problem. And as a result, there’s an enormous amount of effort that’s needed to fix, clean, maintain or generally deal with the idiosyncrasies of an automaton.
Where are all the robots? - by Rohit - Strange Loop Canon
If you know what you’re good at, you can create a clear value proposition for a defense user. You can be clear on the skills and network you bring from tech and get them excited about sharing their skills and networks in defense with you.
Meet John Dulin: The Founder Harnessing AI to Keep America Safe | by Isaac Hasson | Feb, 2023 | Medium
I think everyone agrees the future of defense is AI. However, we also believe the future of AI is defense.
Meet John Dulin: The Founder Harnessing AI to Keep America Safe | by Isaac Hasson | Feb, 2023 | Medium
One of my biggest takeaways was the degree to which national defense was truly a bipartisan issue. Democrats and Republicans both believe that technology can be used to protect our families, friends, and neighbors at home as well as members of our military and intelligence communities overseas.
Anduril & Defense Tech - by Elad Gil - Elad Blog
The takeaway is that serving a customer need well is often more important (and harder) to think about than defensibility. In many cases defensibility emerges over time - particularly if you build out a proprietary data set or become an ingrained workflow, or create defensibility via sales or other moats.
Defensibility & Competition - by Elad Gil - Elad Blog
“Respect each other” is translated into “find a way to include and agree with every person’s opinion”
The maze is in the mouse. What ails Google. And how it can turn… | by Praveen Seshadri | Feb, 2023 | Medium
almost everyone is working only for other Googlers, and the feedback loop is based on what your colleagues and managers think of your work.
The maze is in the mouse. What ails Google. And how it can turn… | by Praveen Seshadri | Feb, 2023 | Medium
The way I see it, Google has four core cultural problems. They are all the natural consequences of having a money-printing machine called “Ads” that has kept growing relentlessly every year, hiding all other sins. (1) no mission, (2) no urgency, (3) delusions of exceptionalism, (4) mismanagement.
The maze is in the mouse. What ails Google. And how it can turn… | by Praveen Seshadri | Feb, 2023 | Medium
In hyper vertical marketplaces, if you’re less vertical than someone else, then you lose to the more vertical player because their product-market fit is more perfect than yours.
LinkedIn is the New Craigslist. The Coming Hyper-verticalization of… | by Jeff Fluhr | Craft Ventures | Medium
Third, LinkedIn does not have industry-specific functionality that is highly valuable when matching workers with employers.
LinkedIn is the New Craigslist. The Coming Hyper-verticalization of… | by Jeff Fluhr | Craft Ventures | Medium
These new specialists may look quite different than LinkedIn — just as Airbnb and Tinder looked quite different than Craigslist — but the common denominator will be the hyper-verticalization of labor/service marketplaces.
LinkedIn is the New Craigslist. The Coming Hyper-verticalization of… | by Jeff Fluhr | Craft Ventures | Medium
Airbnb picked off “vacation rentals” and “shared rooms”. Tinder picked off “personals”. At StubHub, the company I founded, we picked off the “tickets” category. These companies all built valuable marketplaces by focusing on a specific vertical within Craigslist.
LinkedIn is the New Craigslist. The Coming Hyper-verticalization of… | by Jeff Fluhr | Craft Ventures | Medium
A communal softness has cloaked American professionalism – and it’s been staring us in the face the entire time.
GUBI - by Nick Simmons - Nick’s Substack
This optionality is derived from the self-supporting function that surrounds their operations, wherein there is a massive disparity between the utility needed to maintain that which is created.
GUBI - by Nick Simmons - Nick’s Substack
The necessity of work is now purely rooted in money (indirect), and leads to ‘work,’ as it is colloquially used, defined by salary, not activity.
GUBI - by Nick Simmons - Nick’s Substack
Something as simple and playful as “WebGL Water” gave Evan and Dylan the intuition that complex graphics in the browser were not far from reality.
Please Build More Silly Things - by Varun Shenoy
What the smartest people do on the weekends is what everyone else will do during the week in ten years.
Please Build More Silly Things - by Varun Shenoy
We want to make people very successful, making a great return [on their equity], that's great, as long as it's at a normal, reasonable level. If the full AGI thing breaks, we want something different for that paradigm.
Exclusive Interview: OpenAI’s Sam Altman Talks ChatGPT And How Artificial General Intelligence Can ‘Break Capitalism’
How the profits of AGI are shared, how access to is shared and how governance is distributed, those are three questions that are going to require new thinking.
Exclusive Interview: OpenAI’s Sam Altman Talks ChatGPT And How Artificial General Intelligence Can ‘Break Capitalism’
And the stuff that I'm excited about for these models is that it's not like, “Oh, how do you replace the experience of going on the web and typing in a search query,” but, “What do we do that is totally different and way cooler?’”
Exclusive Interview: OpenAI’s Sam Altman Talks ChatGPT And How Artificial General Intelligence Can ‘Break Capitalism’