Stephanie Y.
1 followers · 276 views
on the atlas · 10
- The Artificiality of Alignment - by jessica dai - Reboot7 savers
- Can We Stop Runaway A.I.? | The New Yorker1 savers
- Dan Doctoroff, Sidewalk Labs CEO, Reckons With His Legacy1 savers
- Uber’s CEO Says He’ll Always Find a Reason to Say His Company Sucks | WIRED1 savers
- Inside the AI Factory: The Humans That Make Tech Seem Human1 savers
- Can Anyone Fix California? | Vanity Fair1 savers
- To Break the 'Urban Doom Loop,' Build Housing and Transit - Bloomberg1 savers
- The Rise of the AI Engineer - by swyx - Latent Space4 savers
- Milan Cvitkovic52 savers
- Curius / Onboarding2704 savers
highlights · 88
about how we — as members of the general public — talk about, write about, think about AI
The Artificiality of Alignment - by jessica dai - Rebootframeworks for reasoning about what constitutes valid governance, about delegating decisions for the collective good, about what it means to truly participate in the public sphere when only some kinds of contributions are deemed legitimate by those in power. Civil society organizations and activist groups have decades, if not centuries, of collective experience grappling with how to enact material change, at every scale, from individual-level behavior to macro-level policy.
The Artificiality of Alignment - by jessica dai - RebootIn healthcare, algorithms could in theory improve clinician decisions, but the organizational structure that shapes AI deployment in practice is complex.
The Artificiality of Alignment - by jessica dai - RebootThere is a rich and nuanced discussion to be had about when and whether algorithms can be used to improve human decision-making, about how to measure the effect of algorithms on human decisions or evaluate the quality of their recommendations, and about what it actually means to improve human decision-making, in the first place.
The Artificiality of Alignment - by jessica dai - Rebootignores all of the other if conditions embedded in that sentence: if we decide to outsource reasoning about consequential decisions — about policy, business strategy, or individual lives — to algorithms. If we decide to give AI systems direct access to resources, and the power and agency to affect the allocation of those resources — the power grid, utilities, computation. All of the AI x-risk scenarios involve a world where we have decided to abdicate responsibility to an algorithm.
The Artificiality of Alignment - by jessica dai - RebootFiguring out how to “align” chatbots is a difficult problem, both technically and normatively. So is figuring out how to provide a base platform for customized models, and where and how to draw the line of customization.
The Artificiality of Alignment - by jessica dai - RebootRather than asking, “how do we create a chatbot that is good?”, these techniques merely ask, “how do we create a chatbot that sounds good”?
The Artificiality of Alignment - by jessica dai - RebootAlmost tautologically, OpenAI, Anthropic, and similar startups exist in order to dominate the marketplace of extremely powerful models in the future.
The Artificiality of Alignment - by jessica dai - Rebootthe existence of financial incentives means that alignment work often turns into product development in disguise rather than actually making progress on mitigating long-term harms
The Artificiality of Alignment - by jessica dai - RebootAll of these technical approaches — and, more broadly, the “intent alignment” framing — are deceptively convenient. Some limitations are obvious: a bad actor may have a “bad intent,”
The Artificiality of Alignment - by jessica dai - RebootThe preference model itself is built through an iterative process of pairwise comparisons
The Artificiality of Alignment - by jessica dai - RebootIdeally, we’d be able to ask all 8 billion people on earth how they feel about all the possible outputs of the base model; in practice, we instead train an additional machine learning model that predicts human preferences. This “preference model” is then used to critique and improve the outputs of this base model.
The Artificiality of Alignment - by jessica dai - RebootThe key goal in this line of work is to develop a model of human preferences, and use them to improve a base “unaligned” model.
The Artificiality of Alignment - by jessica dai - Rebootindividual researchers motivated by x-risk. This community has developed an extensive vocabulary around theories of AI safety and alignment
The Artificiality of Alignment - by jessica dai - RebootSo OpenAI and Anthropic might be trying to conduct research, push the technical envelope, and possibly even build superintelligence, but they’re undeniably also building products — products that carry liability, products that need to sell, products that need to be designed such that they claim and maintain market share.
The Artificiality of Alignment - by jessica dai - RebootIf A.G.I. does develop, he argues, then it’s likely to happen in multiple places around the same time. The systems would then be put to economic use by the companies or organizations that developed them. The market would curtail their powers; investors, wanting to see their companies succeed, would go slow and add safety features.
Can We Stop Runaway A.I.? | The New Yorkerwe worry too much about the singularity.
Can We Stop Runaway A.I.? | The New Yorkerin 2021, a hundred and ninety-three countries adopted a Recommendation on the Ethics of Artificial Intelligence, created by the United Nations Educational, Scientific, and Cultural Organization (unesco). The recommendations focus on data protection, mass surveillance, and resource efficiency (but not computer superintelligence).
Can We Stop Runaway A.I.? | The New YorkerHe doubts that we’ll recognize the approach of superhuman A.I. before it’s too late.
Can We Stop Runaway A.I.? | The New YorkerAn “evolving” or self-programming A.I. might invent a similar method and hide its weak points or its capabilities from auditors or even its creators, evading detection.
Can We Stop Runaway A.I.? | The New YorkerAnd yet tracking and forecasting progress toward A.G.I. or superintelligence is complicated by the fact that key steps may occur in the dark.
Can We Stop Runaway A.I.? | The New YorkerBut, for the most part, regulations haven’t targeted the research and development of A.I. Even if they did, it’s not clear that we’d know when to tap the brakes. We may not know when we’re nearing a cliff until it’s too late.
Can We Stop Runaway A.I.? | The New YorkerA growing area of research called A.I. alignment seeks to lessen the danger by insuring that computer systems are “aligned” with human goals.
Can We Stop Runaway A.I.? | The New Yorkerto prevent the development of “nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us”
Can We Stop Runaway A.I.? | The New YorkerHe answers with a highly diplomatic tautology. “There’s vision, but what really needs to happen is you have to execute.” I take him to mean that the Adams administration has appointed smart, clear-thinking people, including veterans of various Doctoroff enterprises. What’s missing is anyone with his maniacal focus, autonomy, and clout.
Dan Doctoroff, Sidewalk Labs CEO, Reckons With His LegacyTwo terms of the de Blasio administration failed to solve many of the problems it hoped to: Rents kept climbing, the homeless population grew, and big projects stalled.
Dan Doctoroff, Sidewalk Labs CEO, Reckons With His Legacythe effort was doomed by a fundamental contradiction between the needs of a democratic society and those of a corporation intent on gathering proprietary data. Maybe it was just local developers and a homegrown tech industry closing ranks against an interloper. Possibly it was another instance of caricature being more effective than nuance.
Dan Doctoroff, Sidewalk Labs CEO, Reckons With His LegacyThe Bloomberg administration is easy to caricature as a club of petulant and self-regarding overachievers who left the city shiny on top, rotten below, and overpriced throughout. It was criticized for tossing the keys to developers and corporations in a neoliberal power grab, even though, like a gang of old-fashioned lefties, it also hugely increased government spending.
Dan Doctoroff, Sidewalk Labs CEO, Reckons With His LegacyIn the new regime, sustainability got linked to density and affordable housing, which brought bike lanes, street trees, waterfront access, and outdoor seating. Private interests had to pitch in to enrich the public realm.
Dan Doctoroff, Sidewalk Labs CEO, Reckons With His LegacyDense cities like New York are environmentally both good and bad
Dan Doctoroff, Sidewalk Labs CEO, Reckons With His LegacyHe’s constantly making his case, striving to win interlocutors over through some combination of charm, insistence, logic, and data — always the data.
Dan Doctoroff, Sidewalk Labs CEO, Reckons With His LegacyDoctoroff had no experience in international sports, urban management, transportation, event planning, or anything even tangentially related to the issue. But he did have money, connections, a sudden passion, and a limitless capacity for homework.
Dan Doctoroff, Sidewalk Labs CEO, Reckons With His LegacyYet for someone who plotted out his own life and New York’s trajectory in multi-decade chunks, coping with a short time horizon has meant redefining the principle that shaped him: the future.
Dan Doctoroff, Sidewalk Labs CEO, Reckons With His LegacyUltimately, if you look at anything that moves in a city, we want to wire it up on demand.
Uber’s CEO Says He’ll Always Find a Reason to Say His Company Sucks | WIREDWe’re building a network that connects riders and eaters with drivers and couriers. If those drivers and couriers happen to be robots, and they’re safe and they’re effective, we will welcome them to the network.
Uber’s CEO Says He’ll Always Find a Reason to Say His Company Sucks | WIREDWill we ever get a clear definition of what’s fair for your drivers?
Uber’s CEO Says He’ll Always Find a Reason to Say His Company Sucks | WIREDFor every occasion in which you might want to use your own car, we’re building an on-demand solution.
Uber’s CEO Says He’ll Always Find a Reason to Say His Company Sucks | WIREDIn both mobility and delivery, the gig-worker model isn’t proven.
Uber’s CEO Says He’ll Always Find a Reason to Say His Company Sucks | WIREDAnd a lot of the safety features we introduced hurt our growth. We had to make trade-offs.
Uber’s CEO Says He’ll Always Find a Reason to Say His Company Sucks | WIREDOne way the AI industry differs from manufacturers of phones and cars is in its fluidity. The work is constantly changing, constantly getting automated away and replaced with new needs for new types of data. It’s an assembly line but one that can be endlessly and instantly reconfigured, moving to wherever there is the right combination of skills, bandwidth, and wages.
Inside the AI Factory: The Humans That Make Tech Seem Human“I think you always need a human to monitor what AIs are doing just because they are this kind of alien entity,” Chen said. Machine-learning systems are just too strange ever to fully trust.
Inside the AI Factory: The Humans That Make Tech Seem HumanChen is skeptical AI will reach a point where human feedback is no longer needed, but he does see annotation becoming more difficult as models improve. Like many researchers, he believes the path forward will involve AI systems helping humans oversee other AI.
Inside the AI Factory: The Humans That Make Tech Seem HumanThe new models are so impressive they’ve inspired another round of predictions that annotation is about to be automated. Given the costs involved, there is significant financial pressure to do so.
Inside the AI Factory: The Humans That Make Tech Seem HumanHaving fewer, better-trained workers producing higher-quality data allows Surge to compensate better than its peers, Chen said, though he declined to elaborate, saying only that people are paid “fair and ethical wages.”
Inside the AI Factory: The Humans That Make Tech Seem HumanThe annotation landscape needs to shift from this low-quality, low-skill mind-set to something that’s much richer and captures the range of human skills and creativity and values that we want AI systems to possess.”
Inside the AI Factory: The Humans That Make Tech Seem HumanOpenAI, Microsoft, Meta, and Anthropic did not comment about how many people contribute annotations to their models, how much they are paid, or where in the world they are located.
Inside the AI Factory: The Humans That Make Tech Seem Humanthe company’s researchers hold weekly annotation meetings in which they rerate data themselves and discuss ambiguous cases, consulting with ethical or subject-matter experts when a case is particularly tricky.
Inside the AI Factory: The Humans That Make Tech Seem HumanRanking a language model’s responses is always going to be somewhat subjective because it’s language. A text of any length will have multiple elements that could be right or wrong or, taken together, misleading.
Inside the AI Factory: The Humans That Make Tech Seem HumanThis circuitous technique is called “reinforcement learning from human feedback,” or RLHF, and it’s so effective that it’s worth pausing to fully register what it doesn’t do.
Inside the AI Factory: The Humans That Make Tech Seem HumanThe point is that they are creating data on human taste, and once there’s enough of it, engineers can train a second model to mimic their preferences at scale, automating the ranking process and training their AI to act in ways humans approve of.
Inside the AI Factory: The Humans That Make Tech Seem Human