LLMs shouldn’t write SQL - by Benn Stancil - benn.substack
[ People can ] work like journalists, collating existing metrics and drawing conclusions by considering them in their totality. Rather than looking for new ways to assess a question, they start by asking, “how do we currently measure that?” [ Or they can ] work like scientists, creating new datasets and aggregating them in novel ways to draw conclusions about specific, nuanced hypotheses. In business contexts, the first type of analysis typically consists of asking a series of Mad Lib-style questions about known metrics, like revenue retention or daily active users or ad spend, and aggregating and filtering those metrics by different dimensions. “Show me total orders in Massachusetts by month compared to the same number from a year ago,” someone might ask. This question will reveal something, like a spike in new orders in February. And that will prompt more questions of the same style—“now show me total orders in Massachusetts by month and by product category”—until people find whateve
LLMs shouldn’t write SQL There's no direct path from a business question to a useful query. Feb 23, 2024 52 23 5 Share There are, very roughly, two ways to analyze data : 1 [ People can ] work like journalists, collating existing metrics and drawing conclusions by considering them in their totality. Rather than looking for new ways to assess a question, they start by asking, “how do we currently measure that?” [ Or they can ] work like scientists, creating new datasets and aggregating them in novel ways to draw conclusions about specific, nuanced hypotheses. In business contexts, the first typ
saved by
related reading
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- Natural Language Is an Unnatural Interfacevarunshenoy.substack.com
- Malleable software in the age of LLMsgeoffreylitt.com
- Writing for LLMs So They Listen · Gwern.netgwern.net
- Taking LLMs Seriously (As Language Models) — LessWronglesswrong.com
- AI startups: Sell work, not softwaresarahtavel.com
- Thoughts on AI in academiatheinfinitesimal.substack.com
- Here’s how I use LLMs to help me write codesimonwillison.net
- Putting Task Expertise into RL Achieves State-of-the-Art Performance on Text-to-SQLthinkingmachines.ai
- Lessons from Human Data Analysts to Improve the AI Varietydatafordoers.substack.com
- The biggest bottleneck for large language model startups is UX | by Davis Treybig | Innovation Endeavors | Mediummedium.com
- How I Use "AI"nicholas.carlini.com