Get the hell out of the LLM as soon as possible | sgnt.ai

Don’t let an LLM make decisions or execute business logic: they suck at that. I build NPCs for an online game, and I get asked a lot “How did you get ChatGPT to do that?” The answer is invariably: “I didn’t, and also you shouldn’t”. In most applications, the LLM should be the user-interface only between the user and an API into your application logic. The LLM shouldn’t be executing any logic. Get the hell out of the LLM as soon as possible, and stay out as long as you can. This is best illustrated by a contrived example: you want to write a chess-playing bot you access over WhatsApp. The user sends a description of what they want to do (“use my bishop to take the knight”), and the bot plays against them. Could you get the LLM to be in charge of maintaining the state of the chess board and playing convincingly? Possibly, maybe. Would you? Hell no, for some intuitive reasons: The chess example illustrates the fundamental problem with using LLMs for core application logic, but this princi

Don’t let an LLM make decisions or execute business logic: they suck at that. I build NPCs for an online game, and I get asked a lot “How did you get ChatGPT to do that?” The answer is invariably: “I didn’t, and also you shouldn’t”. In most applications, the LLM should be the user-interface only between the user and an API into your application logic. The LLM shouldn’t be executing any logic. Get the hell out of the LLM as soon as possible, and stay out as long as you can. Y Tho? This is best illustrated by a contrived example: you want to write a chess-playing bot you access over WhatsApp. Th

saved by

• Jordan Sucher

related reading

• LLM Powered Autonomous Agents | Lil'Loglilianweng.github.io
• Malleable software in the age of LLMsgeoffreylitt.com
• Guardian Angels: LLM Personalization for Productivity and Security · Gwern.netgwern.net
• LLM Chess: Benchmarking Reasoning and Instruction-Following in LLMs through Chessarxiv.org
• Why Chatbots Are Not the Future of Interfaceswattenberger.com
• Against LLM Reductionism — LessWronglesswrong.com
• Building LLM applications for productionhuyenchip.com
• Building Effective AI Agents \ Anthropicanthropic.com
• The Future of Everything is Lies, I Guessaphyr.com
• What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
• What We Learned from a Year of Building with LLMs (Part I) – O’Reillyoreilly.com
• LLMs Are Not All You Need | Pineconepinecone.io