flâneur

What Building "Copilot for X" Really Takes

latent.space · 1,600 words · saved by 1 readers

Note from swyx: I’m delighted to present our first guest post! This comes from the Codeium team, who I’ve had the good fortune of getting to know in the past year. This small team blew away my expectations by creating a complete Copilot clone in one month (not exaggerating - I have disabled GitHub Copilot completely to use them, in part to beta-test Hey Github, but also Codeium seems a bit faster!). Since “Copilot for X” is at the top of every AI product thinker’s wishlist (example), I invited them to share their learnings! TL;DR: To build a “Copilot for X”, you must: Estimate inference scale: while trading off latency and output quality Build first party: Third-party APIs will wreck your unit economics Figure out Realtime Infra: ChatGPT, Dall-E, etc are too slow to keep flow Optimize prompts: Context window is limited; what info most improves output? Merge model outputs and UX: Raw model output is unintuitive; long tail of UX. Eventually, you will want to go beyond “Copilot for X” bec

Note from swyx: I’m delighted to present our first guest post! This comes from the Codeium team, who I’ve had the good fortune of getting to know in the past year. This small team blew away my expectations by creating a complete Copilot clone in one month (not exaggerating - I have disabled GitHub Copilot completely to use them, in part to beta-test Hey Github, but also Codeium seems a bit faster!). Since “Copilot for X” is at the top of every AI product thinker’s wishlist (example), I invited them to share their learnings! TL;DR: To build a “Copilot for X”, you must: Estimate inference…

related reading