Here comes the Muybridge camera moment but for text. Photoshop too (Interconnected)
What I like to do (and what I also do for clients) is to string together weak signals and see where it takes me. I get to new places when I think out loud. The process is… meandering. And technical. And lengthy. So feel free to skip to the tl;dr at the bottom if you want to know where I end up. There’s an AI-adjacent technique called “embeddings.” A word, or a phrase, or a paragraph is mathematically converted into coordinates. Just like a location on a map is described by lat and long. Only the “map” in this case is a map of concepts. So if two phrases mean roughly the same thing, their coordinates are close together. If they mean different things, they’re further away. Simon Willison has a great deep-dive into embeddings (2023). But let me give you an example so you can get a feel for this… I built an embeddings-powered search engine for my unofficial BBC In Our Time archive site, Braggoscope. There are a 1,000 episodes on all kinds of cultural and historical topics, so it’s a good c
Here comes the Muybridge camera moment but for text. Photoshop too 10.49, Friday 31 May 2024 Link to this post Can you measure the velocity of concepts over a piece of text, e.g. 0.5 concepts/word? Yes. Or rather, well, something like that, possibly one day soon, it’s interesting. I want to unpack that thought. Hey, an editorial note: This post is for me, not for you haha What I like to do (and what I also do for clients) is to string together weak signals and see where it takes me. I get to new places when I think out loud. The process is… meandering. And technical. And lengthy. So feel free
Explore this link on the map →saved by
related reading
- Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
- Prism: mapping interpretable concepts and features in a latent space of language | thesephist.comthesephist.com
- Embeddings: What they are and why they mattersimonwillison.net
- Introducing text and code embeddings | OpenAIopenai.com
- Quantifying Truesight With SAEs · Gwern.netgwern.net
- An intuitive introduction to text embeddings - Stack Overflowstackoverflow.blog
- An intuitive introduction to text embeddings - Stack Overflowstackoverflow.blog
- TextFXtextfx.withgoogle.com
- Vector embeddings | OpenAI APIdevelopers.openai.com
- An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversiontextual-inversion.github.io
- Gwern visits BAIR – Yuxi on the Wiredyuxi.ml
- Getting creative with embeddingswattenberger.com