Clouded Judgement 9.4.26 - Recurrent Depth - by Jamin Ball
cloudedjudgement.substack.com · 2,350 words · saved by 1 readers
Every week I’ll provide updates on the latest trends in AI and software
Every week I’ll provide updates on the latest trends in cloud software companies. Follow along to stay up to date! Recurrent Depth This week we got a peak into OpenAIs new model Astra. I thought this was an important model because it’s rumored to use a new technique called recurrent depth. In this post I want to discuss what that means and what the implications are. But first - a quick timeline of some of the larger breakthroughs in model architecture: 2017: The Transformer era. The leap here was self-attention mechanism. Before 2017 recurrent neural networks (RNNs) processed text…
saved by
related reading
- My picture of the present in AI — LessWronglesswrong.com
- The Short Case for Nvidia Stock | YouTube Transcript Optimizeryoutubetranscriptoptimizer.com
- (Some of) The Models, They Just Don't Want to Learn | Tildeblog.tilderesearch.com
- Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
- As Rocks May Think | Eric Jangevjang.com
- The Big LLM Architecture Comparisonmagazine.sebastianraschka.com
- ali (@waterloo_intern) on Xx.com
- The Decade of Deep Learning | Leo Gaobmk.sh
- Non_Interactive – Software & MLnonint.com
- What I've Learned About AI in the Past Two Months.sheracaolity.ghost.io
- AINews | AINewsnews.smol.ai
- trees are harlequins, words are harlequins - I don't think you're drawing the right lesson from...nostalgebraist.tumblr.com