Intuition on AI Scale for Biologists - by Jason Steiner

The news has been awash with mega funding rounds for AI companies. It is not uncommon to see 9 figure funding for startup and billion dollar partnerships are being struck with the leading companies like Anthropic and OpenAI. Inflection AI has said they have/will be spending $1B on a compute cluster1. GPT-4 or versions of Google Gemini have been cited to cost over $100M in training costs alone and have been reported to have over 1T parameters at the least2. Dario Amodei has stated that it is well within expectation to be training models that are 10 or 100 times larger within the next few years. In the life sciences, the company EvolutionaryScale, started by an ex-Meta team that built the ESM protein folding model have been reported to plan on spending (provided they can continue to raise capital) several hundred million dollars on compute costs over the next few years to test scaling laws in biology3. Perhaps the biggest outstanding question is what will happen? How much better can thin

The news has been awash with mega funding rounds for AI companies. It is not uncommon to see 9 figure funding for startup and billion dollar partnerships are being struck with the leading companies like Anthropic and OpenAI. Inflection AI has said they have/will be spending $1B on a compute cluster1. GPT-4 or versions of Google Gemini have been cited to cost over $100M in training costs alone and have been reported to have over 1T parameters at the least2. Dario Amodei has stated that it is well within expectation to be training models that are 10 or 100 times larger within the next few years.