flâneur

Do the Returns to Software R&D Point Towards a Singularity? – Epoch AI

epochai.org · 2,804 words · saved by 1 readers

The returns to R&D are crucial in determining the dynamics of growth and potentially the pace of AI development. Our new paper offers new empirical techniques and estimates for this crucial parameter. Improvements in AI have predominantly been driven by two factors. First, advancements in hardware performance and substantial investments in larger clusters have increased the computing power for training AI models. This has resulted in improved performance given the abundance of data that we can use to train larger AI systems. Second, progress on the “software" side (training techniques, architectures, algorithm implementations, etc.) has resulted in the compute being used more efficiently (see our work on algorithmic progress). This means that AI model performance surpasses what we’d expect from merely increasing computing resources. At Epoch, we have extensively researched each of these trends. The combination of the scaling of compute and improvements in training techniques has effect

Do the returns to software R&D point towards a singularity? | Epoch AI Returns to R&D and hyperbolic technological progress Improvements in AI have predominantly been driven by two factors. First, advancements in hardware performance and substantial investments in larger clusters have increased the computing power for training AI models. This has resulted in improved performance given the abundance of data that we can use to train larger AI systems. Second, progress on the “software” side (training techniques, architectures, algorithm implementations, etc.) has resulted in the compute being us

related reading