Compute trend analysis

Summary

This Epoch AI analysis tracks historical trends in the monetary cost of training machine learning systems, examining how dollar costs have evolved alongside compute scaling. It provides empirical data on training cost trajectories to inform forecasts about future AI development economics and accessibility.

Key Points

• •Training costs for frontier ML models have grown dramatically over time, with top models now requiring billions of dollars in compute expenditure.
• •Despite hardware efficiency improvements reducing cost-per-FLOP, total training costs have risen due to exponentially increasing model scale.
• •Cost trends have implications for who can develop frontier AI systems, concentrating capabilities among well-resourced actors.
• •Historical data enables forecasting of future training costs, relevant for governance and safety planning timelines.
• •Declining per-unit compute costs mean capable models become more accessible over time, raising proliferation concerns.