[2007.05558] The Computational Limits of Deep Learning
paperAuthors
Credibility Rating
Good quality. Reputable source with community review or editorial standards, but less rigorous than peer-reviewed venues.
Rating inherited from publication venue: arXiv
Foundational analysis examining the computational scaling requirements of deep learning progress, arguing that current trends are economically and environmentally unsustainable, which directly informs discussions of AI capabilities development and resource constraints relevant to AI safety planning.
Paper Details
Metadata
Summary
This paper by Thompson et al. documents deep learning's heavy dependence on computational power for recent progress across applications like Go, image classification, and translation. The authors demonstrate that progress across diverse domains is strongly correlated with increases in computing resources and argue that extrapolating current trends reveals this reliance is becoming economically, technically, and environmentally unsustainable. They conclude that continued progress requires either dramatically more computationally-efficient deep learning methods or a shift toward alternative machine learning approaches.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Deep Learning Revolution Era | Historical | 44.0 |
Cached Content Preview
[2007.05558] The Computational Limits of Deep Learning
-->
Computer Science > Machine Learning
arXiv:2007.05558 (cs)
[Submitted on 10 Jul 2020 ( v1 ), last revised 27 Jul 2022 (this version, v2)]
Title: The Computational Limits of Deep Learning
Authors: Neil C. Thompson , Kristjan Greenewald , Keeheon Lee , Gabriel F. Manso View a PDF of the paper titled The Computational Limits of Deep Learning, by Neil C. Thompson and 3 other authors
View PDF
Abstract: Deep learning's recent history has been one of achievement: from triumphing over humans in the game of Go to world-leading performance in image classification, voice recognition, translation, and other tasks. But this progress has come with a voracious appetite for computing power. This article catalogs the extent of this dependency, showing that progress across a wide variety of applications is strongly reliant on increases in computing power. Extrapolating forward this reliance reveals that progress along current lines is rapidly becoming economically, technically, and environmentally unsustainable. Thus, continued progress in these applications will require dramatically more computationally-efficient methods, which will either have to come from changes to deep learning or from moving to other machine learning methods.
Comments:
33 pages, 8 figures
Subjects:
Machine Learning (cs.LG) ; Machine Learning (stat.ML)
Cite as:
arXiv:2007.05558 [cs.LG]
(or
arXiv:2007.05558v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2007.05558
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: Neil Thompson [ view email ]
[v1]
Fri, 10 Jul 2020 18:26:17 UTC (1,871 KB)
[v2]
Wed, 27 Jul 2022 17:26:18 UTC (1,717 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled The Computational Limits of Deep Learning, by Neil C. Thompson and 3 other authors View PDF
TeX Source
view license
Current browse context: cs.LG
< prev
|
next >
new
|
recent
| 2020-07
Change to browse by:
cs
stat
stat.ML
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
3 blog links
( what is this? )
DBLP - CS Bibliography
listing | bibtex
Kristjan H. Greenewald
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer ( What is the Explorer? )
Connected Papers Toggle
Connected Papers ( What is Connected Papers? )
Litmaps Toggle
Litmaps ( What is Litmaps? )
... (truncated, 5 KB total)676a61fd3e474b32 | Stable ID: MTYyMzBmZT