Back
How long before superintelligence?
webnickbostrom.com·nickbostrom.com/superintelligence
An early and influential paper by Nick Bostrom that helped establish superintelligence as a serious academic topic; predates his book 'Superintelligence' and is useful for understanding the historical development of AI safety concerns.
Metadata
Importance: 72/100working paperprimary source
Summary
An early analysis by Nick Bostrom examining timelines and pathways to superintelligent AI, exploring mechanisms such as whole brain emulation, recursive self-improvement, and collective intelligence. The paper argues that superintelligence could arrive within decades and that its implications deserve serious philosophical and strategic attention. It serves as a foundational text in the academic discourse on transformative AI risk.
Key Points
- •Outlines multiple distinct pathways to superintelligence including brain emulation, biological enhancement, and AI-driven recursive self-improvement.
- •Argues that the transition to superintelligence may happen rapidly once a sufficient threshold of capability is reached.
- •Emphasizes that superintelligence would represent a qualitative leap beyond human cognition, not merely incremental improvement.
- •Raises early concerns about the difficulty of controlling or aligning a superintelligent system with human values.
- •Situates the superintelligence question within broader existential risk discourse, calling for proactive research and preparation.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Self-Improvement and Recursive Enhancement | Capability | 69.0 |
Cached Content Preview
HTTP 200Fetched Mar 15, 202649 KB
How long before superintelligence?
HOW
LONG BEFORE SUPERINTELLIGENCE?
(1997) Copyright
[Revised 25 October, 1998, and a postscript
added]
[Second postscript added 28 August
2000]
[Third postscript added 30 October
2005]
[Fourth postscript added 12 March 2008]
Nick Bostrom
Oxfrord Future of Humanity Institute
Faculty of Philosophy & Oxford Martin School
University of Oxford
https://nickbostrom.com
[Originally published in Int. Jour.
of Future Studies , 1998, vol. 2]
[Reprinted in Linguistic and
Philosophical Investigations, 2006, Vol. 5, No. 1, pp. 11-30.]
Abstract
This paper outlines the case for believing that we will
have superhuman artificial intelligence within the first third of the next
century. It looks at different estimates of the processing power of the
human brain; how long it will take until computer hardware achieve a similar
performance; ways of creating the software through bottom-up approaches
like the one used by biological brains; how difficult it will be for neuroscience
figure out enough about how brains work to make this approach work; and
how fast we can expect superintelligence to be developed once there is human-level
artificial intelligence.
Definition of "superintelligence"
By a "superintelligence" we mean an intellect
that is much smarter than the best human brains in practically every field, including
scientific creativity, general wisdom and social skills. This definition leaves open how
the superintelligence is implemented: it could be a digital computer , an ensemble
of networked computers, cultured cortical tissue or what have you. It also leaves open
whether the superintelligence is conscious and has subjective experiences.
Entities such as companies or the scientific community are not
superintelligences according to this definition. Although they can perform a number of
tasks of which no individual human is capable, they are not intellects and there are many
fields in which they perform much worse than a human brain - for example, you can't have
real-time conversation with "the scientific community".
Moore's law and present supercomputers
Moore's law <A> states that processor speed doubles every eighteen
months. The doubling time used to be two years, but that changed about fifteen years ago.
The most recent data points indicate a doubling time as short as twelve months. This would
mean that there will be a thousand-fold increase in computational power in ten years.
Moore's law is what chip manufacturers rely on when they decide what sort of chip to
develop in order to remain competitive.
If we estimate the computational capacity of the human brain, and allow
ourselves to extrapolate available processor speed according to Moore's law (
... (truncated, 49 KB total)Resource ID:
02dc187c15139435 | Stable ID: OWEzZjZiZD