Emergent Abilities in Large Language Models: An Explainer

[Skip to main content](https://cset.georgetown.edu/article/emergent-abilities-in-large-language-models-an-explainer/#main)

### CSET

# Emergent Abilities in Large Language Models: An Explainer

Thomas Woodside

April 16, 2024

A recent topic of contention among artificial intelligence researchers has been whether large language models can exhibit unpredictable ("emergent") jumps in capability as they are scaled up. These arguments have found their way into policy circles and the popular press, often in simplified or distorted ways that have created confusion. This blog post explores the disagreements around emergence and their practical relevance for policy.

#### Summary

Claims of “emergent capabilities” in large language models (LLMs) have been discussed by [journalists](https://www.wired.com/story/how-quickly-do-large-language-models-learn-unexpected-skills/), [researchers](https://arxiv.org/abs/2206.07682), and even [members of Congress](https://republicans-science.house.gov/_cache/files/8/a/8a9f893d-858a-419f-9904-52163f22be71/191E586AF744B32E6831A248CD7F4D41.2023-12-14-aisi-scientific-merit-final-signed.pdf). But these conversations have often been hampered by confusion about what exactly the term means and what it implies.

The idea of “emergence” comes from the study of complex systems. It describes systems that cannot be explained simply by looking at their parts, such as complex social networks. The field of deep learning, including LLMs, inherently involves emergence, since the internal properties of neural networks are difficult to predict simply by looking at their millions or billions of individual parameters.

A related—but distinct—definition of emergence has become common in the context of LLMs. According to this definition, emergence refers to the capabilities of LLMs that appear suddenly and unpredictably as model size, computational power, and training data scale up. Researchers have found many examples of such “emergent capabilities.” The topic has garnered attention because of the potential for the unpredictable emergence of risky capabilities (e.g., effective autonomous hacking) in the future. It has also been exaggerated in the popular press.

Some researchers dispute that LLMs exhibit sudden and unpredictable changes in their capabilities as they increase in scale. They have devised alternative metrics to measure the same capabilities that show smooth increases with scale and could potentially help predict capability increases into the future. However, devising such metrics is often not straightforward, especially for complex tasks, making it unclear how reliable this approach will be for making predictions.

While there are some methods for estimating the capabilities of LLMs before they are trained—and these will hopefully improve so that researchers can predict even seemingly sudden jumps in capabilities—researcher and policymaker ability to predict the future of LLMs is far from assured. Previous attempts to predict

... (truncated, 27 KB total)