Neel Nanda

Quick Assessment

Overview

Neel NandaPersonNeel NandaComprehensive biographical profile of Neel Nanda covering his role as DeepMind's mechanistic interpretability team lead, key contributions (TransformerLens, Gemma Scope, grokking paper), and his ev...Quality: 26/100 is a Senior Research Scientist and Mechanistic Interpretability Team Lead at Google DeepMindOrganizationGoogle DeepMindComprehensive overview of DeepMind's history, achievements (AlphaGo, AlphaFold with 200M+ protein structures), and 2023 merger with Google Brain. Documents racing dynamics with OpenAI and new Front...Quality: 37/100, where he leads research into reverse-engineering neural networks to understand how they implement algorithms. He studied Mathematics at Trinity College, Cambridge (2017–2020), then worked at AnthropicOrganizationAnthropicComprehensive reference page on Anthropic covering financials ($380B valuation, $14B ARR at Series G growing to $19B by March 2026), safety research (Constitutional AI, mechanistic interpretability...Quality: 74/100 as a language model interpretability researcher under Chris OlahPersonChris OlahComprehensive biographical profile of Chris Olah covering his unconventional career path, foundational contributions to mechanistic interpretability (feature visualization, circuit analysis, sparse...Quality: 27/100 (2021–2022), before joining DeepMind's mechanistic interpretability team in 2023.¹²

He is the creator of TransformerLens, an open-source library that has become a widely used tool in the interpretability research community, and a co-author of "A Mathematical Framework for Transformer Circuits," an influential Anthropic paper introducing a mathematical vocabulary for analyzing transformer behavior. His team at DeepMind produced Gemma Scope, a collection of over 400 openly released sparse autoencoders for Gemma 2 models.³ As of 2025, Nanda had accrued over 13,000 citations on Google Scholar, an h-index of 31, and had co-authored papers appearing at NeurIPS, ICLR, and ICML.⁴⁵

Nanda has also been active in training the next generation of interpretability researchers, having mentored approximately 50 junior researchers through programs including SERI MATS, with 15 co-authored papers published at top ML venues and 7 mentees subsequently placed at major AI companies.⁶

Background

Neel NandaPersonNeel NandaComprehensive biographical profile of Neel Nanda covering his role as DeepMind's mechanistic interpretability team lead, key contributions (TransformerLens, Gemma Scope, grokking paper), and his ev...Quality: 26/100 studied Mathematics at Trinity College, Cambridge from 2017 to 2020.¹ Following graduation, he joined AnthropicOrganizationAnthropicComprehensive reference page on Anthropic covering financials ($380B valuation, $14B ARR at Series G growing to $19B by March 2026), safety research (Constitutional AI, mechanistic interpretability...Quality: 74/100 in 2021 as a researcher in the interpretability team, working under Chris OlahPersonChris OlahComprehensive biographical profile of Chris Olah covering his unconventional career path, foundational contributions to mechanistic interpretability (feature visualization, circuit analysis, sparse...Quality: 27/100 on language model interpretability.² He left Anthropic in 2022 and did a period of independent mechanistic interpretability research before joining Google DeepMind's mechanistic interpretability team in 2023.¹²

At DeepMind, Nanda joined expecting to be an individual contributor. When the prior team lead stepped down, he took on the role of team lead despite having no prior management experience.⁷⁸ As of 2025, he holds the title of Senior Research Scientist and Mechanistic Interpretability Team Lead.⁴⁷

Nanda maintains an active presence in the AI alignment research community through LessWrongOrganizationLessWrongLessWrong is a rationality-focused community blog founded in 2009 that has influenced AI safety discourse, receiving $5M+ in funding and serving as the origin point for ~31% of EA survey respondent...Quality: 44/100 and the Alignment Forum, where he publishes tutorials and research updates.

Research Contributions

TransformerLens Library

Nanda created TransformerLens, an open-source Python library for interpretabilityResearch AreaInterpretabilityMechanistic interpretability has extracted 34M+ interpretable features from Claude 3 Sonnet with 90% automated labeling accuracy and demonstrated 75-85% success in causal validation, though less th...Quality: 66/100 research on transformer models.⁹ The library was originally created in 2022 and is now maintained by a broader contributor community under the TransformerLensOrg GitHub organization, with Bryce Meyer as a key maintainer.¹⁰

As of 2025, the repository has approximately 3,100 GitHub stars, 513 forks, and 112 contributors, with 796 closed pull requests.¹⁰ The library provides:

• Programmatic access to model activations at each layer
• Hook functions for intervention experiments
• Support for loading 50+ open-source language models from Hugging Face
• Visualization utilities for attention patterns

The library is formally cited as: Nanda, N. & Bloom, J. (2022). TransformerLens.¹⁰ A companion CircuitsVis repository provides additional visualization tools and has accrued 297 stars and 40 forks.¹¹ Version 2.0 removed Hooked SAE functionality, which was migrated to the separate SAELens library.¹⁰

Alternative interpretability libraries exist (e.g., baukit, pyvene, nnsight), though TransformerLens's hook-based API and extensive model support have made it a common choice for circuit-analysis research.

Transformer Circuits Research

Nanda is a co-author of "A Mathematical Framework for Transformer Circuits" (2021), published by the Anthropic interpretability team.¹² The paper lists Nelson ElhagePersonNelson ElhageResearch Scientist at Anthropic. Key contributor to Anthropic's interpretability research including 'Toy Models of Superposition'., Neel NandaPersonNeel NandaComprehensive biographical profile of Neel Nanda covering his role as DeepMind's mechanistic interpretability team lead, key contributions (TransformerLens, Gemma Scope, grokking paper), and his ev...Quality: 26/100, and Catherine OlssonPersonCatherine OlssonAnthropic co-founder. Previously at Google Brain and OpenAI. Focused on interpretability research and AI safety. as core research contributors, with additional contributors including Tom HenighanPersonTom HenighanAnthropic co-founder. Previously at OpenAI where he worked on scaling laws research. Key technical contributor., Nicholas Joseph, Ben MannPersonBen MannAnthropic co-founder. Previously at Google. Key technical contributor to Anthropic's infrastructure., and others; Chris Olah handled correspondence.¹³ Nanda has described it as "the coolest paper I've ever had the privilege of working on" and "the most awesome paper I've been involved in."¹⁴

The research:

• Introduced a mathematically equivalent framework for understanding transformer operations
• Identified "induction heads" as circuits that enable in-context learning
• Demonstrated that attention mechanisms compose to perform multi-step reasoning
• Showed that induction heads only develop in models with at least two attention layers
• Provided mathematical descriptions of how models track positional and semantic information

The paper built on earlier work from AnthropicOrganizationAnthropicComprehensive reference page on Anthropic covering financials ($380B valuation, $14B ARR at Series G growing to $19B by March 2026), safety research (Constitutional AI, mechanistic interpretability...Quality: 74/100's interpretability team examining circuits in vision models.

Grokking and Modular Arithmetic

Nanda co-authored "Progress Measures for Grokking via Mechanistic Interpretability" (January 2023, with Lawrence ChanPersonLawrence ChanAI safety researcher. Previously at MIRI and Redwood Research. Works on alignment and interpretability., Tom Lieberum, Jess Smith, and Jacob SteinhardtPersonJacob SteinhardtJacob Steinhardt is an assistant professor of statistics at UC Berkeley. His research focuses on making machine learning systems more reliable and safe, including work on distribution shift, advers...), which received an ICLR 2023 Spotlight designation.¹⁵ The paper investigates the "grokking" phenomenon—a delayed generalization observed in small transformers trained on modular arithmetic—and fully reverse-engineers the algorithm learned, using discrete Fourier transforms and trigonometric identities.¹⁵

Additional Research Areas

Nanda has published work on:

• Indirect Object Identification: The paper "Interpretability in the Wild" (from Redwood ResearchOrganizationRedwood ResearchA nonprofit AI safety and security research organization founded in 2021, known for pioneering AI Control research, developing causal scrubbing interpretability methods, and conducting landmark ali...Quality: 78/100, with Kevin Wang, Arthur Conmy, and Alexandre Variengien) reverse-engineers a 26-head circuit in GPT-2 Small for identifying indirect objects in sentences (e.g., predicting "Mary" in "After John and Mary went to the shops, John gave a bottle of milk to ___"). Nanda hosted a multi-part walkthrough of this paper with its authors and used it extensively as a case study in his research agenda posts.¹⁶
• Sparse Autoencoders (SAEs): Nanda's DeepMind team has produced a series of SAE papers including Gated SAEs (NeurIPS 2024, led by Sen Rajamanoharan)¹⁷, JumpReLU SAEs¹⁷, and SAEBench (ICML 2025)¹⁸, as well as "Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control" (ICLR 2025)¹⁸.
• Gemma Scope: A collection of over 400 freely available open SAEs for Gemma 2 9B and Gemma 2 2B, authored with Tom Lieberum, Sen Rajamanoharan, Arthur Conmy, Lewis Smith, Nic Sonnerat, Vikrant Varma, Janos Kramar, and others.³ The goal was to enable the broader safety research community to explore model internals using open-source models and tools.

Publication Record

As of 2025, Nanda's Google Scholar profile lists over 13,000 total citations and identifies him as Mechanistic Interpretability Team Lead at Google DeepMind.⁴ His Semantic Scholar profile lists 62 publications, an h-index of 31, and 7,759 total citations including 880 highly influential citations.⁵ Peer-reviewed venues where his work has appeared include:

Sources: DBLP¹⁸, Semantic Scholar⁵

Educational Content

Nanda publishes interpretability tutorials and explanations on his blog at neelnanda.io and through video content. His "200 Concrete Open Problems in Mechanistic Interpretability" sequence, published on the Alignment Forum from December 28, 2022 through January 19, 2023, outlines research directions for the field.¹⁹²⁰ The sequence covers topics including: Concrete Steps to Get Started, The Case for Analysing Toy Language Models, Looking for Circuits in the Wild, Interpreting Algorithmic Problems, Exploring Polysemanticity and Superposition, Analysing Training Dynamics, Techniques Tooling and Automation, Image Model Interpretability, Interpreting Reinforcement Learning, and Studying Learned Features in Language Models.²⁰ A Manifold Markets prediction market tracking progress on these problems resolved at 14% solved by 2025.²¹

Nanda has written guides for newcomers to interpretability research, including walkthroughs of TransformerLens usage and explanations of foundational papers. These materials are distributed as blog posts, Jupyter notebooks, and video tutorials.

His materials have been incorporated into the ARENA (Alignment Research Engineer AcceleratorOrganizationAlignment Research Engineer AcceleratorTraining program that upskills software engineers to become alignment researchers.) curriculum, a 4–5 week ML bootcamp focused on AI safety. The interpretability chapter of ARENA is largely based on Nanda's open-source material; during ARENA 2.0, 6 of 15 participants in Nanda's SERI MATS cohort rated ARENA content among their top 3 most helpful resources.²² ARENA has continued to run annually (ARENA 7.0 ran January–February 2026), citing the work of Nanda and the Google DeepMind Interpretability Team in its interpretability curriculum.²³

Through the SERI MATS mentorship program, Nanda has mentored approximately 50 junior researchers over roughly three years, supervised 30+ MATS papers, and co-authored 15 papers published at top ML venues. As of his 2024 MATS applications post, 7 mentees had gone on to work at major AI companies.⁶

Research Approach

Nanda's work emphasizes:

• Circuit discovery: Identifying specific subnetworks responsible for model behaviors
• Mechanistic explanations: Describing algorithms implemented by neural networks in mathematical or computational terms
• Open-source tooling: Building software infrastructure to enable interpretability experiments
• Reproducible examples: Providing code and notebooks that others can run and modify

His stated view is that understanding neural network internals is necessary for evaluating AI safety properties, though he acknowledges that current interpretability techniques may not directly transfer to more capable future systems.²⁴ Since approximately mid-2024, Nanda has described a strategic shift: his team's focus has moved from basic science (ambitious reverse-engineering of circuits) toward applying interpretability skills to real-world safety problems such as model monitoring.²⁵ He has stated that the team "does not particularly care" whether it is called the "Mechanistic Interpretability Team"—the priority is applying available skills to the most impactful safety problems.²⁵

Perspectives on Interpretability and Alignment

In posts and presentations, Nanda has outlined reasons why mechanistic interpretabilityResearch AreaMechanistic InterpretabilityMechanistic interpretability aims to reverse-engineer neural networks to understand internal computations, with $100M+ annual investment across major labs. Anthropic extracted 30M+ features from Cl...Quality: 59/100 research may contribute to AI alignment. He attributes four potential applications to interpretability:

1. Failure diagnosis: Identifying the mechanisms behind unexpected model behaviors
2. Capability evaluation: Determining what tasks models can perform by examining their internal algorithms
3. Deception detection: Searching for representations that indicate models are optimizing for objectives different from their training signal
4. Verification: Checking whether specific safety properties hold in a model's learned algorithms

He notes that these applications remain speculative and that the field is in early stages of developing techniques that scale to frontier models.²⁶ Researchers have raised specific questions about whether circuit-level analysis scales to deception detection in practice; a LessWrong critique notes that "when researchers claim to identify a subnetwork performing a specific task, we should be very cautious — it's easy to find circuits in networks that do arbitrary things without doing anything more impressive than performance-guided network compression."²⁷ Nanda engaged with that critique and is thanked in the post for correcting a mistake in an earlier draft.²⁷

Nanda stated in 2025 that there have been "very exciting, widely popularized results from mechanistic interpretability that subsequent research has shown are either wrong or significantly weaker than originally claimed," and that the most ambitious vision of the field is "probably dead"—he no longer sees a path to deeply and reliably understanding what AI systems are thinking before competitive pressures push deployment of superhuman AI.²⁸ He now describes interpretability as functioning like "model biology"—a tool for determining whether a model's behavior warrants concern—and advocates for a "Swiss cheese" approach of layering multiple safeguards rather than relying on any single alignment technique.²⁸

On AI TimelinesConceptAI TimelinesForecasts and debates about when transformative AI capabilities will be developedQuality: 95/100, Nanda has stated a deliberate policy of not publicly disclosing his p(doom) or specific timeline estimates, citing concern that prominent researchers' views cause others to anchor without examining whether interpretability expertise translates to timeline expertise.⁸ He has said he considers AGI arriving in the next 10–20 years, sooner, or later as all plausible scenarios, and argues that safety work should proceed as though all are realistic.⁸

Current Work

At Google DeepMindOrganizationGoogle DeepMindComprehensive overview of DeepMind's history, achievements (AlphaGo, AlphaFold with 200M+ protein structures), and 2023 merger with Google Brain. Documents racing dynamics with OpenAI and new Front...Quality: 37/100, Nanda holds the title of Senior Research Scientist and Mechanistic Interpretability Team Lead.⁷⁴ Team outputs during his tenure include Gemma Scope (400+ open-weight SAEs released July 2024)³, Gated SAEs (NeurIPS 2024)¹⁷, JumpReLU SAEs¹⁷, and SAEBench (ICML 2025)¹⁸. Recent papers from the team also include "Chain-of-Thought Reasoning In The Wild Is Not Always Faithful" and "Learning Multi-Level Features with Matryoshka Sparse Autoencoders" (both 2025).¹⁸

In 2025, Nanda was named to MIT Technology Review's Innovators Under 35 list for his mechanistic interpretability research and work to build the field.²⁹

Limitations and Open Questions

Nanda has identified challenges for mechanistic interpretability research:³⁰

• Current techniques are labor-intensive and may not scale to models with hundreds of billions of parameters
• Many discovered circuits are descriptive rather than predictive, explaining behavior post-hoc without enabling intervention
• It remains unclear whether interpretability of current models will provide insights applicable to future AI systems with different architectures or training methods

External critiques of the broader research program include the concern that circuits identified through activation-based methods may not correspond to causally meaningful computational units, and that the dominant focus on circuits-style interpretability narrows the field's approach to model verification and deceptive alignmentRiskDeceptive AlignmentComprehensive analysis of deceptive alignment risk where AI systems appear aligned during training but pursue different goals when deployed. Expert probability estimates range 5-90%, with key empir...Quality: 75/100 detection.²⁷ Nanda himself has acknowledged that some earlier high-profile results in the field were "wrong or significantly weaker than originally claimed."²⁸

Questions about whether interpretability should be prioritized relative to other AI AlignmentApproachAI AlignmentComprehensive review of AI alignment approaches finding current methods (RLHF, Constitutional AI) show 75%+ effectiveness on measurable safety metrics for existing systems but face critical scalabi...Quality: 91/100 such as scalable oversightResearch AreaScalable OversightProcess supervision achieves 78.2% accuracy on MATH benchmarks (vs 72.4% outcome-based) and is deployed in OpenAI's o1 models, while debate shows 60-80% accuracy on factual questions with +4% impro...Quality: 68/100 or Constitutional AIApproachConstitutional AIConstitutional AI is Anthropic's methodology using explicit principles and AI-generated feedback (RLAIF) to train safer models, achieving 3-10x improvements in harmlessness while maintaining helpfu...Quality: 70/100 remain open; Nanda argues for treating these as complementary rather than competing.²⁸

External Links

• neelnanda.io — personal website, blog posts, and tutorials
• TransformerLens GitHub — library source code and documentation
• 200 Concrete Open Problems in Mechanistic Interpretability — Alignment Forum sequence (December 2022–January 2023)
• Google Scholar profile — full publication list
• LessWrong posts — research updates and explanations