Feature Visualization - Distill.pub

[Distill](https://distill.pub/)

# Feature Visualization

How neural networks build up their understanding of images

**Edges** (layer conv2d0)

**Textures** (layer mixed3a)

**Patterns** (layer mixed4a)

**Parts** (layers mixed4b & mixed4c)

**Objects** (layers mixed4d & mixed4e)

 Feature visualization allows us to see how GoogLeNet

- **Going deeper with convolutions** [\[PDF\]](https://arxiv.org/pdf/1409.4842.pdf)

  C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich.

  Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1--9. 2015.

\[1\]

, trained on the ImageNet

- **Imagenet: A large-scale hierarchical image database** [\[PDF\]](http://www.image-net.org/papers/imagenet_cvpr09.pdf)

  J. Deng, W. Dong, R. Socher, L. Li, K. Li, L. Fei-Fei.

  Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp. 248--255. 2009.

  [DOI: 10.1109/cvprw.2009.5206848](https://doi.org/10.1109/cvprw.2009.5206848)

\[2\]

 dataset, builds up its understanding of images over many layers.
 Visualizations of all channels are available in the [appendix](https://distill.pub/2017/feature-visualization/appendix/).


### Authors

### Affiliations

[Chris Olah](https://colah.github.io/)

[Google Brain Team](https://g.co/brain)

[Alexander Mordvintsev](https://znah.net/)

[Google Research](https://research.google.com/)

[Ludwig Schubert](https://schubert.io/)

[Google Brain Team](https://g.co/brain)

### Published

Nov. 7, 2017

### DOI

[10.23915/distill.00007](https://doi.org/10.23915/distill.00007)

There is a growing sense that neural networks need to be interpretable to humans.
The field of neural network interpretability has formed in response to these concerns.
As it matures, two major threads of research have begun to coalesce: feature visualization and attribution.

![](https://distill.pub/2017/feature-visualization/images/objectives/neuron.png)![](https://distill.pub/2017/feature-visualization/images/objectives/channel.png)

**Feature visualization** answers questions about what a network — or parts of a network — are looking for by generating examples.


![](https://distill.pub/2017/feature-visualization/images/attribution-1.png)![](https://distill.pub/2017/feature-visualization/images/attribution-2.jpg)

**Attribution**

1

As a young field, neural network interpretability does not yet have standardized terminology.
Attribution has gone under many different names in the literature — including “feature visualization”! — but recent work seems to prefer terms like “attribution” and “saliency maps”.

studies what part of an example is responsible for the network activating a particular way.


This article focuses on feature visualization.
While feature visualization is a powerful tool, actually getting it to work involves a number of details.
In this article, we examine the major issues and explore common approaches to solving them.
We find that remarkably 

... (truncated, 58 KB total)