Researchers Visualized Neural Network Calculations

Visualization of the neural network learning cycle

A team of Graphcore project engineers built graphs of the activity of neural network nodes and their connections in the process of image recognition training, which the researchers described in their blog .

The image above demonstrates the full cycle of training and recognition of the Microsoft Research RESNET-34 neural network in December 2016. The system itself was deployed on the basis of IPU - an intelligent graphics processor, as the creators call it, in the middle of 2016. The obtained data was colored in order to isolate the various density of computations produced by the neural network.

All the images obtained by researchers were not only very complex, but also frighteningly similar to real biological objects. The purpose of the engineers was to visually show what is happening inside the neural network and why even some scientists are confused by the principle of their work.
')
Graphcore images constructed are technical graphs of Microsoft's RESNET neural network. In 2015, RESNET won an image recognition competition called ImageNet.

The following image was taken after conducting 50 cycles of Graphcore image recognition neural network training:



The Graphcore IPU system works with the Poplar framework. The framework is written in C ++ and is focused on working with graphs in the course of machine learning of a neural network. Poplar libraries are open source-development, which in the future can be used in conjunction with TensorFlow and MXNet, which can work out of the box with the Graphcore IPU. A set of debugging and analysis tools can be customized using both C ++ and Python.

IPC Graphcore is applicable not only for image recognition, but also for processing a large array of data. For example, developers lead visualization of astrophysical data processing at their IPU under the control of a neural network:



Or here is an image of the AlexNet deep neural network, built using TensorFlow:



AlexNet is also the winner of ImageNet, but 2012. For comparison, the structure of the neural network based on Microsoft Research RESNET is given:



IPU was developed specifically for work with neural networks, and the developers hope that the result of their work will mark the beginning of a new stage in machine learning. The Graphcore team notes the greater efficiency of networks at IPU, as well as greater learning speed than competitors.