Geekly Articles each Day
⬆️ ⬇️

PyBrain work with neural networks in Python



Within the framework of one project, I faced the need to work with neural networks, considered several options, I liked PyBrain the most . I hope its description will be interesting for many to read.



PyBrain is one of the best Python libraries to study and implement a large variety of algorithms associated with neural networks. It is a good example of combining compact Python syntax with a good implementation of a large set of different algorithms from the field of machine intelligence.



Created for:

')





About the library



PyBrian is a modular library designed to implement various machine learning algorithms in Python. Its main purpose is to provide the researcher with flexible, easy-to-use, but at the same time powerful tools for implementing tasks from the field of machine learning, testing and comparing the effectiveness of various algorithms.

The name PyBrain is an abbreviation of English: Python-Based Reinforcement Learning, Artificial Intelligence and Neural Network Library.

As stated on one site: PyBrain - swiss army knife for neural networking (PyBrain is a Swiss army knife in the area of ​​neural network computing).



The library is built on a modular basis, which allows it to be used both for students to learn the basics and for researchers who need to implement more complex algorithms. The general structure of the procedure for its use is shown in the following diagram:





The library itself is an open source product and is free for use in any project with only one reservation. When used for scientific research, they are asked to add the following book to the list of cited information sources (which people do):

Tom Schaul, Justin Bayer, Daan Wierstra, Sun Yi, Martin Felder, Frank Sehnke, Thomas Rückstieß, Jürgen Schmidhuber. PyBrain. To appear in: Journal of Machine Learning Research, 2010.


Main features



The main features of the library (for version 0.3) are:



Network



PyBrain operates with network structures that can be used to build virtually all the complex algorithms supported by the library. An example is:



Instruments



Additionally, there are software tools that allow you to implement related tasks:



Library installation



Before installing Pybrain, the creators recommend installing the following libraries:

Setuptools is a batch manager for Python that greatly simplifies the installation of new libraries. To install it, it is recommended to download and execute (python ez_setup.py) this script.

After installation you will be able to use the command
easy_install
to install new libraries.

Immediately use them and install the two necessary packages: 

 $ easy_install scipy $ easy_install matplotlib


Next, PyBrain itself is installed.






Library basics



Neural network creation


Creating a neural network with two inputs, three hidden layers and one output: 

 >>> from pybrain.tools.shortcuts import buildNetwork >>> net = buildNetwork(2, 3, 1)


As a result, in the net object there is a created neural circuit initialized with random weights.



Activation function


The activation function is set as follows: 

 net.activate([2, 1])


The number of elements transmitted to the network must be equal to the number of inputs. The method returns the answer in the singular form, if the current circuit has one output, and an array, in the case of a larger number of outputs.



Retrieving network information


In order to obtain information about the current network structure, each of its elements has a name. This name can be given automatically or by other criteria when creating a network.

For example, for the net network names are given automatically: 

 >>> net['in'] <LinearLayer 'in'> >>> net['hidden0'] <SigmoidLayer 'hidden0'> >>> net['out'] <LinearLayer 'out'>


Hidden layers are named with the layer number added to the name.



Opportunities when creating a network


Of course, in most cases, the created neural network should have other characteristics than the default ones. There are various possibilities for this. For example, by default, the hidden layer is created using the sigmoid activation function , and you can use the following constants to specify its other type: 



 >>> from pybrain.structure import TanhLayer >>> net = buildNetwork(2, 3, 1, hiddenclass=<b>TanhLayer</b>) >>> net['hidden0'] <TanhLayer 'hidden0'>




It is also possible to specify the type of the output layer: 

 >>> from pybrain.structure import SoftmaxLayer >>> net = buildNetwork(2, 3, 2, hiddenclass=TanhLayer, outclass=SoftmaxLayer) >>> net.activate((2, 3)) array([ 0.6656323, 0.3343677])


Additionally it is possible to use bias 

 >>> net = buildNetwork(2, 3, 1, bias=True) >>> net['bias'] <BiasUnit 'bias'>




Handling Data (Building a DataSet)


The created network should process the data that this section is dedicated to. A typical data set is a set of input and output values. PyBrain uses the pybrain.dataset module to work with them, and the SupervisedDataSet class is also used below.



Data setting


The SupervisedDataSet class is used for typical teacher training. It supports output and output arrays. Their sizes are set when creating a class instance:

Record type: 

 >>> from pybrain.datasets import SupervisedDataSet >>> ds = SupervisedDataSet(2, 1)


means that a data structure is created to store two-dimensional input data and one-dimensional output.



Adding Samples


The classic task in training a neural network is to learn the XOR function, then the data set used to create such a network is shown. 

 >>> ds.addSample((0, 0), (0,)) >>> ds.addSample((0, 1), (1,)) >>> ds.addSample((1, 0), (1,)) >>> ds.addSample((1, 1), (0,))




Examination of the sample structure


To obtain data arrays in their current set, it is possible to use standard Python functions for working with arrays. 

 >>> len(ds)


will print 4, since this is the number of elements.

Iteration over a set can also be organized in the usual way for arrays: 

 >>> for inpt, target in ds: print inpt, target 


 ... [ 0. 0.] [ 0.] [ 0. 1.] [ 1.] [ 1. 0.] [ 1.] [ 1. 1.] [ 0.]


Also, each set of fields can be directly accessed using its name: 

 >>> ds['input'] 


 array([[ 0., 0.], [ 0., 1.], [ 1., 0.], [ 1., 1.]]) 




 >>> ds['target'] 


 array([[ 0.], [ 1.], [ 1.], [ 0.]])




You can also manually free the memory occupied by the sample by completely removing it: 

 >>> ds.clear() >>> ds['input'] 


 array([], shape=(0, 2), dtype=float64) 


 >>> ds['target'] 


 array([], shape=(0, 1), dtype=float64)




Network training on samples


In PyBrain, the concept of trainers is used to train networks with a teacher. The trainer receives a copy of the network and a copy of the sample set and then trains the network on the set received.

The classic example is backpropagation. To simplify the implementation of this approach in PyBrain, there is a class BackpropTrainer. 

 >>> from pybrain.supervised.trainers import BackpropTrainer


The training set of samples (ds) and the target network (net) have already been created in the examples above, now they will be merged. 

 >>> net = buildNetwork(2, 3, 1, bias=True, hiddenclass=TanhLayer) >>> trainer = BackpropTrainer(net, ds)


The coach received a link to the network structure and can train it. 

 >>> trainer.train() 


 0.31516384514375834


Calling the train () method performs one iteration (epoch) of training and returns the value of the quadratic error (double proportional to the error).

If you do not need to organize a cycle for each epoch, then there is a method of training a network up to convergence: 

 >>> trainer.trainUntilConvergence()


This method will return an array of errors for each epoch.



More examples of the implementation of different networks



In the article
Tom Schaul, Martin Felder, et.al. PyBrain, Journal of Machine Learning Research 11 (2010) 743-746.
An example of creating a network with loading data from a .mat file is given. 



 # Load Data Set. ds = SequentialDataSet.loadFromFile('parity.mat') # Build a recurrent Network. net = buildNetwork(1, 2, 1, bias=True, hiddenclass=TanhLayer, outclass=TanhLayer, recurrent=True) recCon = FullConnection(net['out'], net['hidden0']) net.addRecurrentConnection(recCon) net.sortModules() # Create a trainer for backprop and train the net. trainer = BackpropTrainer(net, ds, learningrate=0.05) trainer.trainEpochs(1000)




A few links:







Conclusion



In conclusion I want to say that this library makes a very good impression, it is convenient to work with it, the descriptions of the algorithms are compact, but do not lose clarity in the wilds of the code.



PS If there are amendments on the names of some terms, then I am ready to listen, not sure about a pair of translations 100% accurate, perhaps there are already well-established names.

Source: https://habr.com/ru/post/148407/

More articles:



All Articles