Neural networks for curious programmers (with an example in c #)

Since the title was marked "for curious programmers," I want to say that my curiosity led to the fact that I, as a developer of mobile games, wrote this post. I am quite sure that there will be programmers who once thought about artificial intelligence and this is a very good chance for them.

After reading a lot of articles about neural networks, I would like to mention some of them that really helped me to master the topic:

java example and useful links
visual implementation using OOP

Since there are a lot of theories on this topic, I would like to start implementation.
')

Implementation

using UnityEngine; using System.Collections; using System.Xml.Serialization; public class Neuron { [XmlAttribute("weight")] public string data; [XmlIgnore] public int[,] weight; //   [XmlIgnore] public int minimum = 50; //  [XmlIgnore] public int row = 64,column = 64; /** *  ,       */ public Neuron() { weight = new int[row,column]; randomizeWeights(); } /** *  ,   * @param input -   * @return  0  1 */ public int transferHard(int[,] input) { int power = 0; for(int r = 0; r < row;r++) { for(int c = 0; c < column;c++) { power += weight[r,c]*input[r,c]; } } //Debug.Log("Power: " + power); return power >= minimum ? 1 : 0; } /** *     * @param input -   * @return n  */ public int transfer(int[,] input) { int power = 0; for(int r = 0; r < row;r++) for(int c = 0; c < column;c++) power += weight[r,c]*input[r,c]; //Debug.Log("Power: " + power); return power; } /** *      */ void randomizeWeights() { for(int r = 0; r < row;r++) for(int c = 0; c < column;c++) weight[r,c] = Random.Range(0,10); } /** *    * @param input -   * @param d -        */ public void changeWeights(int[,] input,int d) { for(int r = 0; r < row;r++) for(int c = 0; c < column;c++) weight[r,c] += d*input[r,c]; } public void prepareForSerialization() { data = ""; for(int r = 0; r < row;r++) { for(int c = 0; c < column;c++) { data += weight[r,c] + " "; } data += "\n"; } } public void onDeserialize() { weight = new int[row,column]; string[] rows = data.Split(new char[]{'\n'}); for(int r = 0; r < row;r++) { string[] columns = rows[r].Split(new char[]{' '}); for(int c = 0; c < column;c++) { weight[r,c] = int.Parse(columns[c]); } } } } 

The class of neurons that contains weight is a binary array of weights, minimum is the threshold of a function. The transferHard function returns the answer to the input vector. Since the answer function is hard, I use it for training. In my opinion, this more effectively teaches neurons. I will be very grateful if there will be feedback on this. The transfer function returns the answer to the input vector but with probability, the sum may be closer to zero or negative if the neuron is trained for another symbol.

 using UnityEngine; using System.Collections; using System.Xml.Serialization; using System.Xml; using System.IO; public class NeuralNetwork { [XmlArray("Neurons")] public Neuron[] neurons; /** *     */ public NeuralNetwork() { neurons = new Neuron[10]; for(int i = 0;i<neurons.Length;i++) neurons[i] = new Neuron(); } /** *   ,    * @param input -   * @return     ,   */ int[] handleHard(int[,] input) { int[] output = new int[neurons.Length]; for(int i = 0;i<output.Length;i++) output[i] = neurons[i].transferHard(input); return output; } /** *   ,     * @param input -   * @return   ,   */ int[] handle(int[,] input) { int[] output = new int[neurons.Length]; for(int i = 0;i<output.Length;i++) output[i] = neurons[i].transfer(input); return output; } /** *   * @param input -   * @return       */ public int getAnswer(int[,] input) { int[] output = handle(input); int maxIndex = 0; for(int i = 1; i < output.Length;i++) if(output[i] > output[maxIndex]) maxIndex = i; return maxIndex; } /** *   * @param input -   * @param correctAnswer -   */ public void study(int[,] input,int correctAnswer) { int[] correctOutput = new int[neurons.Length]; correctOutput[correctAnswer] = 1; int[] output = handleHard(input); while(!compareArrays(correctOutput,output)) { for(int i = 0; i < neurons.Length;i++) { int dif = correctOutput[i]-output[i]; neurons[i].changeWeights(input,dif); } output = handleHard(input); } } /** *    * @param true -   , false -   */ bool compareArrays(int[] a,int[] b) { if(a.Length != b.Length) return false; for(int i = 0;i<a.Length;i++) if(a[i] != b[i]) return false; return true; } void prepareForSerialization() { foreach(Neuron n in neurons) n.prepareForSerialization(); } void onDeserialize() { foreach(Neuron n in neurons) n.onDeserialize(); } public void saveLocal() { prepareForSerialization(); XmlSerializer serializer = new XmlSerializer(this.GetType()); FileStream stream = new FileStream(Application.dataPath + "/NeuralNetwork.txt", FileMode.Create); XmlWriter writer = new XmlTextWriter(stream, new System.Text.ASCIIEncoding()); using(writer) { serializer.Serialize(writer, this); } } public static NeuralNetwork fromXml() { string xml = ""; FileStream fStream = new FileStream(Application.dataPath + "/NeuralNetwork.txt", FileMode.OpenOrCreate); if(fStream.Length > 0) { byte[] tempData = new byte[fStream.Length]; fStream.Read(tempData, 0, tempData.Length); xml = System.Text.Encoding.ASCII.GetString(tempData); } fStream.Close(); if(string.IsNullOrEmpty(xml)) return new NeuralNetwork(); NeuralNetwork data; XmlSerializer serializer = new XmlSerializer(typeof(NeuralNetwork)); using(TextReader reader = new StringReader(xml)) { data = serializer.Deserialize(reader) as NeuralNetwork; } data.onDeserialize(); return data; } } 

The NeuralNetwork class contains an array of neurons, each of which is designed for a specific symbol. An array of ten elements is created in the constructor, because the example is made for recognizing numbers (0-9). If you want to use the network for letter recognition, change the size of the array accordingly. The handleHard function is called to train neurons, returns an array of zeros and ones. The getAnswer function, the network response for the input vector, uses the handle function to get an array of responses, each element of the array contains the neuron's response with probability. The getAnswer function selects the index of the element that contains the highest probability and returns the ego as an answer.

After studying a lot of literature, I learned that the activator can be implemented using Sigmoidal transfer function, which strengthens weak signals and keeps strong ones, but I cannot understand how this will affect the improvement of character recognition.

I would like to see an example of character recognition using the Radial Basis function, so it says almost everywhere that recognition will be improved with this method.

Conclusion

In conclusion, I would like to say that for a better understanding of the codes of neural networks, I advise you to read some literature and try to solve problems of this type on your own, starting with a primitive single-layer perceptron.

I wanted to see different reviews on the topic and on the post.