Development of a fuzzy neural network NEFClass M

An analysis of the deficiencies of the NEFClass system shows that they are caused by the imperfection of the learning algorithm of fuzzy sets NEFClass. In order to fix this, it is necessary to replace the empirical learning algorithm with a rigorous numerical optimization algorithm. Both the original and the modified NEFClass model is based on the architecture of a fuzzy perceptron. The architectural differences between the original and the modified model consist in the form of membership functions of fuzzy sets, the t-norm function for calculating the activation of neuron rules, and also in the form of an aggregating function (t-form) determining the activation of output neurons. The use of numerical optimization methods requires the differentiability of the membership functions of fuzzy sets - a condition that triangular membership functions do not satisfy. Therefore, in the modified model, fuzzy sets have a Gaussian membership function.

The requirement of differentiability also dictates the type of t-norm (intersection) for calculating the activation of neuron rules. The NEFClass system uses the minimum function for this; in the modification is the product of the corresponding values. Finally, the type of aggregation function (t-form) for the modified model is limited only by a weighted sum. The reason is that the maximum function, which is used in the original system, does not satisfy the condition of differentiability.

The main change concerns the learning algorithm of fuzzy sets. The target function in the modified NEFClass system is minimization of the root-mean-square error on a training sample by analogy with classical neural networks.

The argument of numerical optimization, aimed at reducing the root-mean-square error in the training set, is the aggregate vector of parameters a and b of all fuzzy sets of the network. A specific method can be any method of unconditional optimization, such as, for example, the gradient method or the conjugate gradient method.
NEFClass M Fuzzy Neural Network Learning Algorithm
Stage 1 - generation of the rule base
For the first stage of this algorithm - learning the rule base - the first stage of the basic algorithm NEFClass is used.
Stage 2 - Learning Fuzzy Sets
The second stage uses the conjugate gradient method of learning of a neural network of direct action.
')
Conducting experiments with the NNC NEFClass M
Initial data
Employees of MGOU were selected as samples for the network and entered into the database (Fig. 1.). Photos contain changes in angle, scale and lighting. The database has 100 images: 10 people for 10 people. Image size is 112x92. Format - one byte per pixel, in the form of grayscale values ​​[0; 255]. To speed up learning, images were reduced 4 times in volume (46x56).


fig.1.

Databases before training were divided into 2 parts. Odd human images were used for training, and even - for testing. Experiments with a fuzzy neural network were conducted in the mathematical editor Mathcad 14.0.

Experiments with NNS NEFClass M
Learning parameters NNS NEFClass M

Parameter	Value
Algorithm for generating rules	best for class
Learning algorithm	conjugate gradients
Aggregation function	weighted amount
The number of terms for each feature	five

We investigate the dependence of the quality of education on the number of rules that are generated at the first stage. As a test, we will conduct testing on a test sample. To do this, set the number of rules 1, 2, 3 or 4 (out of 5) for each class. Network configuration: 40 output neurons, 5 terms for each trait.
Experimental results
Table 1 - Dependence of the quality of education on the number of rules

Input neurons	Neuron Rules	Recognition (training set)			Recognition (check sample)
		Ok	Err	?	Ok	Err	?
200	40	73.5	26.5	0.0	70.0	30.0	0.0
200	80	92.5	7.5	0.0	86.0	18.0	0.0
200	120	98.5	1.5	0.0	91.0	9.0	0.0
200	160	100.0	0.0	0.0	93.5	6.5	0.0
400	40	76.5	22.0	1.5	67.5	32.0	1.5
400	80	92.0	7.0	1.0	82.5	16.5	1.0
400	120	98.0	2.5	0.5	97.5	0.5	1.5
400	160	100.0	0.0	0.0	98.5	0.0	1.5

where “Err” is the errors of the second kind, “?” - unrecognized images (the error of the first kind).

Of the algorithms studied, the neural network NEFClass M coped with the problem of recognition best of all, however, there are a large number of errors of the first kind in this network. The classical neural network NEFClass showed an identical result of the percentage of such errors, but at the same time lags behind the NEFClass M due to the large number of unrecognized images. This is due to the use of triangular membership functions, which in the input data range can produce zero values ​​for the rules neuron inputs (which is not the case when using Gaussian membership functions).

From the above analysis it follows that to increase the probability of recognition, it is necessary to use the neural network NEFCIass M. The neural network NEFCIass M showed the minimum number of classification errors, but it has an increased value of unrecognized images. It is necessary to optimize the search in the knowledge base and increase the set of face images to reduce the errors of unrecognized images.

Conclusion
Based on the analysis performed and the experiments performed, it should be noted about the best network configuration and preprocessing parameters for image recognition tasks, namely:
1. the number of Fourier coefficients for the qualitative correlation of the image to a particular class: 80-200, depending on the number of classes;
2. the number of hidden neurons: 80-200, depending on the number of output neurons;
3. The most optimal layer activation functions are hyperbolic tangent;
4. learning algorithm - gradient with adaptation of the learning step

PS My first article on Habré. My work is in graduate school.