TensorFlow and Logistic Regression

After a short but very bloody war, I still managed to compile and build TensorFlow for a GPU with CUDA capability = 3.0. Now you can immerse yourself in it thoroughly, because machine learning with a GPU is quick, easy and pleasant, and without a GPU it is sometimes a huge waste of time.

Let's try to program the simplest logistic regression.

Let's start straightforward - importing tensorflow:

import tensorflow as tf 

Next, load the data:

 train_X, train_Y, test_X, test_Y = load_data() 

The function load_data, which loads your data, you will need to implement yourself.

The data structure is extremely simple:
- train_X and test_X are numpy-arrays of NxF dimension, where N is the number of analyzed instances (in other words, data points), and F is the number of attributes of each instance.
- train_Y and test_Y are also numpy-arrays, but now with the dimension of NxC, where C is the number of classes (in the simplest case, 2)
Thus, if train_Y [i] contains the vector [1 0], this means that the i-th instance belongs to class 0 (since it is 1 in the zero element of the vector).
')
For convenience, remember the important dimensions:

 num_features = train_X.shape[1] num_classes = train_Y.shape[1] 

Now we define the character variables for future calculations:

 X = tf.placeholder("float", [None, num_features]) Y = tf.placeholder("float",[None, num_classes]) 

Here we create a character variable X of the type “float” with the dimension “at least how much” on num_features. The number of data points does not affect the essence of the calculations, so you can not fix them in the model. But then we will be able to run calculations on the same model with both 100 points and 10 million.

CY is completely similar situation.

You also need to create variables to store model parameters:

 W = tf.Variable(tf.zeros([num_features,num_classes])) B = tf.Variable(tf.zeros([num_classes])) 

And now we describe the model:

 pY = tf.nn.softmax(tf.matmul(X, W) + B) 

In this case, we use the simplest linear model of the form y = Wx + B, wrapped in softmax. The calculation result, that is, the predicted Y, will be stored in pY, so it will also have the dimension NxC.

Next, we determine the loss function:

 cost_fn = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pY, Y)) 

You can, of course, the loss function and more clearly describe:

 cost_fn = -tf.reduce_sum(Y * tf.log(pY)) 

Then create an optimizer:

 opt = tf.train.AdamOptimizer(0.01).minimize(cost_fn) 

Instead of Adam, anyone can use the good old gradient descent:

 opt = tf.train.GradientDescentOptimizer(0.01).minimize(cost_fn) 

So far, we have only described the model and its parameters, but no calculations have yet been carried out. And now, finally, we turn to the calculations. First, create a session and initialize all variables.

 sess = tf.Session() init = tf.initialize_all_variables() sess.run(init) 

Now let's go:

 num_epochs = 40 for i in range(num_epochs): sess.run(opt, feed_dict={X:train_X, Y:train_Y}) 

Here, the optimizer previously described is simply called to which data is transferred: the character variable X receives data from train_X, and Y from data from train_Y. Thus, there is no longer any uncertainty in the model and all calculations are feasible.

In this case, we carry out 40 iterations, in each of which all available training data are calculated.
If there is a lot of data, then you can divide them into blocks (batch'i) and teach them sequentially, that is, each epoch will include a cycle for all blocks.

To assess the quality of the model, you must enter the evaluation criterion:

 accuracy = tf.reduce_mean(tf.cast(tf.equal(tf.argmax(pY,1), tf.argmax(Y,1)), "float")) 

Here the real class and the predicted class are compared, after which the result of the comparison is converted from the type boolean to the type float (true - 1, false - 0), and then the average is taken for all these comparisons.
Note that the previous line only defines the criterion, and does not calculate it. The above formula, as well as the optimizer, depends on the character variables that need to be filled with data. Therefore, to perform the calculation, run run again:

 accuracy_value = sess.run(accuracy, feed_dict={X:test_X, Y:test_Y}) 

However, now we are already using test data, not training data.

That's all. It was a snap.