How to make friends Tensorflow and C ++

Google TensorFlow is an increasingly popular machine learning library with a focus on neural networks. She has one great feature, she can work not only in Python programs, but also in C ++ programs. However, as it turned out, in the case of C ++, you need to tinker a bit to properly prepare this dish. Of course, the majority of developers and researchers who use TensorFlow work in Python. However, sometimes it is necessary to abandon this scheme. For example, you have trained your model and want to use it in a mobile application or robot. Or maybe you want to integrate TensorFlow into an existing C ++ project. If you are interested in how to do this, welcome under cat.

Compiling libtensorflow.so

To compile tensorflow, google build system Bazel is used . Therefore, to start have to put it. In order not to clutter up the system, I put bazel in a separate folder:

like that

git clone https://github.com/bazelbuild/bazel.git ${BAZELDIR} cd ${BAZELDIR} ./compile.sh cp output/bazel ${INSTALLDIR}/bin

Now we are going to build TensorFlow. Just in case: official installation documentation is here . Previously, to get a library, you had to do something like this .

But now it's a little easier.

 git clone -b r0.10 https://github.com/tensorflow/tensorflow Tensorflow cd Tensorflow ./configure bazel build :libtensorflow_cc.so

Let's go drink tea. The result will be waiting for us here.
')

 bazel-bin/tensorflow/libtensorflow_.so

Getting header files

We got the library, but in order to use it, headers are also needed. But not all heders are easily accessible. Tensorflow uses the protobuf library to serialize the computation graph. The objects to be serialized are described in the Protocol Buffers language, and then the C ++ code of the objects themselves is generated using the console utility. For us, this means that we will have to generate the headers from the .proto files on our own (maybe I just didn’t find these headers in the source code and they can be not generated, if anyone knows where they are, write in the comments). I generate these heders

So here's the script

 #!/bin/bash mkdir protobuf-generated/ DIRS="" FILES="" for i in `find tensorflow | grep .proto$` do FILES+=" ${i}" done echo $FILES ./bazel-out/host/bin/google/protobuf/protoc --proto_path=./bazel-Tensorflow/external/protobuf/src --proto_path=. --cpp_out=protobuf-generated/ $FILES

A complete list of folders that the compiler needs to specify as containing header files

 Tensorflow Tensorflow/bazel-Tensorflow/external/protobuf/src Tensorflow/protobuf-generated Tensorflow/bazel-Tensorflow Tensorflow/bazel-Tensorflow/external/eigen_archive

From version to version, the list of folders changes as the structure of tensorflow sources changes.

Loading graph

Now that we have headers and a library, we can connect TensorFlow to our C ++ program. However, we are waiting for a slight disappointment, without Python, we still can not do, since at the moment the functionality for the construction of the graph is not available from C ++. Therefore, our plan is as follows:

Create a graph in Python and save it to a .pb file

 import numpy as np import tempfile import tensorflow as tf session = tf.Session() #     tf.train.write_graph(session.graph_def, 'models/', 'graph.pb', as_text=False)

Load the saved graph in C ++

 #include "tensorflow/core/public/session.h" using namespace tensorflow; void init () { tensorflow::GraphDef graph_def; tensorflow::Session* session; Status status = NewSession(SessionOptions(), &session); if (!status.ok()) { std::cerr << "tf error: " << status.ToString() << "\n"; } //   status = ReadBinaryProto(Env::Default(), "models/graph.pb", &graph_def); if (!status.ok()) { std::cerr << "tf error: " << status.ToString() << "\n"; } //     TensorFlow status = session->Create(graph_def); if (!status.ok()) { std::cerr << "tf error: " << status.ToString() << "\n"; } }

Calculating the values of operations of a graph in C ++ looks like this:

 void calc () { Tensor inputTensor1 (DT_FLOAT, TensorShape({size1, size2})); Tensor inputTensor2 (DT_FLOAT, TensorShape({size3, size3})); // -  for (int i...) { for (int j...) { inputTensor1.matrix<float>()(i, j) = value1; } } std::vector<std::pair<string, tensorflow::Tensor>> inputs = { { "tensor_scope/tensor_name1", inputTensor1 }, { "tensor_scope/tensor_name2", inputTensor2 } }; //    -   std::vector<tensorflow::Tensor> outputTensors; //          auto status = session->Run(inputs, { "op_scope/op_with_outputs_name" // ,   }, { "op_scope/op_without_outputs_name", //     }, &outputTensors); if (!status.ok()) { std::cerr << "tf error: " << status.ToString() << "\n"; return 0; } //  - for (int i...) { outputs [0].matrix<float>()(0, i++); } }

Saving and loading the state of the graph

Sometimes you want to interrupt the training of the model and continue it on another device or just later. Or, for example, just save the state of the pre-trained graph for later use. In C ++, there is no standard path. But, it turns out, it’s pretty easy to organize this functionality yourself.

First you need to add to the graph the operations of reading and loading the values of variables

 import numpy as np import tempfile import tensorflow as tf session = tf.Session() #     session.run(tf.initialize_all_variables()) #         for variable in tf.trainable_variables(): tf.identity (variable, name="readVariable") tf.assign (variable, tf.placeholder(tf.float32, variable.get_shape(), name="variableValue"), name="resoreVariable") tf.train.write_graph(session.graph_def, 'models/', 'graph.pb', as_text=False)

In C ++, the save and load operations of the state of the graph look like this

 //   void saveGraphState (const std::string fileSuffix) { std::vector<tensorflow::Tensor> out; std::vector<string> vNames; //     int node_count = graph_def.node_size(); for (int i = 0; i < node_count; i++) { auto n = graph_def.node(i); if ( n.name().find("readVariable") != std::string::npos ) { vNames.push_back(n.name()); } } //     Status status = session->Run({}, vNames, {}, &out); if (!status.ok()) { std::cout << "tf error1: " << status.ToString() << "\n"; } //      int variableCount = out.size (); std::string dir ("graph-states-dir"); std::fstream output(dir + "/graph-state-" + fileSuffix, std::ios::out | std::ios::binary); output.write (reinterpret_cast<const char *>(&variableCount), sizeof(int)); for (auto& tensor : out) { int tensorSize = tensor.TotalBytes(); //   protobuf TensorProto p; tensor.AsProtoField (&p); std::string pStr; p.SerializeToString(&pStr); int serializedTensorSize = pStr.size(); output.write (reinterpret_cast<const char *>(&serializedTensorSize), sizeof(int)); output.write (pStr.c_str(), serializedTensorSize); } output.close (); } //  bool loadGraphState () { std::string dir ("graph-states-dir"); std::fstream input(dir + "/graph-state", std::ios::in | std::ios::binary); if (!input.good ()) return false; std::vector<std::pair<string, tensorflow::Tensor>> variablesValues; std::vector<string> restoreOps; int variableCount; input.read(reinterpret_cast<char *>(&variableCount), sizeof(int)); for (int i=0; i<variableCount; i++) { int serializedTensorSize; input.read(reinterpret_cast<char *>(&serializedTensorSize), sizeof(int)); std::string pStr; pStr.resize(serializedTensorSize); char* begin = &*pStr.begin(); input.read(begin, serializedTensorSize); TensorProto p; p.ParseFromString (pStr); std::string variableSuffix = (i==0?"":"_"+std::to_string(i)); variablesValues.push_back ({"variableValue" + variableSuffix, Tensor ()}); Tensor& t (variablesValues.back ().second); t.FromProto (p); restoreOps.emplace_back ("resoreVariable" + variableSuffix); } input.close (); std::vector<tensorflow::Tensor> out; Status status = session->Run(variablesValues, {}, restoreOps, &out); if (!status.ok()) { std::cout << "tf error2: " << status.ToString() << "\n"; } return true; };

Little bit video

Approximately as described in the article, I am training a model of a two-dimensional quadrocopter so far. It looks like this:

The task of the drones is to fly to the center of the cross and be there, for this they can turn the engines on or off (the DQN algorithm is used). In the video, they are in an environment with a rather large friction, so they move slowly. At the moment I am working on flying in an environment without friction and flying over obstacles. When getting a good result, I plan another article.

Source: https://habr.com/ru/post/308002/

All Articles