Apache Thrift RPC Server. We are friends with C ++ and Java

Hello colleagues.

I want to put in this topic instructions on how to quickly screw Thrift, to their handicrafts.

Thrift is a technology for organizing interprocess communication between system components. It was developed somewhere in the bowels of Facebook. Put this is a cross-language framework for creating RPC services, on a binary protocol. With this solution, you can "make friends" components written in different languages ​​C #, C ++, Delphi, Erlang, Go, Java, PHP, Python, Ruby, etc. The description of the signatures of services and data is carried out using a special IDL - language. The technology, in its essence, is similar to COM, but without all this binding with the registration of components. Also, let's not forget that COM is a Windows-only technology, while Thrift is cross-platform.



In general, I decided to experiment, try to take part of the loaded-computational logic from Java to C ++, in the hope that native C ++ code will be slightly more productive, and try Thrift RPC in one hoping that it is faster than REST.

As it should be, there were no tambourines and rakes!



And so, first we need to put all this:

1. We put support for Boost, because everything is tied to it

$ sudo apt-get install libboost-dev libboost-test-dev libboost-program-options-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev 

2. download thrift tarball apache.softded.ru/thrift/0.9.0/thrift-0.9.0.tar.gz

unpack, run configure, and then build.

')

 $ ./configure $ make $ sudo make install 

It seems to be all ... You can even try to generate the code from the tutorial that comes with the thrift tarball

 $ thrift --gen cpp tutorial.thrift 

the thrift command will generate a C ++ wrapper, and carefully put them in the gen-cpp directory. The same can be done for Java, PHP, etc. ...

We try to compile and collect the source files

 $ g++ -Wall -I/usr/local/include/thrift *.cpp -L/usr/local/lib -lthrift -o something 

Oops, get: error: 'uint32_t' does not name a type

It turns out there is a small cockroach in the thrift libraries associated with uint32_t. It is treated by adding "#include <stdint.h>" in "Thrift.h", or (best of all) using special compiler options -DHAVE_NETINET_IN_H -DHAVE_INTTYPES_H



Now it looks like this:

 $ g++ -DHAVE_INTTYPES_H -DHAVE_NETINET_IN_H -Wall -I/usr/local/include/thrift *.cpp -L/usr/local/lib -lthrift -o something 

That's it, there is an executable file called something.

We start, and we get: error while loading shared libraries: libthrift.so.0

Perhaps there are some elegant methods for solving this problem, but I solved it in the forehead by copying thrift files from / usr / local / lib to / lib



Everything, the example started. So everything is in place and everything works.



Now you can write your RPC server.

Its task is to be a key-value repository. Store long (several hundred thousand) bitmasks. Add them (AND), and give the client an array of indices that make up units. Yes, almost the same Redis can, but it does not suit me.



The full code is here: github.com/2anikulin/fast-hands.git



We describe the data and service signatures in the thrift definition file :

 namespace cpp fasthands namespace java fasthands namespace php fasthands namespace perl fasthands exception InvalidOperation { 1: i32 what, 2: string why } service FastHandsService { i32 put(1:i32 key, 2:binary value), binary get(1:i32 key), list <i32> bitAnd(1:list<i32> keys) throws (1:InvalidOperation ouch) } 

And generate envelopes.

C ++ implementation

This code creates and runs the RPC server.

 #define PORT 9090 #define THREAD_POOL_SIZE 15 int main() { printf("FastHands Server started\n"); shared_ptr<TProtocolFactory> protocolFactory(new TBinaryProtocolFactory()); shared_ptr<FastHandsHandler> handler(new FastHandsHandler()); shared_ptr<TProcessor> processor(new FastHandsServiceProcessor(handler)); shared_ptr<ThreadManager> threadManager = ThreadManager::newSimpleThreadManager(THREAD_POOL_SIZE); shared_ptr<PosixThreadFactory> threadFactory = shared_ptr<PosixThreadFactory>(new PosixThreadFactory()); threadManager->threadFactory(threadFactory); threadManager->start(); TNonblockingServer server(processor, protocolFactory, PORT, threadManager); server.serve(); printf("done.\n"); return 0; } 

In the class FastHandsHandler - all our applied functionality is implemented.

This is the header file.

 class FastHandsHandler : virtual public FastHandsServiceIf { public: FastHandsHandler(); int32_t put(const int32_t key, const std::string& value); void get(std::string& _return, const int32_t key); void bitAnd(std::vector<int32_t> & _return, const std::vector<int32_t> & keys); private: void appendBitPositions(std::vector<int32_t> & positions, unsigned char bits, int offset); private: std::map<int32_t, std::string> m_store; }; 

We try to collect and get another error: c ++ undefined reference to apache :: thrift :: server :: TNonblockingServer

The fact is that, unlike the tutorial, my server is asynchronous, and uses the TNonblockingServer class. For the code to be collected, you need to add additional libraries -lthriftnb -levent



ie, the build will now look like this:

 g++ -DHAVE_INTTYPES_H -DHAVE_NETINET_IN_H -Wall -I/usr/local/include/thrift *.cpp -L/usr/local/lib -lthrift -lthriftnb -levent -o something 

Now everything is fine. The code is compiled, linked, the output file named something

We generate wrappers for java, and we write here such client

 import fasthands.FastHandsService; import fasthands.InvalidOperation; import org.apache.thrift.TException; import org.apache.thrift.transport.TFramedTransport; import org.apache.thrift.transport.TTransport; import org.apache.thrift.transport.TSocket; import org.apache.thrift.transport.TTransportException; import org.apache.thrift.protocol.TBinaryProtocol; import org.apache.thrift.protocol.TProtocol; import java.nio.ByteBuffer; import java.util.ArrayList; import java.util.List; public class JavaClient { public static void main(String [] args) { TTransport transport = new TFramedTransport(new TSocket("localhost", 9090)); TProtocol protocol = new TBinaryProtocol(transport); final FastHandsService.Client client = new FastHandsService.Client(protocol); final List<Integer> filters = new ArrayList<Integer>(); try { transport.open(); int count = 12500; byte bt[] = new byte[count]; for (int i =0; i < count; i++) { bt[i] = (byte)0xFF; } for (int i = 0; i < 50; i++) { client.put(i, ByteBuffer.wrap(bt)) ; filters.add(i); } List<Integer> list = client.bitAnd(filters); System.out.println(list.size()); } catch (TTransportException e) { e.printStackTrace(); } catch (TException e) { e.printStackTrace(); } transport.close(); } } 

What is the result.

An interesting technology, and not a bad way to screw the transport functionality to the bare code in C ++. But I will not say that it is much faster than REST, binary data is perfectly transmitted via http. Regarding the performance of code rendered from Java in C ++, the miracle did not happen, it is faster 1.2 - 1.4 times, because serialization + expenses on the transport layer.