Browser Strategy "Paths of History." Architecture and project evolution

In this article I will talk about the development and evolution of the technical part of the browser game “ Paths of History ”.
I will pay attention to the choice of programming language, database, technology and architecture. I'll tell you about hosting.

Paths of History is a massively browser-based strategy game. The project began with the enthusiasm of one person and grew to a serious project with a considerable audience.

To develop the engine, C ++ language was chosen for three reasons:

1. he is fast, which is important for this project;
2. flexible, which allows you to implement some features optimally;
3. I know him better than others.

The essence of the engine is the receipt of the request, the formation and return of the page.
MySQL chose the database only because it is quite popular and such projects are often made using MySQL. At that moment I had no experience with databases.
Immediately there was a question of architecture. The following model has been selected:
The engine is divided into two parts (let's call them D1, D2).
D1 receives the request, transmits to one of the 8 free threads. The stream analyzes the request, requests the necessary data from the database, forms the page and returns it. D1 does not know how to make changes to the database. To reduce the number of database queries, a lot of data is cached on D1.
')
In some cases, D1 receives a request to change the state of the world (a building in the player’s city has been ordered, troops have been sent, etc.). In this case, D1 transmits a request for D2 (communication on sockets). More than one D1 can be connected to D2 (each of which has 8 threads each, and therefore can simultaneously transmit 8 instructions to D2). D2 performs only one instruction at a time, the rest are waiting in the queue. Execution of one instruction for the database is carried out as one transaction. If the instruction is successfully processed, changes are made to the database, if an instruction is invalid, the transaction is canceled and all its changes are rolled back. It is important to note that inadmissible instructions are cut off on D1, but it happens that the instruction became invalid after transferring it to D2 (for example, in the previous instruction, the city spent resources, and in this one it tries to order a building for non-existent resources, but both instructions came almost simultaneously ). The whole system can work without the D2, but only in the “read only” mode - nothing can be changed, all event timers are running, but when they are finished they “hang”. If after this you turn on D2, the system will be restored, as if there was no failure, all events will be processed in the correct order.

Initially, Apache web server was used. It was chosen because it is popular and has a build for Windows. D1 was connected to Apache using ISAPI technology, that is, as a dll library. Apache accepted requests and transferred them to the library connected to itself. Apache itself was rather slow. Therefore, at some point the project was transferred to a bunch of nginx + FastCGI.

The nginx web server is very convenient both in configuration and in use. The speed of return pages increased. In addition, nginx distributes static content very quickly.
How does FastCGI work? The engine from the library dll has been converted into a standalone application. The application accepts requests from the web server through sockets, processes them, generates pages and, through the same sockets, returns the pages to the web server. At the same time, the sockets remain open and new requests are received. Learn more about C ++ development using the FastCGI protocol here .

Now about hosting.
Prior to launching the project, everything worked on a regular home computer over the usual home cable Internet.
At that time there was no financial opportunity to rent a server in the data center, so the first game world was launched all on the same home computer. This created a number of inconveniences: access to the network was unstable, sometimes the lights in the house were turned off, the provider did not give out traffic at the stated tariff and often carried out technical work. The project began to fill with players, the load grew. On the service of the 1st world was put another computer with a different Internet connection. Now one world works easily on the 1st server, but at that time everything was not optimized, and the computers used were weak.

Soon the next world was discovered. Two more computers and two network connections were used. Already preparing to launch the world 3. All these computers were located in my home in the living room. As the number of servers grew, so did the number of problems. I could not leave the house any longer, because something was constantly falling. In addition to force majeure problems, there were also regular bugs. If any abnormal situation arises, the application immediately fell on the assert, without trying to somehow get out of the situation. This solution was chosen specifically. This made me always first of all fight with bugs, and not drag them through the entire development period.
The project began to generate revenue, and a server was rented at the data center. The site of the game and both worlds were transferred to it. It has become much easier to administer the system, but costs have increased. The third world was also launched on this server, but after the DDoS attack, I transferred it to a separate server so that the first two worlds were out of danger.

The development was conducted and tested on Windows. But the code was written immediately without reference to this OS, and, in the future, it took just one day to fix the code and compile the project under FreeBSD.
The POSIX library was chosen for working with threads. To create graphic files, I used the FreeImage library.

System monitoring.
Initially, the monitoring system was using monitors! Any server “crash” could be detected as a window with an error or no outgoing traffic on the chart. Even at night I had to wake up several times and look at all the monitors.
This could not last long and a special php script was written that constantly polls the server, collects status data from them and, if necessary, sends an email or SMS to the phone. This script was launched on a free hosting, where it works to this day. Thanks to him, he always manages to quickly find out about the problems and, if possible, eliminate them immediately.

In the following articles I will talk about the development of the project from idea to release, about technical solutions in the engine and data storage formats in the database, about data backup and protection against attacks, about the mechanism of the formation of pages.

Basis D1:

void* operateRequest(void* listen_socket) { // FCGX_Request request; assert(!FCGX_InitRequest(&request, *(int*)listen_socket, 0)); Session* s = new Session; //      while(FCGX_Accept_r(&request) == 0) { stringstream out; stringstream header; header << "Content-type: text/html"; //   string query; string addr; string referer; string post; string cookie; string agent; int content_lenght = 0; for(char** envp = request.envp; *envp; ++envp) { string v = *envp; string::size_type e = v.find('='); string p = v.substr(0, e); if(p == "REQUEST_URI") query = v.substr(e + 2, v.length()); if(p == "REMOTE_ADDR") addr = v.substr(e + 1, v.length()); if(p == "HTTP_COOKIE") cookie = v.substr(e + 1, v.length()); if(p == "HTTP_REFERER") referer = v.substr(e + 1, v.length()); if(p == "CONTENT_LENGTH") content_lenght = toInt(v.substr(e + 1, v.length())); if(p == "HTTP_USER_AGENT") agent = v.substr(e + 1, v.length()); } //   maximize(content_lenght, 9999); char p[10000]; FCGX_GetStr(p, content_lenght, request.in); p[content_lenght] = 0; post = p; // .  header   s->work(header, out, addr, cookie, referer, query, post); //    header << "\r\n\r\n" << out.str(); FCGX_PutStr(header.str().c_str(), int(header.str().length()), request.out); FCGX_Finish_r(&request); } return 0; } int main() { assert(initSocketSystem()); assert(!FCGX_Init()); int listen_socket = FCGX_OpenSocket(":8000", 400); assert(listen_socket >= 0); //  for(int i = 0; i < threads; ++i) { pthread_t thread; assert(pthread_create(&thread, 0, operateRequest, (void*)&listen_socket) == 0); } while(true) sleep(1000); return 0; } 

Basis D2:

 void operateCommand(asComType com, Socket& sock) { // .        pthread_mutex_lock(&ascs); bool res; //     assert(sql.put("BEGIN")); switch(com) { case ASC_TOWNUPDATE: { //      int id = sock.readVal<int>(); res = asUpdateTown(id); } break; //  //… //… //… } //          assert(sql.put(res ? "COMMIT" : "ROLLBACK")); //  sock.sendVal(res); //   pthread_mutex_unlock(&ascs); } void* clientThread(void* client_socket) { Socket& sock = *(Socket*)client_socket; asComType com; int bytes; //    while((bytes = sock.readVal(com)) && bytes >= 0) { //  operateCommand(com, sock); } delete &sock; return 0; } int main() { while(Socket* client = sock.listen()) { //        pthread_t thread; assert(pthread_create(&thread, 0, clientThread, (void*)client) == 0); } return 0; } 

The entire code presented in places is specially simplified for clarity. Some classes and functions are omitted.