Reverse engineering "Kazakov 3", part of the network: create a local server

Recently, in a conversation with colleagues, I discussed various games of the RTS genre, and I wondered why the release of the third Kazakov passed by me. A couple of minutes and one search query later, I remembered - besides the extremely raw early release, the reincarnation of this classic strategy was distinguished by the impossibility of a multiplayer game without a permanent connection to the official server. Numerous requests from players to “add LAN” on forums of varying degrees of freshness hint that changes should not be expected.

Well, if the mountain does not go to Mohammed ...

TL; DR: Server, instructions for use and source code are available on GitHub .
')

The first steps

In order to understand the general structure of the protocol, the first four packets, overheard between the game client and the official server, are enough. So, we have:

• A network protocol built on top of normal TCP / IP
• without encryption or obfuscation ,
• in which the numbers are transmitted in order from the youngest to the oldest .

The packet header always takes exactly 14 bytes and contains the size of the payload (1), the command code (2) and two player identifiers for addressing packets (3.4). Take a simple example - a private message in the game lobby chat:



It also shows that the lines are preceded by their length (5). It is noteworthy that, depending on the specific command code, the data format varies and the size of the lines is indicated by one, then two, or even four bytes.

Consider a public announcement of the creation of a new game room:



The title also begins with the size, command and sender (1,2,3), but the recipient identifier is missing (4). In this message, this means “send to everyone”, but for other teams it can mean “to everyone in their room.” The first line (5) contains the name, password, and information about the type of game and client version of the creator (host) of the room. To separate values, the tab character \ t is used, it is also 09h. Then follows a line with information about the room (6), which is required for its display in the list. It contains the status, the number of live players, computer opponents, closed slots and two more values. Here, the vertical bar plays the role of a separator. It is followed by two constants of 4 bytes each (hereinafter referred to as the Int type number ), then a string with the host name of the computer of the room creator (7).


Now for the more interesting moments.

Translation difficulties

Initially, I analyzed packets in Wireshark with tcp && data filters so that no “empty” ACK packets would flash before my eyes. At some point, it came out to me sideways: it turned out that Wireshark mistakenly receives packets, the TCP payload of which starts with bytes 05h 00h, for DCERPC protocol packets . In particular, this affected the packets with the notification of the player entering the room, because they always contain exactly five bytes after the header. This causes Wireshark to tag TCP traffic not as Data , but as [Malformed Packet: DCERPC] and hides the packet:



In this case, applying a newer tcp.payload filter would be correct . It displays all TCP packets with a payload, regardless of how Wireshark interprets this load.

Evolution of serialization

When reversing the network protocol, it was obvious that different people working on it at different times had different priorities. There are three types of strings that transmit variables:

• compact rows
  The length of the line is indicated by one byte, the lines contain the names of the variables, the fields are separated by vertical bars: ps=1337|pw=42|pg=12
• intermediate rows
  The length of the string is indicated by two bytes (hereinafter referred to as the Short type number ), the values ​​are separated by a tab, the names are absent: "MyRoom"\t"secret"\t0AFFE
• generous lines
  Special strings, which are actually an array of strings with alternating variable names and their values. Such a kind of associative array in one-dimensional form. The length of each (sic!) Of the lines in it is indicated by a number of type Int . A detailed description of the format can be found in the comments to the source code of the server.
  
  Example

  
  Numbers indicating the lengths of the lines are marked in red, separators in yellow.
  

The last example is taken from a package that regulates the transit of the role of the host to one of the players when the creator of the room leaves the game. As far as I understand, this functionality was modified after the release of the game. It is obvious that at that time the issue of data transfer efficiency was no longer as acute as at the time of writing the network protocol in the original engine.

Sloppy buffers

At the end of many packets, the payload ends with four or six zero bytes. But in certain packages (in particular, with the command codes 0xc8 and 0x19d ), data sometimes pops up among them unexpectedly. I could not understand where they come from and why they are needed, until in one of these packages I discovered a fragment of one of the messages in the chat lobby.

Apparently, the official server does not always reset the buffer to which it writes a reply packet before sending, and the remnants of past packets may be lost in the enclosing bytes. Fortunately, in the game client itself, this does not lead to errors. But something about the pace of development and the level of quality control it still says.

Less you know - stronger server

... or " the worse the better ." After gaining a critical mass of knowledge about the protocol used, the volume of the source code ceased to grow and began to melt. The interfaces were simplified, and the data that the server had to save (for example, to transfer the state of the lobby to newly arrived players) was no longer analyzed and began to be stored as strings. It is not necessary to know the origin and destination of each byte in the packet; enough to understand how and to whom to redirect.

The server responds to most requests with data received from other clients earlier, or it provokes the necessary client with a special package and redirects the response to the requester. Where possible, I omitted the optional strings, replacing them with a null byte. First of all, this concerns data on the rating of players, the number of points and victories.

In some cases, too wide packet retransmission even caused errors on the client side. In particular, I noticed this on the “request-answer” pair with the command codes 0x1b3 and 0x1b4 , duplicating information about the player’s points and the client’s system.

Stand in line

A separate difficulty for me was presented by the aspect of launching and synchronizing the game. Having dealt with packet relays at boot, I noticed that during the game all clients send packets with the command code 0x4b0 . In this case, in one identifier field is the number of the room creator, and the second field is empty. But if you approach logically and understand it as “the client is everything in the specified room,” then a game becomes out of sync.

Instead, the server itself should monitor the source of the packet and check whether it is a host or a regular player. In the first case, the package is sent to all players in the room except the host itself, in the latter case - exclusively to the host. In this case, the package with the gaming team, apparently, is entered into the queue of what is happening and then sent back from the host, but in the context of all other game events. This ensures the same order of execution of teams for all players. The host sends out game packages constantly and in a certain tact, regardless of the number of events, and all the rest - only when the actions of the player.

By the way, this is why when changing the host during the game a warning is shown that your orders may be lost - if the old host did not manage to enter them into the public queue before disconnecting, then the new host has no opportunity to learn about these commands.

Rudimentary LAN

Contrary to public opinion, the client still has the functions necessary for a full-fledged game on the local network. When analyzing network traffic, I noticed that among the many TCP packets, the creator of the room also sends notifications about the number of players and the status of the room via UDP .

In addition, when disassembling the client, I came across a function that is called when an exception occurs and displays the error text. Walking through its challenges and analyzing the text transmitted to it, you can determine the real names of those procedures in which exceptions occur. A little surprised at the result, I played with strings ,

There are two options: either the developers have this humor, or they have used the abbreviation LAN for everything related to a multiplayer game.

C ++, Asio and unscathed legs

Since I initially set myself the goal of writing a cross-platform server and expanding my knowledge of network programming, for implementation I chose the C ++ language and Asio library. The latter also allowed me to abandon multithreading and related data access features in favor of asynchronous and simpler code. I took the source code of one of the examples in the library repository as a basis.

The most interesting aspect of my development was the problem of the availability of the data buffer during asynchronous sending of packets. At the same time, I tried to minimize the number of allocations and copy data in memory. In addition, the server should have a fairly large buffer for receiving packets, since TCP payload size when transferring card data before starting a game can exceed 800 kilobytes.

As a result, I implemented the process of reading, creating and sending packets as follows:

1. When a new client is connected, the server creates an object of the Session class containing a rather large (1 MiB) buffer of type std :: vector <unsigned char> ( hereinafter referred to as Buffer ).
2. After an asynchronous read operation is completed, its address is transferred to this buffer by the main packet processing function. The next read operation will be initiated only after the completion of this function, ensuring the safety of the data in the buffer during processing.
3. At the beginning of processing, an object of the Packet class is created that provides an interface for reading and serializing data. Through it, the server's response is written in the same buffer of the Session object, which is assigned to the sender of the packet.
4. After all operations on the package have been completed, the send function allocates a buffer using std :: make_shared <Buffer> . In this case, the Buffer constructor is passed an iterator of the same buffer to the Session , taking into account the exact size of the response packet (this is monitored by the Packet during recording). Those. in one operation, we first allocate enough memory for the packet and the control pointer block (and at a time ), then copy into it exactly the number of bytes from the large buffer that should be sent.
5. The new buffer is transmitted using the received pointer of type std :: shared_ptr <Buffer> ( hereinafter referred to as BufPtr ) to Session objects of all those clients to whom the packet should be sent. There it is placed in local queues of type std :: deque <BufPtr> . Each copy of the pointer, which is in the queue for any of the clients, increases the reference count. After that, the packet processing ends, and the initial Session buffer is ready to accept the next packet.
6. After the asynchronous write (send) operation of the packet is completed, the pointer is erased from the destination client’s local Session queue, lowering the reference count. As soon as the packet is sent to all customers who have it in the queue, the counter will be reset and the smart pointer will free memory itself.

No matter how you judge, the addition of C ++ lambda expressions, various containers and smart pointers to a standard has significantly reduced the complexity of designing such systems; when writing the server, not a single shot was shot.

Exposure Test

After finishing the work on the server and testing the main functions, I decided to check the server stability and synchronization after the host transit after a long time. For this, I created a room in the local network with three players and five complex AI . For the test, I chose the maximum parameters regarding the size of the map, the number of deposits, the population and the time of non-aggression. I started the game and after a couple of minutes I stopped the process of playing on the host’s computer in the task manager in order to simulate an unexpected shutdown. Then the two remaining clients were left to themselves all night.

Experienced players probably already guessed what happened next. In short, the server passed the test, but the client did not. The next day, the screen greeted me with a game timer frozen at about eight o'clock, as well as several error messages, among which was the familiar Out of memory. Everything is logical, the two remaining AIs divided the map in half and fought with thousands of armies. The process of the game took about 3.5 GB of RAM and rested against its 32-bit boundaries . The server, in turn, continued to work on its 11 MB of RAM.

Conclusion

I hope it was interesting for you to follow the process of copying the “ black box ” of the official server. I also hope that the game developers will take pity on the players and finally add the possibility of multiplayer games on the local network, with blackjack and fast UDP and without having to have a low ping to Hetzner . The last argument, by the way, played with new colors in view of recent events .

If you have any questions or suggestions regarding the reverse or server source code , welcome to comments. See you again!