Creating a listening application for viewing mobile MMORPG traffic
This is the second part of a series of articles about the analysis of network traffic mobile MMORPG. Sample cycle topics:
In this part, I will describe the creation of a listening application (sniffer), which will allow us to filter events by their type and source, display information about the message and save them for analysis, and also climb a little into the executable game file (“binary”) to find supporting information and add Protocol Buffers support to the application. Interested please under the cat.
To be able to repeat the steps described below, you will need:
As we remember from the previous article , the game communicates via the TCP protocol, and within the session it does this with only one server and on one port. To be able to analyze the game traffic, we need to perform the following tasks:
')
These actions are implemented in the
Great, now we have two buffers with packet data from the client and server. We recall the format of events between the game and the server:
Based on this, you can create a class of event
The
Now that we have an event class, we can add methods in
Create a form class that will trigger a wiretap:
Done! Now you can add a pair of tables for managing the list of events (through it
Although it is now possible to analyze game events without problems, there is a huge amount of manual work to determine not only the meaning of all event codes, but also the structure of the payload, which will be quite difficult, especially if it changes depending on some fields. I decided to look for some information in the executable file of the game. Since the game is cross-platform (available on Windows, iOS and Android), the following options are available for analysis:
Having no idea which architecture to choose for Android and iOS, I started with an .exe file. We load a binary in IDA, we see the choice of architectures.
The purpose of our search is some very useful lines, which means decompiling the assembler is not included in the plans, but just in case, select “executable 80386”, since the options “Binary File” and “MS-DOS executable” are clearly not suitable. Click "OK", wait until the file is loaded into the database, and it is desirable to wait until the end of the file analysis. The end of the analysis can be recognized by the fact that in the status bar at the bottom left there will be the following state:
Go to the Strings tab (View / Open subviews / Strings or
Finally, closer to the end of the table comes across what we were looking for.
This is a list of event codes between the client and the server. This can make life easier for us when parsing the network protocol of the game. But we will not stop there! It is necessary to check whether it is possible to somehow get the numerical value of the event code. We see the “familiar” from the previous article codes
Indeed, immediately after the string values ​​of the codes comes the sequence
Unfortunately, the Windows version of the game lags behind the mobile versions by a lot of versions, and therefore the information from the .exe is not relevant, and although it can help, you should not rely on it completely. Next, I decided to study the dynamic library with Android, since I saw on one forum that there, unlike iOS binaries, there is a lot of meta-information about classes. But alas, searching the file for
Without hope, I’m doing the same search in the iOS binary as incredible, but it turned out to have the same data as in .exe! Upload the file to IDA.
I choose the first proposed option, because I'm not sure which one is needed. Again we are waiting for the end of the file analysis (the binary is almost 4 times the size of the .exe, the analysis time, of course, has also increased). Open the window with lines, which this time turned out to be 51k. Through
For some reason, IDA put the whole
Finally, it is worth remembering how, in the comments to the last article, aml suggested that the message structure of the game is an implementation of Protocol Buffers . If you look closely at the code in the binary file, you can see that the description of
Let's write a parser template for 010Editor to get all the code values.
As a result, we get something like this:
More interesting! Noticed
The results are extremely unexpected. Apparently, many types of data are described in the executable game file, including enumerations and message formats between the server and the client. Here is an example of a description of a type describing the position of an object in the world:
A quick search revealed two large places with descriptions of types, although with a more careful study, other small places will most likely come to light. Cutting them out, I wrote a small C # script to separate the descriptions by files (this is similar in structure to the description of the event code list) - it is easier to analyze them in 010Editor.
I will not analyze the format of the description of structures in detail, because either it is specific to the game in question, or it is a common format in
Well, the last thing left for us is to use the information received in our listening application: parse messages using the
Debug output allows you to create a brief description from this:
Now you can create the corresponding class using the
For comparison, here is a template of the same event from the previous article:
When inheriting from the
That's all. The next step is to redirect game traffic to our proxy server for the purpose of injection, substitution, and package cutting.
- Parsing message format between server and client.
- Writing a listening application to view game traffic in a convenient way.
- Interception of traffic and its modification using a non-HTTP proxy server.
- The first steps to your own ("pirated") server.
In this part, I will describe the creation of a listening application (sniffer), which will allow us to filter events by their type and source, display information about the message and save them for analysis, and also climb a little into the executable game file (“binary”) to find supporting information and add Protocol Buffers support to the application. Interested please under the cat.
Required Tools
To be able to repeat the steps described below, you will need:
- Wireshark for packet analysis;
- .NET;
- PcapDotNet library for working with WinPcap;
- protobuf-net library for working with Protocol Buffers.
Writing a listening application
As we remember from the previous article , the game communicates via the TCP protocol, and within the session it does this with only one server and on one port. To be able to analyze the game traffic, we need to perform the following tasks:
')
- intercept mobile device packets;
- filter game packages;
- add the data of the next packet to the buffer for further processing;
- get game events from buffers as it is filled.
These actions are implemented in the
Sniffer class, which uses the PcapDotNet library to intercept packets. In the Sniff method we transfer the IP address of the adapter (in fact, this is the address of the PC from which Wi-Fi for the mobile device is distributed within the same network), the IP address of the mobile device and the IP address of the server. Due to the inconstancy of the last two (after many months of monitoring different platforms and servers, it turned out that the server is selected from a pool of ~ 50 servers, each of which still have 5-7 possible ports), I transmit only the first three octets. The use of this filtering is seen in the IsTargetPacket method. public class Sniffer { private byte[] _data = new byte[4096]; public bool Active { get; set; } = true; private string _adapterIP; private string _target; private string _server; private List<byte> _serverBuffer; private List<byte> _clientBuffer; private LivePacketDevice _device = null; private PacketCommunicator _communicator = null; private Action<Event> _eventCallback = null; public void Sniff(string ip, string target, string server) { _adapterIP = ip; _target = target; _server = server; _serverBuffer = new List<byte>(); _clientBuffer = new List<byte>(); IList<LivePacketDevice> allDevices = LivePacketDevice.AllLocalMachine; for (int i = 0; i != allDevices.Count; ++i) { LivePacketDevice device = allDevices[i]; var address = device.Addresses[1].Address + ""; if (address == "Internet " + _adapterIP) { _device = device; } } _communicator = _device.Open(65536, PacketDeviceOpenAttributes.Promiscuous, 1000); _communicator.SetFilter(_communicator.CreateFilter("ip and tcp")); new Thread(() => { Thread.CurrentThread.IsBackground = true; BeginReceive(); }).Start(); } private void BeginReceive() { _communicator.ReceivePackets(0, OnReceive); do { PacketCommunicatorReceiveResult result = _communicator.ReceivePacket(out Packet packet); switch (result) { case PacketCommunicatorReceiveResult.Timeout: continue; case PacketCommunicatorReceiveResult.Ok: OnReceive(packet); break; } } while (Active); } public void AddEventCallback(Action<Event> callback) { _eventCallback = callback; } private void OnReceive(Packet packet) { if (Active) { IpV4Datagram ip = packet.Ethernet.IpV4; if (IsTargetPacket(ip)) { try { ParseData(ip); } catch (ObjectDisposedException) { } catch (EndOfStreamException e) { Console.WriteLine(e); } catch (Exception) { throw; } } } } private bool IsTargetPacket(IpV4Datagram ip) { var sourceIp = ip.Source.ToString(); var destIp = ip.Destination.ToString(); return (sourceIp != _adapterIP && destIp != _adapterIP) && ( (sourceIp.StartsWith(_target) && destIp.StartsWith(_server)) || (sourceIp.StartsWith(_server) && destIp.StartsWith(_target)) ); } private void ParseData(IpV4Datagram ip) { TcpDatagram tcp = ip.Tcp; if (tcp.Payload != null && tcp.PayloadLength > 0) { var payload = ExtractPayload(tcp); AddToBuffer(ip, payload); ProcessBuffers(); } } private byte[] ExtractPayload(TcpDatagram tcp) { int payloadLength = tcp.PayloadLength; MemoryStream ms = tcp.Payload.ToMemoryStream(); byte[] payload = new byte[payloadLength]; ms.Read(payload, 0, payloadLength); return payload; } private void AddToBuffer(IpV4Datagram ip, byte[] payload) { if (ip.Destination.ToString().StartsWith(_target)) { foreach (var value in payload) _serverBuffer.Add(value); } else { foreach (var value in payload) _clientBuffer.Add(value); } } private void ProcessBuffers() { ProcessBuffer(ref _serverBuffer); ProcessBuffer(ref _clientBuffer); } private void ProcessBuffer(ref List<byte> buffer) { // TODO } public void Suspend() { Active = false; } public void Resume() { Active = true; } } Great, now we have two buffers with packet data from the client and server. We recall the format of events between the game and the server:
struct Event { uint payload_length <bgcolor=0xFFFF00, name="Payload Length">; ushort event_code <bgcolor=0xFF9988, name="Event Code">; byte payload[payload_length] <name="Event Payload">; }; Based on this, you can create a class of event
Event : public enum EventSource { Client, Server } public enum EventTypes : ushort { Movement = 11, Ping = 30, Pong = 31, Teleport = 63, EnterDungeon = 217 } public class Event { public uint ID; public uint Length { get; protected set; } public ushort Type { get; protected set; } public uint DataLength { get; protected set; } public string EventType { get; protected set; } public EventSource Direction { get; protected set; } protected byte[] _data; protected BinaryReader _br = null; public Event(byte[] data, EventSource direction) { _data = data; _br = new BinaryReader(new MemoryStream(_data)); Length = _br.ReadUInt32(); Type = _br.ReadUInt16(); DataLength = 0; EventType = $"Unknown ({Type})"; if (IsKnown()) { EventType = ((EventTypes)Type).ToString(); } Direction = direction; } public virtual void ParseData() { } public bool IsKnown() { return Enum.IsDefined(typeof(EventTypes), Type); } public byte[] GetPayload(bool hasDatLength = true) { var payloadLength = _data.Length - (hasDatLength ? 10 : 6); return new List<byte>(_data).GetRange(hasDatLength ? 10 : 6, payloadLength).ToArray(); } public virtual void Save() { var path = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Packets", EventType); Directory.CreateDirectory(path); File.WriteAllBytes(path + $"/{ID}.dump", _data); } public override string ToString() { return $"Type {Type}. Data length: {Length}."; } protected ulong ReadVLQ(bool readFlag = true) { if (readFlag) { var flag = _br.ReadByte(); } ulong vlq = 0; var i = 0; for (i = 0; ; i += 7) { var x = _br.ReadByte(); vlq |= (ulong)(x & 0x7F) << i; if ((x & 0x80) != 0x80) { break; } } return vlq; } } The
Event class will be used as the base class for all game events. Here is an example class for the Ping event: public class Ping : Event { private ulong _pingTime; public Ping(byte[] data) : base(data, EventSource.Client) { EventType = "Ping"; DataLength = 4; _pingTime = _br.ReadUInt32(); } public override string ToString() { return $"Pinging server at {_pingTime}ms."; } } Now that we have an event class, we can add methods in
Sniffer : private void ProcessBuffer(ref List<byte> buffer) { if (buffer.Count > 0) { while (Active) { if (buffer.Count > 4) // 4 ... { var eventLength = BitConverter.ToInt32(buffer.Take(4).ToArray(), 0) + 6; // ... - .. + 4 + 2 if (eventLength >= 6 && buffer.Count >= eventLength) { var eventData = buffer.Take(eventLength).ToArray(); var ev = CreateEvent(eventData, direction); buffer.RemoveRange(0, eventLength); continue; } } break; } } } private Event CreateEvent(byte[] data, EventSource direction) { var ev = new Event(data, direction); var eventType = Enum.GetName(typeof(EventTypes), ev.Type); if (eventType != null) { try { // (, <code>Ping</code>). var className = "Events." + eventType; Type t = Type.GetType(className); ev = (Event)Activator.CreateInstance(t, data); } catch (Exception) { // - . ev = new Event(data, direction); } finally { } } _eventCallback?.Invoke(ev); return ev; } Create a form class that will trigger a wiretap:
public partial class MainForm : Form { private Sniffer _sniffer = null; private List<Event> _events = new List<Event>(); private List<ushort> _eventTypesFilter = new List<ushort>(); private bool _showClientEvents = true; private bool _showServerEvents = true; private bool _showUnknownEvents = false; private bool _clearLogsOnRestart = true; private uint _eventId = 1; private void InitializeSniffer() { _sniffer = new Sniffer(); _sniffer.AddEventCallback(NewEventThreaded); _sniffer.Sniff("192.168.137.1", "192.168.137.", "123.45.67."); } private void NewEventThreaded(Event ev) { events_table.Invoke(new NewEventCallback(NewEvent), ev); } public delegate void NewEventCallback(Event ev); private void NewEvent(Event ev) { ev.ID = _eventId++; _events.Add(ev); LogEvent(ev); } private void LogEvent(Event ev) { if (FilterEvent(ev)) { var type = ev.GetType(); events_table.Rows.Add(1); events_table.Rows[events_table.RowCount - 1].Cells[0].Value = ev.ID; events_table.Rows[events_table.RowCount - 1].Cells[1].Value = ev.EventType; events_table.Rows[events_table.RowCount - 1].Cells[2].Value = Enum.GetName(typeof(EventSource), ev.Direction); events_table.Rows[events_table.RowCount - 1].Cells[3].Value = ev.ToString(); } } private void ReloadEvents() { events_table.Rows.Clear(); events_table.Refresh(); foreach (var ev in _events) { LogEvent(ev); } } private bool FilterEvent(Event ev) { return ( (ev.Direction == EventSource.Client && _showClientEvents) || (ev.Direction == EventSource.Server && _showServerEvents) ) && (_eventTypesFilter.Contains(ev.Type) || (!ev.IsKnown() && _showUnknownEvents)); } } Done! Now you can add a pair of tables for managing the list of events (through it
_eventTypesFilter filled) and for viewing in real time (the main table events_table ). For example, I filtered by the following criteria ( FilterEvent method):- show events from the client;
- show events from the server;
- showing unknown events;
- showing selected known events.
We study the executable file of the game
Although it is now possible to analyze game events without problems, there is a huge amount of manual work to determine not only the meaning of all event codes, but also the structure of the payload, which will be quite difficult, especially if it changes depending on some fields. I decided to look for some information in the executable file of the game. Since the game is cross-platform (available on Windows, iOS and Android), the following options are available for analysis:
- The .exe file (I have the path C: / Program Files / WindowsApps /% appname% /;
- iOS binary file that is encrypted by Apple, but with JailBreak you can get the decrypted version using Crackulous or similar;
- dynamic library (shared object) Android. Located along the path / data / data /% app-vendor-name% / lib /.
Having no idea which architecture to choose for Android and iOS, I started with an .exe file. We load a binary in IDA, we see the choice of architectures.
The purpose of our search is some very useful lines, which means decompiling the assembler is not included in the plans, but just in case, select “executable 80386”, since the options “Binary File” and “MS-DOS executable” are clearly not suitable. Click "OK", wait until the file is loaded into the database, and it is desirable to wait until the end of the file analysis. The end of the analysis can be recognized by the fact that in the status bar at the bottom left there will be the following state:
Go to the Strings tab (View / Open subviews / Strings or
Shift + F12 ). The process of generating strings may take some time. In my case ~ 47k lines were found. Line address addresses have a prefix of the form .data , .rdata and others . In my case, all the “interesting” lines were in the .rdata section, the size of which was ~ 44.5k records. Looking through the table you can see:- error messages and query segments at the entry stage;
- error lines and initialization information of the game, game engine, interfaces;
- a lot of trash;
- list of game tables on the client side;
- used values ​​of the game engine in the game;
- list of effects;
- a huge list of interface localization keys;
- etc.
Finally, closer to the end of the table comes across what we were looking for.
This is a list of event codes between the client and the server. This can make life easier for us when parsing the network protocol of the game. But we will not stop there! It is necessary to check whether it is possible to somehow get the numerical value of the event code. We see the “familiar” from the previous article codes
CMSG_PING and SMSG_PONG , having codes 30 ( 1E 16 ) and 31 ( 1F 16 ), respectively. Double click on the line to go to this place in the code.Indeed, immediately after the string values ​​of the codes comes the sequence
0x10 0x1E and 0x10 0x1F . Well, that means you can parse the entire table and get a list of events and their numerical value, which will further simplify the analysis of the protocol.Unfortunately, the Windows version of the game lags behind the mobile versions by a lot of versions, and therefore the information from the .exe is not relevant, and although it can help, you should not rely on it completely. Next, I decided to study the dynamic library with Android, since I saw on one forum that there, unlike iOS binaries, there is a lot of meta-information about classes. But alas, searching the file for
CMSG_PING values CMSG_PING not return any results.Without hope, I’m doing the same search in the iOS binary as incredible, but it turned out to have the same data as in .exe! Upload the file to IDA.
I choose the first proposed option, because I'm not sure which one is needed. Again we are waiting for the end of the file analysis (the binary is almost 4 times the size of the .exe, the analysis time, of course, has also increased). Open the window with lines, which this time turned out to be 51k. Through
Ctrl + F we search for CMSG_PING and ... we do not find it. By entering the code character by character, you can notice the following result:For some reason, IDA put the whole
Opcode.proto object in one line. Double click on this place in the code and see that the structure is described in the same way as in the .exe file, so you can cut it out and convert it to Enum .Finally, it is worth remembering how, in the comments to the last article, aml suggested that the message structure of the game is an implementation of Protocol Buffers . If you look closely at the code in the binary file, you can see that the description of
Opcode also in this format.Let's write a parser template for 010Editor to get all the code values.
Updated Packed * type code for 010Editor
Small changes in types include checking field labels to skip missing ones.
uint PeekTag() { if (FTell() == FileSize()) { return 0; } Varint tag; FSkip(-tag.size); return tag._ >> 3; } struct Packed (uint fieldNumber) { if (PeekTag() != fieldNumber) { break; } Varint key <bgcolor=0xFFBB00>; local uint wiredType = key._ & 0x7; local uint field = key._ >> 3; local uint size = key.size; switch (wiredType) { case 1: double value; size += 8; break; case 5: float value; size += 4; break; default: Varint value; size += value.size; break; } }; struct PackedString(uint fieldNumber) { if (PeekTag() != fieldNumber) { break; } Packed length(fieldNumber); char str[length.value._]; }; struct Code { Packed size(2) <bgcolor=0x00FF00>; PackedString code_name(1) <bgcolor=0x00FF00>; Packed code_value(2) <bgcolor=0x00FF00>; Printf("%s = %d,\n", code_name.str, code_value.value._); // Enum }; struct Property { Packed size(5) <bgcolor=0x00FF00>; PackedString prop_name(1) <bgcolor=0x00FF00>; while (FTell() - 0x176526B - prop_name.length.value._ < size.value._) { Code codes <name="Codes">; } }; struct { FSkip(0x176526B); PackedString object(1) <bgcolor=0x00FF00>; PackedString format(2) <bgcolor=0x00FFFF>; Property prop; } file; As a result, we get something like this:
More interesting! Noticed
pb in the description of the object? It would be necessary to look for other lines, what if there are many more such objects?The results are extremely unexpected. Apparently, many types of data are described in the executable game file, including enumerations and message formats between the server and the client. Here is an example of a description of a type describing the position of an object in the world:
A quick search revealed two large places with descriptions of types, although with a more careful study, other small places will most likely come to light. Cutting them out, I wrote a small C # script to separate the descriptions by files (this is similar in structure to the description of the event code list) - it is easier to analyze them in 010Editor.
class Program { static void Main(string[] args) { var br = new BinaryReader(new FileStream("./BinaryFile.partX", FileMode.Open)); while (br.BaseStream.Position < br.BaseStream.Length) { var startOffset = br.BaseStream.Position; var length = ReadVLQ(br, out int size); var tag = br.ReadByte(); var eventName = br.ReadString(); br.BaseStream.Position = startOffset; File.WriteAllBytes($"./parsed/{eventName}", br.ReadBytes((int)length + size + 1)); } } static ulong ReadVLQ(BinaryReader br, out int size) { var flag = br.ReadByte(); ulong vlq = 0; size = 0; var i = 0; for (i = 0; ; i += 7) { var x = br.ReadByte(); vlq |= (ulong)(x & 0x7F) << i; size++; if ((x & 0x80) != 0x80) { break; } } return vlq; } } I will not analyze the format of the description of structures in detail, because either it is specific to the game in question, or it is a common format in
Protocol Buffers (if anyone knows for sure, please indicate in the comments). From what I could find:- the description also comes in the
Protocol Buffersformat; - The description of each field contains its name, number and data type for which its own type table was used:
string TypeToStr (uint type) { switch (type) { case 2: return "Float"; case 4: return "UInt64"; case 5: return "UInt32"; case 8: return "Boolean"; case 9: return "String"; case 11: return "Struct"; case 14: return "Enum"; default: local string s; SPrintf(s, "%Lu", type); return s; } }; - if the data type is an enumeration or structure, then there was a link to the desired object.
Well, the last thing left for us is to use the information received in our listening application: parse messages using the
protobuf-net library. Connect the library via NuGet, add using ProtoBuf; and you can create classes to describe the messages. Take one of the examples from the previous article: character movement. The parsed format description when highlighting segments looks like this:Debug output allows you to create a brief description from this:
Field 1 (Type 13): time Field 2 (Struct .pb.CxGS_Vec3): position Field 3 (UInt64): guid Field 4 (Struct .pb.CxGS_Vec3): direction Field 5 (Struct .pb.CxGS_Vec3): speed Field 6 (UInt32): state Field 10 (UInt32): flag Field 11 (Float): y_speed Field 12 (Boolean): is_flying Field 7 (UInt32): emote_id Field 9 (UInt32): emote_duration Field 8 (Boolean): emote_loop Now you can create the corresponding class using the
protobuf-net library. [ProtoContract] public class MoveInfo : ProtoBufEvent<MoveInfo> { [ProtoMember(3)] public ulong GUID; [ProtoMember(1)] public ulong Time; [ProtoMember(2)] public Vec3 Position; [ProtoMember(4)] public Vec3 Direction; [ProtoMember(5)] public Vec3 Speed; [ProtoMember(6)] public ulong State; [ProtoMember(7, IsRequired = false)] public uint EmoteID; [ProtoMember(8, IsRequired = false)] public bool EmoteLoop; [ProtoMember(9, IsRequired = false)] public uint EmoteDuration; [ProtoMember(10, IsRequired = false)] public uint Flag; [ProtoMember(11, IsRequired = false)] public float SpeedY; [ProtoMember(12)] public bool IsFlying; public override string ToString() { return $"{GUID}: {Position}"; } } For comparison, here is a template of the same event from the previous article:
struct MoveEvent { uint data_length <bgcolor=0x00FF00, name="Data Length">; Packed move_time <bgcolor=0x00FFFF>; PackedVector3 position <bgcolor=0x00FF00>; PackedVector3 direction <bgcolor=0x00FF00>; PackedVector3 speed <bgcolor=0x00FF00>; Packed state <bgcolor=0x00FF00>; }; When inheriting from the
Event class, we can override the ParseData method, deserializing the package data: class CMSG_MOVE_INFO : Event { private MoveInfo _message; [...] public override void ParseData() { _message = MoveInfo.Deserialize(GetPayload()); } public override string ToString() { return _message.ToString(); } } That's all. The next step is to redirect game traffic to our proxy server for the purpose of injection, substitution, and package cutting.
Source: https://habr.com/ru/post/457480/
All Articles