How I wrote my chat
Hi, Habr!
In the article I wrote about how I designed the chat. About its architecture and technical decisions taken during its development.
Chat is a client-server application with p2p elements.
With support:
')
Project Source Code: GitHub
So rushed.
At the beginning when I wrote the first version, I immediately wrote an asynchronous version of the client and server. But, for some reason, I completely forgot about the need to synchronize data. And so, as the chat never experienced a serious load, I realized this only after introducing file transfer to the chat. After that, everything was immediately remembered and a bunch of lockers were inserted everywhere. What, of course, was not the best solution. To be precise, it was just a little better than a program without synchronization.
Now, on the client and on the server, a single data access mechanism is used. The model is blocked completely. I must say that this is not the best solution for the server.
The idea is quite simple: there is a context that contains a model in a private static field. In the constructor, it calls Monitor.Enter. On the model itself, or on a separate synchronization object. The context also implements the IDisposable interface and in the Dispose method it releases this model by calling the Monitor.Exit method.
A generic class used for both server and client. In the example, the model is not contained in the context itself, but in the class that creates it.
As a result, if you want to access data, you do not want to block them, and you no longer think about synchronization. The main thing to remember is to use the using construct. For the server, this is not the best solution. half of the teams work with the maximum of two users, and as a result, all are blocked.
The context in the program can create only one entity (ServerModel - (unexpectedly) for the server, and ClientModel - for the client). It is a class containing a static private model (itself), an API as well as a client connection and a peer - for a client model or a server for a server one. (API, client, etc. are contained as static fields). Also, the client model, in contrast to the server model, also contains events. On which the user interface will be signed. In general, these classes act as basic to access anything.
As an example, I will give the server model (it is smaller). Pay attention to the Get () method that creates the context.
The API in this case is the chat logic. It is a class that stores in itself commands that can be executed on our side, and a set of methods that send commands to the other side (to the Client, if we consider ourselves from the server side. For a client, this is a server or another feast). The methods contain the most complex commands, or simply frequently used.
The whole system works as follows: as soon as a client or server receives a data packet, it passes it to the API for analysis. (The server accepts messages from its connection, and they, in turn, pull one method from the server, indicating that the data has been received). The API simply reads the first two bytes of the message and looks for a command in the dictionary with the desired id, and returns it. Or an empty command that does nothing if there is no such id. Next, the received packet is transmitted to the command, and the id of the connection that sent it, and it is executed.
API also has its own interface, it was not originally there. Appeared after I decided to write another implementation of it, it was assumed that this would be a protected API. But then I just became not interested, and I did not abandon the project for a long time. Two months. After returning to it, I no longer wanted to do all this, and I started implementing P2P.
The client, by the way, knows how to choose the API that the server uses, and if there is no one, it disconnects from the server and says that it does not support the server API. This is implemented quite simply - after the server accepts the connection, it immediately sends a string with the name of its API, and the client actually expects this string and sets the necessary interface. Well, or does not install if this does not support. After this action, there is already a request for registering a user on the server.
Server method that processes received packets:
The full code of the API class (in this case, the server class):
Each team implements the command interface. For the IServerAPICommand server, for the IClientAPICommand client, at this stage they could be reduced to 1 interface, but I don’t want to do this for some reason. It also contains its Id and data necessary for its execution, which is described by the MessageContent class. However, the team may not need data. And she herself is responsible for deserializing the set of bytes into an instance of the class.
Sample command. In this case, this is the command to add a file to the room:
I recently started adding voice chat, maybe at the time of publishing it will still be in the demo version. But already managed to tinker with playback and recording sound.
The first option was the WinApi waveIn * waveOut * function. It was the easiest option, so I started with it. But it did not happen to them, because on the version of framework 3.5, the marshaling on the x64 platform inadequately worked, and when it started up, the application simply fell without any exceptions. When assembling under x86, everything was fine.
Next was an attempt to connect DirectSound, but he had found his bug with a way to notify about the completion of playing a piece of data. After googling this topic, it turned out that Mircosoft abandoned DirectSound long ago and are working with XAudio2. Moreover, its use would lead to the need to compile 2 versions of x86 and x64.
Since I didn’t want to write a wrapper for XAudio2 myself, I remembered OpenAL. For which, in addition, there is a wrapper (OpenTK), also with open source. From OpenTK, only the audio library itself was neatly cut out. Which now works in the program.
Since I had to work with OpenGL ES 2, I became friends immediately with OpenAL. Especially if we consider that there are examples on it on the official OpenTK website.
To write data, I use a fairly simple scheme from the example. When you start recording, a timer is started, the response period of which is adjusted for the time over which the buffer should be filled to half. After that, the data from it is read and sent by the ClientPlayVoiceCommand team to everyone who can listen to us.
The code can be viewed in the TCPChat \ Engine \ Audio \ OpenAL branch.
Initially, the chat was one main room where all users were located. Of the data transfer protocols, only TCP was used. So - as it already provides the reliability of data transmission, it only remained to break its continuous stream into messages.
This was done by simply adding the size of the message to its beginning.
That is, the package is the following:
The first 4 bytes are the size of the message.
5-6 bytes - the command identifier.
The remaining data is serialized by MessageContent.
Further, in one forum I was offered to introduce file transfer and voice communication. I coped with the files almost immediately, although in the first version the files were transmitted through the server, which was generally terrible. After that, I thought about the fact that it would be good to transfer them to the line. As you know with the NAT problem, this is not so easy to do.
I was busy for a long time and tried to implement a NAT traversal using TCP, then I wouldn’t have to be soared about the unreliability of UDP. With him, nothing happened. After that, it was decided to use UDP and UDP hole punching technology.
But first we had to solve the problems of reliability. As usual, I started working on my protocol over UDP to ensure reliable message delivery to me. And I did it all the same, but it worked only locally. When testing it in real conditions, he apparently loaded the network so much that my computer completely hung up.
After that, I began to look for already implemented libraries and came across satyu on Habré, where a similar problem was solved with the help of Lidgren.Network. She was chosen.
NAT traversal is implemented at the API level. The scheme is simple, you just need the peers to find out the real addresses at which the server sees them. After that, one feast should throw a message to another. This message may not come, but it will create a rule on the router that messages from the address to which it was sent need to be delivered to this particular computer. After that, with the help of the server, another feast learns that it is already possible for it to connect and, in fact, connects.
And so, the sequence of actions:
Black arrows indicate sending commands. Red initialization of Lidgren.Network connections.
All this algorithm is hidden in AsyncPeer, and it is enough to call the SendMessage method, and if the client is not connected, it will connect and send a message. Or immediately send if already connected.
Voice commands initiate a connection a little differently. When creating a voice room, the server creates a connection map in the room. In which it is written where every user in the room should connect. And voice playback commands are sent using the AsyncPeer.SendMessageIfConnected method, which simply throws out the message if there is no connection.
Finally, a little about the program interface.
It is developed using WPF and MVVM pattern. Very flexible technology, although in version 3.5 it’s a bit damp, but it nonetheless makes it possible to bypass damp places. As an example, some Command properties are still not dependent, and for them I had to write a wrapper CommandReference.
CommandReference in this case contains a real command - a dependent property on which you can hang the binding. The wrapper itself is hosted in static resources. And it is used in the right place, really causing the command to come.
I also had to use AttachedProperty instead of wrappers, which in turn change the necessary ones.
To notify the interface of changes, it was decided to use the event model, while all the necessary information is transmitted in the events to display the changes.
In other matters, there is nothing more to write about interfaces, WPF as WPF.
In the article I wrote about how I designed the chat. About its architecture and technical decisions taken during its development.
Chat is a client-server application with p2p elements.
With support:
- Private messages
- Rooms
- File transfer
- Voice chat.
')
Project Source Code: GitHub
So rushed.
Model
Data and their synchronization.
API.
Record and play sound.
Data and their synchronization.
At the beginning when I wrote the first version, I immediately wrote an asynchronous version of the client and server. But, for some reason, I completely forgot about the need to synchronize data. And so, as the chat never experienced a serious load, I realized this only after introducing file transfer to the chat. After that, everything was immediately remembered and a bunch of lockers were inserted everywhere. What, of course, was not the best solution. To be precise, it was just a little better than a program without synchronization.
Now, on the client and on the server, a single data access mechanism is used. The model is blocked completely. I must say that this is not the best solution for the server.
The idea is quite simple: there is a context that contains a model in a private static field. In the constructor, it calls Monitor.Enter. On the model itself, or on a separate synchronization object. The context also implements the IDisposable interface and in the Dispose method it releases this model by calling the Monitor.Exit method.
A generic class used for both server and client. In the example, the model is not contained in the context itself, but in the class that creates it.
Code
public abstract class ModelContext<TModel> : IDisposable { #region consts private const int TimeOut = 10000; #endregion #region fields private static object syncObject = new object(); protected TModel model; #endregion #region initialization protected ModelContext(TModel initialModel) { if (!Monitor.TryEnter(syncObject, TimeOut)) throw new InvalidOperationException("model lock timeout"); model = initialModel; } public void Dispose() { model = default(TModel); Monitor.Exit(syncObject); } #endregion } } As a result, if you want to access data, you do not want to block them, and you no longer think about synchronization. The main thing to remember is to use the using construct. For the server, this is not the best solution. half of the teams work with the maximum of two users, and as a result, all are blocked.
The context in the program can create only one entity (ServerModel - (unexpectedly) for the server, and ClientModel - for the client). It is a class containing a static private model (itself), an API as well as a client connection and a peer - for a client model or a server for a server one. (API, client, etc. are contained as static fields). Also, the client model, in contrast to the server model, also contains events. On which the user interface will be signed. In general, these classes act as basic to access anything.
As an example, I will give the server model (it is smaller). Pay attention to the Get () method that creates the context.
Code
public class ServerModel { #region static model private static ServerModel model; /// <summary> /// API /// </summary> public static IServerAPI API { get; private set; } /// <summary> /// /// </summary> public static AsyncServer Server { get; private set; } /// <summary> /// using /// </summary> /// <example>using (var server = SeeverModel.Get()) { ... }</example> /// <returns> .</returns> public static ServerContext Get() { if (Interlocked.CompareExchange(ref model, null, null) == null) throw new ArgumentException("model do not inited yet"); return new ServerContext(model); } #endregion #region consts public const string MainRoomName = "Main room"; #endregion #region properties public Dictionary<string, Room> Rooms { get; private set; } public Dictionary<string, User> Users { get; private set; } #endregion #region constructor public ServerModel() { Users = new Dictionary<string, User>(); Rooms = new Dictionary<string, Room>(); Rooms.Add(MainRoomName, new Room(null, MainRoomName)); } #endregion #region static methods public static bool IsInited { get { return Interlocked.CompareExchange(ref model, null, null) != null; } } public static void Init(IServerAPI api) { if (Interlocked.CompareExchange(ref model, new ServerModel(), null) != null) throw new InvalidOperationException("model already inited"); Server = new AsyncServer("ServerErrors.log"); API = api; } public static void Reset() { if (Interlocked.Exchange(ref model, null) == null) throw new InvalidOperationException("model not yet inited"); if (Server != null) { Server.Dispose(); Server = null; } API = null; } #endregion } API
The API in this case is the chat logic. It is a class that stores in itself commands that can be executed on our side, and a set of methods that send commands to the other side (to the Client, if we consider ourselves from the server side. For a client, this is a server or another feast). The methods contain the most complex commands, or simply frequently used.
The whole system works as follows: as soon as a client or server receives a data packet, it passes it to the API for analysis. (The server accepts messages from its connection, and they, in turn, pull one method from the server, indicating that the data has been received). The API simply reads the first two bytes of the message and looks for a command in the dictionary with the desired id, and returns it. Or an empty command that does nothing if there is no such id. Next, the received packet is transmitted to the command, and the id of the connection that sent it, and it is executed.
API also has its own interface, it was not originally there. Appeared after I decided to write another implementation of it, it was assumed that this would be a protected API. But then I just became not interested, and I did not abandon the project for a long time. Two months. After returning to it, I no longer wanted to do all this, and I started implementing P2P.
The client, by the way, knows how to choose the API that the server uses, and if there is no one, it disconnects from the server and says that it does not support the server API. This is implemented quite simply - after the server accepts the connection, it immediately sends a string with the name of its API, and the client actually expects this string and sets the necessary interface. Well, or does not install if this does not support. After this action, there is already a request for registering a user on the server.
Server method that processes received packets:
Code
public class DataReceivedEventArgs : EventArgs { public byte[] ReceivedData { get; set; } public Exception Error { get; set; } } public interface IServerAPICommand { void Run(ServerCommandArgs args); } public class ServerCommandArgs { public string ConnectionId { get; set; } public byte[] Message { get; set; } } private void DataReceivedCallBack(object sender, DataReceivedEventArgs e) { try { if (e.Error != null) throw e.Error; if (!isServerRunning) return; IServerAPICommand command = ServerModel.API.GetCommand(e.ReceivedData); ServerCommandArgs args = new ServerCommandArgs { Message = e.ReceivedData, ConnectionId = ((ServerConnection)sender).Id, }; command.Run(args); } catch (Exception exc) { ServerModel.Logger.Write(exc); } } The full code of the API class (in this case, the server class):
Code
/// <summary> /// API. /// </summary> public class StandardServerAPI : IServerAPI { /// <summary> /// API. /// </summary> public const string API = "StandartAPI v2.0"; private Dictionary<ushort, IServerAPICommand> commandDictionary = new Dictionary<ushort, IServerAPICommand>(); /// <summary> /// API. /// </summary> /// <param name="host"> API.</param> public StandardServerAPI() { commandDictionary.Add(ServerRegisterCommand.Id, new ServerRegisterCommand()); commandDictionary.Add(ServerUnregisterCommand.Id, new ServerUnregisterCommand()); commandDictionary.Add(ServerSendRoomMessageCommand.Id, new ServerSendRoomMessageCommand()); commandDictionary.Add(ServerSendPrivateMessageCommand.Id, new ServerSendPrivateMessageCommand()); commandDictionary.Add(ServerGetUserOpenKeyCommand.Id, new ServerGetUserOpenKeyCommand()); commandDictionary.Add(ServerCreateRoomCommand.Id, new ServerCreateRoomCommand()); commandDictionary.Add(ServerDeleteRoomCommand.Id, new ServerDeleteRoomCommand()); commandDictionary.Add(ServerInviteUsersCommand.Id, new ServerInviteUsersCommand()); commandDictionary.Add(ServerKickUsersCommand.Id, new ServerKickUsersCommand()); commandDictionary.Add(ServerExitFormRoomCommand.Id, new ServerExitFormRoomCommand()); commandDictionary.Add(ServerRefreshRoomCommand.Id, new ServerRefreshRoomCommand()); commandDictionary.Add(ServerSetRoomAdminCommand.Id, new ServerSetRoomAdminCommand()); commandDictionary.Add(ServerAddFileToRoomCommand.Id, new ServerAddFileToRoomCommand()); commandDictionary.Add(ServerRemoveFileFormRoomCommand.Id, new ServerRemoveFileFormRoomCommand()); commandDictionary.Add(ServerP2PConnectRequestCommand.Id, new ServerP2PConnectRequestCommand()); commandDictionary.Add(ServerP2PReadyAcceptCommand.Id, new ServerP2PReadyAcceptCommand()); commandDictionary.Add(ServerPingRequestCommand.Id, new ServerPingRequestCommand()); } /// <summary> /// API. /// </summary> public string Name { get { return API; } } /// <summary> /// . /// </summary> /// <param name="message"> , .</param> /// <returns> .</returns> public IServerAPICommand GetCommand(byte[] message) { ushort id = BitConverter.ToUInt16(message, 0); IServerAPICommand command; if (commandDictionary.TryGetValue(id, out command)) return command; return ServerEmptyCommand.Empty; } /// <summary> /// . /// </summary> /// <param name="container"></param> public void IntroduceConnections(string senderId, IPEndPoint senderPoint, string requestId, IPEndPoint requestPoint) { using (var context = ServerModel.Get()) { var content = new ClientWaitPeerConnectionCommand.MessageContent { RequestPoint = requestPoint, SenderPoint = senderPoint, RemoteInfo = context.Users[senderId], }; ServerModel.Server.SendMessage(requestId, ClientWaitPeerConnectionCommand.Id, content); } } /// <summary> /// . /// </summary> /// <param name="nick"> .</param> /// <param name="message">.</param> public void SendSystemMessage(string nick, string message) { var sendingContent = new ClientOutSystemMessageCommand.MessageContent { Message = message }; ServerModel.Server.SendMessage(nick, ClientOutSystemMessageCommand.Id, sendingContent); } /// <summary> /// . /// </summary> /// <param name="nick"> , .</param> public void CloseConnection(string nick) { ServerModel.Server.CloseConnection(nick); using (var server = ServerModel.Get()) { foreach (string roomName in server.Rooms.Keys) { Room room = server.Rooms[roomName]; if (!room.Users.Contains(nick)) continue; room.Remove(nick); server.Users.Remove(nick); var sendingContent = new ClientRoomRefreshedCommand.MessageContent { Room = room, Users = room.Users.Select(n => server.Users[n]).ToList() }; foreach (string user in room.Users) { if (user == null) continue; ServerModel.Server.SendMessage(user, ClientRoomRefreshedCommand.Id, sendingContent); } } } } } Each team implements the command interface. For the IServerAPICommand server, for the IClientAPICommand client, at this stage they could be reduced to 1 interface, but I don’t want to do this for some reason. It also contains its Id and data necessary for its execution, which is described by the MessageContent class. However, the team may not need data. And she herself is responsible for deserializing the set of bytes into an instance of the class.
Sample command. In this case, this is the command to add a file to the room:
Code
public interface IServerAPICommand { void Run(ServerCommandArgs args); } public class ServerCommandArgs { public string ConnectionId { get; set; } public byte[] Message { get; set; } } abstract class BaseCommand { protected static T GetContentFormMessage<T>(byte[] message) { using (MemoryStream messageStream = new MemoryStream(message)) { messageStream.Position = sizeof(ushort); BinaryFormatter formatter = new BinaryFormatter(); T receivedContent = (T)formatter.Deserialize(messageStream); return receivedContent; } } } class ServerAddFileToRoomCommand : BaseServerCommand, IServerAPICommand { public void Run(ServerCommandArgs args) { MessageContent receivedContent = GetContentFormMessage<MessageContent>(args.Message); // . if (receivedContent.File == null) throw new ArgumentNullException("File"); if (string.IsNullOrEmpty(receivedContent.RoomName)) throw new ArgumentException("RoomName"); if (!RoomExists(receivedContent.RoomName, args.ConnectionId)) return; using (var context = ServerModel.Get()) // , { Room room = context.Rooms[receivedContent.RoomName]; if (!room.Users.Contains(args.ConnectionId)) { ServerModel.API.SendSystemMessage(args.ConnectionId, " ."); return; } if (room.Files.FirstOrDefault(file => file.Equals(receivedContent.File)) == null) room.Files.Add(receivedContent.File); var sendingContent = new ClientFilePostedCommand.MessageContent { File = receivedContent.File, RoomName = receivedContent.RoomName }; // // , foreach (string user in room.Users) ServerModel.Server.SendMessage(user, ClientFilePostedCommand.Id, sendingContent); } } [Serializable] public class MessageContent { string roomName; FileDescription file; public string RoomName { get { return roomName; } set { roomName = value; } } public FileDescription File { get { return file; } set { file = value; } } } public const ushort Id = (ushort)ServerCommand.AddFileToRoom; } Record and play sound.
I recently started adding voice chat, maybe at the time of publishing it will still be in the demo version. But already managed to tinker with playback and recording sound.
The first option was the WinApi waveIn * waveOut * function. It was the easiest option, so I started with it. But it did not happen to them, because on the version of framework 3.5, the marshaling on the x64 platform inadequately worked, and when it started up, the application simply fell without any exceptions. When assembling under x86, everything was fine.
Next was an attempt to connect DirectSound, but he had found his bug with a way to notify about the completion of playing a piece of data. After googling this topic, it turned out that Mircosoft abandoned DirectSound long ago and are working with XAudio2. Moreover, its use would lead to the need to compile 2 versions of x86 and x64.
Since I didn’t want to write a wrapper for XAudio2 myself, I remembered OpenAL. For which, in addition, there is a wrapper (OpenTK), also with open source. From OpenTK, only the audio library itself was neatly cut out. Which now works in the program.
Since I had to work with OpenGL ES 2, I became friends immediately with OpenAL. Especially if we consider that there are examples on it on the official OpenTK website.
To write data, I use a fairly simple scheme from the example. When you start recording, a timer is started, the response period of which is adjusted for the time over which the buffer should be filled to half. After that, the data from it is read and sent by the ClientPlayVoiceCommand team to everyone who can listen to us.
The code can be viewed in the TCPChat \ Engine \ Audio \ OpenAL branch.
Network
Initially, the chat was one main room where all users were located. Of the data transfer protocols, only TCP was used. So - as it already provides the reliability of data transmission, it only remained to break its continuous stream into messages.
This was done by simply adding the size of the message to its beginning.
That is, the package is the following:
The first 4 bytes are the size of the message.
5-6 bytes - the command identifier.
The remaining data is serialized by MessageContent.
Further, in one forum I was offered to introduce file transfer and voice communication. I coped with the files almost immediately, although in the first version the files were transmitted through the server, which was generally terrible. After that, I thought about the fact that it would be good to transfer them to the line. As you know with the NAT problem, this is not so easy to do.
I was busy for a long time and tried to implement a NAT traversal using TCP, then I wouldn’t have to be soared about the unreliability of UDP. With him, nothing happened. After that, it was decided to use UDP and UDP hole punching technology.
But first we had to solve the problems of reliability. As usual, I started working on my protocol over UDP to ensure reliable message delivery to me. And I did it all the same, but it worked only locally. When testing it in real conditions, he apparently loaded the network so much that my computer completely hung up.
After that, I began to look for already implemented libraries and came across satyu on Habré, where a similar problem was solved with the help of Lidgren.Network. She was chosen.
NAT traversal is implemented at the API level. The scheme is simple, you just need the peers to find out the real addresses at which the server sees them. After that, one feast should throw a message to another. This message may not come, but it will create a rule on the router that messages from the address to which it was sent need to be delivered to this particular computer. After that, with the help of the server, another feast learns that it is already possible for it to connect and, in fact, connects.
And so, the sequence of actions:
- Client 1 tells the server that it wants to connect to Client 2. (ServerP2PConnectRequestCommand command)
- The server delegates its task to the P2PService class.
- P2PService looks at whether such clients have already connected to it and whether it already knows their addresses. If not, it asks for those who have not connected to connect (ClientConnectToP2PServiceCommand command)
- After they are connected, P2PService sends the command to wait for a connection to one of them. In this case, this is Client 2. (ClientWaitPeerConnectionCommand)
- The client who received the command also receives the address that will be connected to it, and sends a message to it. He starts to wait for a connection and sends a command to the server that he is ready to accept the connection. (ServerP2PReadyAcceptCommand)
- After receiving the readiness command, the server tells the other client (Client 1) that it can connect. (ClientConnectToPeerCommand)
- Communication between customers is established.
Black arrows indicate sending commands. Red initialization of Lidgren.Network connections.
All this algorithm is hidden in AsyncPeer, and it is enough to call the SendMessage method, and if the client is not connected, it will connect and send a message. Or immediately send if already connected.
Voice commands initiate a connection a little differently. When creating a voice room, the server creates a connection map in the room. In which it is written where every user in the room should connect. And voice playback commands are sent using the AsyncPeer.SendMessageIfConnected method, which simply throws out the message if there is no connection.
User interface.
Finally, a little about the program interface.
It is developed using WPF and MVVM pattern. Very flexible technology, although in version 3.5 it’s a bit damp, but it nonetheless makes it possible to bypass damp places. As an example, some Command properties are still not dependent, and for them I had to write a wrapper CommandReference.
CommandReference in this case contains a real command - a dependent property on which you can hang the binding. The wrapper itself is hosted in static resources. And it is used in the right place, really causing the command to come.
I also had to use AttachedProperty instead of wrappers, which in turn change the necessary ones.
To notify the interface of changes, it was decided to use the event model, while all the necessary information is transmitted in the events to display the changes.
In other matters, there is nothing more to write about interfaces, WPF as WPF.
Source: https://habr.com/ru/post/228021/
All Articles