Over the past decade, the availability of the Internet has increased significantly. Therefore, the number of applications that work in conjunction client-server, also increased significantly. But what to do if access to the network is, but not always? It is with such a requirement from the customer that we faced in one of the projects. All who are interested in the solution developed by us, please under the cat.

Initial data

So, a little more about the task. There are n geographically distant offices. Each office employs several employees, processing data. Data may overlap between offices. In the office most of the time may not have access to the Internet. Accordingly, a situation may arise when the same object was changed by two employees. Since the changes are equivalent and it is impossible to apply one of them and reject the other, there is another requirement - a conflict resolution mechanism. Well, as a cherry on a cake, relatively old laptops with a small amount of RAM are used as workstations in offices.

Before bravely rushing to solve a problem, we, of course, tried to find out whether it would not be cheaper to resolve the issue of network availability than to look for a software solution. But here the customer was adamant and insisted on his internal reasons for such a decision.
')
Leaving the organizational solution, we moved to the technical. Much further depended on the choice of a DBMS for a local DB. The first choice was SQL CE, but after some time, due to big performance problems, the final choice fell on MySql.

If we discard the solutions found in the open spaces of the codeproject, then there were 3 alternatives:

• Database replication
• MS Sync Framework
• Your bike option

The variant with replication is not suitable for many criteria: starting from the requirements for the visual resolution of conflicts and ending with various DBMS on clients and server (although, of course, this can be solved).

MS Sync Framework inspired great hopes after reading the list of features. After the first acquaintance, the hopes became much less. The main problem was that it was weakly focused on work at the level of domain objects (and this is how it is most convenient for the user to work - to resolve the conflict for the entities with which he works in the application, and not for their individual pieces, which correspond to separate tables).

Therefore, we went to meet the adventure on the way to point 3!

Software model

So, the final architecture included a client implemented on WPF and a server implemented as a WCF service.

For each entity, a class inherited from the base class Entity was created in the domain structure of the application:

public class Entity { public virtual Guid ID { get; set; } public virtual DateTime Timestamp { get; set; } public virtual EEntityState State { get; set; } public virtual bool IsDeleted { get; set; } } public enum EEntityState { Normal, Created, Modified, Deleted } 

• ID - object identifier, using Guid allows you to solve the problem of assigning identifiers when creating new objects on various system clients.
• Timestamp - the time the object was last modified. This field is updated ONLY on the server when synchronizing user changes. Based on this field, the server sends to the client objects that have been modified since the previous synchronization.
• State - the state of the object. This field is updated ONLY on the client. Based on this field, the client selects objects that have been modified (created, edited, or deleted) since the last synchronization.
• IsDeleted - objects in the system are not deleted (another customer requirement). Therefore, instead of the actual deletion, the object is marked as deleted and does not participate in further selections during the work of the client.

Each object in the system has three representations:

• in the form of a domain entity
• in the form of DTO (Data Transfer Object) - in this view, the object is transmitted during synchronization between the client and the server. In such an object there are only simple fields. All links / collections of links are replaced with identifiers of objects to which the link points.
• in the form of PE (Presentation Entity) - in this view the object is displayed on the form.

Server part

Initially, on the service side for each entity two methods are implemented:

• Get - get objects changed after the specified date;
• Save - save changes for transferred objects

During testing, it was decided to add a third method - GetCount , which would return the number of objects changed after the specified date. The Get method, accordingly, will receive parameters for obtaining N objects with offset M. This was done due to the fact that if the connection to the server was bad, the connection timeout often took off. It would be possible to simply increase the timeout, but the option with paging data seems more correct.

In addition to the methods for processing entities, the following methods are also present on the server:

• BeginSynchronization - initiates the start of synchronization. At this point, a synchronization check is performed (only one client can synchronize at a time; the session is considered active if it a) has not completed, b) has been active for the last 20 minutes). If successful, a new session is created.
• EndSynchronization - end of synchronization. This operation closes the current synchronization session, and the client receives the synchronization date (which it uses during subsequent synchronizations).
• Noop - the operation of maintaining the connection. This operation is used at the time of conflict resolution on the client side, so that the connection does not break on timeout, as well as to update the status of the current synchronization session.

Conflict resolution

To solve the problem with the resolution of conflicts in the system, the following editing model was implemented.

In the application, each editor can be used in two versions:

• direct editing of the entity;
• conflict resolution during synchronization.

The differences between these options are in the form on which editing takes place - when editing an entity, the form contains an editor and save / cancel buttons (some forms may also have additional buttons, but this does not affect the essence of the matter), while saving this form, the object is saved in DB and closing dialogue. When resolving conflicts on the form, two editors are displayed at the same time (for a local object and for an object received from the server), management of the state of the object (deleted / not deleted), besides, it is necessary to visually select the different fields inside the editor, and when closing the window you do not need to save changes in the database.

To do this, in the application, each editor was designed as a separate control. This control contained only logic for editing (calling child editors, updating screen state, etc.). The interface of this editor provided two methods for setting / getting an editable object ( PE ).

 public interface IEditor { event Action EditedObjectChanged; IWindow Window { get; set; } ObjectValidationResult Validate(); void SetEditedObject(PresentationEntity pe); PresentationEntity GetEditedObject(); } 

For ease of use, its generic version was introduced:

 public interface IEditor<TPE> : IEditor where TPE: PresentationEntity { TPE DisplayObject { get; set; } } 

Then he turned into a form that included the logic for translating PE into the domain object and further saving the PE into the database.

The conflict resolution dialog simultaneously displays two editors, in each of which either the PE of the local entity or the PE of the server entity are installed respectively.

For some editors, users had the opportunity to change not only the fields of the object itself, but also the fields of the child objects (for example, editing the child list). Accordingly, as a result of the work of the conflict resolution form, not only the object for which the form was called, but also the objects of the child collections could change its state. Therefore, all objects of the child collections in the form view had the following general class:

 public class ChildListItemPE : PresentationsEntity { public Guid ObjectID { get; set; } public ChildEditorPE EditorPE { get; set; } } 

Where ChildEditorPE is PE for the editor of this entity.

 public class ChildEditorPE : ValidatablePresentationEntity { public Guid ObjectID { get; set; } public EEntityState State { get; set; } } 

Thus, after closing the conflict resolution form, we go through the collections of the edited object (using reflection) and update all objects whose editors have a state other than Normal . As a result, we had a modified object state and object identifiers that were changed while editing the main object.

In addition, during the operation of the conflict resolution form, we call the Noop service method on a timer in a parallel thread to support our session.

The logic of highlighting the state of the form element is based on the assumption that the editors of the local object and the object received from the server have an identical structure, as well as on the condition of a limited number of control types used (i.e. if you want to display differences for the slider, then you need to add code which will check the equality of the current values, as well as set styles for the states).

For each element of the local object editor, a corresponding element is searched for in the object editor received from the server. Next, the presence of the established binding is checked on the corresponding field (for example, Text for TextBox or ItemsSource for DataGrid) - this is another assumption, all fields in editors that affect the state of the object are associated with the model through Binding. Next, we get the values ​​of the fields in the element belonging to the editor of the local object, and in the element belonging to the editor to the object received from the server. Based on the comparison of these values, we establish the necessary style.

Here are some examples of conflict resolution forms.

Elements on the form have a red background color in case the value in the local version of the object differs from the value in the object received from the server.



Lists have a green background if the number of objects and their composition is the same in the local version and in the version obtained from the server. Items in the list also change the background color depending on their state.

Synchronization

The first version of synchronization included the following steps:

• synchronization start - the client initializes the connection to the server;
• receiving from the server a list of changed objects (that is, objects that other users have changed);
• getting a list of local changed objects (i.e. objects that the current user has changed);
• objects that have been changed only on the server are immediately saved to the local database;
• Objects that have been modified only locally are added to the collection of objects to be sent to the server.
• Objects that have been modified both locally and on the server are marked for conflict resolution:
  o A local copy of the object participates in conflict resolution, and an object created on the basis of the received DTO . In this case, for an object created on the basis of DTO , additional references to other objects and collections are processed (so that they point not to local objects, but to objects received from the server);
• send information about changed local entities to the server;
• start the save process;
• we complete the synchronization process.

In the process, it became necessary to restore the local user base in case of unsuccessful synchronization (for example, the connection with the server was lost, or the user canceled the synchronization). The first option considered is to wrap everything in a transaction. But since in the synchronization process we have access to the database from different threads (interaction with the server is carried out in one thread, the forms work in another (and the forms, when displaying an object in theory, can refer to the database for some reference data from the database), then this option was dropped. Therefore, the synchronization process was supplemented with two steps - in the beginning we create a backup of the local database, and in case of an error / cancellation, we restore the database from the previously created backup.

The next problem we faced was the timeout of the operation while saving the changes on the server - in the case of a large number of changes of entities of the same type, the operation of saving them could take longer than the set timeout allowed. A simple solution would be to increase the timeout. Another solution could be the introduction of paging (by analogy with data acquisition). But we decided to go along an alternative path, which allowed us to resolve the issue with transactionality and on the server side (so that we could apply either all changes from the synchronization session, or not apply any).

All changes made in the Save methods were translated into a JSON object, which was saved to a special table. This object contained information about the session to which it belongs, the type of entity that matches, and the state of the corresponding DTO .

 public class IntermediateItem { public virtual Guid SessionId { get; set; } public virtual string DTOType { get; set; } public virtual string Content { get; set; } } 

After the end of the main part of synchronization (receiving objects, resolving conflicts and sending changes to the server), the client called the CommitChanges service method , which, in turn, sent a message to the Windows service, which already backtracked the changes from the intermediate table to DTO , translating DTO objects in domain entities and their preservation.

The allocation of this operation in a separate win-service was done due to the fact that the preservation operation can take quite a long time. The client at this time periodically calls the server's GetCommitStatus method in order to check the status of the operation.

Instead of conclusion

It is likely that at one time we invented the bicycle. We also understand that the final decision is not without flaws. But in the end we got a stable and workable version of the solution for synchronizing offline parts of the system. But still for future projects (and oddly enough, requests with similar requirements continue to come in regularly) we would be happy to learn other solutions, as well as to improve our model, therefore we will be glad to receive comments and comments.

Disadvantages:

• one synchronization at a time - as you can see from the described process, only one user can synchronize at a time. This causes some inconvenience during the initial synchronization (that is, when the user does not have a local database). Such a session usually takes 2-3 hours (depending on the speed of access to the server and the speed of the computer). But given the fact that the number of users of the system is less than 20, and the frequency of new users tends to zero, this does not cause problems. The average duration of a synchronization session is 20-30 minutes, and taking into account the different geography of offices (including different time zones), synchronization intersections do not often occur;
• adding entities to the system requires a large number of auxiliary objects (in fact, it’s not that the lack of a specific solution, but the lack of all patterns is their redundancy);
• the need to monitor the correct updating of the state of objects;
• if the client “fell off” from synchronization and did not close its session, then the next client will be able to connect only after the timeout for “not activity” of the previous session expires (20 minutes).

Buns:

• the transactional nature of synchronization operations (both on the server side and on the client side);
• conflict resolution with visualization of differences;
• well and the main thing - the customer is satisfied!

Partially listed disadvantages can be eliminated:

• for example, routine operations when adding new entities can be mostly automated using code generation;
• updating the state of objects can be implemented at the repository level;
• To track closed sessions, you can enter an additional service method, which is called at short intervals (for example, after 15 seconds) and updates the status of the synchronization session. In this case, you can reduce the timeout "no activity" from 20 minutes to the mentioned 15 seconds. Of course, this will increase the traffic and load on the server, but since the number of users is not large and will not change significantly, then this option is quite viable.