Cooking ORM, without departing from the plate

This article is not a call for ~~extremism in the~~ development of bicycles. The purpose of a post is to understand the mechanism well, it is often necessary to create it from scratch. This is especially true of such a shaky topic as ORM.

Why is this all?

Industrial products such as MS Entity Framework, NHibernate are complex, have great functionality and in fact are a separate thing in themselves. Often, to obtain the desired behavior from such ORMs, an individual is needed who is well versed in the intricacies of such systems, which is not good in team development.

The main theoretical source of the post is Martin Fowler’s book Patterns of Enterprise Application Architecture (P of EAA) .
')
There is a lot of information about what is actually ORM, DDD, TTD, ~~traffic police~~ - on this I will try not to stop, my goal is practical. It assumes two parts of the article, at the very end there is a link to the full source code in the git hub.

What do we want to get?

We will immediately define the terms: tracking means tracking changes in business objects within a single transaction, to further synchronize the data stored
in RAM and bd content.

What are the main types of ORM?

There are two types of ORMs for using tracking in the .NET world: with change tracking and without change tracking.
Regarding the approach to the generation of sql queries: with the explicit use of queries and based on query generators from object models.

For example, among the well-known ORM without tracking and with the explicit use of queries, the most vivid example is the Dapper ORM from Stack Overflow . The main functionality of such ORMs is mapping the relational model of the database to the object model, while the client clearly defines how his query to the database will look.

The basic concepts of MS Entity Framework and NHibernate in the use of tracking and query generators from object models. I will not consider here the advantages and disadvantages of these approaches. All yogurts are equally useful and true in combining approaches.

The customer (that is, I) wanted to create an ORM using tracking (with the ability to turn off) and based on query generators from object models. We will generate sql queries from lambda - C # language expressions, from scratch, without using Linq2Sql, LinqToEntities (yes, only hardcore!).

Out of the box, in the MS Entity Framework there is a nuisance with batch updating and deleting data: you must first get all the objects from the database, then update / delete in the cycle, and then apply the changes to the database. As a result, we get more calls to the database than necessary. The problem is described here . We will solve this problem in the second part of the article.

Determine the main ingredients

We will immediately determine how the client code will interact with the library being developed. Highlight the main components of the ORM that will be available from the client code.
The concept of client interaction with ORM will be based on the idea of inheriting from an abstract repository. You also need to define a superclass to restrict the set of business objects with which ORM will work.

Each business object must be unique within its type, so a descendant must explicitly redefine its identifier.

Empty repository and business object superclass

public abstract class Repository<T> : IRepository<T> where T : EntityBase,new() { //   }

 //   public abstract class EntityBase : IEntity { //      public abstract int Id { get; set; } public object Clone() { return MemberwiseClone(); } }

The Clone () method is useful to us for copying an object, when tracking, this will be slightly lower.

Let us have a client business object that stores information about the user - the Profile class.

To use a business object in ORM, you need three steps:

Bind a business object to a table in a database based on attributes

Profile class

  //       EntityBase [TableName("profile")] public class Profile : EntityBase { //        [FieldName("profile_id")] public override int Id { get; } [FieldName("userinfo_id")] public int? UserInfoId { get; set; } [FieldName("role_id")] public int RoleId { get; set; } public string Info { get; set; } }

Define a repository for each business object.

Profile repository will look like

  public class ProfileRepository : Repository<Profile> { //         public ProfileRepository() : base("PhotoGallery") { //  CRUD  } }

To form a connection string to the database

The connection string may look like this.

 <connectionStrings> <add name="PhotoGallery" providerName="System.Data.SqlClient" connectionString="server=PC\SQLEXPRESS; database=db_PhotoGallery"/> </connectionStrings>

For tracking changes made to use the pattern UnitOfWork. The essence of UnitOfWork is to track actions performed on domain objects to further synchronize data stored in RAM with the contents of the database. In this case, the changes are recorded at one time - all at once.

Interface UnitOfWork

  public interface IUnitOfWork : IDisposable { void Commit(); }

It would seem that this is all. But you need to take into account two points:

Change tracking should serve the entire current business transaction and should be accessible to all business objects.
A business transaction must be performed within a single thread, so you need to associate the work unit with the currently running thread using the local stream storage.

If a UnitOfWork object is already associated with a business transaction flow, then it should be placed in this object. In addition, from a logical point of view, the unit of work belongs to this session.

Let's use the static class Session

  public static class Session { //   private static readonly ThreadLocal<IUnitOfWork> CurrentThreadData = new ThreadLocal<IUnitOfWork>(true); public static IUnitOfWork Current { get { return CurrentThreadData.Value; } private set { CurrentThreadData.Value = value; } } public static IUnitOfWork Create(IUnitOfWork uow) { return Current ?? (Current = uow); } }

If you need to do without tracking, then you do not need to create an instance of the class UnitOfWork, just like calling Session.Create.

So, after determining all the elements necessary for interaction with ORM, we give an example of working with ORM.

An example of working with ORM

  var uow = new UnitOfWork(); using (Session.Create(uow)) { var profileRepo = new ProfileRepository(); //   uow.Commit(); }

Start cooking

Everything we talked about earlier concerned the public part. Now consider the internal part. For further development, it is necessary to determine what the structure of the object for tracking is.

Do not confuse the business object and the object for tracking:

A business object has its own type, whereas a tracking object must be suitable for mass manipulations of many business objects, that is, it must not depend on the specific type
A tracking object is an entity that exists within a specific business transaction; among a multitude of business objects, its uniqueness must be determined within the framework of this transaction.

From which it follows that such an object must have the properties:

Unique within its type
Unchangeable

In essence, the tracking object is a container for storing business objects. As noted earlier, all client business objects must be ancestors of the EntityBase superclass and must have an object identifier redefined for them. The identifier provides uniqueness within the type, that is, the table in the database.

Implement container object for tracking

  internal struct EntityStruct { //  internal Type Key { get; private set; } internal EntityBase Value { get; private set; } internal EntityStruct(Type key, EntityBase value) : this() { Key = key; Value = value; } public override bool Equals(object obj) { if (!(obj is EntityStruct)) { throw new TypeAccessException("EntityStruct"); } return Equals((EntityStruct)obj); } public bool Equals(EntityStruct obj) { if (Value.Id != obj.Value.Id) { return false; } return Key == obj.Key; } public override int GetHashCode() { //   ,        // return (unchecked(25 * Key.GetHashCode()) ^ Value.Id.GetHashCode()) & 0x7FFFFFFF; } }

Tracking business objects

Registration of objects for tracking will occur at the stage of obtaining these objects from the repository.

After receiving business objects from the database, they need to be registered as objects for tracking.
Such objects have two states: immediately after receiving from the database and after receiving, before the changes are committed.

The first will be called "clean", the second "dirty" objects.

Example

  var uow = new UnitOfWork(); using (Session.Create(uow)) { var profileRepo = new ProfileRepository(); // ""      , //  "" var profiles = profileRepo.Get(x=>x.Info = " "); // ""  foreach (var profile in profiles) { profile.Info = " "; } //  uow.Commit(); }

The important point is that to save "clean" objects, copying operations are necessary, which can have a detrimental effect on performance.

In the general case, tracking objects should be registered for each type of operation, thus there should be objects for update, delete, insert operations.

It is necessary to take into account that it is necessary to register only really changed objects, for which we need to perform comparison operations by value (the implementation of the EntityStruct structure with the overridden Equals method is shown above). Ultimately, the comparison operation will be reduced to a comparison of their hashes.

Events of registration of tracking objects will be fired from the abstract repository class functional in its CRUD methods.

Implementing tracking object registration functionality

  internal interface IObjectTracker { //            ICollection<EntityStruct> NewObjects { get; } ICollection<EntityStruct> ChangeObjects { get; } //  void RegInsertedNewObjects(object sender, AddObjInfoEventArgs e); void RegCleanObjects(object sender, DirtyObjsInfoEventArgs e); } internal class DefaultObjectTracker : IObjectTracker { //  - "" ,  - ""  private readonly Dictionary<EntityStruct, EntityStruct> _dirtyCreanPairs; public ICollection<EntityStruct> NewObjects { get; private set; } public ICollection<EntityStruct> ChangeObjects { get { //    return _dirtyCreanPairs.GetChangesObjects(); } } internal DefaultObjectTracker() { NewObjects = new Collection<EntityStruct>(); //   boxing/unboxing    EqualityComparer _dirtyCreanPairs = new Dictionary<EntityStruct, EntityStruct>(new IdentityMapEqualityComparer()); } public void RegInsertedNewObjects(object sender, AddObjInfoEventArgs e) { NewObjects.Add(e.InsertedObj); } public void RegCleanObjects(object sender, DirtyObjsInfoEventArgs e) { var objs = e.DirtyObjs; foreach (var obj in objs) { if (!_dirtyCreanPairs.ContainsKey(obj)) { // ""       MemberwiseClone() var cloneObj = new EntityStruct(obj.Key, (EntityBase)obj.Value.Clone()); _dirtyCreanPairs.Add(obj, cloneObj); } } } }

Functional detection of client-modified objects

  public static ICollection<EntityStruct> GetChangesObjects ( this Dictionary<EntityStruct, EntityStruct> dirtyCleanPairs ) { var result = new List<EntityStruct>(); foreach (var cleanObj in dirtyCleanPairs.Keys) { if (!(cleanObj.Key == dirtyCleanPairs[cleanObj].Key)) { throw new Exception("incorrect types"); } if (ChangeDirtyObjs(cleanObj.Value, dirtyCleanPairs[cleanObj].Value, cleanObj.Key)) { result.Add(cleanObj); } } return result; } public static bool ChangeDirtyObjs(EntityBase cleanObj, EntityBase dirtyObj, Type type) { var props = type.GetProperties(); //     foreach (var prop in props) { var cleanValue = prop.GetValue(cleanObj, null); var dirtyValue = prop.GetValue(dirtyObj, null); //    ,      if (!cleanValue.Equals(dirtyValue)) { return true; } } return false; }

It should be noted that the business objects of a single transaction can be from different databases. It is logical to assume that for each database a separate tracking instance must be defined (a class that implements IObjectTracker, for example, DefaultObjectTracker).

The current transaction must be "know" in advance of for which databases the tracking will be performed. At the stage of creating an instance of UnitOfWork, it is necessary to initialize instances of tracking objects (an instance of the DefaultObjectTracker class) using the specified connections to the database in the configuration file.

Change the UnitOfWork class

  internal interface IDetector { //  -    ,  -   Dictionary<string, IObjectTracker> ObjectDetector { get; } } public sealed class UnitOfWork : IUnitOfWork, IDetector { private readonly Dictionary<string, IObjectTracker> _objectDetector; Dictionary<string, IObjectTracker> IDetector.ObjectDetector { get { return _objectDetector; } } public UnitOfWork() { _objectDetector = new Dictionary<string, IObjectTracker>(); foreach (ConnectionStringSettings conName in ConfigurationManager.ConnectionStrings) { //        _objectDetector.Add(conName.Name, new DefaultObjectTracker()); } } }

Information about which instance of tracking to which database will correspond should be available to all instances of the repository as part of the transaction. It is convenient to create a single access point in the static class Session.

Session class will look like

  public static class Session { private static readonly ThreadLocal<IUnitOfWork> CurrentThreadData = new ThreadLocal<IUnitOfWork>(true); public static IUnitOfWork Current { get { return CurrentThreadData.Value; } private set { CurrentThreadData.Value = value; } } public static IUnitOfWork Create(IUnitOfWork uow) { return Current ?? (Current = uow); } //        //     internal static IObjectTracker GetObjectTracker(string connectionName) { var uow = Current; if (uow == null) { throw new ApplicationException(" Create unit of work context and using Session."); } var detector = uow as IDetector; if (detector == null) { throw new ApplicationException("Create unit of work context and using Session."); } return detector.ObjectDetector[connectionName]; } } }

Data access

The data access functional will directly call the access methods of the database. This functionality will be used by the abstract repository class in its CRUD methods. In the simple case, the data access class includes CRUD methods for working with data.

DbProvider class implementation

  internal interface IDataSourceProvider : IDisposable { State State { get; } //     ,      void Commit(ICollection<EntityStruct> updObjs); ICollection<T> GetByFields<T>(BinaryExpression exp) where T : EntityBase, new(); } internal class DbProvider : IDataSourceProvider { private IDbConnection _connection; internal DbProvider(IDbConnection connection) { _connection = connection; State = State.Open; } public State State { get; private set; } public ICollection<T> GetByFields<T>(BinaryExpression exp) where T : EntityBase, new() { //    select-   exp Func<IDbCommand, BinaryExpression, string> cmdBuilder = SelectCommandBulder.Create<T>; ICollection<T> result; using (var conn = _connection) { using (var command = conn.CreateCommand()) { command.CommandText = cmdBuilder.Invoke(command, exp); command.CommandType = CommandType.Text; conn.Open(); result = command.ExecuteListReader<T>(); } } State = State.Close; return result; } public void Commit(ICollection<EntityStruct> updObjs) { if (updObjs.Count == 0) { return; } //  -    update-   exp // -   var cmdBuilder = new Dictionary<Func<IDbCommand, ICollection<EntityStruct>, string>, ICollection<EntityStruct>>(); cmdBuilder.Add(UpdateCommandBuilder.Create, updObjs); ExecuteNonQuery(cmdBuilder, packUpdDict, packDeleteDict); } private void ExecuteNonQuery(Dictionary<Func<IDbCommand, ICollection<EntityStruct>, string>, ICollection<EntityStruct>> cmdBuilder) { using (var conn = _connection) { using (var command = conn.CreateCommand()) { var cmdTxtBuilder = new StringBuilder(); foreach (var builder in cmdBuilder) { cmdTxtBuilder.Append(builder.Key.Invoke(command, builder.Value)); } command.CommandText = cmdTxtBuilder.ToString(); command.CommandType = CommandType.Text; conn.Open(); if (command.ExecuteNonQuery() < 1) throw new ExecuteQueryException(command); } } State = State.Close; } private ICollection<T> ExecuteListReader<T>(EntityStruct objs) where T : EntityBase, IEntity, new() { Func<IDbCommand, EntityStruct, string> cmdBuilder = SelectCommandBulder.Create; ICollection<T> result; using (var conn = _connection) { using (var command = conn.CreateCommand()) { command.CommandText = cmdBuilder.Invoke(command, objs); command.CommandType = CommandType.Text; conn.Open(); result = command.ExecuteListReader<T>(); } } State = State.Close; return result; } private void Dispose() { if (State == State.Open) { _connection.Close(); State = State.Close; } _connection = null; GC.SuppressFinalize(this); } void IDisposable.Dispose() { Dispose(); } ~DbProvider() { Dispose(); } }

The DbProvider class requires an existing connection to the database. We delegate the creation of a connection and additional infrastructure to a separate class based on the factory method. Thus, it is only necessary to create instances of the DbProvider class through the auxiliary factory class.

DbProvider Factory Method

  class DataSourceProviderFactory { static DbConnection CreateDbConnection(string connectionString, string providerName) { if (string.IsNullOrWhiteSpace(connectionString)) { throw new ArgumentException("connectionString is null or whitespace"); } DbConnection connection; DbProviderFactory factory; try { factory = DbProviderFactories.GetFactory(providerName); connection = factory.CreateConnection(); if (connection != null) connection.ConnectionString = connectionString; } catch (ArgumentException) { try { factory = DbProviderFactories.GetFactory("System.Data.SqlClient"); connection = factory.CreateConnection(); if (connection != null) { connection.ConnectionString = connectionString; } } catch (Exception) { throw new Exception("DB connection has been failed."); } } return connection; } public static IDataSourceProvider Create(string connectionString) { var settings = ConfigurationManager.ConnectionStrings[connectionString]; var dbConn = CreateDbConnection(settings.ConnectionString, settings.ProviderName); return new DbProvider(dbConn); } public static IDataSourceProvider CreateByDefaultDataProvider(string connectionString) { var dbConn = CreateDbConnection(connectionString, string.Empty); return new DbProvider(dbConn); } }

Registration of tracking objects should occur in the CRUD methods of the repository, which in turn delegates the functionality to the data access layer. Thus, it is necessary to implement the IDataSourceProvider interface taking into account tracking We will register objects on the basis of the mechanism of events that will be excited in this particular class. The proposed new implementation of the IDataSourceProvider interface should “be able” both to initialize the registration events for tracking and to access the database. In this case, it is convenient to decorate the DbProvider class.

TrackerProvider class implementation

  internal class TrackerProvider : IDataSourceProvider { private event EventHandler<DirtyObjsInfoEventArgs> DirtyObjEvent; private event EventHandler<UpdateObjsInfoEventArgs> UpdateObjEvent; private readonly IDataSourceProvider _dataSourceProvider; private readonly string _connectionName; private readonly object _syncObj = new object(); private IObjectTracker ObjectTracker { get { lock (_syncObj) { //     return Session.GetObjectTracker(_connectionName); } } } public TrackerProvider(string connectionName) { _connectionName = connectionName; _dataSourceProvider = DataSourceProviderFactory.Create(_connectionName); //    RegisterEvents(); } public State State { get { return _dataSourceProvider.State; } } private void RegisterEvents() { //       if (Session.Current == null) { throw new ApplicationException("Session has should be used. Create a session."); }; //    DirtyObjEvent += ObjectTracker.RegCleanObjects; UpdateObjEvent += ObjectTracker.RegUpdatedObjects; } public ICollection<T> GetByFields<T>(BinaryExpression exp) where T : EntityBase, IEntity, new() { //        DbProvider var result = _dataSourceProvider.GetByFields<T>(exp); var registratedObjs = result.Select(r => new EntityStruct(typeof(T), r)).ToList(); //   ""  var handler = DirtyObjEvent; if (handler == null) return result; handler(this, new DirtyObjsInfoEventArgs(registratedObjs)); return result; } public void Commit(ICollection<EntityStruct> updObjs) { //     DbProvider _dataSourceProvider.Commit(updObjs, delObjs, addObjs, packUpdObjs, deleteUpdObjs); } public void Dispose() { _dataSourceProvider.Dispose(); } }

Subtotals

Let's figure out how our public classes will look now.

As noted above, the repository class must delegate its functionality to the IDataSourceProvider interface implementations. When initializing the repository class, based on the connection string passed to the constructor, you need to create the necessary IDataSourceProvider implementation depending on the tracking usage. It is also necessary to take into account that the data access class can “lose” the connection with the database at any time, for which we will monitor this connection using the property.

The UnitOfWork class, as mentioned earlier, in its constructor must create a list of objects of the DefaultObjectTracker class for all available in the connection string of the database. It is logical that all changes should also be committed to all changes: for each tracking instance, a method of fixing its changes will be called.

Public - classes will take the form

 public abstract class Repository<T> : IRepository<T> where T : EntityBase, IEntity, new() { private readonly object _syncObj = new object(); private IDataSourceProvider _dataSourceProvider; //c   ""    private IDataSourceProvider DataSourceProvider { get { lock (_syncObj) { if (_dataSourceProvider.State == State.Close) { _dataSourceProvider = GetDataSourceProvider(); } return _dataSourceProvider; } } } private readonly string _connectionName; protected Repository(string connectionName) { if (string.IsNullOrWhiteSpace(connectionName)) { throw new ArgumentNullException("connectionName"); } _connectionName = connectionName; var dataSourceProvider = GetDataSourceProvider(); if (dataSourceProvider == null) { throw new ArgumentNullException("dataSourceProvider"); } _dataSourceProvider = dataSourceProvider; } private IDataSourceProvider GetDataSourceProvider() { //      DbProvider' //    ////  - TrackerProvider return Session.Current == null ? DataSourceProviderFactory.Create(_connectionName) : new TrackerProvider(_connectionName); } public ICollection<T> Get(Expression<Func<T, bool>> exp) { return DataSourceProvider.GetByFields<T>(exp.Body as BinaryExpression); } } public sealed class UnitOfWork : IUnitOfWork, IDetector { private readonly Dictionary<string, IObjectTracker> _objectDetector; Dictionary<string, IObjectTracker> IDetector.ObjectDetector { get { return _objectDetector; } } public UnitOfWork() { _objectDetector = new Dictionary<string, IObjectTracker>(); foreach (ConnectionStringSettings conName in ConfigurationManager.ConnectionStrings) { _objectDetector.Add(conName.Name, new DefaultObjectTracker()); } } public void Commit() { SaveChanges(); } private void SaveChanges() { foreach (var objectDetector in _objectDetector) { //         var provider = new TrackerProvider(objectDetector.Key); provider.Commit( objectDetector.Value.ChangeObjects, objectDetector.Value.DeletedObjects, objectDetector.Value.NewObjects, objectDetector.Value.UpdatedObjects, objectDetector.Value.DeletedWhereExp); } } }

In the continuation of the article I will consider working with related entities, generating sql queries based on expression trees, methods for batch data deletion / modification (like UpdateWhere, RemoveWhere).

Completely all the source code of the project, without simplifications, lies here .

Source: https://habr.com/ru/post/317860/

All Articles