Client-side Linq to NHibernate
Practically any .NET developer somehow uses Linq technology in his practice. Linq allows you to write beautiful and concise code for receiving objects from a data source with the ability to define criteria for obtaining and / or transforming the requested objects on the fly. Linq support is present in almost all popular ORM frameworks, including NHibernate. NHibernate provides a Linq provider with which we can write a request at design time (Design-Time), but in order to make a request at runtime, you will have to tinker with Reflection. However, if there is a need to form a request in an external process, for example, in the client part of the service, then Reflection will not save, the client part, as a rule, does not know (and should not know anything) about the server ORM.
Below we will discuss how to create an API for writing Linq queries to NHibernate in a situation where the query is written in one process and executed in another. Also, we implement our own IQueryProvider, which will translate requests from the source application to the executing application.
1. IEnumerable vs IQueryable
2. Linq in NHibernate
3. Linq requests without an ISession object
4. Linq database query through NHibernate from external process
5. We write test application
Conclusion
Links
For starters, you should briefly recall the IEnumerable and IQueryable interfaces. They wrote about them here and here . It is also helpful to read about expression trees and how they work .
How the IEnumerable query is executed:
1. The data source is represented as IEnumerable (enumerable); in the case of collections, this is an optional action.
2. The enumerated data source is wrapped (decorated) by an WhereListIterator iterator
3. The first WhereListIterator iterator is decorated by the next WhereListIterator iterator.
4. In the List constructor, the WhereListIterator is “top-level”. In a rough approximation, we can say that the filling of the internal List container takes place through a foreach loop traversing the resulting WhereListIterator. When requesting the next element, the “top-level” decorator calls the whole chain of decorators, each element of which determines which element can be pushed up and which should be skipped.
How is the execution of IQueryable query for example collections:
1. The data source is wrapped by an EnumerableQueryable object (let's call it “A”), inside of which a ConstantExpression expression is created with the reference to the source object closed (IQueryProvider is also created, which in the case of IEnumerable will look at the original collection).
2. Object “A” is decorated with a new object EnumerableQueryable (let's call it “B”), from object A, the Expression property is taken, which is decorated with an expression MethodCallExpression, where Queryable.Where is specified as the called method; the request provider in the new object sets the request provider from object A. Object A is no longer needed.
3. The object B obtained in the previous step with the expression MemberExpression is decorated with the new object EnumerableQueryable (let's call it “C”), from object B the property with the type Expression is taken, which is decorated with the expression MethodCallExpression, where Queryable.Where is specified as the called method; the request provider in the new object sets the request provider from object B. Object B is no longer needed.
4. The main action takes place at the stage of accessing the results of the query:
The IQueryable interface is an IEnumerable inheritor, therefore, an object of type IQueryable can also be passed to the List constructor, before executing the foreach loop (again a rough approximation), the GetEnumerator method will be called on the transferred IQueryable object. During a call to GetEnumerator (), the query provider will compile the resulting MethodCallExpression and return, as in the case of IEnumerable, a chain of decorator methods that will return the next element on request.
Here we come to the difference between IEnumerable and IQueryable.
The advantage of expression trees is that it is inherently a high-level API for generating IL code, which we can build / edit at runtime, compile into a delegate, and call.
A typical scenario for working with Linq in NHibernate looks like this:
Calling session.Query () will return an object of type NhQueryable. Further refinement of the Where conditions will decorate the expression from the NhQueryable source object. Wrapping the original query is exactly the same as in the case of a query to the collection. Differences begin from the moment you call ToList ().
At the time of the GetEnumerator () method call, the constructed expression tree will not be compiled, but translated into an sql query. The Remotion.Linq library is responsible for the Linq query translation mechanism in SQL, it parses the expression tree received from NHibernate. At the parsing stage, calculations of closures in the tree nodes occur, for example:
The lambda expression m => m.AgeLimit> = ageLimit will create a closure on the local variable ageLimit. When parsing this lambda expression, the expression tree of the form m.AgeLimit> = ageLimitInCurrentState will be evaluated in the expression m.AgeLimit> = 16 and already in this form the expression will be given to the Linq2Sql translator.
Let's take a closer look at the IQueryable and IQueryProvider interfaces .
In the IQueryable interface, there is an Expression property, which provides the current expression-decorator (data source decorator, or another expression decorator), as well as the Provider property with the IQueryProvider type, through which you can get the current query provider.
The IQueryProvider interface provides CreateQuery (Expression expression) methods for decorating the lower level expression and Execute (Expression expression) for querying the data source.
All Linq methods can be divided into two groups:
Calling the session.Query () method returns an object of type NhQueryable, whose Provider property is denoted by an object of type INhQueryProvider.
A strongly simplified implementation of NhQueryable looks like this.
Further wrapping the NhQueryable object with methods from the System.Linq.Queryable class (such as Where, Select, Skip, Take, etc.) will use the same data provider that was created in the NhQueryable object.
Unlinking a request from a session involves getting rid of the call to Session.Query () and NhQueryable in the root of the expression tree, respectively. In order to be able to use linq, a stub source is needed that returns an IQueryable object.
Define it:
Now we can write something like:
For ease of use, let's wrap the creation of a query into a repository class:
Now write about this code
and referring to the Expression property of the query object, we get the following expression tree:
Above, I wrote that the expression tree can be edited at runtime, so we can bypass the resulting tree and replace the MockDataSource with NhQueryable. NhQueryable can be obtained by calling session.Query ().
To perform a tree traversal, use the Visitor pattern. In order not to write the expression tree visitor "from scratch", we use this basic implementation . In the successor, override the VisitConstant and VisitMethodCall methods, and also override the Visit method, which will be the access point for editing the expression:
How to use it:
On the line new NhibernateExpressionVisitor (). Visit (query.Expression, nhQueryable), the RemoteQueryable stub was replaced with NhQueryable.
By themselves, such gestures are not reasonable, if the code that collects such an expression is dynamically in the same process with NHibernate, then we will look at how to render the code generating the linq request from the fake source to an external process.
The process from which requests come, let's call it “client”, and the process that executes requests is “server”
For the full implementation of linq, it is necessary to define a client request provider that will provide a fake stub. But before we define the interface through which you can access the server process:
The responsibility for the process of serialization of the query results (objects or collections of objects) will be assigned to the transport layer, or rather to the implementation of IChannelProvider. Next, you need to modify the RemoteQueryable constructor and the RemoteRepository class so that you can pass the server process data provider (IChannelProvider) to these classes
To simplify the process of sending the request, we will give it to the transport level as a string. Next, we define the client query provider (RemoteQueryableProvider) with the IQueryProvider interface, as well as the DTO class (QueryDto) to send the request to the server:
The factory method QueryDto.CreateMessage () serializes the resulting expression using the Serialize.Linq library. The QueryDto class also contains the RequestedTypeName and RequestedTypeAssemblyName properties to identify the type of entity being requested and to properly restore the expression on the server. The very object of the QueryDto class is serialized into a string using the Json.NET library. The serialized request is transmitted to the server via the IChannelProvider proxy object.
')
1. Handling closures in an expression.
In the Linq section in Nhibernate, it was not by chance that I mentioned the process of calculating closures in expressions. Since the formation of the request is now in an external process, before sending the request, it is necessary to calculate the values ​​of the closures in the constructed expression. We are interested in (so far) only closures in predicates for example.
To calculate the value of the links and convert the values ​​to ConstantExpression, we will write a client-side ExpressionVisitor, which will modify the expression
and modify the Execute methods of the RemoteQueryableProvider class, taking into account the functionality of the ClientExpressionVisitor
2. The use of entity properties in queries that are not specified in the mapping.
If in NHibernate we write a condition of the form Where (x => x.UnmappedProperty == 4)), then the validator of the NHibernate query will not miss such an expression. To solve this problem, we introduce API post requests, i.e. queries to the data that have already been received as a result of sql-selection.
and extension for easy query wrapping
Now you can write a query like:
and send it to the server.
Both server and client code must be in the same assembly, but the code contains references to types from the NHibernate assembly. It is necessary to untie the extra dependency for the client. We write a helper for working with Nhibernate types to remove the hard link to the Nhibernate.dll assembly. The very same assembly NHibernate.dll on the server side will be loaded via Reflection.
On the server side, we define the RemoteQueryExecutor class, which will accept the serialized QueryDto, restore it, and execute it:
After receiving the request, on the server we restore QueryDto with the Json.NET deserializer. The serialized expression inside the DTO object is restored using the Serialize.Linq library . Then we modify the expression with the help of the NhibernateExpressionVisitor visitor - we replace the fake root with NhQueryable, as explained above.
The resulting expression is divided into two queries:
The query to the database will work according to the already known scheme, without compilation. The query to the results of the sample is compiled and processes the objects by the methods specified in the expressions. The query for the results of the previous query is compiled and executed as a regular EnumerableQuery.
We implement a test client-server application. For the client side, we use the WPF technology, on the server side, we will use SQLite as the database, for communication between the processes, we will use WCF with HTTP bindings. As an object model, use the WorkItem class.
Behind the scenes, leave the NHibernate mapping setting and the nhibernate.cfg.xml configuration, as well as the WCF setting for data transfer and set the goal - to display 200 WorkItem objects in the ListView, loading data from the database as necessary.
WPF for lists provides a mechanism for data virtualization at the UI layer, which can be further developed for virtualization at the level of the ViewModel data collection. We take the example from this article as a basis and modify the example for paging downloading data from the database to the client.
We implement our IItemsProvider and replace the implementation from the example with our class DemoWorkItemProvider. In the FetchCount () and FetchRange () methods, we will use Linq queries using the RemoteQueryable API. In the FetchRange method, we specify a query only for the range of data that is required for display.
Few rule UI and ListView style for displaying WorkItem. We start the server and the client, when clicking on the Refresh button, the client sends a Linq request for the number of items and two subsequent requests for receiving the first and second pages of the list to the server. When scrolling down the list, the next page is loaded from the database via the
RemoteQueryablyProvider -> WCF -> HTTP -> WCF -> RemoteQueryExecutor -> NHibernate -> SQLite chain
Pros of the RemoteQueryable API:
On this, perhaps, you can stop. The next step may be to modify the request on the server side, taking into account user authorization or optimization of the code that works with Reflection, or the connection Dynamic.Linq, but this is the topic of a separate article.
1. Repositories with code and examples from the article on GitHub
2. IEnumerable and IQueryable, what is the difference?
3. Principles of operation of IQueryable and LINQ data providers
4. Closures in the C # programming language
5. Serialize.Linq source codes
6. NHibernate sources
7. Remotion.Linq on codeplex
8. IQueryProvider on MSDN
9. How to: Implement an Expression Tree Visitor
Below we will discuss how to create an API for writing Linq queries to NHibernate in a situation where the query is written in one process and executed in another. Also, we implement our own IQueryProvider, which will translate requests from the source application to the executing application.
Content
1. IEnumerable vs IQueryable
2. Linq in NHibernate
3. Linq requests without an ISession object
4. Linq database query through NHibernate from external process
5. We write test application
Conclusion
Links
1. IEnumerable vs IQueryable
For starters, you should briefly recall the IEnumerable and IQueryable interfaces. They wrote about them here and here . It is also helpful to read about expression trees and how they work .
IEnumerable
How the IEnumerable query is executed:
1. The data source is represented as IEnumerable (enumerable); in the case of collections, this is an optional action.
2. The enumerated data source is wrapped (decorated) by an WhereListIterator iterator
3. The first WhereListIterator iterator is decorated by the next WhereListIterator iterator.
4. In the List constructor, the WhereListIterator is “top-level”. In a rough approximation, we can say that the filling of the internal List container takes place through a foreach loop traversing the resulting WhereListIterator. When requesting the next element, the “top-level” decorator calls the whole chain of decorators, each element of which determines which element can be pushed up and which should be skipped.
Pseudocode
// List ctor public List<T>(IEnumerable<T> source) { // IEnumerator WhereListIterator. var enumerator = source.GetEnumerator(); // WhereListIterator' while(enumerator.MoveNext()) { items.Add(enumerator.Current) } } Iqueryable
How is the execution of IQueryable query for example collections:
1. The data source is wrapped by an EnumerableQueryable object (let's call it “A”), inside of which a ConstantExpression expression is created with the reference to the source object closed (IQueryProvider is also created, which in the case of IEnumerable will look at the original collection).
2. Object “A” is decorated with a new object EnumerableQueryable (let's call it “B”), from object A, the Expression property is taken, which is decorated with an expression MethodCallExpression, where Queryable.Where is specified as the called method; the request provider in the new object sets the request provider from object A. Object A is no longer needed.
3. The object B obtained in the previous step with the expression MemberExpression is decorated with the new object EnumerableQueryable (let's call it “C”), from object B the property with the type Expression is taken, which is decorated with the expression MethodCallExpression, where Queryable.Where is specified as the called method; the request provider in the new object sets the request provider from object B. Object B is no longer needed.
4. The main action takes place at the stage of accessing the results of the query:
The IQueryable interface is an IEnumerable inheritor, therefore, an object of type IQueryable can also be passed to the List constructor, before executing the foreach loop (again a rough approximation), the GetEnumerator method will be called on the transferred IQueryable object. During a call to GetEnumerator (), the query provider will compile the resulting MethodCallExpression and return, as in the case of IEnumerable, a chain of decorator methods that will return the next element on request.
Pseudocode
public List<T>(IQueryable<T> source) { // IEnumerator WhereListIterator. var enumerator = source.GetEnumerator(); // WhereListIterator' while(enumerator.MoveNext()) { items.Add(enumerator.Current) } } public class EnumerableQueryable<T> : IQueryable<T> { private Expression expression; private IEnumerable enumerableResult; public IEnumerator GetEnumerator() { if (enumerableResult == null) enumerableResult = expression.Compile().Invoke(); return enumerableResult.GetEnumerator(); } } Here we come to the difference between IEnumerable and IQueryable.
- When working with IEnumerable, the data source is decorated with iterator objects .
- When working with IQueryable, the data source is decorated using expression trees .
The advantage of expression trees is that it is inherently a high-level API for generating IL code, which we can build / edit at runtime, compile into a delegate, and call.
2. Linq in NHibernate
A typical scenario for working with Linq in NHibernate looks like this:
var activeMasterEntities = session // NhQueryable<T>, ConstantExpression , . .Query<Entity>() // IQueryable, MethodCallExpression , ConstantExpression. .Where(e => e.IsMaster == true) // IQueryable, MethodCallExpression , MethodCallExpression. .Where(e => e.IsActive == true) // .ToList() Calling session.Query () will return an object of type NhQueryable. Further refinement of the Where conditions will decorate the expression from the NhQueryable source object. Wrapping the original query is exactly the same as in the case of a query to the collection. Differences begin from the moment you call ToList ().
At the time of the GetEnumerator () method call, the constructed expression tree will not be compiled, but translated into an sql query. The Remotion.Linq library is responsible for the Linq query translation mechanism in SQL, it parses the expression tree received from NHibernate. At the parsing stage, calculations of closures in the tree nodes occur, for example:
int stateCoefficient = 0.9; int ageLimitInCurrentState = 18 * stateCoefficient; var availableMovies = session .Query<Movie>() .Where(m => m.AgeLimit >= ageLimitInCurrentState) .ToList() The lambda expression m => m.AgeLimit> = ageLimit will create a closure on the local variable ageLimit. When parsing this lambda expression, the expression tree of the form m.AgeLimit> = ageLimitInCurrentState will be evaluated in the expression m.AgeLimit> = 16 and already in this form the expression will be given to the Linq2Sql translator.
Convert Linq to Sql
IQueryable and IQueryProvider
Let's take a closer look at the IQueryable and IQueryProvider interfaces .
In the IQueryable interface, there is an Expression property, which provides the current expression-decorator (data source decorator, or another expression decorator), as well as the Provider property with the IQueryProvider type, through which you can get the current query provider.
The IQueryProvider interface provides CreateQuery (Expression expression) methods for decorating the lower level expression and Execute (Expression expression) for querying the data source.
All Linq methods can be divided into two groups:
- Decorator methods that wrap the expression of the previous IQueryable object (Where (), Select (), OrderBy (), etc.). The decorator methods use the CreateQuery () method of the IQueryProvider object and return IQueryable.
- Processor methods that execute the query (Count (), First (), Last (), Sum (), etc.).
Processor methods use the IQueryProvider object's Execute () method and return the result.
Calling the session.Query () method returns an object of type NhQueryable, whose Provider property is denoted by an object of type INhQueryProvider.
A strongly simplified implementation of NhQueryable looks like this.
NhQueryable
public class NhQueryable<T> : QueryableBase<T> { public NhQueryable(ISessionImplementor session) { // INhQueryProvider Provider = QueryProviderFactory.CreateQueryProvider(session); // . Expression = Expression.Constant(this); } } Further wrapping the NhQueryable object with methods from the System.Linq.Queryable class (such as Where, Select, Skip, Take, etc.) will use the same data provider that was created in the NhQueryable object.
3. Linq requests without an ISession object
Unlinking a request from a session involves getting rid of the call to Session.Query () and NhQueryable in the root of the expression tree, respectively. In order to be able to use linq, a stub source is needed that returns an IQueryable object.
Define it:
RemoteQueryable
public class RemoteQueryable<T> : IQueryable<T> { public Expression Expression { get; set; } public Type ElementType { get; set; } public IQueryProvider Provider { get; set; } public RemoteQueryable() { Expression = Expression.Constant(this); } } Now we can write something like:
var query = new RemoteQueryable<Entity>().Where(e => e.IsMaster); For ease of use, let's wrap the creation of a query into a repository class:
Remoterepository
public static class RemoteRepository { public static IQueryable<TResult> CreateQuery<TResult>(IChannelProvider provider) { return new RemoteQueryable<TResult>(provider); } } Now write about this code
var query = RemoteRepository.CreateQuery<Entity>() .Where(e => e.IsMaster) .Where(e => e.IsActive); and referring to the Expression property of the query object, we get the following expression tree:
Above, I wrote that the expression tree can be edited at runtime, so we can bypass the resulting tree and replace the MockDataSource with NhQueryable. NhQueryable can be obtained by calling session.Query ().
To perform a tree traversal, use the Visitor pattern. In order not to write the expression tree visitor "from scratch", we use this basic implementation . In the successor, override the VisitConstant and VisitMethodCall methods, and also override the Visit method, which will be the access point for editing the expression:
NhibernateExpressionVisitor
public class NhibernateExpressionVisitor : ExpressionVisitor { protected IQueryable queryableRoot; public new Expression Visit(Expression sourceExpression, IQueryable queryableRoot) { this.queryableRoot= queryableRoot; return Visit(sourceExpression); } protected override Expression VisitMethodCall(MethodCallExpression m) { var query = m; var constantArgument = query.Arguments.FirstOrDefault(e => e is ConstantExpression && e.Type.IsGenericType && e.Type.GetGenericTypeDefinition() == typeof(EnumerableQuery<>)); if (constantArgument != null) { var constantArgumentPosition = query.Arguments.IndexOf(constantArgument); var newArguments = new Expression[query.Arguments.Count]; for (int index = 0; index < newArguments.Length; index++) { if (index != constantArgumentPosition) newArguments[index] = query.Arguments[index]; else newArguments[index] = queryableRoot.Expression; } return Expression.Call(query.Object, query.Method, newArguments); } return base.VisitMethodCall(query); } protected override Expression VisitConstant(ConstantExpression c) { if (c.Type.IsGenericType && typeof(RemoteQueryable<>).IsAssignableFrom(c.Type.GetGenericTypeDefinition())) return queryableRoot.Expression; return c; } } How to use it:
var query = RemoteRepository.CreateQuery<Entity>() .Where(e => e.IsMaster) .Where(e => e.IsActive); using (var session = CreateSession()) { var nhQueryable = session.Query<Entity>(); var nhQueryableWithExternalQuery = new NhibernateExpressionVisitor().Visit(query.Expression, nhQueryable); var result = nhQueryable.Provider.Execute(nhQueryableWithExternalQuery); } On the line new NhibernateExpressionVisitor (). Visit (query.Expression, nhQueryable), the RemoteQueryable stub was replaced with NhQueryable.
By themselves, such gestures are not reasonable, if the code that collects such an expression is dynamically in the same process with NHibernate, then we will look at how to render the code generating the linq request from the fake source to an external process.
4. Linq database query through NHibernate from external process
The process from which requests come, let's call it “client”, and the process that executes requests is “server”
Client part
For the full implementation of linq, it is necessary to define a client request provider that will provide a fake stub. But before we define the interface through which you can access the server process:
public interface IChannelProvider { T SendRequest<T>(string request); } The responsibility for the process of serialization of the query results (objects or collections of objects) will be assigned to the transport layer, or rather to the implementation of IChannelProvider. Next, you need to modify the RemoteQueryable constructor and the RemoteRepository class so that you can pass the server process data provider (IChannelProvider) to these classes
public RemoteQueryable(IChannelProvider channelProvider) { Expression = Expression.Constant(this); Provider = new RemoteQueryableProvider<T>(channelProvider); } public static IQueryable<TResult> CreateQuery<TResult>(IChannelProvider provider) { return new RemoteQueryable<TResult>(provider); } To simplify the process of sending the request, we will give it to the transport level as a string. Next, we define the client query provider (RemoteQueryableProvider) with the IQueryProvider interface, as well as the DTO class (QueryDto) to send the request to the server:
RemoteQueryProvider
public class RemoteQueryProvider : IQueryProvider { public IQueryable CreateQuery(Expression expression) { var enumerableQuery = new EnumerableQuery<T>(expression); var resultQueryable = ((IQueryProvider)enumerableQuery).CreateQuery(expression); return new RemoteQueryable<T>(this, resultQueryable.Expression); } public IQueryable<TElement> CreateQuery<TElement>(Expression expression) { var enumerableQuery = new EnumerableQuery<TElement>(expression); var resultQueryable = ((IQueryProvider)enumerableQuery).CreateQuery<TElement>(expression); return new RemoteQueryable<TElement>(this, resultQueryable.Expression); } public object Execute(Expression expression) { var serializedQuery = SerializeQuery(expression); return channelProvider.SendRequest<object>(serializedQuery); } public TResult Execute<TResult>(Expression expression) { var serializedQuery = SerializeQuery(expression); return this.channelProvider.SendRequest<TResult>(serializedQuery); } public RemoteQueryableProvider(IChannelProvider channelProvider) { this.channelProvider = channelProvider; } private static string SerializeQuery(Expression expression) { var newQueryDto = QueryDto.CreateMessage(expression, typeof(T)); var serializedQuery = JsonConvert.SerializeObject(newQueryDto, new JsonSerializerSettings { ReferenceLoopHandling = ReferenceLoopHandling.Ignore, TypeNameHandling = TypeNameHandling.All }); return serializedQuery; } } Queryery to
public class QueryDto { public ExpressionNode SerializedExpression { get; set; } public string RequestedTypeName { get; set; } public string RequestedTypeAssemblyName { get; set; } public static QueryDtoCreateMessage(Expression expression, Type type) { var serializedExpression = expression.ToExpressionNode(); return new QueryDto(serializedExpression, type.FullName, type.Assembly.FullName); } private QueryDto(ExpressionNode serializedExpression, string requestedTypeName, string requestedTypeAssemblyName) { this.SerializedExpression = serializedExpression; this.RequestedTypeName = requestedTypeName; this.RequestedTypeAssemblyName = requestedTypeAssemblyName; } protected QueryDto() { } } The factory method QueryDto.CreateMessage () serializes the resulting expression using the Serialize.Linq library. The QueryDto class also contains the RequestedTypeName and RequestedTypeAssemblyName properties to identify the type of entity being requested and to properly restore the expression on the server. The very object of the QueryDto class is serialized into a string using the Json.NET library. The serialized request is transmitted to the server via the IChannelProvider proxy object.
')
The devil is in the details
1. Handling closures in an expression.
In the Linq section in Nhibernate, it was not by chance that I mentioned the process of calculating closures in expressions. Since the formation of the request is now in an external process, before sending the request, it is necessary to calculate the values ​​of the closures in the constructed expression. We are interested in (so far) only closures in predicates for example.
Pseudocode
RemoteRepository.Query<WorkItem>() .Where(w => w.Priority == EnvironmentSettings.MaxPriority)) // EnvironmentSettings To calculate the value of the links and convert the values ​​to ConstantExpression, we will write a client-side ExpressionVisitor, which will modify the expression
ClientExpressionVisitor
internal class ClientExpressionVisitor : ExpressionVisitor { public Expression Evaluate(Expression expression) { return base.Visit(expression); } private Expression EvaluateIfNeed(Expression expression) { var memberExpression = expression as MemberExpression; if (memberExpression != null) { if (memberExpression.Expression is ParameterExpression) return expression; var rightValue = GetValue(memberExpression); return Expression.Constant(rightValue); } var methodCallExpression = expression as MethodCallExpression; if (methodCallExpression != null) { var obj = ((ConstantExpression)methodCallExpression.Object).Value; var result = methodCallExpression.Method.Invoke(obj, methodCallExpression.Arguments.Select(ResolveArgument).ToArray()); return Expression.Constant(result); } return expression; } protected override Expression VisitBinary(BinaryExpression b) { Expression left = this.EvaluateIfNeed(this.Visit(b.Left)); Expression right = this.EvaluateIfNeed(this.Visit(b.Right)); Expression conversion = this.Visit(b.Conversion); if (left != b.Left || right != b.Right || conversion != b.Conversion) { if (b.NodeType == ExpressionType.Coalesce && b.Conversion != null) return Expression.Coalesce(left, right, conversion as LambdaExpression); else return Expression.MakeBinary(b.NodeType, left, right, b.IsLiftedToNull, b.Method); } return b; } private static object ResolveArgument(Expression exp) { var constantExp = exp as ConstantExpression; if (constantExp != null) return constantExp.Value; var memberExp = exp as MemberExpression; if (memberExp != null) return GetValue(memberExp); return null; } private static object GetValue(MemberExpression exp) { var constantExpression = exp.Expression as ConstantExpression; if (constantExpression != null) { var member = constantExpression.Value .GetType() .GetMember(exp.Member.Name) .First(); var fieldInfo = member as FieldInfo; if (fieldInfo != null) return fieldInfo.GetValue(constantExpression.Value); var propertyInfo = member as PropertyInfo; if (propertyInfo != null) return propertyInfo.GetValue(constantExpression.Value); } var expression = exp.Expression as MemberExpression; if (expression != null) return GetValue(expression); return null; } } and modify the Execute methods of the RemoteQueryableProvider class, taking into account the functionality of the ClientExpressionVisitor
RemoteQueryableProvider methods
public object Execute(Expression expression) { var partialEvaluatedExpression = this.expressionEvaluator.Evaluate(expression); var serializedQuery = SerializeQuery(partialEvaluatedExpression); return channelProvider.SendRequest<object>(serializedQuery); } public TResult Execute<TResult>(Expression expression) { var partialEvaluatedExpression = this.expressionEvaluator.Evaluate(expression); var serializedQuery = SerializeQuery(partialEvaluatedExpression); return this.channelProvider.SendRequest<TResult>(serializedQuery); } 2. The use of entity properties in queries that are not specified in the mapping.
If in NHibernate we write a condition of the form Where (x => x.UnmappedProperty == 4)), then the validator of the NHibernate query will not miss such an expression. To solve this problem, we introduce API post requests, i.e. queries to the data that have already been received as a result of sql-selection.
Postqueryable
internal class PostQueryable<T> : BaseQueryable<T> { public PostQueryable(IChannelProvider channelProvider) : base(channelProvider) { } public PostQueryable(AbstractQueryProvider provider, Expression expression) : base(provider, expression) { } public PostQueryable() { Expression = Expression.Constant(this); } } and extension for easy query wrapping
Ex
public static class Ex { public static IQueryable<T> PostQuery<T>(this IQueryable<T> sourceQuery) { var query = Expression .Call(null, typeof (PostQueryable<T>).GetMethod(nameof(PostQueryable<T>.WrapQuery)), new [] {sourceQuery.Expression}); return sourceQuery.Provider.CreateQuery<T>(query); } } Now you can write a query like:
Pseudocode
int stateCoefficient = 0.9; int ageLimitInCurrentState = 18 * stateCoefficient; var availableMovies = session .Query<Movie>() .Where(m => m.AgeLimit >= ageLimitInCurrentState) .PostQuery() .Where(m => m.RatingInCurrentState > 8) // unmapped- RatingInCurrentState .ToList() and send it to the server.
Server part
Both server and client code must be in the same assembly, but the code contains references to types from the NHibernate assembly. It is necessary to untie the extra dependency for the client. We write a helper for working with Nhibernate types to remove the hard link to the Nhibernate.dll assembly. The very same assembly NHibernate.dll on the server side will be loaded via Reflection.
NHibernateTypesHelper
internal static class NHibernateTypesHelper { private static readonly Assembly nhibernateAssembly; public static Type SessionType { get; private set; } public static Type LinqExtensionType { get; private set; } public static bool IsSessionObject(object inspectedObject) { return SessionType.IsInstanceOfType(inspectedObject); } static NHibernateTypesHelper() { nhibernateAssembly = AppDomain.CurrentDomain.GetAssemblies() .FirstOrDefault(asm => asm.FullName.Contains("NHibernate")) ?? Assembly.Load("NHibernate"); if (nhibernateAssembly == null) throw new InvalidOperationException("Caller invoking server-side types, but the NHibernate.dll not found in current application domain"); SessionType = nhibernateAssembly.GetTypes() .Single(p => p.FullName.Equals("NHibernate.ISession", StringComparison.OrdinalIgnoreCase)); LinqExtensionType = nhibernateAssembly.GetTypes() Single(p => p.FullName.Equals("NHibernate.Linq.LinqExtensionMethods", StringComparison.OrdinalIgnoreCase)); } } On the server side, we define the RemoteQueryExecutor class, which will accept the serialized QueryDto, restore it, and execute it:
RemoteQueryExecutor
public static class RemoteQueryExecutor { public static object Do(string serializedQueryDto, object sessionObject) { var internalRemoteQuery = DeserializeQueryDto(serializedQueryDto); var deserializedQuery = DeserializedQueryExpressionAndValidate(internalRemoteQuery); var targetType = ResolveType(internalRemoteQuery); return Execute(deserializedQuery, targetType, sessionObject); } private static TypeInfo ResolveType(QueryDto internalRemoteQuery) { var targetAssemblyName = internalRemoteQuery.RequestedTypeAssemblyName; var targetAssembly = GetAssemblyOrThrownEx(internalRemoteQuery, targetAssemblyName); var targetType = GetTypeFromAssemblyOrThrownEx(targetAssembly, internalRemoteQuery.RequestedTypeName, targetAssemblyName); return targetType; } private static Expression DeserializedQueryExpression(QueryDto internalRemoteQuery) { var deserializedQuery = internalRemoteQuery.SerializedExpression.ToExpression(); return deserializedQuery; } private static TypeInfo GetTypeFromAssemblyOrThrownEx(Assembly targetAssembly, string requestedTypeName, string targetAssemblyName) { var targetType = targetAssembly.DefinedTypes .FirstOrDefault(type => type.FullName.Equals(requestedTypeName, StringComparison.OrdinalIgnoreCase)); if (targetType == null) throw new InvalidOperationException(string.Format("Type with name '{0}' not found in assembly '{1}'", requestedTypeName, targetAssemblyName)); return targetType; } private static Assembly GetAssemblyOrThrownEx(QueryDto internalRemoteQuery, string targetAssemblyName) { var targetAssembly = AppDomain.CurrentDomain.GetAssemblies() .FirstOrDefault(asm => asm.FullName.Equals(internalRemoteQuery.RequestedTypeAssemblyName, StringComparison.OrdinalIgnoreCase)); if (targetAssembly == null) throw new InvalidOperationException(string.Format("Assembly with name '{0}' not found in server app domain", targetAssemblyName)); return targetAssembly; } private static QueryDto DeserializeQueryDto(string serializedQueryDto) { var internalRemoteQuery = JsonConvert .DeserializeObject<QueryDto>(serializedQueryDto, new JsonSerializerSettings { ReferenceLoopHandling = ReferenceLoopHandling.Ignore, TypeNameHandling = TypeNameHandling.All }); return internalRemoteQuery; } private static object Execute(Expression expression, Type targetType, object sessionObject) { var queryable = GetNhQueryableFromSession(targetType, sessionObject); var nhibernatePartialExpression = ExpressionModifier.GetNhibernatePartialExpression(expression, queryable); var resultFromStorage = queryable.Provider.Execute(nhibernatePartialExpression); var requestedCollection = resultFromStorage as IEnumerable<object>; if (requestedCollection == null) return resultFromStorage; var resultCollectionType = requestedCollection.GetType(); if (resultCollectionType.IsGenericType) targetType = resultCollectionType.GetGenericArguments().Single(); var enumerableQueryable = (IQueryable)Activator .CreateInstance(typeof(EnumerableQuery<>).MakeGenericType(targetType), new[] { requestedCollection }); var postQueryPartialExpression = ExpressionModifier .GetPostQueryPartialExpression(expression, enumerableQueryable); if (postQueryPartialExpression == null) return resultFromStorage; return enumerableQueryable.Provider.Execute(postQueryPartialExpression); } private static IQueryable GetNhQueryableFromSession(Type targetType, object sessionObject) { var finalQueryMethod = ResolveQueryMethod(targetType); var queryable = (IQueryable) finalQueryMethod.Invoke(null, new object[] {sessionObject}); return queryable; } private static MethodInfo ResolveQueryMethod(Type targetType) { var queryMethod = typeof(LinqExtensionMethods).GetMethods(BindingFlags.Public | BindingFlags.Static) .Where(m => m.IsGenericMethod) .Where(m => m.Name.Equals("Query")) .Single(m => m.GetParameters().Length == 1 && NHibernateTypesHelper.SessionType.IsAssignableFrom(m.GetParameters().First().ParameterType)); var finalQueryMethod = queryMethod.MakeGenericMethod(targetType); return finalQueryMethod; } } After receiving the request, on the server we restore QueryDto with the Json.NET deserializer. The serialized expression inside the DTO object is restored using the Serialize.Linq library . Then we modify the expression with the help of the NhibernateExpressionVisitor visitor - we replace the fake root with NhQueryable, as explained above.
The resulting expression is divided into two queries:
- Directly query to the database
- Query the results of the sample from the database
The query to the database will work according to the already known scheme, without compilation. The query to the results of the sample is compiled and processes the objects by the methods specified in the expressions. The query for the results of the previous query is compiled and executed as a regular EnumerableQuery.
Request drawings
5. We write test application
We implement a test client-server application. For the client side, we use the WPF technology, on the server side, we will use SQLite as the database, for communication between the processes, we will use WCF with HTTP bindings. As an object model, use the WorkItem class.
[DataContract] public class WorkItem : BaseEntity { [DataMember] public virtual string Text { get; set; } [DataMember] public virtual int Priority { get; set; } } Behind the scenes, leave the NHibernate mapping setting and the nhibernate.cfg.xml configuration, as well as the WCF setting for data transfer and set the goal - to display 200 WorkItem objects in the ListView, loading data from the database as necessary.
WPF for lists provides a mechanism for data virtualization at the UI layer, which can be further developed for virtualization at the level of the ViewModel data collection. We take the example from this article as a basis and modify the example for paging downloading data from the database to the client.
We implement our IItemsProvider and replace the implementation from the example with our class DemoWorkItemProvider. In the FetchCount () and FetchRange () methods, we will use Linq queries using the RemoteQueryable API. In the FetchRange method, we specify a query only for the range of data that is required for display.
DemoWorkItemProvider
public class DemoWorkItemProvider : IItemsProvider<WorkItem> { public int FetchCount() { return RemoteRepository.CreateQuery<WorkItem>(new DemoChannelProvider()) .Count(); } public IList<WorkItem> FetchRange(int startIndex, int count) { return RemoteRepository.CreateQuery<WorkItem>(new DemoChannelProvider()) .Skip(startIndex) .Take(count) .ToList(); } } Few rule UI and ListView style for displaying WorkItem. We start the server and the client, when clicking on the Refresh button, the client sends a Linq request for the number of items and two subsequent requests for receiving the first and second pages of the list to the server. When scrolling down the list, the next page is loaded from the database via the
RemoteQueryablyProvider -> WCF -> HTTP -> WCF -> RemoteQueryExecutor -> NHibernate -> SQLite chain
Demonstration of work
Pros of the RemoteQueryable API:
- 1. Clean and clear code in the FetchRange and FetchCount methods
- 2. The ability to build dynamic queries on the client using Dynamic.Linq (for example, to filter data in tables)
- 3. The one and only WCF service method for processing a request.
- 4. Optimization of data retrieval - from the database only those records that were actually requested by the client will be selected.
Conclusion
On this, perhaps, you can stop. The next step may be to modify the request on the server side, taking into account user authorization or optimization of the code that works with Reflection, or the connection Dynamic.Linq, but this is the topic of a separate article.
Links
1. Repositories with code and examples from the article on GitHub
2. IEnumerable and IQueryable, what is the difference?
3. Principles of operation of IQueryable and LINQ data providers
4. Closures in the C # programming language
5. Serialize.Linq source codes
6. NHibernate sources
7. Remotion.Linq on codeplex
8. IQueryProvider on MSDN
9. How to: Implement an Expression Tree Visitor
Source: https://habr.com/ru/post/302594/
All Articles