LINQ extensions for Azure Table Storage that implement Or and Contains

Hello! I am glad to present you the fifth article from the series “The internal structure and architecture of the AtContent.com service”. In it, I will talk about how to make working with Azure Table Storage more functional and convenient.







The Windows Azure platform provides a very powerful set of tools for implementing your ideas. And among them - Azure Table Storage - non-relational database with unlimited volume. The big advantage of this repository is that you can do quite complex queries to it. But beyond that there are some inconveniences. For example, using LINQ, it is impossible to execute queries that have Or or Contains logic without additional modifications.



When we had to deal with this, we studied the problem and it turned out that using the REST API, you can make requests in which there is Or. And we really needed such requests. Now don't rewrite all the code from LINQ to the REST API for the sake of it. Digging a little deeper, we found a solution!

')

So, to save a convenient way to work with Azure Table Storage - LINQ - and to complement its ability to make Contains queries, it was necessary to parse the mechanisms that occur in the depths of LINQ when the query is transformed into a REST API. The problem itself is that we dynamically modify the LINQ query only with Where. The expression is added to the query with the modifier And. Therefore, it is easy to compose a query with a variable number of parameters connected by the operator And. If you need to do the same for Or, you will have to involve lambda expressions for help.



Of course, you can just make a few requests and then combine them into one, but such extravagance affects the final cost of the solution for your customers or subscribers. Even with the cost of one request in hundredths of a cent, the difference between one request and, say, ten is obvious. Therefore, we proceed to our decision.



The first attempt ended in failure. Search engines could not help in this matter and therefore had to act through the experiment. In IQuerable. Where () you can substitute Expression <Func <T, bool >>. With grief in half, having made such an expression and testing it, I was disappointed. The resulting expression after Invoke () included the class names and such a query to Azure Table Storage returned an exception, of course.



But that did not stop me. The prospect of using the REST API instead of LINQ prevented me from falling asleep. Just a couple of days of intensive reading of MSDN and an elegant solution was born that can be applied very simply.

public static class LinqExtension { public static Expression<Func<T, bool>> Contains<T>(string ObjectStringName, string FieldStringNameToCompare, IList<String> ListOfValues) { Expression ExprBody = null; ParameterExpression ParameterObject = Expression.Parameter(typeof(T), ObjectStringName); var PropertyFieldToCompare = Expression.Property(ParameterObject, FieldStringNameToCompare); foreach (var ValueToCompare in ListOfValues) { ConstantExpression ValueConst = Expression.Constant(ValueToCompare, typeof(string)); BinaryExpression EqualTermExpression = Expression.Equal(PropertyFieldToCompare, ValueConst); if (ExprBody == null) { ExprBody = EqualTermExpression; } else { ExprBody = Expression.Or(ExprBody, EqualTermExpression); } } if (ExprBody == null) { ExprBody = Expression.IsTrue(Expression.Constant(false)); } var FinalExpression = Expression.Lambda<Func<T, bool>>(ExprBody, new ParameterExpression[] { ParameterObject }); return FinalExpression; } public static IQueryable<T> Contains<T>(this IQueryable<T> obj, string ObjectStringName, string FieldStringNameToCompare, IList<String> ListOfValues) { var Expression = Contains<T>(ObjectStringName, FieldStringNameToCompare, ListOfValues); if (Expression != null) return obj.Where<T>(Expression); return obj; } } 

It allows you to apply to the query operation Contains. Thus, an analogue of the IN operation in SQL is obtained. This has greatly expanded the applicability of Azure Table Storage in our project.



In this implementation, it should be noted that the lambda expression is constructed in stages and that very line that kept me awake for a couple of days looks like this:

 ConstantExpression ValueConst = Expression.Constant(ValueToCompare, typeof(string)); 

It indicates that when executing Invoke () you need to substitute a constant, and not the value of the object.

The implementation given here can work only with string values, but nothing prevents you from expanding these methods yourself for use with any types.



The solution described in the article is included in the CPlase Open Source library, which is being prepared for publication.



Read in the series:

• " AtContent.com. Internal structure and architecture ",
• " The mechanism of messaging between roles and instances ",
• " Caching data on an instance and managing caching ",
• " Effective cloud queue management (Azure Queue) ",
• " Practical tips for splitting data into parts, generating PartitionKey and RowKey for Azure Table Storage ."

I also invite you to try our service.