.Net Core, 1C, dynamic compilation, Scripting API

Good day to the people of Habratchane! Today I will continue to torment you with the great and mighty Ruslish. This is a continuation of the articles:

» Development → Cross-platform use of .Net classes from unmanaged code. Or analogue IDispatch on Linux
» Development → Cross-platform use of .Net classes in 1C through Native VK. Or replacing COM with Linux
» Cross-platform use of .Net classes in 1C through Native VK. Or replacing COM with Linux II
» Asynchronous programming in 1C via .Net Native VK
1C, Linux, Excel, Word, OpenXML, ADO and Net Core

Currently in .Net Core 2 dynamic compilation options
')
1. This is an analogue of CodeDom Microsoft.CodeAnalysis.CSharp.CSharpCompilation
2. Roslyn Scripting Api. Examples here

var compilation = Microsoft.CodeAnalysis.CSharp.CSharpCompilation.Create("a") .WithOptions(new Microsoft.CodeAnalysis.CSharp.CSharpCompilationOptions(Microsoft.CodeAnalysis.OutputKind.DynamicallyLinkedLibrary)) .AddReferences( Microsoft.CodeAnalysis.MetadataReference.CreateFromFile(typeof(object).GetTypeInfo().Assembly.Location)) .AddSyntaxTrees(Microsoft.CodeAnalysis.CSharp.CSharpSyntaxTree.ParseText( @" using System; public class C { public C(){} public string M() { return ""Hello Roslyn.""; } }")); var fileName = @"d:\NetStandart\TestCoreNetApp\src\TestCoreNetApp\bin\Debug\netcoreapp1.0\a.dll"; compilation.Emit(fileName); var a = System.Runtime.Loader.AssemblyLoadContext.Default.LoadFromAssemblyPath(fileName); Type  = a.GetType("C"); var obj = Activator.CreateInstance(); var res = .GetMethod("M").Invoke(obj, null); Console.WriteLine(res.ToString()); 

This approach is good when you need to update the library. But this creates a DLL with all the consequences.

I like the second way more. Take the example of getting a delegate.

 string words = "        "; string pattern = @"\w+"; var scr = Microsoft.CodeAnalysis.Scripting.ScriptOptions.Default; var mscorlib = Assembly.Load(System.Runtime.Loader.AssemblyLoadContext.GetAssemblyName(@"c:\Users\Smirnov_SA\.nuget\packages\Microsoft.NETCore.Portable.Compatibility\1.0.1\ref\netcore50\mscorlib.dll")); scr =scr.WithReferences(mscorlib, typeof(MulticastDelegate).GetTypeInfo().Assembly, typeof(System.Runtime.CompilerServices.IStrongBox).GetTypeInfo().Assembly, typeof(MatchEvaluator).GetTypeInfo().Assembly, typeof(Regex).GetTypeInfo().Assembly) .WithImports("System", "System.Text.RegularExpressions"); string  = @"return (MatchEvaluator)((match) => { string x = match.Value; // If the first char is lower case... if (char.IsLower(x[0])) { // Capitalize it. return char.ToUpper(x[0]) + x.Substring(1, x.Length - 1); } return x; });"; var result = Microsoft.CodeAnalysis.CSharp.Scripting.CSharpScript.EvaluateAsync(, scr).Result; MatchEvaluator evaluator = (MatchEvaluator)result; Console.WriteLine(Regex.Replace(words, pattern, evaluator)); 

Be sure to include links to:

mscorlib.dll
System.Private.CoreLib.ni.dll
System.Runtime.dll

The following libraries are used for compilation:

"Microsoft.CodeAnalysis.CSharp": "2.0.0-beta3",
"Microsoft.CodeAnalysis.CSharp.Scripting": "2.0.0-beta3",
"Microsoft.CodeAnalysis.Scripting.Common": "2.0.0-beta3",
Microsoft.CodeAnalysis.Scripting

By dynamic compilation. At one time he worked with spare parts for cars. And there are a lot of suppliers and customers. At the same time prices for millions of positions. And everyone is ready to give data in its own format. It was easier for each client to write code in the directory and use it through Run or Calculate. True, when working with millions of prices, this approach was inhibited.

One way or another, I had to write DLLs and work through COM.

With the help of dynamic compilation, you can store the text of the code and apply it depending on the conditions. Including dynamically form the conditions, and the compiled delegate can be cached for reuse.
The compilation speed is quite high (except for the first time seconds 5).

Let's go to 1C. So the algorithm on 1C looks like this

  = "return (MatchEvaluator)((match) => |{ | string x = match.Value; |// If the first char is lower case... |if (char.IsLower(x[0])) |{ |// Capitalize it. |return char.ToUpper(x[0]) + x.Substring(1, x.Length - 1); |} |return x; | |});";  = "        ";  = "\w+"; // MatchEvaluator evaluator = (MatchEvaluator)(); ScriptOptions=("Microsoft.CodeAnalysis.Scripting.ScriptOptions","Microsoft.CodeAnalysis.Scripting"); CSharpScript=("Microsoft.CodeAnalysis.CSharp.Scripting.CSharpScript","Microsoft.CodeAnalysis.CSharp.Scripting"); scr = (ScriptOptions.Default); mscorlib = (.("mscorlib.dll",)); Private_CoreLib=(.("System.Private.CoreLib.ni",)); System_Runtime=(.("System.Runtime",)); RegularExpressions=(.("System.Text.RegularExpressions",)); Regex=(RegularExpressions.GetType("System.Text.RegularExpressions.Regex")); scr =(scr.WithReferences(mscorlib.(), Private_CoreLib.(), System_Runtime.(), RegularExpressions.())); scr =(scr.WithImports("System", "System.Text.RegularExpressions")); evaluator = ((CSharpScript.EvaluateAsync(, scr.())).Result); (Regex.Replace(, , evaluator.())); 

In general, nothing special. Received the assembly, gave links to them compiled, called.

Let us turn to more complex examples. So in the previous article I showed examples of using DocumentFormat.OpenXml on the example of reading Excel and Word.

There was a problem in speed due to bringing the string to the object through the function and the call rate itself from 1C is 5 times slower than its native code.

Therefore, we will render most of the code in .Net. A similar option is on the large .Net
.Net in 1C. On the example of using HTTPClient, AngleSharp. Convenient parsing of sites using the AngleSharp library, including authorization ala JQuery using CSS selectors. Dynamic compilation (example at the end of the article).

Create a class and copy it into the layout. The essence of the class is to read the data cells and group them by line number.

 public class  { public string ; public string ; public int ; public string ; } public class ExcelReader { static Regex  = new Regex("[A-Za-z]+"); OpenXmlElementList ; void (string ,  ) { . = ; var match = .Match(); var  = match.Value; var  = int.Parse(.Substring(.Length)); . = ; . = ; } void (List<> , Cell cell) { var  = cell.CellReference.InnerText; var text = cell.CellValue?.Text; var DataType = cell.DataType; string res = text; if (DataType != null && DataType == CellValues.SharedString) { int ssid = int.Parse(text); res = [ssid].InnerText; } if (res == null) return; var result = new (); (, result); result. = res; .Add(result); } public List<> ReadExcel(string fileName) { List<>  = new List<>(); using (FileStream fs = new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)) { using (SpreadsheetDocument doc = SpreadsheetDocument.Open(fs, false)) { var workbookPart = doc.WorkbookPart; //    //     var pt = workbookPart.GetPartsOfType<SharedStringTablePart>(); var sstpart = pt.First(); var sst = sstpart.SharedStringTable;  = sst.ChildElements; var workbook = workbookPart.Workbook; //    var sheet = workbook.Descendants<Sheet>().First(); var worksheetPart = (DocumentFormat.OpenXml.Packaging.WorksheetPart)workbookPart.GetPartById(sheet.Id); var worksheet = worksheetPart.Worksheet; var cells = worksheet.Descendants<Cell>(); // One way: go through each cell in the sheet foreach (Cell cell in cells) { (, cell); } } } return ; } static string (List<> ) { var  = ""; var  = 0; foreach (var  in ) { var  = .; var  = .Length; if ( > ) {  = ;  = ; } else if ( ==  && string.Compare(, , true) > 0)  = ; } return ; } public static object (string fileName) { var res = new ExcelReader(); var  = res.ReadExcel(fileName); var  = .GroupBy( => .).Select( => new {  = .Key,  = .ToArray() }).OrderBy( => .); var  = (); return new {  = .ToList(),  =  }; } } return new Func<string, object>(ExcelReader.); 

We have described a reading class and return a reference to the delegate that takes the path to the file and returns an anonymous class. All the same, in 1C we will work with him through reflection. Please note that 2 classes are described here.

Now let's call this code from 1C.

 ScriptOptions=("Microsoft.CodeAnalysis.Scripting.ScriptOptions","Microsoft.CodeAnalysis.Scripting"); CSharpScript=("Microsoft.CodeAnalysis.CSharp.Scripting.CSharpScript","Microsoft.CodeAnalysis.CSharp.Scripting"); scr = (ScriptOptions.Default); mscorlib = (.("mscorlib.dll",)); Private_CoreLib=(.("System.Private.CoreLib.ni",)); System_Runtime=(.("System.Runtime",)); RegularExpressions=(.("System.Text.RegularExpressions",)); OpenXml=(.("DocumentFormat.OpenXml.dll")); Linq=(.("System.Linq", )); FileSystem=(.("System.IO.FileSystem", )); Regex=(RegularExpressions.GetType("System.Text.RegularExpressions.Regex")); scr =(scr.WithReferences(mscorlib.(), Private_CoreLib.(), System_Runtime.(), RegularExpressions.(),OpenXml.(),Linq.(),FileSystem.())); scr =(scr.WithImports("System", "System.Collections.Generic", "System.Linq", "System.IO", "DocumentFormat.OpenXml", "DocumentFormat.OpenXml.Packaging", "DocumentFormat.OpenXml.Spreadsheet", "System.Text.RegularExpressions")); =("").();  = ((CSharpScript.EvaluateAsync(, scr.())).Result); = (.DynamicInvoke()); (.); = ; =.; (.,); =.; =(.); =(.(.())); =1; //       .MoveNext()  = (.Current); =.;  <  =+1; .(); ; =+1; =.(); //     =(.); =(.(.()));  .MoveNext()  = (.Current); =.; =.; //        64  1  26   //         =.(); .(.(),); ; ; 

Now Excel's processing speed has increased significantly, and the compilation costs are commensurate with the cost of reading files.

Let's look at the process of reading Word. Without further ado, I took a ready class here . Moreover, everything is in English.

 //     //https://code.msdn.microsoft.com/office/CSOpenXmlGetPlainText-554918c3/sourcecode?fileId=71592&pathId=851860130 public class GetWordPlainText : IDisposable { // Specify whether the instance is disposed. private bool disposed = false; // The word package private WordprocessingDocument package = null; /// <summary> /// Get the file name /// </summary> private string FileName = string.Empty; /// <summary> /// Initialize the WordPlainTextManager instance /// </summary> /// <param name="filepath"></param> public GetWordPlainText(string filepath) { this.FileName = filepath; if (string.IsNullOrEmpty(filepath) || !File.Exists(filepath)) { throw new Exception("The file is invalid. Please select an existing file again"); } this.package = WordprocessingDocument.Open(filepath, true); } /// <summary> /// Read Word Document /// </summary> /// <returns>Plain Text in document </returns> public string ReadWordDocument() { StringBuilder sb = new StringBuilder(); OpenXmlElement element = package.MainDocumentPart.Document.Body; if (element == null) { return string.Empty; } sb.Append(GetPlainText(element)); return sb.ToString(); } /// <summary> /// Read Plain Text in all XmlElements of word document /// </summary> /// <param name="element">XmlElement in document</param> /// <returns>Plain Text in XmlElement</returns> public string GetPlainText(OpenXmlElement element) { StringBuilder PlainTextInWord = new StringBuilder(); foreach (OpenXmlElement section in element.Elements()) { switch (section.LocalName) { // Text case "t": PlainTextInWord.Append(section.InnerText); break; case "cr": // Carriage return case "br": // Page break PlainTextInWord.Append(Environment.NewLine); break; // Tab case "tab": PlainTextInWord.Append("\t"); break; // Paragraph case "p": PlainTextInWord.Append(GetPlainText(section)); PlainTextInWord.AppendLine(Environment.NewLine); break; default: PlainTextInWord.Append(GetPlainText(section)); break; } } return PlainTextInWord.ToString(); } #region IDisposable interface public void Dispose() { Dispose(true); GC.SuppressFinalize(this); } protected virtual void Dispose(bool disposing) { // Protect from being called multiple times. if (disposed) { return; } if (disposing) { // Clean up all managed resources. if (this.package != null) { this.package.Dispose(); } } disposed = true; } #endregion public static string GetText(string FileName) { using (var pt = new GetWordPlainText(FileName)) { return pt.ReadWordDocument(); } } } return new Func<string,string>(GetWordPlainText.GetText); 

But back to Ruslish:

 ScriptOptions=("Microsoft.CodeAnalysis.Scripting.ScriptOptions","Microsoft.CodeAnalysis.Scripting"); CSharpScript=("Microsoft.CodeAnalysis.CSharp.Scripting.CSharpScript","Microsoft.CodeAnalysis.CSharp.Scripting"); scr = (ScriptOptions.Default); mscorlib = (.("mscorlib.dll",)); Private_CoreLib=(.("System.Private.CoreLib.ni",)); System_Runtime=(.("System.Runtime",)); RegularExpressions=(.("System.Text.RegularExpressions",)); OpenXml=(.("DocumentFormat.OpenXml.dll")); Linq=(.("System.Linq", )); FileSystem=(.("System.IO.FileSystem", )); Regex=(RegularExpressions.GetType("System.Text.RegularExpressions.Regex")); scr =(scr.WithReferences(mscorlib.(), Private_CoreLib.(), System_Runtime.(),OpenXml.(),FileSystem.())); scr =(scr.WithImports("System", "System.Text", "System.IO", "DocumentFormat.OpenXml", "DocumentFormat.OpenXml.Packaging")); =("").();  = ((CSharpScript.EvaluateAsync(, scr.())).Result);  = .DynamicInvoke(); = ; .(); .(); 

The main task is to provide links to the used assemblies and namespace.

In the next article, I will create a dynamic wrapper over objects using events, by analogy with
.NET (C #) for 1C. Dynamic compilation of a wrapper class for using .Net events in 1C via Add Handler or Processing an External Event