Part I. InterSystems GlobalsDB .Net - reconnaissance in force with peeping under the hood

Finally, instead of persuading them to wait a little longer, the question “Is there InterSystems GlobalsDB / Caché Extreme under Microsoft .Net?” Can be answered in the affirmative. The new version of Caché 2012.2 (Field Test) and GlobalsDB v2012.296 has added support for this platform.
I will try my favorite for many developers on one-sixth sushi style, that is, without reading install notes and other things, to explore what the GlobalsDB distribution for Windows actually represents.

Downloading the distribution for Windows and .Net. We get the globals_setup_2012.296.exe size of 25 megabytes. The InstallShield installer starts. I launched it not in administrator mode under Windows 7x64, running in VMWare Fusion under Mac OS X.

The installation was predictable, I specified C: \ usr \ isc \ globals as the home directory. For the purity of the experiment, I must say that I already have this Caché operating system.

I study what was put. There are three folders in \ usr \ isc \ globals: bin, dev, mgr - standard for Caché structure. The binaries contain the binaries and the DBMS itself, in mgr the configuration and database files, plus, which is important, the system log, in our case globalsdb.log
')
Go to. \ Dev - a folder with various tools for the developer. We are interested in. \ Dev \ dotnet, consisting of bin, help and samples. In. \ Bin is the assembly of InterSystems.GlobalsDB, in the help we notice chm analogue of javadocs, in samples there is a Visual Studio project with a small example of use.

I press on the project. My Visual Studio 10 starts up and starts converting the project from 2008 into its format - the InterSystems developers are still sitting on the eight, although in the new release of Caché there are already builds for .Net 4. The conversion ends successfully, I launch Debug for execution - and about a miracle earned!

My surprise is due to the fact that in the case of GlobalsDB for Java, it was necessary to start the database via. \ Bin \ globalsdb start and for accessing the database it was necessary to coax the GLOBALS_HOME and PATH environment variables.

Explore the issue using Sysinternals Process Explorer. I discover six cache.exe processes that started from C: \ usr \ isc \ globalsdb (by the way, this is the value for GLOBALS_HOME). I look at the environment of my started Visual Studio - it already has GLOBALS_HOME and the desired path in PATH is% GLOBALS_HOME% \ bin. I look at the list of Windows services - I discover the GlobalsDB service. In general, the installer took care of the little things - a good trend. By the way, start / stop through the% GLOBALS_HOME% \ bin \ globalsdb start or stop command is equivalent to starting or stopping the service and vice versa.

Well, the console application works, however, the example is rather a test suite of the developer GlobalsDB, which methodically checked that all the overloads of the GlobalsDB API methods under the .Net work. Therefore, to deal with the API, create your new project. At the same time, let's see what needs to be done in VS in order to pick up GlobalsDB.

Create a new project under the console application. Add a reference to InterSystems.GlobalsDB, which lies in% GLOBALS_HOME% \ dev \ dotnet \ bin. In my case, this is the C: \ usr \ isc \ globals \ dev \ dotnet \ bin \ InterSystems.GlobalsDB.dll assembly.

using System; using System.Diagnostics; using InterSystems.Globals; namespace globals { class Program { static void Main(string[] args) { } } } 

System.Diagnostics will be used to measure the speed of work.

Now let's see in the example how the connection is made and transfer it to our code:

 static void Main(string[] args) { Connection connection = ConnectionContext.GetConnection() connection.Close(); Console.ReadLine(); } 

We start. We get the exception. Everything is clear, my project collects under x32 by default. Go to the project Properties-> Build and set the Platform target to x64. Run - all is well.

Go ahead - try to write something in the database. This is something called in GlobalsDB globals, or using the full name — global variables, that is, Caché global variables, which, unlike local variables in the memory of a particular database process, live on disk and thus are accessible to all database connections. . In GlobalsDb we can work only with globala.

Globals each perceives because of their corruption - someone calls them multidimensional variables, others see them as trees or key-value stores, I prefer to represent them as associative arrays or MultiMap. If you do not indulge in fantasies, then the global is the atomic structure of data storage in Caché (for example, the table will be the atomic structure for relationals), which has a name, consists of elements called nodes. Each node consists of the left part, which is called a subscript (often called an index or key) and the right part - the actual stored data, which can be either a simple type (numbers, strings), or a list, the elements of which are again either simple types or lists .
Examples of globals (pseudocode).

a. For those who think that this is a key value or a collection with access by key:

test(1) = “some text or number limited to 3641144 bytes”
test(“any string limited to 500 bytes”) = 3.14


b. For those who think that this tree:

test(1,”post”) = “root”
test(1,2,”post”) = “branch A”
test(1,3,”post”) = “branch B”

c. An empty subscript is also a subscript:

test = 2

d. Similar to tabular data storage using lists:

orders(1)=list(1001,”20.12.2012”,3,,999.99)
orders(2)=list(1002,”04.12.2012”,4,,249.99)

Let's try now to save the data in the global. To do this, you need to specify the name of the global, create a new global node, specify a subscript for the node (key or index, as you can see from the example) and data that is stored in this node. To work with global nodes in the GlobalsDB API, an object of the NodeReference class is used. In fact, this is a pointer to a node in the global. Since the node is identified by its subscript, we can assume that the NodeReference is a link to some subscript. The link to the global, which initially points to the head node, we get through the connection:

NodeReference testArray = connection.CreateNodeReference ("test"); // global name is test

To begin, use the simple NodeReference.Set () method. In generalized form, it can be perceived as a NodeReference.Set (data, subscript) and, accordingly, after its execution, we have a global node in the database of this form:

globalName (subscript) = data

Thus, the code looks like this:

  static void Main(string[] args) { Connection connection = ConnectionContext.GetConnection(); NodeReference node = connection.CreateNodeReference("test"); node.Set("some text or number limited to 3641144 bytes",1); node.Set(3.14, "any string or primitive type or several values limited to 500 bytes"); connection.Close(); Console.ReadLine(); } 

We do. We get the exception: Must connect before calling Connection.CreateNodeReference (). That's right, we are looking at the example code - we are responsible for establishing the connection ourselves, not the factory. Why is that? I do not know. Here you can see one thing that there are issues related to multi-threading work with the connection and for the future it is important to keep this in mind if necessary. Add a connection:

if (! connection.IsConnected ()) connection.Connect ();

Now everything is fine. We notice that hints for GlobalsDB API work. Add examples to work with multidimensional subscript and limit values ​​for subscript and data on the right:

  static void Main(string[] args) { Connection connection = ConnectionContext.GetConnection(); if (!connection.IsConnected()) connection.Connect(); NodeReference node = connection.CreateNodeReference("test"); node.Set("some text or number limited to 3641144 bytes",1); node.Set(3.14, "any string limited to 500 bytes"); node.Set("root",new object[2] {1,"post"}) ; node.Set("branch A",new object[3] {1,1,"post"}) ; node.Set("branch B",new object[3] {1,2,"post"}) ; node.Set(2); node.Close(); node = connection.CreateNodeReference("limit"); String longestKey = new String('k', 500); String longestValue = new String('v', 3641144); node.Set(longestValue, longestKey); connection.Close(); Console.ReadLine(); } 

How to see what happened globals insertion into the database? You can use one of the NodeReference reading methods. However, for debugging, you can use this technique, which is not documented in GlobalsDB, but is familiar to those who worked with Caché (especially as there are no administration utilities in GlobalsDB yet). So, go to% GLOBALS_HOME% and execute on the command line:

 C:\usr\isc\globals\bin>cache -s ..\mgr -U Data DATA>_ 

As a result, we get a terminal session to the database. What it is? This is a standard database process running in terminal mode, which allows us to execute commands in the language Caché Object Script. Independent processes that work with the database is one of the features of Caché that allows you to use it as an application server — launch processes where, for example, user code of the application will work. In the case of GlobalsDB, this process becomes the CLR process in which our .Net code is executed.

How to view the created nodes? To do this, there are commands write (abbreviated w) and zwrite (zw). Example:

 DATA>zw ^test ^test=2 ^test(1)="some text or number limited to 3641144 bytes" ^test(1,1,"post")="branch A" ^test(1,2,"post")="branch B" ^test(1,"post")="root" ^test("any string limited to 500 bytes")=3.1400000000000001243 

Deleting a node or the entire global - kill command (abbreviated k):

 DATA>kill ^test 

Data Record:

 DATA>s ^test(1,"name") = "Athens" 

Insert in a loop:

 DATA>for i=1:1:100 { s ^test(i) = i } 

Exiting the terminal - the halt command (abbreviated h):

 DATA>h C:\usr\isc\globals\bin> 

Measurement time:

 DATA>s st=$zh for i=1:1:100000 { s ^test(i)=i} w $zh-st,! .064444 

Now we’ll try to insert 100,000 records from .Net and check the time:

  static public void testInserts(int loop) { Connection connection = ConnectionContext.GetConnection(); if (!connection.IsConnected()) connection.Connect(); NodeReference node = connection.CreateNodeReference("test"); node.Kill(); Stopwatch dbTimer = Stopwatch.StartNew(); for (int i = 0; i < loop; i++) { node.Set(i+" item", i); } dbTimer.Stop(); Console.WriteLine(dbTimer.ElapsedMilliseconds + " milliseconds to save " + loop + " items"); Console.WriteLine("It is " + dbTimer.Elapsed.Seconds + " seconds"); node.Close(); connection.Close(); } 

Results for a series of tests:

 for (int i = 100000; i <= 1100000; i += 200000) testInserts(i); ConnectionContext.GetConnection().Close(); 

 C:\src\globals\globals\bin\Release>.\globals.exe 430 ms to save 100000 items. 232558 records/second 1223 ms to save 300000 items. 245298 records/second 1958 ms to save 500000 items. 255362 records/second 2754 ms to save 700000 items. 254175 records/second 3559 ms to save 900000 items. 252880 records/second 4436 ms to save 1100000 items. 247971 records/second 

Over 200,000 records per second. All this happens with logging enabled and in transactions. On the other hand, the data are quite mock. On the third, it happens under a virtual machine, which has 5 meters of free disk space. From the fourth, at such speeds, everything very much depends on the specifics - the size of the records and the randomness of the key, the intensity of the transaction flow, the ability to process per packet, the need for transactions, and so on.

First of all, it’s enough - GlobalsDB is installed, the project is working, general speed conclusions are made.
In the future, we will examine such important topics as navigation through global nodes and node traversal operations, work with lists, transactions, counters, and locks. Let us consider separately the interesting question of how GlobalsDB behaves when working with several threads.