Use Apache Ignite at home

Continuing the theme “accessible language about Ignite / GridGain ”, started in the previous post ( Why Apache Ignite is needed ), let's look at examples of using the product “for mere mortals”.

Terabytes of data, clusters of hundreds of machines, big data , high load, machine learning , microservices and other scary words - all this is available Ignite. But this does not mean that it is not suitable for smaller goals.

Today we will look at how Ignite can easily store any of your objects, share them over the network, and provide .NET and Java interaction.

We will talk more about the .NET API, but some things are applicable in Java.

Embeddable database

Ignite does not require installation and configuration, it can run directly in your process, and from version 2.1 it can store data on disk . Add to this a fast and compact built-in serialization and get the easiest way to save any of your objects to disk.

var cfg = new IgniteConfiguration { //      PersistentStoreConfiguration = new PersistentStoreConfiguration() }; var ignite = Ignition.Start(cfg); ignite.SetActive(true); //      persistence var cache = ignite.GetOrCreateCache<int, MyClass>("foo"); cache[cache.GetSize() + 1] = new MyClass(); 

That's all the code! We run several times and make sure that the data is not lost.

At the same time to MyClass no requirements! This can be anything, any nesting, including delegates, anonymous types and methods , expression trees , dynamic objects, and so on. Can your serializer do this? I do not think. The only exception is that you cannot do IntPtr , but this is for your own good. Serializing pointers is a bad idea.

Moreover, serialized objects are not a “black box”, IBinaryObject allows IBinaryObject to selectively receive and modify individual fields of objects.

 using (var ignite = Ignition.Start()) { var cache = ignite.CreateCache<int, object>("c"); //  Type,     ! cache[1] = typeof(int); Console.WriteLine(ReferenceEquals(typeof(int), cache[1])); // true //  . var greeting = "Hi!"; cache[2] = (Action) (() => { Console.WriteLine(greeting); }); ((Action) cache[2])(); // , : Hi! //    ! //    Binary -    . ICache<int, IBinaryObject> binCache = cache.WithKeepBinary<int, IBinaryObject>(); //     BinaryObject. IBinaryObject binDelegate = binCache[2]; //   target0 -  ,   . IBinaryObject target = binDelegate.GetField<IBinaryObject>("target0"); //     greeting. target = target.ToBuilder().SetField("greeting", "Woot!").Build(); //       . binCache[2] = binDelegate.ToBuilder().SetField("target0", target).Build(); //     ,  ,  . ((Action)cache[2])(); // Woot! //  . cache[3] = new { Foo = "foo", Bar = 42 }; //   . Console.WriteLine(binCache[3]); // ...[<Bar>i__Field=42, <Foo>i__Field=foo] //  . dynamic dynObj = new ExpandoObject(); dynObj.Baz = "baz"; dynObj.Qux = 1.28; cache[4] = dynObj; Console.WriteLine(binCache[4]); // _keys=[Baz, Qux], _dataArray=[baz, 1.28, ] } 

Of course, you can work with all this data both in key-value mode ( ICache.Put , ICache.Get ), and through SQL , LINQ , full-text search .

I note that, in addition to the embedded mode, you can work with Ignite via ODBC and JDBC .

Interprocess communication

Google Chrome browser uses a separate process for each tab. Implementing this approach with Ignite is very simple: data can be easily and transparently exchanged through a cache, synchronized code execution through distributed data structures, and exchanged messages via messaging .

Processes can be equal. For example, the navigation history can be stored in the Replicated cache, so that each tab, when clicking on a link, updates the history, can read it, and the fall of one of the processes does not affect the integrity of the data. In the code, it all looks just like working with a collection.

 var history = ignite.GetCache<Guid, HistoryItem>("history"); history.Put(Guid.NewGuid(), new HistoryItem(url, DateTime.UtcNow)); //   10   history.AsCacheQueryable() .Select(x => x.Value) .OrderByDescending(x => x.Date) .Take(10); 

Ignite provides thread-safe data access across the entire cluster.

Cross-platform interaction

Ignite has full-fledged APIs in Java, .NET, C ++. You can run on Windows, Linux, Mac. For example, part of your application can be written in Java and run on Linux, the other part - on .NET and under Windows.

The serialization protocol is universal, objects written on one platform can be read on another. The data structures mentioned above are also cross-platform.

Moreover, it is possible to transparently call from the .NET services written in Java,:

 public class MyJavaService implements Service { public String sayHello(String x) { return "Hello, " + x; } } ignite.services().deployClusterSingleton("myJavaSvc", new MyJavaService()); 

 interface IJavaService //     { string sayHello(string str); //  ,     } var prx = ignite.GetServices().GetServiceProxy<IJavaService>("myJavaSvc"); string result = prx.sayHello("Vasya"); 

As elsewhere, instead of strings, you can transfer any objects.

A detailed guide to building a cross-platform Ignite cluster, using the example of a simple chat, is here (in English).

Conclusion

Ignite makes work with any data fast and enjoyable: serialize, save to memory or to disk, transfer over the network, ensure consistency and thread safety. Even the smallest projects can benefit from the product without worrying about a possible increase in load in the future.