Realistic Realm. 1 year experience

Realm has long been known among mobile (and not only) developers. Unfortunately, there are almost no articles on this database in Runet. Let's fix this situation.

A year ago, a line appeared in the build.gradle of our project:

classpath "io.realm:realm-gradle-plugin:0.87.5" 

During this year, the Realm code has grown to version 3.3, acquired many features and fixed a bunch of bugs, implemented a new functionality and got a cloud backend. Let's talk more about Realm in the realities of Andoroid development and discuss the subtle points that arise when using it.

About Us

We are developing an application for communication within the team, something between a telegram and slack. The Android application is written in Kotlin, from the very beginning the offline-first approach was used, i.e. when all the data displayed on the screen is taken from the cache. Having tried several different databases, we stopped at Realm and actively used it throughout the year. This article has grown from an internal document on the use of Realm. The article is not a translation of the documentation and does not pretend to be complete, it is rather a collection of recipes and analysis of subtle points. For full understanding we strongly recommend reading the official documentation . We will tell you about our experience and what bumps stuffed this year. All the code for the article is written in Kotlin, you can find it on Github .

Realm as a startup

If we talk about Realm as a company , then this is a Danish startup founded in 2011. Previously, the project was called tight.db. During its existence, attracted 29M $ investment. The company plans to earn on the basis of the Realm Mobile Platform , the database itself is free and open source. Realm for Android appeared in 2014 and has been constantly evolving since then. Some updates break backward compatibility, but fixes can be done quite easily and quickly.

Realm as a database

Realm is a database for multiple platforms. About themselves they write:

The Realm Mobile Platform is a next-generation data layer for applications. Realm is reactive, concurrent, lightweight, and native objects.

In short, this is a native no-sql database for Android (Java, Kotlin), iOS (Objective-C, Swift), Xamarin (C #) and JavaScript (React Native, Node.js).
There is also a backend that allows you to synchronize data from all sources.
')
Key features include zero copy , MVCC and ACID . There is no built-in mechanism for aging and data cleaning.

Realm has very good documentation and lots of examples on github .
Realm employees periodically monitor StackOverflow , you can also get an issue on github .

Hello world

Hello world for Android looks like this:

Add to build.gradle

 build.gradle (Project level) classpath "io.realm:realm-gradle-plugin:3.3.0" build.gradle (App level) apply plugin: 'realm-android' 

In Application we will configure Realm Configuration

 Realm.init(this) val config = RealmConfiguration.Builder() .build() Realm.setDefaultConfiguration(config) 

And you can start working with the database:

 val realm = Realm.getDefaultInstance() realm.executeTransaction { realm -> val dataObject = realm.createObject(DataObject::class.java) dataObject.name = "A" dataObject.id = 1 } val dataObject = realm.where(DataObject::class.java).equalTo("id", 1).findFirst() dataObject.name // => A realm.executeTransaction { realm -> val dataObjectTransaction = realm.where(DataObject::class.java).equalTo("id", 1).findFirst() dataObjectTransaction.name = "B" } dataObject.name // => B 

Comparison with other databases

On Habré there is an article from April 8, 2016, where 9 ORMs are compared for Android , including Realm. Realm is in the lead, here are the graphics:


On its website Realm provides the following statistics :


There are three main features to consider:
Live Objects - All objects obtained from Realm are, in fact, a proxy to the database. Due to this, zero copy is achieved (objects are not copied from the database)
Transactions - All changes to the associated data objects must be made inside the transaction.
Open \ Close - The need to open / close the database instance

Live objects

All objects from Realm can be received synchronously or asynchronously.

Synchronous read

 fun getFirstObject(realm: Realm, id: Long): DataObject? { return realm.where(DataObject::class.java).equalTo("id", id).findFirst() } 

Call the Realm method and block the stream until we get an object or null. You cannot use objects received in other threads, therefore, for use in the main thread, you need to block ui or use asynchronous requests. Fortunately, Realm provides us with a proxy, and not the object itself, so everything happens fairly quickly. You can work with the object immediately upon receipt.

Asynchronous read

Very unobvious case. What do you think will happen in this code:

 val firstObject = realm.where(DataObject::class.java).findFirstAsync() log(firstObject.id) 

The correct answer is: get the error java.lang.IllegalStateException
In asynchronous reading, although we get the object right away, we cannot work with it until it loads. You need to check this with the isLoaded () function or call the load () blocking function. It looks quite uncomfortable, so it’s better to use rx here. Convert to observable and get the loaded object in OnNext. Asynchronous operations are available only in streams with Looper.

 fun getObjectObservable(realm: Realm, id: Long): Observable<DataObject> { return realm.where(DataObject::class.java).findFirstAsync().asObservable() } 

Main features of Realm objects

• Obtaining objects from the database is very fast, there is no deserialization as such, reading from disk occurs only when accessing a specific field.
• For this purpose, there is a requirement to make all fields private and contact via getters.
• The copyFromRealm () method allows you to get untied, fully assembled objects, just like a normal ORM. True and all Realm chips become unavailable. The input is the depth of deserialization, the default is MAX_INT
• In the debugger all fields will be null. To get any value we need to contact via getter
• All objects are Live, i.e. live. Changes are distributed instantly within a single thread. For more complex cases, see below (multithreading).
• Objects are filtered by fields, and you specify the field names with your hands as a string. For example: .equalTo ("id", 1). This complicates refactoring and leads to errors in the naming of fields for filtering. Unfortunately, Realm does not generate variables with field names, so it is better to hide all samples inside the function:
  
  

   fun findFirstDataObject(id: Long, realm: Realm) : DataObject = realm.where(DataObject::class.java).equalTo("id", id).findFirst() 

  
  or use the cmelchior field name generator
  
• This item has changed right at the time of writing (a vivid example of how the project is developing)
  It was: It is impossible to use DiffUtil when changing an object, it is impossible to understand what fields the object has changed. Those. if you received a notification about the change of the object, it is impossible to understand what has changed. This is due to the fact that both objects (old and new) are live objects and refer to the same data, they will always be equal.
  It became: You can use RealmObjectChangeListener to understand what has changed:
  
  RealmObjectChangeListener

   Person p = realm.where(Person.class).findFirst(); p.addChangeListener(new RealmObjectChangeListener<Person>() { @Override public void onChange(Person person, ObjectChangeSet changeSet) { if (changeSet.isDeleted()) { hide(); // Object was deleted } else { // Use information about which fields changed to only update part of the UI if (changeSet.isFieldChanged("name")) { updateName(person.getName()); } } } }); 

  
  
• Any object is available only as long as the instance realm-a is open from which we received it. You can check isValid methods. When accessing an invalid object, we get an exception
• Objects are available only in the stream in which they were created. You cannot call from another thread, we get an exception

Similarly, lists (RealmResult) of objects (query results) are proxied to Realm, this leads to the following:

• Getting lists is very fast, in fact we only get count. All requests lazy, we can get a large list of complex objects very quickly
• Lists are read-only; any modification methods result in an exception.
  
• Since we can quickly and cheaply receive all the elements, you can forget about the problem of pagination. We always give a complete list of items, when scrolling, we turn to objects, and they quickly come out of the database. If we need to load the data, we start the download, get the data, save it to Realm, again get the full list with the loaded elements and display it
  
• Until recently (prior to version 3.0) there was a problem with redrawing all the elements of the list. If we use the list for the adapter, then changing one element causes a complete redrawing of the entire list. Using DiffUtils and comparing which objects have changed will not work, because These are live objects. In Realm 3.0, OrderedCollectionChangeSet appeared, which tells us DeletionRanges, InsertionRange, ChangeRanges. It has finally become possible to understand what objects have changed and how.
  
  
  CollectionChangeListener Example

   private final OrderedRealmCollectionChangeListener<RealmResults<Person>> changeListener = new OrderedRealmCollectionChangeListener() { @Override public void onChange(RealmResults<Person> collection, OrderedCollectionChangeSet changeSet) { // `null` means the async query returns the first time. if (changeSet == null) { notifyDataSetChanged(); return; } // For deletions, the adapter has to be notified in reverse order. OrderedCollectionChangeSet.Range[] deletions = changeSet.getDeletionRanges(); for (int i = deletions.length - 1; i >= 0; i--) { OrderedCollectionChangeSet.Range range = deletions[i]; notifyItemRangeRemoved(range.startIndex, range.length); } OrderedCollectionChangeSet.Range[] insertions = changeSet.getInsertionRanges(); for (OrderedCollectionChangeSet.Range range : insertions) { notifyItemRangeInserted(range.startIndex, range.length); } OrderedCollectionChangeSet.Range[] modifications = changeSet.getChangeRanges(); for (OrderedCollectionChangeSet.Range range : modifications) { notifyItemRangeChanged(range.startIndex, range.length); } } }; 

  
  
  
  

Transactions

Modifying objects bound to Realm is possible only inside a transaction, when changing outside of a transaction, we get an error. On the one hand, it is not very convenient, on the other hand - it disciplines and does not allow to change objects in any part of the code, only in a certain layer (database). You also need to remember that transactions within another transaction are prohibited.

How not to do:

 val user = database.getUser(1) button.setOnClickListener { user.name = "Test" } 

How can you:

 val user = database.getUser(1) button.setOnClickListener { database.setUserName(user, "Test") } 

Transactions can be made synchronously and asynchronously. Let's take a closer look at each of the options:

Synchronous transactions:

 fun syncTransaction() { Realm.getDefaultInstance().use { it.executeTransaction { val dataObject = DataObject() it.insertOrUpdate(dataObject) } } } 

You can also perform transactions between beginTransaction and commitTransaction, however, it is recommended that you use executeTransaction.

Unfortunately, synchronous transactions do not support onError callback, so error handling is on your conscience. There is an issue to add onError callback since June 2016.

Asynchronous transactions

Asynchronous transactions are triggered by the asyncTransaction method. We give the transaction and callback onSuccess and onError to the input, we get the RealmAsyncTask object at the output, with which we can check the status or cancel the transaction. Asynchronous transactions run only in threads with Looper. An example of an asynchronous transaction:

 Realm.getDefaultInstance().use { it.executeTransactionAsync({ it.insertOrUpdate(DataObject(0)) }, { log("OnSuccess") }, { log("onError") it.printStackTrace() }) } 

A couple of important nuances:

You cannot assign an object not bound to Realm via a setter. You must first put the object in the database, and then attach the linked copy. Example:

 val realm = Realm.getDefaultInstance() val parent = realm.where(Parent::class.java).findFirst() val children = Children() // parent.setChildren(children) <-- Error val childrenRealm = realm.copyToRealmOrUpdate(children) parent.setChildren(childrenRealm) /// Ok 

Many transactions are best merged into one. In Realm there is an internal queue for transactions (100 in size) and if you exceed it, an exception will fall.

All asynchronous transactions run on the same executor.

 // Thread pool for all async operations (Query & transaction) static final RealmThreadPoolExecutor asyncTaskExecutor = RealmThreadPoolExecutor.newDefaultExecutor(); 

If you have many asynchronous operations in a short time, you will get a RejectedExecutionException error. The way out of this situation would be to use a separate stream and start synchronous transactions in it or merge several transactions into one.

Open / close realm

We receive all objects from the database using a specific instance of Realm-a, and we can work with them while this instance is open. As soon as we call realm.close (), any attempt to read an object will result in an exception for us. If we do not close Realm in time, it will lead to memory leaks, since the garbage collector does not know how to work correctly with the resources used by Realm.

The official documentation is recommended to open / close Realm:

• for Activity: onCreate / onDestroy
• for Fragment: onCreateView / onDestroyView

However, if you want to make the logic of working with Realm from Activity \ Fragments into presenters, you will have to use (prokidyd) life-cycle methods.

If you need to somehow change the data or add new ones, the easiest way is to get a new instance, write the data and then close it. In Kotlin, you can use .use ()

 Realm.getDefaultInstance().use { // it = realm instance} 

For reading objects using Rx, you can use “isolated” instances and close them in doOnUnsubscribe (or use Observable.using)

 // Use doOnUnsubscribe val realm = Realm.getDefaultInstance() realm.where(DataObject::class.java).findAllSorted("id").asObservable().doOnUnsubscribe { realm.close() } // Use Observable.using Observable.using(Realm.getDefaultInstance(), realm -> realm.where(DataObject::class.java).equalTo("id", id) .findFirstAsync() .asObservable() .filter(realmObject -> realmObject.isLoaded()) .cast(DataObject::class.java), Realm::close); 

There is also a feature associated with closing Realm in onDestroy \ onDestroyView. Sometimes after closing Realm, FragmentManagerImpl.moveToState → ViewGroup.removeView → ... → RecyclerViewAdapter.getItemCount () is called and the list.size () method is called from an invalid collection. So here you need to check isValid () or untie the adapter from recyclerView

If you are using Kotlin Android Extensions , then you can work with view (from kotlinx.android.synthetic. *) From Fragment only from the onViewCreated () method, it is better to configure all listeners in this method in order not to get NPE.

After analyzing the three most important features, let's go over the less important ones:

Notifications, RxJava

Realm supports notifications about changes in data, both for the object itself and for embedded objects (all linked objects). This is implemented using RealmChangeListener (the object itself comes to us), RealmObjectChangeListener (a modified object comes in and ObjectChangeSet for it, you can understand which fields have changed) or using RxJava (you get an object in onNext, in the case of an asynchronous request you need to check isLoaded (), only in streams with Looper).

RxJava2 hasn’t been delivered yet, the issue has been hanging since September 2016 , when they are implementing it is unknown, use Interop .

Similarly, you can listen to changes in collections or the entire instance Realm. Listening to changes inside transactions is prohibited.

Rx example:

 fun getObjectObservable(realm: Realm, id: Long): Observable<DataObject?> { return realm.where(DataObject::class.java).equalTo("id", id).findFirstAsync() .asObservable<DataObject?>().filter({ it?.isLoaded }).filter { it?.isValid } } 

Multithreading and asynchrony

Realm is an MVCC database. Wikipedia says about MVCC :

“Management of parallel access using multi-version (MVCC - MultiVersion Concurrency Control) is one of the mechanisms for providing parallel access to the database, which is to provide each user with a so-called“ snapshot ”of the database, which has the property that changes made by the user to the database are invisible to others users until the transaction is committed. This method of control allows you to ensure that writing transactions do not block readers, and reading transactions do not block writers. ”

In practice, it looks like this: we can listen to changes to an object or use RxJava to receive changed objects in onNext. If changes occur in stream A, and we work with an object in stream B, then stream B will know about the changes after the Realm instance is closed in stream A. Changes are transmitted through the Looper. If there is no Looper in stream B, then the changes will not come (you can check with isAutoRefresh ()). The way out of this situation is to use the waitForChange () method.

As for asynchronous calls and transactions, it is better not to use them at all. It is more convenient to translate actions to a separate stream and perform synchronous operations there. There are several reasons :

• You can not mix asynchronous transactions and synchronous, if mixed, all transactions will become synchronous
• Asynchronous calls cannot be used in streams without Looper
• For long-term transactions, you need to open a separate instance realm, otherwise the realm can be closed during the transaction and you will get an exception
• All actions in an asynchronous transaction occur on a separate internal executor, as a result you cannot use external realm objects, an executor may overflow, changes to the realm object are not propagated between threads and other inconveniences

Testing

Previously, Realm.java was final and for testing you needed powerMock or other similar tools. At the moment, Realm.java has ceased to be final and you can safely use the usual mockito. Examples of tests in the demo project or on the official repository

One Realm is good, but three is better.

Working with Realm, we always mean the standard realm, but there are also In-Memory Realm and Dynamic Realm.

Standard Realm - can be obtained using Realm.getDefaultInstance () methods or using a specific Realm.getInstance (config) configuration, there can be an unlimited number of configurations, these are essentially separate databases.

In-Memory Realm is a Realm that stores all recorded data in memory without writing it to disk. As soon as we close this instance, all data will be lost. Suitable for short-term data storage.

Dynamic Realm - used mainly during migration, allows working with realm objects without using the generated RealmObject classes, access is carried out by field names.

Inheritance and polymorphism

Realm does not support inheritance. Any realm object must either inherit from RealmObject or implement an interface with a RealmModel marker and be marked with the @RealmClass annotation. It is impossible to inherit from existing Realm objects. It is recommended to use composition instead of inheritance. A very serious problem, the issue has been hanging since January 2015 , but it's still there.

Kotlin

Realm out of the box works with Kotlin .
Data classes do not work, you need to use the usual open class.
Also worth noting is Kotlin-Realm-Extensions , convenient extensions for working with RealmObject.

Realm mobile platform

At first, Realm was presented only to databases for different platforms, now they have rolled out the server for synchronization between all devices. Now the platform consists of:

• Realm Mobile Database - a database for data storage
  
• Realm Object Server - server that is responsible for automatic synchronization and event handling.
• Realm Data Integration API - to connect and synchronize data with existing databases (Oracle, MongoDB, Hadoop, SAP HANA, Postgres and Redis)

Debugging

We have several tools for debugging:

• RealmLog - displays the log, there are different levels of logging
  
• Realm browser - you need to view the database from a computer. It works only on Mac. To view the database on Windows, you can use Stetho Realm
  There are also several Android libraries for easy viewing of data on the device .
  
• WriteCopyTo() — .
• NDK Debugging — Crashlytics NDK Crash Reporting
  

Architecture

Realm is great for MV * architectures when the entire implementation hides behind the database interface. All calls and selections occur in the database module (repository), the Observable with automatically closing realm is sent to the top when unsubscribe. Or we take as input the instance realm and perform all the actions with it. When recording objects, we open the realm, write data and close it, only the object to be saved is input. See both examples on github .
Alas, the use of Realm (without copyFromRealm) imposes serious restrictions on the use of clean architecture. It’s impossible to use different data models for different layers, the whole meaning of live objects and proxy lists disappears. Also, difficulties will arise when creating independent layers and opening / closing Realm, since this operation is tied to the Activity \ Fragment life cycle. A good option would be an isolated layer of receiving data, converting objects and storing them in a database.

Realm is very useful when building offline-first applications, when we receive all the data for display from the database.

useful links

To continue exploring and debriefing, we recommend the following articles:

Three articles from @Zhuinden :
Basics of Realm: A guide to using Realm 1.2.0
How to use it wrong
Realm 1.2.0 + Android Data Binding

Two articles about Realm integration from @Viraj.Tank
Safe-Integration of Realm in Android production code
, with MVP

Multithreading, detailed analysis:
Designing a Database: Realm Threading Deep Dive
Docs - Auto-Refresh
Docs - Threading

A recent article on FairBear:
How to make friends with Realm

Conclusion

Realm harder than it seems at first glance. However, all the disadvantages are more than covered by its power and convenience. Live objects, notifications and Rx, convenient API and many other things make it easy to create applications. From competitors, you can select the requery, ObjectBox and GreenDao. Realm fully reveals itself when building offline-first applications, when we get all the data from the cache and we need complex samples, as well as constant data updates.
You can find all the above code on Github.