How to make friends with Realm

In this article, we would like to share the experience of using the increasingly popular library for data storage - Realm. Before any project at the beginning of development there is a question what to use for data storage - something checked or to try tools from the category for hipsters .

We are a small startup developing a kids launcher. Although we are a startup and we have a small team, but we pay great attention to the quality of the code. In two years of development, the requirements, functionality, and technologies chosen by us have changed quite a bit. Up to the point that we switched from a fully native application to a hybrid one, based on Cordova. Also, one of these changes was the transition from BaaS from Facebook Parse to Realm. In this article we want to talk about the problems that we encountered when switching to Realm and whether it is worth trying new libraries if we’ve already made friends with the old ones.

Realm is a library designed to facilitate data storage on a device, an analogue of ORM, with its own core and specificity. Currently, its developers have given a solution for cloud storage of the same objects. Everything is built on the ideology of "living objects", in this system you do not need to think about synchronization between threads, devices, users, everything is done for you - any change is automatically applied to all clients of this object. In addition, the developers of this database promise a high sampling rate, thus sampling from the database can be done almost on the UI stream.

What was before Realm

Before that, we used Parse . In it, objects were represented as dictionaries that were serialized as json's and either sent to the cloud or stored on the device. Synchronization rested entirely on the shoulders of the application developer, but the SDK provided quite a few possibilities, besides, it was open-source, which we used sometimes and tried to facilitate the interaction between the server and the device by forking the SDK.

Of the problems we encountered when using Parse, we can note:

1) The difficulty of debugging - anonymous classes from Bolts are very difficult to test and profile, it is impossible to understand what is coming from. In addition, it was these classes that we spent in narrow places up to 40% of the execution time.
2) Quite often, with a large amount of data, the processing speed left much to be desired.
3) A lot of your code for synchronization between repositories.

Tip: initially think about synchronizing your repositories and databases, otherwise the incredible will happen later.

We wanted to make the application as responsive as possible so that the user waited for the minimum time when receiving or changing data. Architecturally it looked like this:

public interface Storage<ModelType, QueryType> { interface Transaction<T> { T transact() throws Exception; } QueryType initQuery(QueryType query); void saveInBackground(ModelType object, @Nullable SaveCallback callback); void saveAllInBackground(List<? extends ModelType> objects, @Nullable SaveCallback callback); void save(ModelType object) throws StorageException; void saveAll(List<? extends ModelType> objects) throws StorageException; void deleteInBackground(ModelType object, @Nullable DeleteCallback callback); void deleteAllInBackground(List<? extends ModelType> object, @Nullable DeleteCallback callback); void deleteAll(List<? extends ModelType> object) throws StorageException; void delete(ModelType object) throws StorageException; void discard() throws StorageException; void discardInBackground(@Nullable DeleteCallback cb); boolean isDiscarded(); } 

 LocalStorage implements Storage<...> CloudStorage implements Storage<...> 

Tip: always hide your storage for the interface, even if you do not see the point. Do not disassemble the Cursor in Fragment'e or View.

The user interacts only with the local repository. All interaction with the server takes place in the background thread and the user does not see any progress bars or other that block progress.

A year ago, Parse announced the closure of its service and we, without hesitation, decided to switch to Realm. We represented the entire transition as " rewrite LocalStorage for the specifics and interface of the Realm SDK ". But everything, as usual, turned out to be a little more difficult ...

Something about the device Realm

The transition itself took quite a bit of time - Realm API is very convenient and requires minimal gestures from the developer. We immediately abandoned the asynchronous methods in our repository, deciding that everything will happen on the calling thread.

Realm and the logic of obtaining objects from it works as follows:

There is a Realm object - this is the entry point into the database, through which both the retrieval of objects and their preservation go. All objects are bound to a Realm instance. Moreover, each Realm is tied to the thread on which you received it. Due to this, synchronization of all database objects is achieved. Everything, once received objects on a specific Realm, is cached in memory and their next receipt is much faster. Inside the Realm objects are synchronized between each other. As the developers of Realm say, these synchronizations are almost instantaneous, but in our opinion, they sometimes occurred with delays.

You can also set different configurations for these instances. Realm developers advise you to use configuration sharing as much as possible. Configuration is an analogue of the database. For example, to store different types of objects in different configurations, or to separate the states of objects and, depending on the state, to move between them. Thus, working with one database will not work with unnecessary objects. In practice, the only thing we could divide into configurations is Realm for logging and for everything else.

Objects Realm'a come in two types. The one that you described in the code and that was generated when building the project. At runtime, they differ in that your object does not look anywhere and it is a simple POJO object, an object of the type of the generated class is a proxy object, the so-called mediator, whose set and get methods look into the database. That is, any change to an object instantly leads to a change in the database.

The specific synchronization at this moment imposes a rather strong limitation - we can only change proxy objects from the stream on which we received them, but more on that below.

In addition, to reset the in-memory cache, you need to call the close method on the Realm object with which you interacted. Moreover, when the instance is received, the object counter on each thread is triggered. That is, the close method works only when we called it as many times as many times called Realm.getInstance () on the same thread.

This logic imposes enormous restrictions on the architecture of the application, which hides the database behind the interface. The documentation also shows a very simple example of using this method.

We nullified their logic with counting out objects by caching out issued instances in the ThreadLocal list. For each stream we issue and hold exactly one realm instance. When you close it, we delete it.

 private final Context mContext; private final ThreadLocal<Realm> mRealm = new ThreadLocal<>(); public void save(RealmObject object) throws StorageException { Realm realm = getRealm(); realm.beginTransaction(); try { realm.copyToRealmOrUpdate(object); realm.commitTransaction(); } catch (Exception e) { realm.cancelTransaction(); throw new StorageException(e); } } private Realm getRealm() { initIfNeeded(mContext); Realm realm = mRealm.get(); if (realm == null) { Log.d(Tags.STORAGE(), "Init Realm on the thread: " + Thread.currentThread().getName()); realm = Realm.getDefaultInstance(); mRealm.set(realm); } return realm; } public void closeConnection() { Realm realm = mRealm.get(); if (realm != null) { Log.d(Tags.STORAGE(), "Clos on finished thread: " + Thread.currentThread().getName()); realm.close(); mRealm.remove(); } } 

It remains to learn how to close it in time ...

Tip: monitor each thread that runs in the application — group them into thread pools. Never do it in the forehead.

 new Thread(new Runnable() {...}).start() 

It helped us a lot. All running operations in other threads, we controlled through thread pools. Thus, we can, without changing the business logic, clean Realm-objects after executing the thread in the pool.

It looks like this:

 class RealmCleanerThreadExecutor(val mStorageManager: StorageManager) { private val TAGS = List("RealmCleanerThreadExecutor") private val mCount: AtomicInteger = new AtomicInteger(0) private val CPU_COUNT = Runtime.getRuntime.availableProcessors private val CORE_POOL_SIZE = CPU_COUNT + 1 val threadFactory = new ThreadFactory() { private val mCount: AtomicInteger = new AtomicInteger(0) def newThread(r: Runnable): Thread = { new Thread(() => { r.run() mStorageManager.getRealm.closeConnection() }, "RealmCleanerThreadExecutor_N" + mCount.incrementAndGet()) } } val executor: ExecutorService = Executors.newFixedThreadPool(CORE_POOL_SIZE, threadFactory) } 

For those using Cordova :

In the heir of CordovaActivity, you need to override the method that creates the CordovaInterfaceImpl, you can pass the Executor we need into the input.

 @Override protected CordovaInterfaceImpl makeCordovaInterface() { return new CordovaInterfaceImpl(this, mRealmCleanThreadExecutor.executor()) { @Override public Object onMessage(String id, Object data) { return MainCordovaActivity.this.onMessage(id, data); } }; } 

For those who use JobManager :

 Context appContext = context.getApplicationContext(); Configuration.Builder builder = new Configuration.Builder(appContext) .threadFactory(realmCleanerThreadExecutor.threadFactory()); mJobManager = new JobManager(builder.build()); 

We don’t close the realm on the UI thread at all; due to this, we use the Realm instance heated up directly on the calling thread.

Thus, we everywhere, after completion of work of a flow, we can reset the cache of Realm objects, without touching the client code.

Tip: transactions that are fairly frequent, but do not have a deadline, we spend on HandlerThread. This reduces the number of threads created.

As an example, we process push notifications on it and we do not clear the Realm attached to it either.

One of the features of the SDK is filtering nested collections using this very SDK. It looks like this

 @Nullable public SessionState getSessionState(String groupUuid) { RealmList<SessionState> sessionsState = getSessionsState(); return sessionsState.where().contains(SessionState.GROUP_UUID,groupUuid).findFirst(); } 

Reduces the amount of our code at times. We immediately took advantage of this opportunity and forgot about looping through and filtering lists by hand. And then we saw the following in their code:

 public RealmQuery<E> where() { if (managedMode) { checkValidView(); return RealmQuery.createQueryFromList(this); } else { throw new UnsupportedOperationException(ONLY_IN_MANAGED_MODE_MESSAGE); } } 

If we create the RealmList ourselves, then managedMode = false and any use of the specific methods of these lists leads to the exception This method is only available in managed mode . As a result, we cannot fully use Realm objects in other threads and, also, we cannot, if it is detail from Realm. In our opinion there are two strategies:

1) Do not transfer objects between threads, work with proxy objects everywhere and be sure that you can use features from the SDK everywhere.

2) Transfer detail objects between threads, but control the code, realizing that there can be both a “real-life object” and a regular java-object not tied to the database.

We have not yet found the perfect solution. Using the filtering capabilities of sdk breaks down the tests, as in the tests we have to completely get wet Realm. You have to create wrappers so that in tests you can replace this behavior.

At the same time, the transfer of detail objects complicates the understanding of the code.

At the moment, we stopped at the option with the constant receipt of objects from Realm instances. In the class field it stores its identifier, in functions we get it from the instance. In this approach, the speed may fall, but you do not need to worry about synchronization.
We would love to listen to the experience of other projects in this place.

As for testing: JUnit and Robolectric are not friendly with Realm, it should be completely locked in the tests. A ticket for this is likely to be forever.

Tip: if you want to easily test and replace the behavior of objects, then ideally make all the creation of new objects in external dependencies. In our example, we had to move methods that create sample objects for Realm objects into the factory.

In fact, this library is noteworthy, it is actively developing, the bugs they learn about are closed in the next couple of days. Yes, and in a large industrial project will not be the perfect technology.

What else would I like to mention:

1) Generated Realm objects hit the DexCount quite painfully, to whom this is important. Each object adds about 20 methods + method to each set \ get method of your source class.
2) Problems with inheritance - multiple inheritance is not supported.
3) Minor difficulties with storing primitives - you have to write wrapper classes (strangely, they have not done this yet).
4) To change the list in a proxy object, it is necessary to open a transaction, that is, it must be placed on the shoulders of the repository and it is rather difficult to do this purely in the code of business logic.
5) It is impossible to override the behavior of the set / get methods since they are generated. We needed this in order to understand whether this object was different from the object on the server. Although this is all solved.

In custody

Initially, this article was planned as a post that any technology can be entered into its architecture, but in the end we got a post with a small part of the problems that we encountered when switching to a completely different ORM in an already finished project. As a result, we are almost satisfied with this transition. The main inconvenience we have is the closing of the instances and working with objects in a multithreaded environment.

Of the advantages compared with other solutions can be identified:

1) Convenience SDK - migration out of the box, synchronization, sampling, sorting, convenient architecture of connections between objects.
2) Speed ​​of work. We didn’t do exact calculations, but we do many operations on the UI stream, considering that there are a lot of animations in the application and other things, the user interface does not lose frames.
3) Ability to subscribe to change objects, as defined, and any lists / selections. Especially handy if you need to update the UI when changing the data set.
4) Very fast help from Realm developers and high-quality documentation.

We very much hope for a lot of comments that we did everything wrong and could have been made easier.

→ You can read