Offline data handling in a mobile application using Couchbase Lite
Welcome, Habrozhiteli!
We, the Information Technology Center company, are creating infrastructure solutions and high-tech software products that support global state initiatives in the Russian Federation and the countries of the Eurasian Economic Union.
In this post we will share with you our experience in the field of mobile application development focused on working with data.
')
When developing mobile applications focused on working with data, we often encounter the need for full-fledged offline application operation, as well as synchronization of changes in data between the mobile application and the backend, with which desktop or web users also work.
The use of “clouds” to synchronize data is far from always permissible, especially when it comes to high-level customers who do not allow such a solution for security reasons and require the deployment of all in-house system components. In this article I will tell you about our experience in solving this problem with the help of a bundle of a full-fledged Couchbase server database and a “lightweight” mobile database Couchbase Lite.
The Couchbase database is a document distributed NoSQL database that provides high performance by recording data primarily into RAM and then eventual persistence on disk storage. Due to the independence and equality of the nodes with the binding of the document to a specific node, Couchbase provides strong consistency. The Couchbase queries are based on indexed views that implement the MapReduce computation model .
The Couchbase Lite database is a lightweight version of Couchbase designed for use in desktop and mobile applications and supporting synchronization with the Couchbase server base. There are implementations of this database under iOS, Android, Java and .NET, so that it can be used not only in mobile, but also in desktop applications. It is worth mentioning that the implementation of Couchbase Lite for iOS currently has several advantages compared to other platforms - for example, there is the possibility of full-text search, as well as tools for automatic mapping of documents into object models.
Couchbase and Couchbase Lite synchronization uses a replication protocol that is “compatible” with the CouchDB protocol. Compatible in quotes - because the authors do not vouch for exact compatibility due to the lack of a detailed description of the CouchDB protocol - due to the insufficiently detailed documentation, the Couchbase Lite developers had to partly reverse it. The protocol is implemented using Sync Gateway - a replication REST service. All clients wishing to synchronize data with each other should work with the database through this service.
I will not retell the installation process Couchbase , especially since it is different for different platforms. We assume that the database is already installed on localhost. We go to the administration interface ( http: // localhost: 8091 / by default) and create a batch called “demo” - the repository of our documents. To do this, open the Data Buckets tab and click the Create New Data Bucket button.
We introduce the name of the bakery “demo” and limit it by the memory quota of 100 megabytes.
If everything went well, then a demo appears in the list of buckets with a green circle, symbolizing its activity.
By clicking on the Documents button, we’ll make sure that the newly created bakt is still empty.
Installation and launch of the Sync Gateway service are described in the documentation. In this article I will only provide the sync-gateway-config.json configuration file, which allows you to repeat the steps indicated in the article:
By launching Sync Gateway with this configuration file, we get the following log, indicating that the bake with the name “demo” is ready for use as a central repository for data synchronization:
If we now update the list of documents in the batch, we will find there some service documents of the Sync Gateway, the identifiers of which begin with _sync.
The console application code is available on GitHub along with the mobile application code and is more intended to demonstrate and test the interaction. This is a small Java application that works with the local Couchbase Lite database, again on the Java platform. The application can create a document with an attached image in the local database (image attachment) and the add date / time attribute (timestamp_added attribute), as well as start replication of changes to the Couchbase server database.
The mobile application displays thumbnails of thumbnails that were added to the console application and saved to the database. I will describe the process of developing a mobile application in this article in more detail. As a platform, iOS was used as having the best support from the Couchbase Lite API. Swift was used as a language.
First, create a simple project like Single View Application:
To connect to the project library couchbase-lite-ios, we use CocoaPods dependency manager. Installing CocoaPods is described here . Initialize CocoaPods in the project directory:
Add the couchbase-lite-ios dependency to the Podfile file:
Install the necessary libraries:
Now re-open the project as a workspace (file CouchbaseSyncDemo.xcworkspace). Add a bridging file to it, so that Objective-C libraries connected using CocoaPods can be used in Swift classes. To do this, add to the project a header file called CouchbaseSyncDemo-Bridging-Header.h with the following content:
Specify this file in the project's Build Settings:
The automatically created ViewController class will inherit from UICollectionViewController:
Open the Main.storyboard file and replace the default ViewController with the Collection View Controller by dragging it from the object library and redirecting the Storyboard Entry Point to it. As a controller class, in the Custom Class section of the Identity Inspector section, we specify the default ViewController. We will also select the Collection View Cell and in the Attribute Inspector settings we assign the cell reuse identifier to it. The screenshot shows the result.
Create a class CouchbaseService, which will be responsible for working with the local database Couchbase Lite, and implement it as a singleton:
In the class constructor, open a database called demo. Let's start continuous inbound replication (pull) - if the application is started under the emulator, and we deployed the server database on the same machine, then you can use localhost as the replication address. The continuous flag means that “continuous” replication will be used using long polling. Also create an images view to extract a list of images:
The view in Couchbase is the indexed and automatically updated result of the map and (optionally) reduce functions on the entire array of documents stored in the bucket. In this case, the view is set only by its map-function, which for each document returns the time it is added to the database as a key. The results are sorted by key in the views, so the pictures in the results of the queries to this view will always be sorted by the date added. The version parameter is the version of the view; it must be changed each time we change the view code. The version change allows us to inform Couchbase that the code for the map and reduce functions has changed, and the view needs to be rebuilt from scratch.
You can query queries in Couchbase. A special type of requests - the so-called live-requests, the results of which are automatically updated array of documents. Thanks to the KVO mechanism in Objective C and Swift, we can subscribe to changes to this array and update the application interface when new data arrives.
Unfortunately, this way of tracking changes only signals the fact that the query results were updated, and not the specific records added or deleted. Such information would minimize the update of the interface - and there is such a mechanism in Couchbase Lite too. This is a subscription to the kCBLDatabaseChangeNotification event, which signals all new revisions added to the database. But in this example, I decided not to consider it, but to use a simpler mechanism for live requests.
So, let's add a function to CouchbaseService to perform a live request to the images view created earlier:
The Couchbase Lite version for iOS differs from other platforms in that it implements an automatic two-way mapping of documents and object models. Here, the dynamic properties of the Objective C language are used, and with the corrected Swift, it looks like this:
The timestamp_added field is dynamically linked to the field of the same name in the document, and using the attachmentNamed: function, you can get binary data attached to the document. To convert a document to its object model, we can use the ImageModel constructor.
Now it remains only to sign ViewController to update the live request and process this update by redrawing the collection. In the images attribute we will store a list of documents converted to object models.
The controller's functions from the UICollectionViewDataSource protocol are fairly standard and do not require explanations, except that here we use the “cell” reuse identifier that we specified in the storyboard for the cell:
Now check what happened. Run the console application. Using the start command, we will begin replication, and the attach command will load several images, each of which will be created a separate document.
The data is replicated to the mobile device and immediately displayed in the application interface:
In this article, I demonstrated data replication between a server database and a mobile device based on Couchbase and Couchbase Lite to create a mobile application that can fully work offline. In the following articles, I am going to take a closer look at the document revision storage mechanism and the Couchbase Lite replication protocol, how it handles situations of disconnection, leaving the application in the background and other “joys” of mobile development.
The source code for the article on GitHub
Couchbase database
Couchbase Lite Database
Sync Gateway Sync Server
MapReduce Calculation Model
Install Couchbase
Install and Run Sync Gateway
Installing CocoaPods
We, the Information Technology Center company, are creating infrastructure solutions and high-tech software products that support global state initiatives in the Russian Federation and the countries of the Eurasian Economic Union.
In this post we will share with you our experience in the field of mobile application development focused on working with data.
')
Couchbase and Couchbase Lite
When developing mobile applications focused on working with data, we often encounter the need for full-fledged offline application operation, as well as synchronization of changes in data between the mobile application and the backend, with which desktop or web users also work.
The use of “clouds” to synchronize data is far from always permissible, especially when it comes to high-level customers who do not allow such a solution for security reasons and require the deployment of all in-house system components. In this article I will tell you about our experience in solving this problem with the help of a bundle of a full-fledged Couchbase server database and a “lightweight” mobile database Couchbase Lite.
The Couchbase database is a document distributed NoSQL database that provides high performance by recording data primarily into RAM and then eventual persistence on disk storage. Due to the independence and equality of the nodes with the binding of the document to a specific node, Couchbase provides strong consistency. The Couchbase queries are based on indexed views that implement the MapReduce computation model .
The Couchbase Lite database is a lightweight version of Couchbase designed for use in desktop and mobile applications and supporting synchronization with the Couchbase server base. There are implementations of this database under iOS, Android, Java and .NET, so that it can be used not only in mobile, but also in desktop applications. It is worth mentioning that the implementation of Couchbase Lite for iOS currently has several advantages compared to other platforms - for example, there is the possibility of full-text search, as well as tools for automatic mapping of documents into object models.
Couchbase and Couchbase Lite synchronization uses a replication protocol that is “compatible” with the CouchDB protocol. Compatible in quotes - because the authors do not vouch for exact compatibility due to the lack of a detailed description of the CouchDB protocol - due to the insufficiently detailed documentation, the Couchbase Lite developers had to partly reverse it. The protocol is implemented using Sync Gateway - a replication REST service. All clients wishing to synchronize data with each other should work with the database through this service.
Install and configure the server part
Couchbase Setup
I will not retell the installation process Couchbase , especially since it is different for different platforms. We assume that the database is already installed on localhost. We go to the administration interface ( http: // localhost: 8091 / by default) and create a batch called “demo” - the repository of our documents. To do this, open the Data Buckets tab and click the Create New Data Bucket button.
We introduce the name of the bakery “demo” and limit it by the memory quota of 100 megabytes.
If everything went well, then a demo appears in the list of buckets with a green circle, symbolizing its activity.
By clicking on the Documents button, we’ll make sure that the newly created bakt is still empty.
Sync Gateway Setup
Installation and launch of the Sync Gateway service are described in the documentation. In this article I will only provide the sync-gateway-config.json configuration file, which allows you to repeat the steps indicated in the article:
{ "interface":":4984", "adminInterface":"0.0.0.0:4985", "log": ["CRUD+", "REST+", "Changes+", "Attach+"], "databases":{ "demo":{ "bucket":"demo", "server":"http://localhost:8091", "users": { "GUEST": {"disabled": false, "admin_channels": ["*"]} }, "sync":`function(doc) {channel(doc.channels);}` } } } By launching Sync Gateway with this configuration file, we get the following log, indicating that the bake with the name “demo” is ready for use as a central repository for data synchronization:
23:27:02.411961 Enabling logging: [CRUD+ REST+ Changes+ Attach+] 23:27:02.412547 ==== Couchbase Sync Gateway/1.0.3(81;fa9a6e7) ==== 23:27:02.412559 Configured Go to use all 8 CPUs; setenv GOMAXPROCS to override this 23:27:02.412604 Opening db /demo as bucket "demo", pool "default", server <http://localhost:8091> 23:27:02.413160 Opening Couchbase database demo on <http://localhost:8091> 23:27:02.601456 Reset guest user to config 23:27:02.601467 Starting admin server on 0.0.0.0:4985 23:27:02.603461 Changes+: Notifying that "demo" changed (keys="{_sync:user:}") count=2 23:27:02.604248 Starting server on :4984 ... If we now update the list of documents in the batch, we will find there some service documents of the Sync Gateway, the identifiers of which begin with _sync.
Console application
The console application code is available on GitHub along with the mobile application code and is more intended to demonstrate and test the interaction. This is a small Java application that works with the local Couchbase Lite database, again on the Java platform. The application can create a document with an attached image in the local database (image attachment) and the add date / time attribute (timestamp_added attribute), as well as start replication of changes to the Couchbase server database.
Mobile app
The mobile application displays thumbnails of thumbnails that were added to the console application and saved to the database. I will describe the process of developing a mobile application in this article in more detail. As a platform, iOS was used as having the best support from the Couchbase Lite API. Swift was used as a language.
Creating a project and connecting dependencies
First, create a simple project like Single View Application:
To connect to the project library couchbase-lite-ios, we use CocoaPods dependency manager. Installing CocoaPods is described here . Initialize CocoaPods in the project directory:
pod init Add the couchbase-lite-ios dependency to the Podfile file:
target 'CouchbaseSyncDemo' do pod 'couchbase-lite-ios', '~> 1.0' end Install the necessary libraries:
pod install Now re-open the project as a workspace (file CouchbaseSyncDemo.xcworkspace). Add a bridging file to it, so that Objective-C libraries connected using CocoaPods can be used in Swift classes. To do this, add to the project a header file called CouchbaseSyncDemo-Bridging-Header.h with the following content:
#ifndef CouchbaseSyncDemo_CouchbaseSyncDemo_Bridging_Header_h #define CouchbaseSyncDemo_CouchbaseSyncDemo_Bridging_Header_h #import "CouchbaseLite/CouchbaseLite.h" #endif Specify this file in the project's Build Settings:
Interface Stub
The automatically created ViewController class will inherit from UICollectionViewController:
class ViewController: UICollectionViewController { Open the Main.storyboard file and replace the default ViewController with the Collection View Controller by dragging it from the object library and redirecting the Storyboard Entry Point to it. As a controller class, in the Custom Class section of the Identity Inspector section, we specify the default ViewController. We will also select the Collection View Cell and in the Attribute Inspector settings we assign the cell reuse identifier to it. The screenshot shows the result.
Initialization and start of replication
Create a class CouchbaseService, which will be responsible for working with the local database Couchbase Lite, and implement it as a singleton:
private let CouchbaseServiceInstance = CouchbaseService() class CouchbaseService { class var instance: CouchbaseService { return CouchbaseServiceInstance } } In the class constructor, open a database called demo. Let's start continuous inbound replication (pull) - if the application is started under the emulator, and we deployed the server database on the same machine, then you can use localhost as the replication address. The continuous flag means that “continuous” replication will be used using long polling. Also create an images view to extract a list of images:
private let pull: CBLReplication private let database: CBLDatabase private init() { // database = CBLManager.sharedInstance().databaseNamed("demo", error: nil) // let syncGatewayUrl = NSURL(string: "http://localhost:4984/demo/") pull = database.createPullReplication(syncGatewayUrl) pull.continuous = true; pull.start() // database.viewNamed("images").setMapBlock({(doc: [NSObject : AnyObject]!, emit: CBLMapEmitBlock!) -> Void in emit(doc["timestamp_added"], nil) }, version: "1") } Couchbase Lite Views
The view in Couchbase is the indexed and automatically updated result of the map and (optionally) reduce functions on the entire array of documents stored in the bucket. In this case, the view is set only by its map-function, which for each document returns the time it is added to the database as a key. The results are sorted by key in the views, so the pictures in the results of the queries to this view will always be sorted by the date added. The version parameter is the version of the view; it must be changed each time we change the view code. The version change allows us to inform Couchbase that the code for the map and reduce functions has changed, and the view needs to be rebuilt from scratch.
You can query queries in Couchbase. A special type of requests - the so-called live-requests, the results of which are automatically updated array of documents. Thanks to the KVO mechanism in Objective C and Swift, we can subscribe to changes to this array and update the application interface when new data arrives.
Unfortunately, this way of tracking changes only signals the fact that the query results were updated, and not the specific records added or deleted. Such information would minimize the update of the interface - and there is such a mechanism in Couchbase Lite too. This is a subscription to the kCBLDatabaseChangeNotification event, which signals all new revisions added to the database. But in this example, I decided not to consider it, but to use a simpler mechanism for live requests.
Work with data
So, let's add a function to CouchbaseService to perform a live request to the images view created earlier:
func getImagesLiveQuery() -> CBLLiveQuery { return database.viewNamed("images").createQuery().asLiveQuery() } The Couchbase Lite version for iOS differs from other platforms in that it implements an automatic two-way mapping of documents and object models. Here, the dynamic properties of the Objective C language are used, and with the corrected Swift, it looks like this:
@objc class ImageModel: CBLModel { @NSManaged var timestamp_added: NSString var imageInternal: UIImage? var image: UIImage? { if (imageInternal == nil) { imageInternal = UIImage(data: self.attachmentNamed("image").content) } return imageInternal } } The timestamp_added field is dynamically linked to the field of the same name in the document, and using the attachmentNamed: function, you can get binary data attached to the document. To convert a document to its object model, we can use the ImageModel constructor.
Interface binding to data
Now it remains only to sign ViewController to update the live request and process this update by redrawing the collection. In the images attribute we will store a list of documents converted to object models.
private var images: [ImageModel] = [] private var query: CBLLiveQuery? override func viewDidAppear(animated: Bool) { query = CouchbaseService.instance.getImagesLiveQuery() query!.addObserver(self, forKeyPath: "rows", options: nil, context: nil) } override func observeValueForKeyPath(keyPath: String, ofObject object: AnyObject, change: [NSObject : AnyObject], context: UnsafeMutablePointer<Void>) { if object as? NSObject == query { images.removeAll() var rows = query!.rows while let row = rows.nextRow() { images.append(ImageModel(forDocument: row.document)) } collectionView?.reloadData() } } The controller's functions from the UICollectionViewDataSource protocol are fairly standard and do not require explanations, except that here we use the “cell” reuse identifier that we specified in the storyboard for the cell:
override func collectionView(collectionView: UICollectionView, numberOfItemsInSection section: Int) -> Int { return images.count } override func collectionView(collectionView: UICollectionView, cellForItemAtIndexPath indexPath: NSIndexPath) -> UICollectionViewCell { let cell = collectionView.dequeueReusableCellWithReuseIdentifier("cell", forIndexPath: indexPath) as! UICollectionViewCell cell.backgroundView = UIImageView(image:images[indexPath.item].image) return cell } Application launch
Now check what happened. Run the console application. Using the start command, we will begin replication, and the attach command will load several images, each of which will be created a separate document.
start CBL started 15, 2015 11:41:14 PM com.github.oxo42.stateless4j.StateMachine publicFire INFO: Firing START push event: PUSH replication event. Source: com.couchbase.lite.replicator.Replication@144c1e50 Transition: INITIAL -> RUNNING Total changes: 0 Completed changes: 0 15, 2015 11:41:15 PM com.github.oxo42.stateless4j.StateMachine publicFire push event: PUSH replication event. Source: com.couchbase.lite.replicator.Replication@144c1e50 Transition: RUNNING -> IDLE Total changes: 0 Completed changes: 0 INFO: Firing WAITING_FOR_CHANGES attach http://upload.wikimedia.org/wikipedia/commons/4/41/Harry_Whittier_Frees_-_What%27s_Delaying_My_Dinner.jpg Saved image with id = 8e357b3c-1c7f-4432-b91d-321dc1c9fd9d push event: PUSH replication event. Source: com.couchbase.lite.replicator.Replication@144c1e50 Total changes: 1 Completed changes: 0 push event: PUSH replication event. Source: com.couchbase.lite.replicator.Replication@144c1e50 Total changes: 1 Completed changes: 1 The data is replicated to the mobile device and immediately displayed in the application interface:
Conclusion
In this article, I demonstrated data replication between a server database and a mobile device based on Couchbase and Couchbase Lite to create a mobile application that can fully work offline. In the following articles, I am going to take a closer look at the document revision storage mechanism and the Couchbase Lite replication protocol, how it handles situations of disconnection, leaving the application in the background and other “joys” of mobile development.
Links
The source code for the article on GitHub
Couchbase database
Couchbase Lite Database
Sync Gateway Sync Server
MapReduce Calculation Model
Install Couchbase
Install and Run Sync Gateway
Installing CocoaPods
Source: https://habr.com/ru/post/257393/
All Articles