All the “joys” of CallKit or how we did the caller ID on iOS 10

2GIS has long wanted to share with users of iPhones their knowledge of the telephone numbers of companies from the directory. The Android platform provided such an opportunity , but under iOS there was no suitable tool for a long time.

In June, we went to WWDC 2016, and at one of the sessions, the guys from Apple let it slip that we could finally do a “gorgeous astonishment” - the number identifier for iOS 10. Our joy knew no bounds, but for the time being: how Apple likes , she provided a feature with a number of restrictions.

Prototype

The first “joy” we encountered is “rich” documentation, namely:
→ CXCallDirectoryExtensionContext
')

@interface CXCallDirectoryExtensionContext : NSExtensionContext @property (nonatomic, weak, nullable) id<CXCallDirectoryExtensionContextDelegate> delegate; - (void)addBlockingEntryWithNextSequentialPhoneNumber:(CXCallDirectoryPhoneNumber)phoneNumber; - (void)addIdentificationEntryWithNextSequentialPhoneNumber:(CXCallDirectoryPhoneNumber)phoneNumber label:(NSString *)label; - (void)completeRequestWithCompletionHandler:(nullable void (^)(BOOL expired))completion; @end 

→ CXCallDirectoryManager

  @interface CXCallDirectoryManager : NSObject @property (readonly, class) CXCallDirectoryManager *sharedInstance; - (void)reloadExtensionWithIdentifier:(NSString *)identifier completionHandler:(nullable void (^)(NSError *_Nullable error))completion; - (void)getEnabledStatusForExtensionWithIdentifier:(NSString *)identifier completionHandler:(void (^)(CXCallDirectoryEnabledStatus enabledStatus, NSError *_Nullable error))completion; @end 

And that's all. Well, it could be worse.

From this we see that the dialer for iOS is an extension of the application that is spinning as a separate process, it can be overloaded and its status can be obtained. It looks like what we need.
In the extension itself, you can add numbers in the form of "phone / name" and add numbers to block.

The first prototype was ready in 30 minutes. One personal phone, wired into an extension, one test phone added to the lock, everything started up the first time, there was no limit to joy. The future looked extremely bright - we already imagined how it all gets into the next release the next day.

Not yet faced with the second "joy": we can not turn on the dialer from the main application. You need to send the user deep into the settings, which is clearly not going to increase the conversion of this feature.

Then they began to add a pack of numbers and the third “joy” turned out: all the numbers needed to be written into the database before they were determined (this is Apple’s famous security - so that we don’t get access to the incoming callerID). And our base is about 4 000 000 numbers with a signature. That is, 140 MB of textual information, or 40 MB, if you shake on the tin, and all this must somehow be delivered to the extension.

Armed with this knowledge, we prepared the data in the form of "phone / name" and began to saw a more real prototype.

Database

At first, they decided to stupidly add all the numbers, and again a surprise - the numbers should be added not anyhow, but in ascending order: 01, 02, 911, etc. Otherwise, the extension falls. In the first beta of 8, xcode extension crashed without any errors at all.

Then it turned out that we are limited to 1,999,999 numbers. Yes, it is 1,999,999, not 2,000,000, which is also not quite equal to our 4,000,000 rooms. At first, they wanted to make three extensions, to fill up to 1,999,999 rooms each and not to blow. Then they decided to divide by regions: Moscow + Peter, the rest of Russia, abroad. But they refused this decision, because it was necessary to come up with a more complex delivery and make the feature even less stable, and the work of several simultaneously working extensions was also not stable. Yes, and force the user to include all three extensions also did not want. In the end, we decided to leave only the numbers of the cities set by the user

At first, we wanted to deliver data via SQLite. We collected a simple database of 100,000 numbers from Novosibirsk, wrote the logic of working with the base, launched a demo project, and ... nothing. There are no errors, everything is OK, but the numbers are not determined.

Having dug this case, we found out that when trying to pull data from SQLite into an ascending order, the database creates a cache of 30 MB and the extension falls out of memory. Digging the Apple forums, they realized that it was better not to get out of 5 MB of RAM. As a result, with a combined base for Moscow, St. Petersburg and a few more cities, it will be necessary to greatly complicate the database queries, build well-optimized memory and fetch speeds, and complicate the testing process. There was no time to do all this, reluctance, and besides, my competences in the assigned technologies were clearly not enough.

We wrote down our stupid, like a log, data format in the form of a bit sequence:

[uint16_t: Block Size] [unsigned long long int: Phone] [String: Name]


Yes, in theory, you need to use the cache, read in 8 KB block and all such cases. But such an algorithm runs through the base of 2,000,000 numbers in 10 seconds in a separate system process, without affecting the main application in any way, and this happens once per update, so they decided not to bother much with optimization.

Hooray! Now we are able to safely parse phone numbers from the database, quietly fitting in the limit of 5 MB of memory. But time passes, and the feature is still not ready.

Data delivery

Next, it was necessary to understand how to deliver this data to extension, that is, in fact, in a separate application. Sew them there will not work, as the user downloads new regions, deletes the old ones, and we also want to update everything, the data becomes outdated, new ones are added, and we are a company about accuracy and relevance.

It turned out that everything was invented for us and there is a wonderful App Groups feature that allows you to fumble data between two applications from one developer.

You can put the file in the main application along the path:

  + (NSString *)extensionDataPath { return [[[NSFileManager defaultManager] containerURLForSecurityApplicationGroupIdentifier:[self extensionGroupName]].path stringByAppendingPathComponent:@"Dialer"]; } 

and in extensions get it through:

  NSString *databasePath = [[DGSCallKitExtensionModel extensionDataPath] stringByAppendingPathComponent:manifest.databaseName]; 

Although there were no delivery problems, and thanks for that.

Then we prepared the data in the right format. If you don’t go deep, a 500 MB file in .tsv format should be scattered across 108 regions, overtaken into a binary format, archived and created Jobs on jenkins, so as not to do all this with your hands and have a ready-made data wrapper for each release without much pain. In short, we also spent a decent amount of time on this - about 90% of the entire development.

The challenge was to deliver this data to the phone (the second 90% of the development).

First, we decided to use the “On demand resources” technology, and at the same time to find out why we need a third, always empty tab in xcode - Resource Tags.



These guys will tell better:

• Documentation ;
• Video from WWDC 2015 ;
• Video from WWDC 2016 ;

In short, Resource Tags for us is just manna from heaven (namely Download Only On Demand). It allows you to tag some application resources with tags, specify their type, and when you fill the application into the store, it will not include them in the binary. Then they can be downloaded using NSBundleResourceRequest and obtained through the [NSBundle mainBundle]. That is, you don’t need to kick other teams at all, figure out how to store them and how to deliver them to the user. And Apple itself stores all the data + provides a very adequate API for obtaining them. What promised quick integration at least here.

But not everything turned out so rosy: in the first release, this technology proved to be extremely lousy, and about 20% of users stupidly could not download. After digging the Apple forums, we found out that we are not the only ones having such a problem, but they haven’t been fixing it for a long time and don’t react to it.

Resource Tags had to cut and deliver data in another way. As a result, we sewed the data into the city update database. Now, along with the upgrade of the city, users receive new database of numbers.

All ahead

At the very least, the dialer got into the AppStore, and then the fourth “joy” was waiting for us.

After a successful installation, we deleted the bases, because why keep what is already in the phone memory. It turned out that everything is not so simple: if the user enters the settings, turns the extensions off and on, then instead of simply turning on, the extensions follow the full update script. My bad, we did not take this into account, and all those who did so lost bases without the possibility of updating them. In the next version, we quickly corrected it and now we leave the data on the phone while it is still relevant.

We constantly receive complaints that the determinant is not working, or questions about how to turn it on. So far, as an intermediate option, we have made a separate item about the determinant in the 2GIS settings.

With iOS 10.3, Apple threw up more problems: if you upgrade to this version, the determinant disappears in the settings until the user either reinstalls the application or rolls in the update. The extension generally behaves unstably. Periodically (for unknown reasons and laws) it turns off or disappears completely from the settings when updating. Sometimes, in the process of updating numbers, the system silently nails extension with error codes:

→ CXErrorCodeCallDirectoryManagerErrorLoadingInterrupted;
→ CXErrorCodeCallDirectoryManagerErrorUnknown.

Back in October, we created a couple of radars at Apple with a request to give us a handle to allow users to turn on the dialer from the application itself, and about bugs from 10.3. The first Apple ticket is ignored since October, and the second is in a sooo long queue.



So in the near future we can hardly make the product better for the user.

How it all works in the end:

1. User downloads city / cities;
2. From the city gets the base numbers in our format;
3. We look at all the databases that are installed by the user (we store them in the general UserDefaults between the extension and the main application);
4. Each base has a hash. If at least one hash does not match or a new one has appeared, we write all new databases to the common storage and mark them as ready to install. This is necessary in case the user did not activate extension, but turned off the application and turn it on later;
5. If the extension is active, reboot it after:
   
   

     [[CXCallDirectoryManager sharedInstance] reloadExtensionWithIdentifier:bundleID completionHandler:^(NSError * _Nullable error) {}]; 

   
   
6. In the extension itself, when it receives:
   
   

     - (void)beginRequestWithExtensionContext:(CXCallDirectoryExtensionContext *)context 

   
   we are looking to see if there are databases ready for installation. If there is, we run through everything and add numbers through:
   
   

     [context addIdentificationEntryWithNextSequentialPhoneNumber:phone label:name]; 

   
   
7. Mark the bases as installed;
8. Repeat the process for each update;

The main problem with the implementation of this feature was the preparation of data and their delivery to the application. If you sew in extensions about 100,000 phones, then the feature can be done in an hour (provided that you have them).

If there is no data in a ready-made format and they need to be delivered and updated in a tricky way, then the integration of this feature will take a lot of time, and because of the complexity of its inclusion, users, unfortunately, will not tell you "many thanks". In the majority of reviews there will be something like “it doesn’t work for me”, “I downloaded the application, but it doesn’t determine anything” and stuff like that.

Instead of conclusion

At the moment, the feature is completed, in the near future there are no plans to finalize it. But I still want to make a selection of the most identifiable numbers - somewhere in the region of 100,000 numbers - and sew them immediately into extension, so that users immediately get the minimum functionality without having to download regions. We also have quite a lot of data on “toxic” numbers: collection agencies, various kinds of polls, various financial pyramids and other objectionable numbers that the Dialer users on Android complained about. We can also deliver them as a separate package to everyone.



In general, I wanted something more stable and more user-friendly, so that even my mother herself could turn it on. In any case, at least 20,000 users turned on extension, and this is a real benefit and a feeling that everything was not in vain.