How we launched 2GIS under CarPlay and are still unraveling

Hello! My name is Vanya, I am writing a 2GIS mobile application for iOS. Today there will be a story about how our navigator appeared in CarPlay. I'll tell you how with such documentation and unfinished tools we created a work product and placed it in the AppStore.

A few words about CarPlay

First, some materials to understand some aspects of the work of CarPlay and the reasons why we made certain decisions.

CarPlay is not an OS inside another OS, as many articles write about it. If roughly, then CarPlay is a protocol for working with an external display of the screen of the head unit; sound from car speakers; touch screens, touch panels, washers and other input devices.

That is, the entire executable code is located directly in the main application (not even in a separate extension!) This is very cool: to get new features, you do not need to update the radio or even the car, you just need to update iOS.

At WWDC 2018 Keynote, we were presented with the possibility of creating navigation applications for CarPlay, which made us very happy. Immediately after the presentation, we sent a request for authorization of development for CarPlay. In the request it was necessary to show that our application can navigate.

While we were waiting for a response from Apple, a lecture was published in which, using the example of the CountryRoads sample application, they talked about working with CarPlay.framework. The lecture did not tell about the pitfalls and intricacies when working with CarPlay, but they mentioned that after connecting to the CarPlay radio tape recorder, the application will run in background mode.

The first stick in the wheels

The work of the application in the background has disappointed us. There were two reasons for this:

1. We do not work in the background. Once left this limitation for technical reasons and for the sake of energy saving.
2. Our map is written in OpenGL (yes, deprecated, yes, not Metal, we all know that), and OpenGL does not work in the background state. At best, you get a black view, and at worst, crash.

With the work in the background, it was still possible to cope, but definitely it was necessary to solve something with the map. That's when the idea came to make it through the standard MKMapView. Until you started throwing stones at us for the idea of ​​using standard Apple maps, I will explain: we were going to use MKMapView, but not Apple maps.

The fact is that MKMapView can load third-party tiles. Tiles are special rectangular containers for textures. We just had a servachok who knows how to give tiles. On GitHub there is a code with implementation.

Apple's answer

We received a response from Apple, in which, in addition to the development permission, we also received the documentation for the elect, the CountryRoads sample code (it was shown at the WWDC lecture) and, most importantly, the private capability-key com.apple.developer.carplay-maps . This key is set in the entitlements file with a value of YES, so that the system understands that you can handle events from CarPlay when you start your application.

Without waiting for the sprint with the development of the story, I climbed to download Xcode Beta. The first attempt to collect 2GIS was a failure. But the draft sample application CoutryRoads managed to build a simulator.

Before each opening of the CarPlay window, the latter had to be customized through this window:

To do this, it was necessary to write a line in the terminal: defaults write com.apple.iphonesimulator CarPlayExtraOptions -bool YES

For some reason, this did not work - I had to run almost on the smallest simulator with a resolution of 800 × 480 points and a scale × 2. At the moment, this setting works and helps a lot.

Having created my sample project and armed myself with documentation, I began to figure out what was happening.
The first thing I realized: navigation applications for CarPlay consist of base view and templates layers.

Base view is your map. On this layer there should be only a map, no other views and controls.

Templates is an almost non-customizable mandatory set of UI elements for displaying routes, maneuvers, lists of all sorts, and so on.

Beta Development

Let's move on to writing code. The first thing to do is to implement a couple of required CPApplicationDelegate methods in the ApplicationDelegate file.

 func application( _ application: UIApplication, didConnectCarInterfaceController controller: CPInterfaceController, to window: CPWindow ) {} func application( _ application: UIApplication, didDisconnectCarInterfaceController controller: CPInterfaceController, from window: CPWindow ) {} 

Let's look at the signature:

With UIApplication, everything is clear.
CPWindow - the heir of UIWindow, the window for the external display of the head unit of the radio.
CPInterfaceController is something like an analogue of a UINavigationController, only from CarPlay.framework.

We now turn directly to the implementation of the method.

 func application( _ application: UIApplication, didConnectCarInterfaceController controller: CPInterfaceController, to window: CPWindow ) { let carMapViewController = CarMapViewController( interfaceController: controller ) let navigationController = UINavigationController( rootViewController: carMapViewController ) window.rootViewController = navigationController } 

In didConnect, you need to write code similar to the one we used to see in didFinishLaunching. CarMapViewController is a base view (controller actually, but ok), as per documentation.

Here is a picture I finally got:

Somewhere around this time it dawned on me that in the new Xcode the new build system is enabled by default and most likely because of this 2GIS is not going to.

I opened Xcode, put legacy (or rather stable, let's call things by their proper names) build system, and my theory was confirmed: 2GIS gathered.

Having set the same capability-key, I launched 2GIS under CarPlay and did not see any logs about switching the application to background mode. It became even more incomprehensible, because Apple engineers from the scene said about the background-mode, but, on the other hand, we were promised a contentView from UIAlertView, and as a result, UIAlertView became deprecated.

Deciding that it should be so, I did not bother with MKMapView. She would have deprived us of offline and made us re-write route drawing.

Single card problem

I didn’t have time to rejoice at the news that there would be our map in CarPlay, as the following problem arose before me: because of the technical features, there could be only one card.
A quick solution to this problem was, though not very elegant.

Usually at the time of using 2GIS on CarPlay, the phone is locked and lies somewhere on the shelf. This means that the map is not very necessary on the phone at that moment (it doesn’t hurt to search, of course). Therefore, we decided to take the card from the main application when connecting the phone to CarPlay and display it on the CarPlay radio screen. And when disconnected, respectively, return back to the application on the phone.

Yes, the solution is this itself, but it is fast, it still works and did not have to kick a couple of other teams to rivet the MVP.

Controls on the map

So, we got our map on the screen of the radio. Now it was necessary to do the first and obvious things for any map: controls of the zoom, current location and movement of the map.

Let's start with the zoom and the current location, because these controls are on the map itself and these are not ordinary UIControl. As I wrote above, on the base view there is only a map.

In order to put these controls on the card, I had to climb into the documentation and sample application again. There I read about the first template - CPMapTemplate.

CPMapTemplate - a transparent template for displaying some controls on the map and an analog navigationBar. It is created and exhibited as follows:

 let mapTemplate = CPMapTemplate() self.interfaceController.setRootTemplate(mapTemplate, animated: false) 

Next, you need to create these controls and put them on the map.

 let zoomInButton = CPMapButton(…) let zoomOutButton = CPMapButton(…) let myLocationButton = CPMapButton(…) self.mapTemplate.mapButtons = [ zoomInButton, zoomOutButton, myLocationButton ] 

But the mapButtons array turned out to be a joke, because how many items do you need for it, it will take only the first three elements and display them on the screen. You will not get any errors in the log or asserts.

Then I climbed up to look at how to make the map move, and I found this in the documentation:

 Navigation apps are designed to work with a variety of car input devices, and CarPlay does not support direct user interaction in the base view (apps do not directly receive tap or drag events). 

Strange, I thought, and it was useful to watch how it was done in the CountryRoads sample application. The answer is through this interface:

It is not very convenient, but there is no other way, the documentation will not lie, right?

Since the location for the controls on the map was over, it was necessary to make a button for transferring the map to the “dragging” mode in this analog navigationBar.

 let panButton = CPBarButton(…) self.mapTemplate.leadingNavigationBarButtons = [panButton] self.mapTemplate.trailingNavigationBarButtons = [] 

But here the leadingNavigationBarButtons and trailingNavigationBarButtons arrays were also not without a joke: how many elements in them or shove, they take only the first two. Also without errors in the log and asserts.

And to activate and deactivate the mode of dragging the card, you must write:

 self.mapTemplate.showPanningInterface(animated: true) self.mapTemplate.dismissPanningInterface(animated: true) 

Building and displaying routes on the map

Next, I proceeded to reuse our already existing API to build routes.

Just for the demo and understanding what and how to do, I decided to take two points and build a route between them. Point A was the user's location, and Point B was our head office in Novosibirsk.

 let choice0 = CPRouteChoice( summaryVariants: ["46 "], additionalInformationVariants: ["  "], selectionSummaryVariants: ["1  7 "] ) let choice1 = CPRouteChoice( summaryVariants: ["46 "], additionalInformationVariants: ["  "], selectionSummaryVariants: [“1  11 "] ) let startItem = MKMapItem(…) let endItem = MKMapItem(…) endItem.name = ",  ” let trip = CPTrip( origin: startItem, destination: endItem, routeChoices: [choice0, choice1] ) let tripPreviewTextConfiguration = CPTripPreviewTextConfiguration( startButtonTitle: " ”, additionalRoutesButtonTitle: “”, overviewButtonTitle: "" ) self.mapTemplate.showTripPreviews( [trip], textConfiguration: tripPreviewTextConfiguration ) 

On the screen we got a control with a description of the route:

Navigation mode

Routes are good, but the main feature of the navigator is still in navigation. To make it appear, you need to write the following:

 func mapTemplate( _ mapTemplate: CPMapTemplate, startedTrip trip: CPTrip, using routeChoice: CPRouteChoice ) { self.navigationSession = self.mapTemplate.startNavigationSession(for: trip) } 

CPNavigationSession is a class with which you can display some UI elements that are needed only in navigation mode.

To display the maneuver, you must:

 let maneuver = CPManeuver() maneuver.symbolSet = CPImageSet( lightContentImage: icon, darkContentImage: darkIcon ) maneuver.instructionVariants = [". "] maneuver.initialTravelEstimates = CPTravelEstimates(…) self.navigationSession?.upcomingManeuvers = [maneuver] 

After that, on the screen of the radio, we get this:

To update the footage to the maneuver, you must:

 let estimates = CPTravelEstimates(…) self.navigationSession?.updateEstimates(estimates, for: maneuver) 

It just works!

When the main functionality for the navigator was ready, I decided to show this craft on the inside presentation. The presentation was a success: everyone was excited about the idea of ​​finishing, testing and launching the navigator as soon as possible.

First of all, we ordered a real head unit with CarPlay support. And here, as they say, the heat has gone.

Provision Profiles

Because of the addition of a new capability-key, profiles need to be regenerated. In normal development, we do not think about it, because Xcode will do everything by itself. But not in the case of a private key.

 Code Signing Error: Automatic signing is unable to resolve an issue with the "v4ios" target's entitlements. Automatic signing can't add the com.apple.developer.carplay-maps entitlement to your provisioning profile. Switch to manual signing and resolve the issue by downloading a matching provisioning profile from the developer website. 

It also broke our CI, as for the local distribution of application versions we use an enterprise account, to which we did not request permission to develop an application for CarPlay. But that's another story.

Debugging

Connect to CarPlay via Bluetooth or Lightning. Practice shows that the second method is much more popular. Our Bluetooth radio tape recorder did not know how, so during development we had to use Wi-Fi debug. If you have tried it on projects harder than hello world, then you know what kind of hell it is.

And then it became very interesting to me why Apple did not do any DevKit (so that Apple-way, it just works and that's all). Without it, collecting a test stand was not very convenient. So far, once in a couple of weeks, something falls off - you have to remember from the pics what to stick with. It is good that the administrator during the assembly of this stand told what and why.

The best framework we ever made

In the end, when everything was assembled on a real device, it became clear that the “2GIS under CarPlay” feature would definitely be. It is time to do beauty.

Viewport issues

It was necessary to set up the map viewport in order to draw routes in the area without extra controls, and not just in the middle. In short, to make it look wrong:

And so:

I expected to get some layoutguide with the current visible area. So that it takes into account both the navigationBar, the view with the route, and the controls on the map. In fact, I got nothing. It is still not clear how to set up a viewport, so we have a hardcode in our code like:

 let routeControlsWidth = self.view.frame.width * 0.48 let zoomControlWidth = self.view.frame.width * 0.15 

Building a passage not only between two points

In the first release, we decided to take our rubricator made via CPGridTemplate:

Favorites and Home / Work via CPListTemplate.

And keyboard search through CPSearchTemplate:

I will not show the code about templates, since it is simple and the documentation about it is well written (at least about something).

 self.interfaceController.pushTemplate(listTemplate, animated: true) self.interfaceController.presentTemplate(alertTemplate, animated: true) 

 self.interfaceController.showTemplate(listTemplate, animated: true) 

 Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'Unsupported object <YourAwesomeGridTemplate: 0x60000060dce0> <identifier: 6CAC7E3B-FE70-43FC-A8B1-8FC39334A61D, userInfo: (null)> passed to pushTemplate:animated:. Allowed classes: {( CPListTemplate, CPGridTemplate, CPSearchTemplate, CPMapTemplate )}' 

Testing and bugs

Testing involved artemenko-aa . One of the first bugs that he found, we still can not fix it.

The fact is that when you disconnect the phone from the CarPlay radio tape recorder, we are sporadically beat Watchdog - without explaining the reason. Even syslogs opened, nothing is clear. So if you have an idea how to fix or understand the cause, then we would like to comment.

The next bug was in the same place, but with a special behavior. I wrote above that the CPApplicationDelegate method didDisconnect is called when the phone is disconnected from CarPlay. And in this method we return the map from the screen of the radio tape recorder back to the main application. Imagine how many problems we would have had if this method had not been called at least once out of five.

It became clear that this is a problem of iOS, and not specifically our application, since the entire system believed that it was connected to CarPlay.

I even wrote it as a radar (like all the other bugs). I was asked to drop logs with such a profile, but I could not respond to support for some time, so they closed the radar.

Once Apple did not plan to do anything, the problem had to be circumvented on its own, as it was reproduced quite often.

And then I remembered that the lion's share of connections to CarPlay goes through Lightning. This means that the phone is charging at the time of connection, and at the time of shutdown it stops charging. And if so, then you can subscribe to the state of the battery and find out exactly when the phone stopped charging and disconnected from CarPlay.

The scheme is tiny, but we had no choice. We went this way, and everything worked!

Fortunately, this crutch from the code has long been removed: Apple developers have fixed everything in one of the iOS releases.

The story of two rejects

The first reject was associated with the metadata. The text of the redesign said that our description (not release notes) did not say that we support CarPlay. As you can guess, neither in the review guidelines, nor the same Google Maps had such a thing. We did not argue (because it is usually longer than editing the metadata), copied the line from the Release Notes in the Description and waited for a new review.

The second rejection was due to the list of cities. 2GIS has a very cool feature - full offline mode. This feature shot us in the leg.

When connecting the application without a set city to CarPlay, we do not show the map, because there is nothing to show. For this, we and zredzhektili. The solution was simple: an alert without buttons, which says that you need to download the city.

What can not talk

Moving the map with gestures

Around the same time, the navigator came out under the CarPlay from Google Maps - and there it was possible to move the map with gestures on the screen. Private APIs, I thought, that's obvious! The guys from Google just came from a nearby building and said they need it. After all, the documentation reads as follows:

 Navigation apps are designed to work with a variety of car input devices, and CarPlay does not support direct user interaction in the base view (apps do not directly receive tap or drag events). 

However, I decided to make sure and went to Google, although it was almost meaningless, because there were no technical articles about CarPlay Navigation Apps. However, I managed to find something useful and, EXTREMELY, on the Apple site .

In the guidelines, I found a video that says the documentation is blatantly lying. The video shows how the map can still be dragged with gestures. I realized that I did not understand anything, and the only thing left for me was to open CarPlay.framework and revise all .h files.

And lo and behold! I find in CPMapTemplate his delegate CPMapTemplateDelegate, in which there are 3 methods that seem to shout that if you implement them, you can get control of the map gestures.

I implemented them and launched the application on the simulator - nothing worked. Not having time to get upset, I realized that the simulator can be of the same quality as the documentation, and assembled on the device. Everything started, fortunately there was no limit!

Fun fact: CarPlay-radio tape recorder needs a quarter of the screen to understand that a pan-gesture has begun. I want to note that UIPanGestureRecognizer needs only 10 points.

The dissimilarity of the UI on different tape recorders

We received a message of support: the user gets only one sadgest in the search, although it could have been more. Strange, I thought, because on all screens fit only one line. Request a screenshot:

And this is completely different from the UI CPSearchTemplate, which I showed above. And this should be taken into account when developing, although it is impossible to understand how many cells in the plate below can fit into the screen.

Speed ​​Limit Control

We looked at the statistics and realized that the navigator for CarPlay is used and it is necessary to bring it at least to the level of the navigator in the main application. First decided to add a speed limit control. No problem, of course, not done.

Question number one: where to post?

Rummaging through the .h files in the CPWindow again, I found a curious layoutGuide:
var mapButtonSafeAreaLayoutGuide: UILayoutGuide

And it turned out to be the right thing. Our control fit in perfectly there:

Question number two: is this, in general, legal?

The fact is that the technical control is on the base view. And according to the documentation, the base view cannot contain anything but a map:

 The base view is where the map is drawn. The base view must be used exclusively to draw a map, and may not be used to display other UI elements. Instead, navigation apps overlay UI elements such as the navigation bar and map buttons using the provided templates. 

But the reviewers missed us in the AppStore, which means that the controls that relate to navigation can still be embedded.

Voice search

In an amicable way, this feature had to be done first of all, but we have accumulated a few technical debt tasks that prevented the voice search for CarPlay from being implemented. And this task was not as simple as it seemed.

Problem one: animation. The fact is that in CPVoiceControlTemplate there is no possibility to make standard animations. Animation for speech recognition and search had to be collected frame by frame from the pictures and indicate how much they go on time.

 for i in 1...12 { if let image = UIImage(named: "carplay_searching_\(i)") { images.append(image) } } let image = UIImage.animatedImage(with: images, duration: 0.96) 

It looks like you can guess, not really, but you do not want to inflate the size of the application.

Problem two: access. Alerts on access to the microphone and speech recognition appear on the phone display. I had to write on the display of the radio tape recorder that the user needed to take the phone in hand, give permission, and only then use the navigator on the radio tape recorder. Very comfortably!

Right-hand drive cars.

We were sent a screenshot in which the UI of the entire application was turned over!

And, of course, the viewport of the card remained the same as we used it, because no one expected that there is a separate setting for right-hand drive cars. I didn’t find how to get around this “correctly”, but noticed that since our control of the speed limit lies in the layoutGuide for the map controls, it moved to the left side.

Ultrafix was not long in coming. They did it rough, but it works.

 let isLeftWheelCar = self.speedControlViewController.view.frame.origin.x > self.view.frame.size.width / 2.0 

, , .

. CarPlay, , . , , Apple .