How we do the Alkoscaner

Hi, Habr.

Today I want to tell in some detail about how we made an application consisting not only of a client server, but also connected with the outside world.
')
Application - Alkoskaner , made for iPhone and Android (from 4.0 so far, release for two is preparing for release). Totally free. Brief essence - aggregator of stocks and special offers for the purchase of alcohol. Currently working in Moscow (million-plus cities will be gradually added), supported by 2500 stores, in the database of about 70,000 special offers.

Initially, everything was planned in one form, as a result, the application evolved well to release, but this also gave rise to certain problems.

Let's start with the server

1. Work speed

In general, a very key factor that is particularly relevant on mobile phones (no one will wait long for an answer, especially when you want to drink now). Initially, everything was fine, but after loading the first combat base, it turned out that there was so much data that they would go to the client forever, especially on the radius of Moscow. I had to rewrite the conclusion that caused significant brakes on the server side. Optimized, optimized and optimized. On a dedicated server, everything worked quite quickly and adequately with Hetzner, but then we moved to Selectk ...

2. Server platform

When choosing a hosting provider, we were guided by the proximity of servers to Moscow to reduce ping in the first place and scaling (preferably automatic), secondly, because during peak mailing periods we have hundreds of users at a time, and queries related to geo-location are quite expensive. For some reason, the Scalaxi never answered the call once, chose Selektel (by the way, I somehow don’t even know other Russian clouds, maybe you can tell?). But it didn’t happen - constant independent server drops without load, soaring under 200% of the CPU load, when there is no one in the application, etc. persuaded to return to Hetzner.

3. Summary

So far everything runs on PHP + MySQL (CodeIgniter framework with its own add-on above it). We are planning to bring part of MySQL to Redis, since architecture is constantly evolving and requests are becoming more expensive.

IPhone part

1. YandexMapKit integration.

Download from gitHub library. We run an example - everything works fine, a lot of things can.
We integrate into the project, using a small description on the gita, how to do it. Compiled and earned everything with a half kick. But after a couple of weeks of working with the UC, all of a sudden, the app began to crumble on adding annotations on the map. Debug as the main tool for the search for bugs did not give anything, searches for descriptions of similar situations in the internet also did not succeed. The issue in GitHub is still answered) github.com/yandexmobile/yandexmapkit-ios/issues/95
They decided everything in the end by demolishing the entire library from the project and re-importing it, even the code was not to be rewritten. Everything went on smoothly, but not one hour of time left like water in the sand, and the sediment remained.

2. Additional points and clustering.

Googling a little and not finding a hint of such tasks, without much hope unsubscribed to the same issues on GitHub with a request to suggest how to implement clustering on the UC. github.com/yandexmobile/yandexmapkit-ios/issues/109 Almost immediately, some of the guys who were already using Yak answered that they did this by hand. For what, rolling up our sleeves, we started. Having scribbled a simple, but ornate algorithm, it turned out to achieve the desired result - with the zoom, additional points turned into full-fledged balloons, and at a distance - again turned into a small red mark.

3. Building a route

And again YAK. Want a route from point to point? Api can't do this. Start YandexNavigator from your App and transfer coordinates there, and it will only build a car route. Do you want to use Yandex.Maps to build a pedestrian? Use Android! He is able, iOS-kit - no.

Android

1. Data exchange with the server

No one could have imagined that on a large amount of data the good old JSON turns into a stunned old woman ...
The first implementation of the parser was the native Androids - JSONParser, which did not show itself from the best side even before the actual amount of data was loaded. But when they loaded ... They got from 15-90 seconds of parsing, which was oh, how not good. With this, JSONParser ate not so little memory, sometimes filling up the application. The first way out of this situation was to increase the heap of the application, using the attribute “Large heap”. But this did not solve the problem with speed.
The guru androd was sent on the right path and dropped the link to the repository https://github.com/johnkil/Android-JSONCompare . The graphs and samples showed the potential speed of parsing, but only rock - only hardcore! To begin with, we took up JSON.simple, which, when implemented within objects, was not very different from native, allowing it to be used on the fly without significant changes in the code. The result was depressing, as was to be expected, with a significant load of 1000+ records, simple started to blunt even worse than JSONParser. Further, the choice fell on GSON, which, in turn, on the implementation of the same was very similar to the native, which immediately alerted, given the previous failure. But it turned out not so bad! 90 seconds turned into a maximum of 30. But even that didn’t suit anyone, so there was only one choice - Jackson. The parser is original, in comparison with the previous ones, but, by trial and error, we still forced it to work for ourselves. And they were shocked.
He showed himself even better than in a dream could dream. The previous 30 seconds have sunk into oblivion and surfaced for 3-5 seconds.

2. Oh, these Yandex.Maps

Considering that the application for Russia, where Google and Apple maps are still very sad, the choice of the map fell on Yandex.cards. First, here is the github.com/yandexmobile/yandexmapkit-android repository itself. It began to use maps quite easily, but “the deeper into the forest, the angrier the woodpeckers”. The first brake point was the installation of points on the map, the points, like the baluns, were custom ones, they were installed perfectly, but when there are more than 50+ on the map, significant memory leaks start, due to which the application began to fall systematically. At first, we assumed that the leaks were somewhere in our code. Walking through the code more than once, optimizing the image we use instead of the flag, re-encoding and compressing it, we found out that nothing had changed ... As a result, I had to restrict myself to random percentage of data (every 3 entries) that fell into a certain radius around the center of the map. But by this time, I really wanted to go testing the application in the field, because fiction was becoming far from scientific. Further - calmer, until the moment when two obvious and very noticeable bugs came out: black tiles or tiles that behave very strange ( https://github.com/yandexmobile/yandexmapkit-android/issues/105 ) or spontaneous crashes clicking on the baluns ( https://github.com/yandexmobile/yandexmapkit-android/issues/122 ). At the same time, the entire library has not been updated for nearly a year, considering how many bugs were detected and corrected by both users and developers ... In general, having armed ourselves with brains and a tambourine tambourine, we forced this library to work without crashing (frequent), but the project manager at some point, he began to seriously think about writing his own mapkit for Yandex.Maps.

3. Port 2.2+

For a quick conversion, it was decided to use the Sherlock library. Its first application left a pleasant residue on the soul. Redoing the application from 4.0 to 2.2 took 15 minutes, while all the functionality remained untouched, there were only minor problems with the styles, or rather with the standard android style indents, but this was solved 5 minutes after the search.
And then - more. Tests on all the phones that were on hand (2.2, 2.3) show no problems, tests from friends and acquaintances who are hundreds of kilometers away - crash, crash, crash ...
Here, tests are still ongoing, suggestions for reasons of falls on every HTC Desire are accepted

4. Animation

Animation, as an action in the system, is a very simple process in terms of implementation. But the observations and test showed an interesting picture. When you call an asynchronous request to the server, an animation appears, which, both in appearance and implementation, is very simple. Although the data is loaded asynchronously, the parsing takes place as it is strangely long, with a delay. After receiving the data, we immediately parse in the doInBackground, and then in onPostExecute we display them (as it should). It seems that a simple animation loads the UI thread, which later loads the application. Measurements with animation that the animation slows down parsing data by about 3 seconds (SGS3). On different devices, this delay time is different, which leaves the question open even now.

Data

All this work made sense only if we could fill the app with up-to-date information about discounts in stores. Now I’ll open a small secret: no, we didn’t suck up to some secret database, as many people who used the application think. There are shops from which we receive data directly, but so far there are few of them. During the development of the collection process, we came to the conclusion that a single process gives unstable quality results, and that every trading network needs to be studied, and now we have more than one process, but actually 20+ collection procedures, by the number of networks, and these procedures are important part of the application. Our methods cannot give 100% accuracy, but according to our own tests, we have 80% accuracy. There are still various ridiculous problems, right up to a fight with guards and blocking the assembler in the room, contrary to Art. 29 of the Constitution of the Russian Federation (which emphasizes its authority in the country), but in general we cope with these incidents, inventing various tricks if necessary.

We will be happy to answer questions and clarifications in the comments.