As I tried to fix the search on the cards for drivers. Part 3 (final)
So, this is the third part of my attempt to rethink the usual map search. The first part is here , and the second one is more technical, but you can run for a better understanding. In short, it sounds like this: I am tired of poking around at the cards while driving, trying to find the nearest gas station among small icons and advertisements. Instead, I would just like to go looking at the application screen. So that it sorts the nearest places according to driving time, shows them in a list, explains which ones are on the way and what kind of traffic to them. Such is the idea.
Actually, by version 3.0 of the application, it finally turned out to implement all the basic functions that I wanted. After the last article in this series, a number of people downloaded it, and even wrote reviews - thanks, I listened to everyone. He worked on the new version of the month two intensively, not to list all minor changes - in fact, this is 80% new application. With a radically improved interface, 2 times faster and much more stable. Again, I invite sympathizers to evaluate and scold. And under the cut again technical moments.
')
Here are links to iPhone and Android
One of the key complaints about the previous version of the application was the incorrect timing - I implemented it on my own through the Open Source Routing Machine, and he considered the net time of the road. During periods of minimal traffic (for example, at night), my numbers coincided with the fact that they issued the same Google maps, but in most cases the assessment was at least extremely optimistic. This leveled out the very meaning of the application itself, and it was necessary to think of something.
There are two ways to solve this problem: turn to third-party api or try to extort traffic weights from somewhere and import to yourself. I really didn’t want to depend on someone, so I spent some time searching for a second solution. The results were disappointing: I have nowhere found a base of traffic weights with global coverage that would be compatible with OpenStreetMap. There are some open bases on pieces of Europe and America, which, in theory, can be stitched with OSM through tambourine jumps - but in the end, on reflection, I decided not to get involved. Of course, the ability to host the navigation with traffic was seducing, but was scared away by incomplete coverage, difficulties with integration, inaccuracies and the very fact that traffic was cached, and not in real time. In short, again spend a lot of time and get a miser in the end.
Realizing that nowadays without the big uncle's api, I began to look for an adequate and cheap uncle. Stumbling, I chose HERE services - these are former Nokia cards, then Microsoft took them to me, leaving them as a separate unit. As I understand it, they now mainly work with corporate clients (for example, on logistics) and have a fairly reasonable and clean api. And most importantly, they have global realtime traffic and fairly generous quotas. Not an obvious choice, but I decided to try.
The integration ended up being pretty simple. A significant role was played here by the general flexibility of the architecture, which I made. If desired, it is now easy to integrate Google into it, even though Yandex traffic jams. I made the traffic from HERE disconnected with the fallback to my old navigation. Finally, by comparing the values ​​without traffic (mine) and with traffic, you can derive a general estimate — empty road, light traffic, medium traffic, and so on.
Maps were the second major claim to the app. I hit an incredible amount of time on drawing my own route thumbnails and the general map, and it even worked - but the generation of raster tiles with a modern design immensely dragged the server. If in testing mode, pictures appeared within half a second, then with a real audience (even a small one), people could wait up to five or even ten (!!!!) seconds. Especially if the piece of the map was large - then my code was chewed and rendered a bunch of vector data. In addition, this process is not particularly parallelized: there were bottlenecks, and all the queues still quickly clogged. In general, sadness.
What to do, it was obvious - throw out thumbnails of the cards to hell. I struggled with myself for a long time (I did all of them for so long), but in the end I made a strong decision and, in general, did the right thing. When the huge and often not loaded pictures left, it became calmer and a lot of space was freed up on the screen. Even filling it with new data and adding inscriptions, I was able to squeeze twice more results into the screen than before.
But with the general map, I acted differently. No one (including myself) could understand why, in fact, it is needed. It was self-made, clumsy, loaded forever and in general was originally created for debug. However, for some reason, the visual image of the isochrone continued to warm my soul and as a result it remained to occupy a place in the interface. Then I decided: let's try to make a full-fledged alternative viewing mode out of this meaningless picture. If someone doesn’t like the list and he reads the map more easily - why not? Especially since I had a couple of ideas that could not be found anywhere else.
Reluctantly, I threw out my picture (the server groaning from the load was grateful for that). Instead, I embedded a full-fledged map through the flutter_map plugin - I took the background tiles from Mapbox - and began to show my position and results points around it. Almost immediately, the need arose to cluster these points, and I quickly sketched a distance-based clustering code. It is rather primitive, but covers 90% of cases. Under all this, I again planted my favorite green blob isochron. Finally, the map legend also became interactive: tap by the number of results focuses the map on points, and tap on time on the isochron. Pretty comfortable.
One of the ideas in which I felt value, but could not articulate it in any way, was the display of the current route and vehicle motion vector. I tried to stick it in the route cards in different ways, but nowhere did it look organically and in place. And finally, almost in despair, I realized: the new card mode is perfect for this chip. Because in the list mode I write directly with the text, along the way some place or not - but on the map it is always incomprehensible. Even with Google or Apple, you observe the constantly spinning sector of the compass and for a long time you do not understand in which direction you are going.
Inspired, I sat down to work. I had to refactor a bunch of code along the way, but after a couple of days the logic was ready. I decided to update the position not every 200 meters, as results, but more often - after 10 meters. Each update I recalculate the motion vector, and so it turns out very accurate, because it does not depend on the accelerometer, but on the previous position. The route (that is, the array of the history of our coordinates) I draw on the map with a line, and the direction of movement - with an arrow. All this is updated almost in real time, and you can not even imagine how much the map has changed and become more convenient.
A separate nuance was the fact that in the first seconds of receiving the location the gps is still being calibrated. Noticed how the point crawls on the map for some time at the beginning? With my logic, these phantom movements would immediately give false conclusions about the direction of motion. And given the fact that the nearest update of the results is already 200 meters away, this would be a great misinformed driver. I solved this problem very simply: before the first update, pretend that we are standing. That is, do not show an arrow on the map (although the point still crawls) or a route. And to unblock this data already after, when we moved a substantial distance, 5 seconds passed and the chances of getting false information are almost nil.
I also added a card that crashes when selecting a result on a map. In fact, these are the same data as in the list, but a preview of the route is added to them (yes, it did return) and the button that starts the navigation.
In general, the interface was thoroughly redesigned. I will not describe how I rewrote the menus, rebuilt the color palette and stuff like that. I will focus on the most interesting moments. All the inscriptions were proportionally enlarged (my falling vision came in handy here - it increased it until I saw it myself from the driver’s seat). Changed the font on the SF Pro Rounded - this is a rounded variation of Apple San Francisco. Download here , sensible font. I strongly recommend in cases where you do not have a solid text, but large dies that should be readable from afar.
After some thought, I made an unclear decision for myself to remove the “Along the Way” filter. Like the time filter, at first it seemed almost the main function of the app. However, at some point I realized that I did not use it. In the list mode you can clearly see which places are on the way, and in the map mode, it is completely confusing. For some time I senselessly dragged this switch through the interfaces, after which I simply hid it and lost nothing. Plus, technically, it produced nuances that were a chore and it is not necessary to decide at all.
Actually, the main problem at the moment of the application is data. I still take them from OSM with all the attendant problems: uneven coverage, a bunch of outdated data, lack of hours of operation, phones, and so on. My backend is built in such a way that it is very easy to integrate any third-party api into it - but that's where to get it? The first (and best) candidate is Google Places, but after the recent rise in price by 1400% (good Lord), I cannot afford them yet. All the rest - TripAdvisor, Foursquare and the like - are either expensive or with clumsy api. Some services (the same Mapbox or HERE) under the guise of their data provide chewed places from OSM, which I myself have.
From all this fraternity, I decided to try to fasten the Yelp - it seems to be inexpensive and the api looked decent. I understood that this is an American portal, respectively, there will be a minimum of data from other parts of the world, but this is at least some progress. And at first everything looked quite good: I integrated everything in a couple of hours, and even read that they claim coverage for almost half of the world. However, I did not have time to rejoice as some circus began. A huge number of places in their data had the wrong coordinates. Obviously, they entered places not through points, but through addresses — and their geocoder arranged coordinates arbitrarily. As a result, I have a gas station in the right place, but, apart from the address, it has almost no data; and they have both reviews, and rating, and hours of work - only the coordinates are generally left-handed. The only properties by which our data can be reduced are the name and address. And often both that, and that is written arbitrarily, with errors, incorrectly formatted and so on. I tried to compare them through my geocoder + fuzzy matching, and, in principle, it worked - although we still lose a certain percentage of places in this way.
But this turned out to be only the first problem. Then the rest fell down: they have a very unstable search by radius, a lot of bugs (if you read the comments of people in their gita, then nothing works there at all) and so on. In the end, I checked the branding requirements - it turns out that you are obliged to use their red (!!!) stars for rating places. After seeing how it looks in my interface, I spat and turned off the entire farce.
As a result, there is not much progress with the data. Actually, the only thing that can (and very easily) be done is to fasten Google Places. There, everything is fine and coated, and with coordinates. Only now it costs a lot of money. Therefore, I ask for your opinion: how would you react to a paid subscription? In the free version it would be as it is now, but for some symbolic amount per month, Google data or Yandex, for example, would be available (you need to read how much it costs). So I probably will not go broke.
In general, I recently figured: for almost a year and a half, I took this application. Of course, not all my free time I spent on him, but most of it for sure. Threw a couple of times for a month, sometimes it seemed that I would never finish it - but I did finish it all the same. And, in principle, now it looks exactly as I wanted from the very beginning. Along the way, I mastered more than one new technology and gained a lot of experience. In general, it was interesting. Now I would be happy if the result was useful to someone.
On the rights of self-promotion: in the gap I made another application for myself, a very small one, a parking aid. I did not translate it into Russian, but there is a one-button interface. Maybe it will be interesting to someone.
Actually, by version 3.0 of the application, it finally turned out to implement all the basic functions that I wanted. After the last article in this series, a number of people downloaded it, and even wrote reviews - thanks, I listened to everyone. He worked on the new version of the month two intensively, not to list all minor changes - in fact, this is 80% new application. With a radically improved interface, 2 times faster and much more stable. Again, I invite sympathizers to evaluate and scold. And under the cut again technical moments.
')
Here are links to iPhone and Android
Traffic jams
One of the key complaints about the previous version of the application was the incorrect timing - I implemented it on my own through the Open Source Routing Machine, and he considered the net time of the road. During periods of minimal traffic (for example, at night), my numbers coincided with the fact that they issued the same Google maps, but in most cases the assessment was at least extremely optimistic. This leveled out the very meaning of the application itself, and it was necessary to think of something.
There are two ways to solve this problem: turn to third-party api or try to extort traffic weights from somewhere and import to yourself. I really didn’t want to depend on someone, so I spent some time searching for a second solution. The results were disappointing: I have nowhere found a base of traffic weights with global coverage that would be compatible with OpenStreetMap. There are some open bases on pieces of Europe and America, which, in theory, can be stitched with OSM through tambourine jumps - but in the end, on reflection, I decided not to get involved. Of course, the ability to host the navigation with traffic was seducing, but was scared away by incomplete coverage, difficulties with integration, inaccuracies and the very fact that traffic was cached, and not in real time. In short, again spend a lot of time and get a miser in the end.
Realizing that nowadays without the big uncle's api, I began to look for an adequate and cheap uncle. Stumbling, I chose HERE services - these are former Nokia cards, then Microsoft took them to me, leaving them as a separate unit. As I understand it, they now mainly work with corporate clients (for example, on logistics) and have a fairly reasonable and clean api. And most importantly, they have global realtime traffic and fairly generous quotas. Not an obvious choice, but I decided to try.
The integration ended up being pretty simple. A significant role was played here by the general flexibility of the architecture, which I made. If desired, it is now easy to integrate Google into it, even though Yandex traffic jams. I made the traffic from HERE disconnected with the fallback to my old navigation. Finally, by comparing the values ​​without traffic (mine) and with traffic, you can derive a general estimate — empty road, light traffic, medium traffic, and so on.
Maps List
Maps were the second major claim to the app. I hit an incredible amount of time on drawing my own route thumbnails and the general map, and it even worked - but the generation of raster tiles with a modern design immensely dragged the server. If in testing mode, pictures appeared within half a second, then with a real audience (even a small one), people could wait up to five or even ten (!!!!) seconds. Especially if the piece of the map was large - then my code was chewed and rendered a bunch of vector data. In addition, this process is not particularly parallelized: there were bottlenecks, and all the queues still quickly clogged. In general, sadness.
What to do, it was obvious - throw out thumbnails of the cards to hell. I struggled with myself for a long time (I did all of them for so long), but in the end I made a strong decision and, in general, did the right thing. When the huge and often not loaded pictures left, it became calmer and a lot of space was freed up on the screen. Even filling it with new data and adding inscriptions, I was able to squeeze twice more results into the screen than before.
Maps - new mode
But with the general map, I acted differently. No one (including myself) could understand why, in fact, it is needed. It was self-made, clumsy, loaded forever and in general was originally created for debug. However, for some reason, the visual image of the isochrone continued to warm my soul and as a result it remained to occupy a place in the interface. Then I decided: let's try to make a full-fledged alternative viewing mode out of this meaningless picture. If someone doesn’t like the list and he reads the map more easily - why not? Especially since I had a couple of ideas that could not be found anywhere else.
Reluctantly, I threw out my picture (the server groaning from the load was grateful for that). Instead, I embedded a full-fledged map through the flutter_map plugin - I took the background tiles from Mapbox - and began to show my position and results points around it. Almost immediately, the need arose to cluster these points, and I quickly sketched a distance-based clustering code. It is rather primitive, but covers 90% of cases. Under all this, I again planted my favorite green blob isochron. Finally, the map legend also became interactive: tap by the number of results focuses the map on points, and tap on time on the isochron. Pretty comfortable.
One of the ideas in which I felt value, but could not articulate it in any way, was the display of the current route and vehicle motion vector. I tried to stick it in the route cards in different ways, but nowhere did it look organically and in place. And finally, almost in despair, I realized: the new card mode is perfect for this chip. Because in the list mode I write directly with the text, along the way some place or not - but on the map it is always incomprehensible. Even with Google or Apple, you observe the constantly spinning sector of the compass and for a long time you do not understand in which direction you are going.
Inspired, I sat down to work. I had to refactor a bunch of code along the way, but after a couple of days the logic was ready. I decided to update the position not every 200 meters, as results, but more often - after 10 meters. Each update I recalculate the motion vector, and so it turns out very accurate, because it does not depend on the accelerometer, but on the previous position. The route (that is, the array of the history of our coordinates) I draw on the map with a line, and the direction of movement - with an arrow. All this is updated almost in real time, and you can not even imagine how much the map has changed and become more convenient.
A separate nuance was the fact that in the first seconds of receiving the location the gps is still being calibrated. Noticed how the point crawls on the map for some time at the beginning? With my logic, these phantom movements would immediately give false conclusions about the direction of motion. And given the fact that the nearest update of the results is already 200 meters away, this would be a great misinformed driver. I solved this problem very simply: before the first update, pretend that we are standing. That is, do not show an arrow on the map (although the point still crawls) or a route. And to unblock this data already after, when we moved a substantial distance, 5 seconds passed and the chances of getting false information are almost nil.
I also added a card that crashes when selecting a result on a map. In fact, these are the same data as in the list, but a preview of the route is added to them (yes, it did return) and the button that starts the navigation.
Interface
In general, the interface was thoroughly redesigned. I will not describe how I rewrote the menus, rebuilt the color palette and stuff like that. I will focus on the most interesting moments. All the inscriptions were proportionally enlarged (my falling vision came in handy here - it increased it until I saw it myself from the driver’s seat). Changed the font on the SF Pro Rounded - this is a rounded variation of Apple San Francisco. Download here , sensible font. I strongly recommend in cases where you do not have a solid text, but large dies that should be readable from afar.
After some thought, I made an unclear decision for myself to remove the “Along the Way” filter. Like the time filter, at first it seemed almost the main function of the app. However, at some point I realized that I did not use it. In the list mode you can clearly see which places are on the way, and in the map mode, it is completely confusing. For some time I senselessly dragged this switch through the interfaces, after which I simply hid it and lost nothing. Plus, technically, it produced nuances that were a chore and it is not necessary to decide at all.
Data
Actually, the main problem at the moment of the application is data. I still take them from OSM with all the attendant problems: uneven coverage, a bunch of outdated data, lack of hours of operation, phones, and so on. My backend is built in such a way that it is very easy to integrate any third-party api into it - but that's where to get it? The first (and best) candidate is Google Places, but after the recent rise in price by 1400% (good Lord), I cannot afford them yet. All the rest - TripAdvisor, Foursquare and the like - are either expensive or with clumsy api. Some services (the same Mapbox or HERE) under the guise of their data provide chewed places from OSM, which I myself have.
From all this fraternity, I decided to try to fasten the Yelp - it seems to be inexpensive and the api looked decent. I understood that this is an American portal, respectively, there will be a minimum of data from other parts of the world, but this is at least some progress. And at first everything looked quite good: I integrated everything in a couple of hours, and even read that they claim coverage for almost half of the world. However, I did not have time to rejoice as some circus began. A huge number of places in their data had the wrong coordinates. Obviously, they entered places not through points, but through addresses — and their geocoder arranged coordinates arbitrarily. As a result, I have a gas station in the right place, but, apart from the address, it has almost no data; and they have both reviews, and rating, and hours of work - only the coordinates are generally left-handed. The only properties by which our data can be reduced are the name and address. And often both that, and that is written arbitrarily, with errors, incorrectly formatted and so on. I tried to compare them through my geocoder + fuzzy matching, and, in principle, it worked - although we still lose a certain percentage of places in this way.
But this turned out to be only the first problem. Then the rest fell down: they have a very unstable search by radius, a lot of bugs (if you read the comments of people in their gita, then nothing works there at all) and so on. In the end, I checked the branding requirements - it turns out that you are obliged to use their red (!!!) stars for rating places. After seeing how it looks in my interface, I spat and turned off the entire farce.
As a result, there is not much progress with the data. Actually, the only thing that can (and very easily) be done is to fasten Google Places. There, everything is fine and coated, and with coordinates. Only now it costs a lot of money. Therefore, I ask for your opinion: how would you react to a paid subscription? In the free version it would be as it is now, but for some symbolic amount per month, Google data or Yandex, for example, would be available (you need to read how much it costs). So I probably will not go broke.
Total
In general, I recently figured: for almost a year and a half, I took this application. Of course, not all my free time I spent on him, but most of it for sure. Threw a couple of times for a month, sometimes it seemed that I would never finish it - but I did finish it all the same. And, in principle, now it looks exactly as I wanted from the very beginning. Along the way, I mastered more than one new technology and gained a lot of experience. In general, it was interesting. Now I would be happy if the result was useful to someone.
PS
On the rights of self-promotion: in the gap I made another application for myself, a very small one, a parking aid. I did not translate it into Russian, but there is a one-button interface. Maybe it will be interesting to someone.
Source: https://habr.com/ru/post/459408/
All Articles