Inevitability of nodocalypse

Each object in OpenStreetMap has a unique number. The basic element of the cards is a point, everything consists of them, and there are a lot of them. On February 9, point identifiers exceeded 2³¹ − 1: the maximum number that can fit into a 32-bit signed int. The impending problem was warned in a year and a half, and all more or less popular programs managed to switch to a long one. That Saturday passed without incident.



Not really.



Everyone knows that in 25 years time stamps will also stop getting into int, but no one worries: there is more than enough time to fix. Similarly, we thought about our tools in May 2011, when Frederic Ramm first raised this issue. The point identifiers grew rapidly, and the chart seemed to hint. In osm2pgsql (the main program for loading data into PostgreSQL), a switch for 64-bit mode was introduced, then they began to discuss whether nodes could be renumbered or unsigned int in some way, from which the conversation turned to the need for negative identifiers and was blown away.



The year has changed, the number of participants OpenStreetMap was approaching a million, the French with might and main imported contours of sheds about fifty peaks each from the inventory, the Germans were developing tagging schemes for curbs, sewer manholes and road markings. Smartphones and tablets have spread among mappers, and convenient editors have written for them. Studies have reported the overwhelming growth of OSM and the astounding quality of the data. Microsoft replenished the Bing satellite imagery array, and thousands of enthusiasts lined it up for days on end. It was December 2012.

')

“Another 62 million points - and your programs may break,” Frederick recalled, “I uploaded several files with large identifiers to SVN so that it would be easier for you to check your tools.” Does it make sense to warn people five days before the New Year, a week after the "apocalypse"? Not before the participants are more pleased with other conversations: about the new js-editor and rating by the number of painted houses.



February came, Paul Norman remarked in the ezine that it was already five million 2³¹ away. Users add about one million points per day - consider yourself. A decrepit program for preparing shoreline shapes will not survive this shock, but it has a substitute. PHP scripts can break down, since 64-bit numbers are recognized only on 64-bit operating systems. It seems to be nothing serious: the wiki page with the list of required versions of programs grew, but from the noticeable - only an additional key requirement for the popular Osmosis, a program for filtering and converting OSM data.



Participants excitedly followed the countdown , loading freshly painted houses and forests in another window.



The first audience was pleased with the site of reference: of course, written in PHP and launched in 32 bits. Shoemaker without shoes. A few hours later, Toby Murray wanted to add car rental to the map, and the editor for the android Vespucci flew out. Then the Japanese launched the POI + iOS editor to mark fast food in Nagoya, but instead of creating a new point, he used the one already drawn with the number 2³¹ − 1 , which belonged to the contour of the house in southern Germany. The house began to look something like this:







Then came the puzzled Osmosis users. It does not matter that they warned about the obligatory key on the forum, in the news blog, in the chat, in the newsletter and in the wiki. Again and again I had to explain what to add to the command line, - both on the forum and in the chat. And in the news blog for the second time . It did not help immediately: since Osmosis is used for regional landings, the latter have not been updated for several days. And along with them, exports to navigators and validation froze.



Finally, another dump of the planet surprised everyone with its size: instead of 26 gigabytes - 80 bytes. A small C ++ script to store the most important file in the project stored identifiers in an int field.



All last week, services gradually straightened out. The shoreline processor was repaired and the line itself was updated (it is processed separately from the rest of the card, since this is a single 300-megabyte multipolygon; the update is run every couple of months). They released a new Osmosis, which, in addition to fixing the bug, included a new powerful tag handler. The authors of Vespucci and POI + fixed the problems with identifiers almost immediately, but in a mobile environment everything is not easy: Google Play and the AppStore posted the corrected versions only on February 16. Planet collected, albeit with a third time. The countdown has ceased to be reversed.



Unlike the history of 25-year-old cookies, the “nodocalypse” was predicted, reminded of it in two months and a few days. All of the programs described are open source; anyone could check the type of the variable in which the point number is stored. Nevertheless, even the tools used by thousands of people were corrected only when errors got into.



Output two. Do not ignore warnings, including your own. And, as a blogger from PJ Online advised a month ago, if exactly after 25 years seven in the morning you are caught driving, take the curb, wait twenty minutes. So, just in case.