Rendering world
MapBox lead programmer Jung Khan (Young Hahn) has published a transcript of his presentation from the FOSS4G conference.
MapBox is known as the developer of the open source cartographic design studio TileMill and the TileStream tile server. Recently, they released a new system, MapBox Streets - an analogue of Google Streets, but on open maps OpenStreetMap. This is a global map service with zooming to the level of individual houses, tunnels, metro stations, etc.
In a presentation titled Rendering the World, the company's lead programmer tells how they managed to generate tiles at this zoom level for the entire planet.
')
If you count the number of required tiles on a global scale, then the data volumes are astronomical. In this system, the scale z0 corresponds to one tile for the entire world map, at the level of z1 the map consists of four fragments, at the level of z2 - 16 fragments and so on. In general, the number of tiles at each level z equals 4 z .
If we assume a completely high server performance, say 1000 tiles per second, and the minimum tile size is 400 bytes, then at the level of z17 we need 200 days of rendering, and the amount of map data is 10.3 terabytes.
With such numbers, even a strong optimization of rendering does not have a decisive effect. If you speed it up twice, the rendering time will increase from only 200 to 100 days, which is unacceptable in both cases.
Another option for the problem may be a simple increase in the number of servers for real-time rendering of specific requests to users who are currently arriving.
But in reality, such a system is difficult to scale. Such servers are slow and expensive to maintain, and real-time rendering requires a large number of them, while MapBox is a small company and cannot afford it.
So how could they render the whole world at the level of z17? They approached the matter wisely. An analysis of geographic data shows that more than half of the earth’s surface actually does not need to be rendered at all. About 60% of the tiles are simple blue squares - the territory of the world ocean. The school teaches that the ocean occupies 70-80% of the surface, but in fact the blue squares are about 60%, because Antarctica and Greenland cover a gigantic area.
These are like letters in the alphabet, 50% of which can be removed from the text, and the text will not lose its meaning from this.
By optimizing the task further, MapBox developers began to analyze what percentage of the earth’s surface does not carry the information load and can be ignored. In addition to the ocean, they have allocated land area, which is not located near the coastline. It turned out that almost always these are also homogeneous "empty" tiles. In general, the coastal territory accounts for only 2% of the tiles. And if you take the cities, roads and any objects of human activity, they are visible only on 1% of the Earth's territory!
Thus, the number of tiles at the level of z is no longer equal to 4 z , since with 1% of usable territory at each scale level, one single useful tile needs to be generated.
The MBTiles tile storage system created by MapBox allows you to store tiles taking into account redundancy, that is, identical tiles are not duplicated, but simply refer to one sample. So instead of 10.3 terabytes, the entire territory of the Earth at the level of z17 in this system takes 200 gigabytes. And instead of 200 days of the work of the tile server, rendering 2% of the usable territory takes only 4 days.
MapBox is known as the developer of the open source cartographic design studio TileMill and the TileStream tile server. Recently, they released a new system, MapBox Streets - an analogue of Google Streets, but on open maps OpenStreetMap. This is a global map service with zooming to the level of individual houses, tunnels, metro stations, etc.
In a presentation titled Rendering the World, the company's lead programmer tells how they managed to generate tiles at this zoom level for the entire planet.
')
If you count the number of required tiles on a global scale, then the data volumes are astronomical. In this system, the scale z0 corresponds to one tile for the entire world map, at the level of z1 the map consists of four fragments, at the level of z2 - 16 fragments and so on. In general, the number of tiles at each level z equals 4 z .
If we assume a completely high server performance, say 1000 tiles per second, and the minimum tile size is 400 bytes, then at the level of z17 we need 200 days of rendering, and the amount of map data is 10.3 terabytes.
With such numbers, even a strong optimization of rendering does not have a decisive effect. If you speed it up twice, the rendering time will increase from only 200 to 100 days, which is unacceptable in both cases.
Another option for the problem may be a simple increase in the number of servers for real-time rendering of specific requests to users who are currently arriving.
But in reality, such a system is difficult to scale. Such servers are slow and expensive to maintain, and real-time rendering requires a large number of them, while MapBox is a small company and cannot afford it.
So how could they render the whole world at the level of z17? They approached the matter wisely. An analysis of geographic data shows that more than half of the earth’s surface actually does not need to be rendered at all. About 60% of the tiles are simple blue squares - the territory of the world ocean. The school teaches that the ocean occupies 70-80% of the surface, but in fact the blue squares are about 60%, because Antarctica and Greenland cover a gigantic area.
These are like letters in the alphabet, 50% of which can be removed from the text, and the text will not lose its meaning from this.
By optimizing the task further, MapBox developers began to analyze what percentage of the earth’s surface does not carry the information load and can be ignored. In addition to the ocean, they have allocated land area, which is not located near the coastline. It turned out that almost always these are also homogeneous "empty" tiles. In general, the coastal territory accounts for only 2% of the tiles. And if you take the cities, roads and any objects of human activity, they are visible only on 1% of the Earth's territory!
Thus, the number of tiles at the level of z is no longer equal to 4 z , since with 1% of usable territory at each scale level, one single useful tile needs to be generated.
The MBTiles tile storage system created by MapBox allows you to store tiles taking into account redundancy, that is, identical tiles are not duplicated, but simply refer to one sample. So instead of 10.3 terabytes, the entire territory of the Earth at the level of z17 in this system takes 200 gigabytes. And instead of 200 days of the work of the tile server, rendering 2% of the usable territory takes only 4 days.
Source: https://habr.com/ru/post/142089/
All Articles