Our "Street View" - the history of kartamania.ru
Before telling how we created our answer to Google , Street View in Russian, in response to the comments made in the comments to our first post , we want to inform you that the project is slowly developing, and we made a new version of the panorama browser.
The main innovation requiring discussion is perspective distortion.
When viewing panoramas based on a cubic projection, we implemented the so-called perspective distortions, which, when the panorama is rotated, create the effect of depth or volume. In addition, the added effect of "mixing" when replacing panoramas during movement.
Before us there were two ways: to buy a ready-made solution in the west, for example, from Immersive Media, or to do everything ourselves.
')
We chose the second option for several reasons.
The first reason - we wanted to try to do it at least in some form. It was necessary to go through all the technical stages in order to understand the problems and nuances on their own experience. We did not plan to make a commercial project of this, for us it was something like a hobby.
The second reason is various unpredictable obstacles (of offline character). For example, what will happen if you will be circling all day in the center of Moscow with cameras on the roof of a car? I can say that at first the sensations were not the most pleasant. :-)
The third reason is the lack of a ready-made solution on the market that fully meets our requirements. For example, the affordable and well-known in the west system from Immersive Media, which Google initially used (and as we understand it, abandoned it later), does not provide the quality we wanted if, after the pilot project, we decided to continue to do it seriously. Their system has 12 cameras with a resolution of just 640 × 480 or 360 × 290 degrees. This is not enough, and the quality is the same as that of our “pilot” installation (only the vertical coverage is greater and, maybe, the sticher and optics are better). Other devices - DVRs, commercially available, also do not fit for a number of reasons.
Assessing the costs and risks when creating a system with high-quality shooting, we decided to make a pilot project with a lightweight head unit, and according to its results, if we decide to continue, we will assemble an installation that will shoot high-quality panoramas.
What are the minimum requirements for a pilot system?
No sooner said than done. We picked up webcams (USB) with the widest possible angle (52 degrees horizontally) with autoexposure and allowing to shoot video with a resolution of 640 Ă— 480. To obtain a panorama, taking into account the overlapping of neighboring frames (this is necessary for gluing) and taking into account the possible temporary synchronization of the cameras during movement, we needed 8 such cameras. We disassembled the finished chambers and collected from them and scrap materials (scrap and soldering iron) just such a thing.
The next problem is fixing the device to the car. Our admin, a student of the old school, assembled the design:
The whole structure for protection from rain, wind and other troubles was hidden in a case:
And this is already on the roof of the car:
Next, we developed software that allowed us to synchronize and save images with reference to GPS coordinates. The whole system worked on two laptops from the vehicle’s onboard network.
Along the way, we avoided two serious problems:
So what's next? After we invented the bicycle in the pilot project, we had new requirements for the head system. Of course, there are a lot more requirements, but here are the main ones:
In our opinion, such a device will allow making panoramas no worse than that of Google (its new version). What we slowly and do ... still a hobby :-)
Thank you all, I hope it was interesting.
The main innovation requiring discussion is perspective distortion.
When viewing panoramas based on a cubic projection, we implemented the so-called perspective distortions, which, when the panorama is rotated, create the effect of depth or volume. In addition, the added effect of "mixing" when replacing panoramas during movement.
The story of how we did it.
Before us there were two ways: to buy a ready-made solution in the west, for example, from Immersive Media, or to do everything ourselves.
')
We chose the second option for several reasons.
The first reason - we wanted to try to do it at least in some form. It was necessary to go through all the technical stages in order to understand the problems and nuances on their own experience. We did not plan to make a commercial project of this, for us it was something like a hobby.
The second reason is various unpredictable obstacles (of offline character). For example, what will happen if you will be circling all day in the center of Moscow with cameras on the roof of a car? I can say that at first the sensations were not the most pleasant. :-)
The third reason is the lack of a ready-made solution on the market that fully meets our requirements. For example, the affordable and well-known in the west system from Immersive Media, which Google initially used (and as we understand it, abandoned it later), does not provide the quality we wanted if, after the pilot project, we decided to continue to do it seriously. Their system has 12 cameras with a resolution of just 640 × 480 or 360 × 290 degrees. This is not enough, and the quality is the same as that of our “pilot” installation (only the vertical coverage is greater and, maybe, the sticher and optics are better). Other devices - DVRs, commercially available, also do not fit for a number of reasons.
Assessing the costs and risks when creating a system with high-quality shooting, we decided to make a pilot project with a lightweight head unit, and according to its results, if we decide to continue, we will assemble an installation that will shoot high-quality panoramas.
What are the minimum requirements for a pilot system?
- The ability to take a circular image at least once per second with a minimum resolution of one frame of 640 Ă— 480. That is, moving at a speed of 50 km / h, we should get pictures every 15 meters.
- Take pictures suitable for glueing in panorama, at speeds up to 60-70 km / h.
Here it is necessary to clarify that when driving at a speed of 50 km / h, the car travels about 14 m / s per second and if cameras that are arranged in a circle take a picture with a difference of at least ½ second, then the shooting points may differ by 7 meters. If we shoot a wide avenue, this may not affect the quality of the panorama, but if we shoot small streets or there are close objects in the field of view of the camera, then a difference of 7 meters can be very significant.
On our panoramas there are such defective pictures, due to system failures and due to the brazen violation of the speed limit - we tried to shoot at speeds up to 100 km / h. - Work system from the onboard vehicle network.
No sooner said than done. We picked up webcams (USB) with the widest possible angle (52 degrees horizontally) with autoexposure and allowing to shoot video with a resolution of 640 Ă— 480. To obtain a panorama, taking into account the overlapping of neighboring frames (this is necessary for gluing) and taking into account the possible temporary synchronization of the cameras during movement, we needed 8 such cameras. We disassembled the finished chambers and collected from them and scrap materials (scrap and soldering iron) just such a thing.
The next problem is fixing the device to the car. Our admin, a student of the old school, assembled the design:
The whole structure for protection from rain, wind and other troubles was hidden in a case:
And this is already on the roof of the car:
Next, we developed software that allowed us to synchronize and save images with reference to GPS coordinates. The whole system worked on two laptops from the vehicle’s onboard network.
Along the way, we avoided two serious problems:
- Limiting the bandwidth of USB 2.0 on laptops with many cameras connected.
- Maximum frame synchronization on different cameras and moreover, on different laptops.
Future plans
So what's next? After we invented the bicycle in the pilot project, we had new requirements for the head system. Of course, there are a lot more requirements, but here are the main ones:
- 16 cameras with a resolution of 1280 Ă— 1024;
- hardware synchronization of cameras;
- high-quality optics.
In our opinion, such a device will allow making panoramas no worse than that of Google (its new version). What we slowly and do ... still a hobby :-)
Thank you all, I hope it was interesting.
Source: https://habr.com/ru/post/26332/
All Articles