Series of publications about collecting signatures

1. Introduction, the site "Bulk 20! 8", preparation for the collection

2. Iron and networks, video surveillance

3. Reaper 2018: a system for collecting signatures

4. Project Management

In the previous post, we told what collection of signatures is and why it is meaningless to be conducted without a specialized IT infrastructure. And this chapter is about the network and computer equipment of the federal network of Navalny headquarters, as well as about developing its own document scanner.

What tasks we solved

Equip workstations for operators, provide each headquarters with stable and secure access to service web applications, make video surveillance of the headquarters premises, organize scanning of documents to verify the documents of signatories.

')

Expected difficulties:

- equipment theft prior to initialization of the set of equipment;

- equipment theft after initialization;

- Disconnection of the Internet (cut off the wire, interfered with the channel);

- human factor (any incorrect actions with the equipment);

- toxic staff staff;

- fast moving (which is worse than a fire);

- the actual fire (and other methods of physical destruction of the headquarters).



We have selected equipment so that any element of the infrastructure is easily replaced, and it is not necessary to carry it from the central headquarters, but you can buy it in local stores. The assembly of equipment consists of the simplest “insert wire” operations. The sequence of actions is described in detail in the instructions (with gifs!).



All configurations can be updated remotely while the Internet is running. If the Internet does not work, staff can reset all the settings using the instructions on the operating device.



Access to the headquarters VPN network is issued on the basis of several factors, so that equipment theft (with certificate, PSK keys) or password leakage does not lead to data leakage. All devices are tracked in a special application and can be blocked remotely.



The basic principles are described below without deep detail. We do not lay out scripts and configurations in open access for security reasons.

Elements of infrastructure

A hypervisor is a server running KVM virtual machines with the right applications.



Owl is a virtual machine with an application for operator workstations.



A video server is a virtual machine that runs an ivideon-server and captures the stream from cameras.



The headquarters computer is a personal computer of the headquarters employee, from which you can access a closed part of our applications and access the Internet.



R-station - operator station based on the Raspberry Pi 3 Model B.



Camera - external or internal ip-camera for video surveillance in the headquarters. It is powered via a PoE-adapter, the connection is handled by local experts.



Router - MikroTik, which is used at headquarters for distributing the Internet and routing traffic between infrastructure elements.



3G modem - Huawei USB modem in LTE mode, provides a backup channel.



Dashboard is a system for registering, automating and monitoring the state of the elements of the staff infrastructure.



Document scanner , also known as Box - DIY document scanner based on USB-camera.

Dashboard

Dashboard is an application that manages configurations of routers and allows you to monitor the work of the staff network. From here you can monitor the status of all devices and quickly disable access in case of equipment withdrawal or other problems. For each headquarters, you can see the status of its communication channels and connected devices. There are logs of the state of routers and actions with them, uptime of all network clients.





Another Dashboard controls the configuration of routers and cameras (below will be the details).

Routers

We use MikroTik SOHO models as headquarters routers. They are inexpensive, they are in any city, they are flexibly configured and allow us to implement the necessary network configuration, including VPN tunnel, channel reservation and complex routing between devices.



Initialization of the new router is organized so as to minimize the risk of unauthorized connection to our network:



- the headquarters informs the support staff of the mac-address of the router;

- the router is connected by wire to the operator station;

- on the station, the router initialization script is launched;

- the script resets the settings of the router and downloads our firmware;

- the router disconnects from the station and connects to the Internet;

- firmware scripts do the rest of the work.



When the Internet appears, Mikrotik establishes a temporary VPN connection, retrieves its individual configuration from our server, establishes it, opens a permanent VPN connection, configures its DHCP server and connects headquarters devices.



If there are any problems with the firmware, you can restart the whole process remotely or, if the problems are fatal, ask the staff to repeat the initialization.



Mikrotik is connected via two channels: via the wire that comes to the office, and via its own 3G / LTE modem. The modem is used as a backup channel. In the firmware of the router there is a code that monitors the status of the channel and switches to the backup channel if the main one does not work.







The router controls the routing of traffic between headquarters devices and restricts access. Operator stations can only go to a specific server running the signature collection application. Cameras do not go anywhere at all, but a video server can take a stream from them. Stations of one headquarters can contact each other inside the subnet and collect the video stream of the scanner.







Initially, we planned to connect staff computers to our network, but then we decided that it was not safe for the rest of the infrastructure, so personal computers connect to another router and do not overlap with this infrastructure.

Operator computers

Verification operators need computers to scan documents, enter subscriber data into the system and check the correctness of this data. Computers should be cheap, reliable, easy to maintain and uninteresting for all sorts of law enforcement agencies.



We could not make a real “thin client” with network booting, since there is not and cannot be a sufficiently reliable Internet in many headquarters. Yes, and no one had such experience. We decided to use the traditional loading of the operating system and in all its applications not to go beyond the browser. This approach allows you to quickly restore the work of operators, even if there is something wrong with the standard kits. And we precisely are able to do web applications.



For our task, the characteristics of modern single-board computers were sufficient. Tests of several models showed that the browser and quite complex web applications work without any problems and that you can not chase performance, so one of the most popular models was chosen - Raspberry Pi 3 Model B. There is a 4-core ARM processor, 1 GB RAM, WiFi and HDMI output.







Raspberry Pi can be found in almost every major city. All useful information is on one micro-SD-card, which in the case of a raid of riot police, you can quickly pull out and eat .



As the operating system took the standard Raspbian. The browser is pre-installed Chromium.



Monitors chose the cheapest, which have HDMI and a resolution of 1920 × 1080.

Document Scanners

To avoid errors and in case of litigation with the electoral commission, we must receive an electronic copy of the document certifying the identity of the citizen who leaves the signature. By documents, we first understand the passport of a citizen of the Russian Federation. At the verification stage, the consent to the processing of personal data of those visitors who were not recorded in advance through the site was also scanned. We made the consent form the size of a passport reversal in order to simplify the task.







There are many scanners specifically designed for ID cards, but they have two drawbacks. First, most often they scan one page of a passport, and we need a U-turn. Secondly, the cost of even the cheapest options starts from a few hundred dollars, which, with our number of headquarters and scanners, turns into several million rubles.



Flatbed scanners take up a lot of space on the table (A4 format + body + power), scan slowly (typical time is more than 15 seconds, and we save every second, because the bandwidth of the headquarters depends on it) and do not show the picture before scanning starts .



In addition, it is not clear how to connect any of these scanners with a browser application. Most often, scanners are connected via USB and work through the TWAIN interface. There is an implementation of such a connection on the HTML5 Web Scanning SDK, but under Raspbian it could not be started.



We contacted Smart Engines, which publish articles on Habré about scanning and automatic recognition of passports, but they were not interested in delivering only scanners, and for the whole complex (with autorecognition) they asked for too much for us.



In the end, we decided to make our scanner based on a USB camera. The camera shows the image in real time, and the picture is accessible from the browser. The scanner body can be made exactly to the size of the passport so that it does not take up too much space and there are no problems with the positioning of the document.





Found the manufacturer, together with him made a couple of prototypes and after testing began production. The scanner body is made of aluminum composite panels and glass. The electronic stuffing is closed by a plastic cover.





_{Scanner parts ready for assembly}

Passport Recognition

We considered the option of automatic recognition of passport data, but in the end it was abandoned. On real passport scans (crumpled, dirty, hand-filled), even commercial accuracy solutions were inferior to manual input.







The most important information for us - the address of the permanent registration - is not automatically recognized at all, because half of the passports contain stamps filled in by hand. Not some stamps do not have information about the subject of the Russian Federation, you need to learn from the owner.



In addition, we considered it important that the operator, who in 20 seconds will have to enter passport data in the subscription list, read them in his passport with his own eyes and drove it into the computer, specifying the passport’s incomprehensible information. This reduces operator errors.



This is how we did not develop a computer vision system.

CCTV

After the opening of several headquarters, attacks and dubious searches began (this is when the police come and carry out all the contents of the headquarters under the pretext of searching for extremist text in leaflets). For shooting similar raids and fixing the signature collection process, we made a video surveillance system. Three cameras were installed in the headquarters - two in the room and one at the entrance.





One relatively powerful server is engaged in collecting and storing camera recordings. Calculations showed that in a month about 40 TB of archival records are being gathered. A server with ten SATA disks of 6 TB each and 50 TB of traffic costs us less than 200 euros per month.



You can save space in the archive and make working with cameras a little more convenient by turning on software motion detection (save only those intervals where there is movement), but in this case you will need 3-4 similar servers due to CPU load. We decided to keep the threads full. It is easier, safer and cheaper.



To collect streams from video cameras, ivideon-server is used, which Ivideon distributes free of charge. To view their desktop application is used, connected to our server.







We manage the configuration of the staff network devices through our own Dashboard application. In its database there is information about the headquarters and ip-addresses of devices. Dashboard creates configuration files for each daemon and a single configuration for their launch through the supervisor, where each daemon is given the path to its individual settings:

command=/opt/ivideon/videoserver -c /video/vs_0001.conf --v=5bccd2a043c14e 

The parameter --v is the checksum of the settings file (vs_0001.conf). After saving the new configurations, it is enough to run supervisorctl update to restart all processes, the configuration of which has changed. Supervisor sees a parameter change with a checksum and understands that this process needs to be restarted.



A separate script deals with the removal of records when they become older than a certain age. Another script bypasses all of the ivideon-server sqlite databases, collects camera timelines (intervals when there is a record from this camera) and adds them to PostgreSQL so that they can be shown in a report on Dashboard.







That's about everything you need to organize your video surveillance service for 230 cameras.

Sharing Scanners

If you have a USB scanner, one operator can use it. What if there are more operators than scanners? Adjust streaming images over the network!



We even thought about buying ip cameras, which themselves can stream the image, but working with them is much more difficult: they need power, additional network equipment (the router ports are already busy, and there are not so many PoE ports). Ip-cameras are configured through some kind of interface, very bad in cheap cameras, it is difficult to automate. It is difficult to find an ip camera module (without a case) with suitable characteristics (wide angle, high resolution, good optics with minimal distortion at the edges). In addition, ip cameras are noticeably more expensive. The use of one scanner by several operators is not the main scenario, but a “plan B”, so I did not want to spend a lot of manpower and resources on it.



From the streaming we needed low communication latency and minimal use of scarce Raspberry Pi resources. We tested several solutions:



- ffmpeg / ffserver (avconv);

- nginx + nginx-rtmp-module + ffmpeg;

- motion (https://motion-project.imtqy.com/ - software motion detection);

- MJPG-Streamer (https://github.com/jacksonliam/mjpg-streamer);

- vlc.



Compiling all of this under Raspbian in total took probably a couple of days. The most convenient and fastest (low-latency) solution was MJPG-Streamer. He took the picture from the cameras in the desired resolution and gave it without transcoding via the built-in web server. The picture was available in the browser without any additional actions. The processor load turned out to be significantly less than with other solutions.



Not all developers had scanners or webcams at hand. For debugging, we used an Interactive Online Biology broadcast from Stanford. It uses the same MJPG-streamer server as ours.

---

The third chapter will describe the process of creating applications for collecting signatures and all that concerns working with physical subscription lists. They also talked about the invention of the form of the subscription list, which made it possible to conveniently synchronize physical media with the electronic base, while remaining within the law.