We collect bad data - 2. 1.5 years later, about how not to publish open data
It is no secret that the Russian authorities recently declare a high degree of openness, publish data and some of them are even useful, but it was not always and not everywhere.
A year and a half ago, I gave a lot of examples of misinterpretation of open data. In the “Collect Bad Data” note there are a few examples when for machine-readable open data they were given sections on websites or PDF documents.
While monitoring the openness of state data in Russia is a thankless task, and that the method of recommending the same Ministry of Economic Development is the form's victory over content, but even in this scenario, one could complain only that government bodies made their life easier by publishing data, and did not complicate it.
')
Whether it is necessary to do a full-fledged survey of the authorities regarding the quality of open data is an open question. Sooner or later it will have to be done even though it takes a lot of time.
But the most outstanding examples can be found now. And of course - bullshit ratings, anti-rating rule. The worst you need to know in person.
Reference: xn - 80aabycqsgabatek.xn - p1ai / Otkritie_dannie
4 data sets and their passports. Open your passport: xn - 80aabycqsgabatek.xn - p1ai / Otkritie_dannie / Pasport_nabora_otkritih_dannih_podvedoms
Written XML, inside the Excel file with a passport. The contents of the Excel file in the screenshot. Inside there is nothing, no links. Contact email on mail.ru, data structures are not, links are not to the site of the department, but to the site of their developers like “roz.sitesoft.ru”.
Link: www.rostransnadzor.ru/rostrans/pressa
It is immediately significant that the press service publishes open data on the site. We can learn a lot of interesting information about Rotransnadzor in the open data section, but for some reason the data itself is presented in the form of sections on the website, MS Word and Excel files. There is not only a list of published data, in general, everything placed does not fully correspond to the form, spirit or goals of disclosure of machine-readable information.
Link: www.fsin.su/opendata
In the list of FSIN 7 data sets:
Of these, only one is alive, the first. The rest are empty signs, in which there is simply nothing - www.fsin.su/opendata/7706562710.2.1
Link: rosavtodor.ru/opendata
Given that they place 19 data sets and among the sets there are even interesting ones, but even the first glance allows us to evaluate the quality of the disclosure.
In the rosavtodor.ru/opendata/7717509757-svoddor set , the data is presented in XML with the set of fields Column1, Column2, and so on. In the scheme (structure) of the data, of course, no field is marked what it means.
Total - the formalities are met, the meaning is slightly more than zero.
Link: www.gfs.ru/opendata
There is little data from the service, it almost does not work with people and very few people need it. But even in this case, having looked into the vacancy dataset, we find that it has not been updated for a year and, judging by the content, the SFS has no vacancies. Although there are 3 vacancies on the site itself - www.gfs.ru/kadrovaya-sluzhba/vakansii/vakansii-federalnoj-gosudarstvennoj-grazhdanskoj-sluzhby
Link: www.minsport.gov.ru
On the website of the Ministry of Sports there is a link from the main page. The link, as expected, gives the message “403. Forbidden. Type of data is, even open, but not for us.
- Outside there were a few particularly prominent departments, publishing MS Word files under the guise of XML files, dummies as CSV files and more. About them separately and with addiction next time.
Why is this situation? The reasons are many. This is the fault of the Ministry of Economic Development, the Council for Open Data, and, most importantly, the entire system of state administration, which has the capacity not only to “imitate an orgasm,” but to imitate activities in principle.
If you know other outstanding cases of such imitation - I invite you to share them all.
Question to all: can we automate the verification and identification of the most odious cases? Automatic check schemes CSV, XSD? Identifying those XML files that are actually DOC files and so on? Maybe there are already open source tools?
And the second question / survey:
A year and a half ago, I gave a lot of examples of misinterpretation of open data. In the “Collect Bad Data” note there are a few examples when for machine-readable open data they were given sections on websites or PDF documents.
While monitoring the openness of state data in Russia is a thankless task, and that the method of recommending the same Ministry of Economic Development is the form's victory over content, but even in this scenario, one could complain only that government bodies made their life easier by publishing data, and did not complicate it.
')
Whether it is necessary to do a full-fledged survey of the authorities regarding the quality of open data is an open question. Sooner or later it will have to be done even though it takes a lot of time.
But the most outstanding examples can be found now. And of course - bullshit ratings, anti-rating rule. The worst you need to know in person.
Rosoboronzakaz
Reference: xn - 80aabycqsgabatek.xn - p1ai / Otkritie_dannie
4 data sets and their passports. Open your passport: xn - 80aabycqsgabatek.xn - p1ai / Otkritie_dannie / Pasport_nabora_otkritih_dannih_podvedoms
Written XML, inside the Excel file with a passport. The contents of the Excel file in the screenshot. Inside there is nothing, no links. Contact email on mail.ru, data structures are not, links are not to the site of the department, but to the site of their developers like “roz.sitesoft.ru”.
Rostransnadzor
Link: www.rostransnadzor.ru/rostrans/pressa
It is immediately significant that the press service publishes open data on the site. We can learn a lot of interesting information about Rotransnadzor in the open data section, but for some reason the data itself is presented in the form of sections on the website, MS Word and Excel files. There is not only a list of published data, in general, everything placed does not fully correspond to the form, spirit or goals of disclosure of machine-readable information.
FSIN of Russia
Link: www.fsin.su/opendata
In the list of FSIN 7 data sets:
- 7706562710.1.1 The list of territorial bodies of the Federal Penitentiary Service of Russia
- 7706562710.2.1 The list of subordinate organizations
- 7706562710.3.1 Plan for scheduled and unscheduled inspections
- 7706562710.4.1 Information on the results of scheduled and unscheduled inspections
- 7706562710.5.1 Statistical information
- 7706562710.6.1 Information about the vacancies of the civil service in the Federal Penitentiary Service of Russia
- 7706562710.7.1 Registers of licenses for specific types of activities
Of these, only one is alive, the first. The rest are empty signs, in which there is simply nothing - www.fsin.su/opendata/7706562710.2.1
Federal Road Agency
Link: rosavtodor.ru/opendata
Given that they place 19 data sets and among the sets there are even interesting ones, but even the first glance allows us to evaluate the quality of the disclosure.
In the rosavtodor.ru/opendata/7717509757-svoddor set , the data is presented in XML with the set of fields Column1, Column2, and so on. In the scheme (structure) of the data, of course, no field is marked what it means.
Total - the formalities are met, the meaning is slightly more than zero.
State Courier Service
Link: www.gfs.ru/opendata
There is little data from the service, it almost does not work with people and very few people need it. But even in this case, having looked into the vacancy dataset, we find that it has not been updated for a year and, judging by the content, the SFS has no vacancies. Although there are 3 vacancies on the site itself - www.gfs.ru/kadrovaya-sluzhba/vakansii/vakansii-federalnoj-gosudarstvennoj-grazhdanskoj-sluzhby
Ministry of Sports of the Russian Federation
Link: www.minsport.gov.ru
On the website of the Ministry of Sports there is a link from the main page. The link, as expected, gives the message “403. Forbidden. Type of data is, even open, but not for us.
- Outside there were a few particularly prominent departments, publishing MS Word files under the guise of XML files, dummies as CSV files and more. About them separately and with addiction next time.
Why is this situation? The reasons are many. This is the fault of the Ministry of Economic Development, the Council for Open Data, and, most importantly, the entire system of state administration, which has the capacity not only to “imitate an orgasm,” but to imitate activities in principle.
If you know other outstanding cases of such imitation - I invite you to share them all.
Question to all: can we automate the verification and identification of the most odious cases? Automatic check schemes CSV, XSD? Identifying those XML files that are actually DOC files and so on? Maybe there are already open source tools?
And the second question / survey:
Source: https://habr.com/ru/post/248521/
All Articles