Hackathon Open Data Day - Member's View

February 21 is the International Open Data Day (ODD). The fact that such open data can be found in Wikipedia . Here is a story about the action, which took place in Moscow.

This year, ODD was held traditionally under the auspices of NP "Information Culture" and the Russian branch of the Open Knowledge Foundation. These organizations were again traditionally represented by Ivan Begtin and Irina Radchenko, respectively. A hackathon was dedicated to data visualization. The formal report has already been compiled by Irina. I would like to reveal the “how it was” from the point of view of the participants.

Hackathon happened in Impact Hub Moscow. It is in the center of Moscow. But disguised considerably, so I (and not only) stood for a long time in the courtyard and were looking for which door to enter. So when I came, the action had already begun. There was almost no place - I hardly found a chair in the front row. All other places were occupied.

I came to the hackathon with a ready-made idea. The idea was to visualize on a school expenditure map . And some data dragged ready, too. I regretted very much later that I did not collect more in advance and did not process it. I also had a more or less development environment (pycharm).
')
In general, having listened to Ivan Begtin's introductory message, my friends and I went to the back tables to do our project. Strangely enough, there were very few people like us - basically everyone else was left to listen. As it turned out later, only a few came to the hakaton - most simply listened or talked. But to do ... But maybe this is good - there were not very many tables, too many developers could not sit.

Data exchange, we and a friend established through Yandex.Disk, the code was practically not exchanged - he was engaged in obtaining data, I visualized. But still, git was entered and the code was periodically saved into it. Chat and share links via chat in Google Hangouts.

Our project is actively using open data. For a start, we took lists of the best schools from the website of the Department of Education of the city of Moscow. Then they connected it with the general register of schools from the Open Data Portal of the Government of Moscow (dataset No. 747). And already this data was connected with the data of public procurement through the site expenses mashape-API. Well, all this data was geocoded through the Yandex API and mapped onto the map with beautiful pie charts and lines.

But back to the timeline. In the informative part, the first was a review report by Ivan Begtin about various visualization tools for budgets and budgets in general. Honestly - listened to in the middle. Already plunged into development. But even so - the report is interesting. The truth is mostly not for developers, but for users - a lot of different visualization tools are listed. Then there was Irina Radchenko and Rostislav Yavorsky , but I somehow missed them at all. Worked with data.

What is the work with data? Well, it is their receipt, study ("intelligence"), transformation, addition, linking with other data, and finally saving in a ready-made format. For this, one of the best tools is OpenRefine (formerly Google Refine). It is convenient to open a large file, see what is there for the data, filter it, change it en masse, add new ones (for example, from geocoding) and then save to json for example.

My friend at that time worked on studying and working with API through mashape - collecting the next portion of data. And I moved from the data to their visualization. Thanks to openrefine, I already had an array in json, and I actively plotted it. Did it on LeafletJS. It was actually my time to speak - I also announced in advance, at the beginning of the free stand to speak. He told about the same - thematic maps on leafletjs for beginners . After me there was a “free stand”, without a presentation from the openstreetmap community and other people.

With the project, not everything worked out, both for me and for a friend. Well, yes for a hackathon this is typical - you should not try to do a lot there. Better a little, but beautiful. I got into the jungle of leaflet-dvf - a plug-in for LeafletJS and dug into it. Time was ticking. His head was boiling.

Then I did it more or less dramatically, but my friend didn’t. As a result, the visualization was almost ready, but there was no data. I began to experiment in the functionality a bit, and it helped a lot - as a result, when the data came and turned out to be not quite what I expected, the experiments helped to adapt them.

In the meantime, the movement began again - live broadcast with Perm. There was also a similar event. But only there with the creation of projects was even quieter than ours. Just gathered, talked. In any case, I told about my project in the video bridge, but from there nobody told me about anything.

This is where the data came in, and I began to adapt my js for them, and the data - for js. Anyway, the time was not enough until the end, and the project was shown with a black and white pie chart instead of a beautiful color one. However, he already liked it.
Conclusions in general: preparing for a hackathon should be better. Ideally, there were no questions, but all that remained was to “sit down and do.” But this is an unattainable ideal. Establish communication and a common development environment - or clearly separate the tasks immediately. Better the second. Hard to limit the functionality of the application - to be on time. And if something happened - not to relax and rest, but to work further, to experiment, not to fall out of the “stream”.