The story of my participation in the Ubuntu Scope Showdown 2016

January 15, 2016 was announced the annual competition Ubuntu Scope Showdown! Participants were asked to develop from scratch within six weeks and publish their scop for Unity 8 in the app store (the official translation of the word scope as “lens” is not at all happy, so let it be just a scop). During this month and a half, about 15 scouts were worked out, and a month and a half later the jury voted (three times longer than the two weeks declared). As a result, the winners were Roman Shchekin, Ricardo Silva, Michael Weimann and Pieter Engelbrecht. About the experience of participation I would like to tell under habrakatom.

Idea

First of all, let me remind you what exactly scope is in relation to Unity 8:

1. from the developer’s point of view, this is nothing more than a plug-in to the working environment;
2. from the user's point of view, this is the start screen that allows you to quickly access the necessary information without launching individual applications, a customizable shell, so to speak, a competitor of widgets.

My attention to the competition was attracted by the prizes - a great nettop with our market prices for iron is just a tasty morsel for me. Yes, and its iron is already outdated, so I wanted to compete for the grand prize. Phones, controllers, other small things are nice, but no more than.

Ok, there is a desire to participate, but you need an idea for skoupa. Again, for reference, I’ll clarify that the main purpose of a scop is to search for information, be it a search in a bookstore, search for information about current films and events, search for images in social networks, etc. remote virtual machine resources). I remember that in 2014 there was a similar competition, and then, due to the lack of an idea, I missed a few weeks, and then abandoned my work on creating a skop on the topic of Dota 2 (which was then carried away). For the past few years I have been diligently studying various online courses (even filled out a page about myself at https://github.com/open-source-society/computer-science ), so it’s not surprising that I had the idea to create a scop for search for MOOC courses from various sources.
')

Sources

I began to implement my plan by exploring the possibilities offered by the “main players” of the online courses market, namely Coursera, edX, Udacity and Udemy. Briefly for each:

Coursera

Solid and well thought out API. Good documentation. In a word - facing people, it is easy and convenient to work, you can try it right in the browser. Give JSON, the content is configured.

edX

There was an element of disappointment waiting for me - there are an infinite number of pages with a description of the API, even some concrete things about the methods and description of the parameters, but in fact nothing works, and when it works, it is not clear. As a temporary solution, I had to use an RSS feed. It has about 100 courses, the content is updated regularly.

Udacity

A good API will be, but later, for now there is only one link where you can get all the available courses. Everything is better than RSS with arbitrary content, so no need to complain.

Udemy

The most convenient, flexible and pleasant API. You can customize almost anything. The only thing I didn’t really understand was why to screw the “authorization” - before using the API, it was suggested to get an access key, which would then be transmitted in the headers. I had a strong feeling that I was the first to receive this key at all, because the support acted extremely clumsily.

Other sources

Later added iversity and OpenLearning. According to the "wealth" API is very close to Udacity, I will not consider in detail.

Overall, I was satisfied. Sources provided all the desired information, namely, course description, details of the course (duration, level of complexity, etc.), links to images and video business cards, information about instructors. Naturally, she [information] was heterogeneous, sometimes incomplete, and organized in different ways. The tasks of searching, structuring and displaying in a form acceptable to the user were needed to be solved. Having given the project the temporary name CourseExplorer, I began to develop.

Toolkit (and some details)

Canonical developed a special toolkit for creating scopes. Physically, each scop is a dynamic library. Moreover, closer to the competition, the list of available language tools was updated: a few months before, the only non-alternative option was C ++, then it was proposed to use JavaScript and Go as part of the competition. Naturally, the emphasis was on new languages ​​and technologies in order to attract newbies. I chose C ++, because “C ++ (Qt)” was among the teamplates, which could not but rejoice experienced Qt developer.

Here I was waited by the first disappointment - in a template there was "naked" C ++. I even created a scoping project in C ++ without Qt to try to see the difference. As it turned out, in a template with Qt in one place is used QJsonDocument. And that's all :) “Laaaaadno”, - I thought, “we will bend our line!”. It was decided to use Qt in all places, and on pure C ++ to move only in places of interaction with the API. By the way about the API: in fact, it was proposed to implement three entities in your project (to inherit from some base classes):
Scope
Scop himself, capable of processing requests for searching and displaying information. In my case, almost did not require revision.
Query
"Workhorse" Scoupa, searches for information and the division into categories. The implementation has come down to the sequential invocation of source client classes (which, in turn, parsed the data returned by the services).
Preview
Display of previously found and specially structured information in a declarative style using a predefined set of controls.
There is no special point about telling the source code, you can see everything here: GitHub

To speed up the display of results on an empty query, caching was applied (in official documentation, this use is called surfacing mode). In addition, a mechanism for mapping the categories provided by the source into internal categories of skoupa was applied in the scoping. In some cases, it was necessary to reduce two dozen categories to ten (for example, Literature and History were reduced to Humanities), and sometimes vice versa - to use one internal category for all categories of the original source (Web design, Desktop development, JavaScript were reduced to one IT & Tech) .

Visual component

At a certain stage, I realized that it was necessary to bring my work into a presentation, correctly title and draw an icon. After thinking for a few minutes, I came to the conclusion that it would be nice to call the skup on the initial letter of the alphabet so that it would always be in the top of the list of installed scopes sorted alphabetically on one of the basic OS screens. Thus was born the name Academy. Unfortunately, the icon did not work out so quickly. The idea had to be found no longer in my head, but in one of the logo creation services. I came across the icon of an atom with particles in orbits. I immediately liked him, as many people are firmly associated with science. I redraw it in one popular graphic editor. It turned out bad, what can I say, in a hurry. However, the first version flew to the store in this form. The next time I decided to take up the icon more thoroughly and redrawn the old concept with a higher quality. It turned out better, perhaps because there was already no such pressure of terms. I wanted to stop at this option, but I was lucky - I accidentally came across a post of one OpenSource designer in G +, he just painted icons for some UP applications. He agreed to draw me too. This is how the final version turned out, which you can see below. The rest of the interface components, I summed up under the color design of the icon, it turned out really nice.

Bug

In some kind of action in my scoop began to lead to the fall of Unity. I didn’t want to believe that this was my doing, so I tested all the stable versions without the latest changes - a bug was present. Fortunately, we were able to quickly contact the developers of Unity 8, together we found the problem. It turned out that the scoop engine cannot correctly respond to absolutely identical results (since it performs a comparison across all the fields inside). It turned out that this edX spoils life, sends two identical descriptions. Simple check based on QSet and problem solved.

Total

So, I filled out the participant form and waited. First I waited for the end of the competition, then the planned two weeks of voting by the jury, and then two more, and two more. Frankly, I was fed up already waiting. During this time, the Academy got into the top scopes by rating, several extremely positive reviews were collected in different languages, and I quietly engaged in optimizations and processed pull-requests with localizations.

And here, the long-awaited announcement of the winners. The distribution of seats has already been announced in the prologue, so you can quickly return to the beginning to see the winners again, and at this time I will be tired of joy, because Roman Shchekin is me!

PS Several screenshots of the finished scoop can be seen here: uappexplorer / academy , the announcement of the results of the developer.ubuntu.com
PSS Maybe it should be on giktaims. It was conceived as easy reading, IT history and nothing more. Criticism is welcome!