Ubuntu Mobile: Post-mortem Analysis

This is what Ubuntu Touch looked like when the project was announced on January 2, 2013. Image: Canonical

Now that Ubuntu’s phones and tablets are gone, I’d like to share my thoughts on why the project failed and what lessons can be learned from this.

To summarize my participation in the project: I used Ubuntu Touch on Nexus 7 constantly and periodically from the moment of its announcement in 2013 and until December 2014, I started working on Click applications in December 2014, I started writing an article in 15 parts. ” Hacking Ubuntu Touch ”on the system in January 2015, was an insider on the Ubuntu Phone Insider program, received Canon's Meizu MX4, organized a contest for UbuContest application developers and was its sponsor, worked on bug-reports and applications until April 2016 and then sold or redid all my remaining devices Events in the middle of 2016 So I think that I can share some thoughts about the project, its problems and where we could work better.
')
Please note that this article does not affect the UBPorts project, which continues to work on the operating system of phones, Unity 8 and other components.

1. He did not hit the lucrative niche

Ubuntu for PC, laptops and servers has made this relatively easy. Almost all of these devices allow you to install any operating system that can work with hardware, so when Ubuntu appeared in 2004, its largest competitor (Microsoft) was very vulnerable. Windows had a bad reputation, a high price, and this system was devouring resources, so that Ubuntu could only be less annoying, cheaper, easier to install, and work better on older computers. And that's exactly what she did. Windows still has a bad reputation, even now it spies on users, and it is still quite expensive. So, Ubuntu Desktop was not needed then and there is no need to do much now to keep and increase the audience of users.

On the Windows server market, Red Hat and especially SUSE were perceived as too conservative solutions, too cumbersome and again too expensive. A Red Hat Enterprise subscription costs a few hundred dollars a year, and this subscription does not even necessarily include live human support. A fast-growing, less expensive alternative with some support from the industry and a huge number of packages in the repositories should have interested many, especially for cloud solutions. The fact that Ubuntu was chosen as an exemplary operating system for OpenStack also helped a lot.

But with mobile devices, everything is different. You cannot simply flash any operating system on your phone or tablet. Each device comes with a custom, specially prepared Android build. When Ubuntu announced its entry into the mobile market in 2013, neither Android nor iOS were vulnerable, unlike the situation on the desktop market. People called for the creation of a third alternative, not because Android and iOS had a bad reputation or some restrictions, or were inconvenient to use, but because they (rightly) feared the monopoly of Google. So the attack on Android and iOS was not as simple as Microsoft and Red Hat in the desktop and server markets.

I remember someone from Canonical saying that the project needed to capture about 1% of the mobile market in order to support itself. At the time, this meant selling about 11 million Ubuntu phones and a couple of million tablets annually. If you managed to earn at least one dollar / euro on software and services for each device, then more than a hundred developers would easily pay for it, which is a lot of money if used correctly. The company Jolla, which develops Sailfish OS, had about 120 employees at some point, I think, but there were marketing and support departments that Canonical already had available. But the sale of 11 million phones and a couple of millions of tablets over the years was a very ambitious goal, given that the number of users of Ubuntu Desktop was somewhere around 20-30 million.

• Opportunity number 1 to achieve one percent. Being so much better than the competition that you are becoming the standard and not even worry about any one percent. I think we all knew that this was impossible, especially it became clear after all the important services (WhatsApp, Google, Twitter, Instagram, etc.) did not even allow their applications to be cloned to run on Ubuntu devices. Canonical did not make its own Telegram client when the first commercial Ubuntu phones entered the market, there was no messenger at all. And this is in 2015, when everyone exchanges text messages all the time. Nobody wanted to pay the same money for an Ubuntu phone if it cannot do the same things as the same Android model, even if it is positioned as a “device for the developer”.
  
  
• Opportunity number 2 to achieve one percent. Positioned in niches with deep pockets. Canonical was too concentrated in the convergence niche, which was not interesting to a lot of people, at the same time it ignored all hackers, makers and people who ate surveillance from Microsoft, Google and the NSA. Few were willing to pay a premium price for a phone, which can turn into a stunted laptop when connected to an external display, but many were willing to pay a premium price for a Blackphone .

2. User inconvenience and distorted priorities

I want to be honest: after receiving the first several over-the-air (OTA) updates, I asked myself: “Will bq and Meizu, and especially their users, put up with this?” The phones slowed down, they needed to be rebooted regularly. Meizu MX4 overheated. The battery meter often showed false data. Mobile data did not work reliably, (national) roaming often did not work at all. The location service was very unreliable. The phone did not always ring the incoming call or you could not make an outgoing call because the UI hid the buttons. It was impossible to rely on the alarm clock. Bluetooth only supported audio devices, and later input devices, but no file transfer, even in basic form. WiFi could not connect to WPA Enterprise networks until the fifth OTA-5 update. It seems to me that at some point the audio player even started deleting files during their indexing. And so on.

The list of things that had to work, but did not work, is very long. To make matters worse, several times the bugs returned through several OTA updates, like a regression. During the existence of the project for phones / tablets, the number of Launchpad bug reports took off like I had never seen before.

The elimination of all these bugs was not the main priority, and the developers spent most of their time supporting more iron (Meizu Pro 5, bq Aquaris 10) and ensuring convergence. Until the last day of the project, the users with whom I spoke were not happy with the device. Only those who used the most basic functionality, like my father, who did not even have the data transfer function enabled and made one call every two days, were satisfied, because the device worked for days without recharging. However, buying a smartphone for 150 euros, and then not using the functions that make it “smart”, does not make much sense.


What the convergence should look like. Image: Canonical

I understand that there were not enough developers to fix everything at once, but instead of choosing to make a good phone OR a good tablet with convergence, we got devices that in reality could not do anything properly. The whole project was constantly accompanied by a kind of aura “These devices are for developers, it’s not necessary to fix everything quickly, we will win in the long run” - until the management realized the obvious things that all this is quite expensive and too much time is already lost. It was then that they began to reduce losses, transferred all the key developers in Snappy around October 2016, allowed phones and tablets to die a silent death and for a few months didn’t say anything to the public.

Yes, and I think that designers have held on to the idea of ​​Scopes for too long. Especially given that no one really could figure out how to use these Scopes on the desktop.

3. The devices were hard to get and they became obsolete.

I think we can all agree that actually getting the device was too difficult. I bought my first Nexus 7 in the store, and the Nexus 4 on eBay, but when the project really went on a serious pace, these devices were already outdated, it became harder to get them, and soon the official image builds stopped for them. The bq devices were at least sold all over Europe, but most often the page was marked “out of stock”. Get MX4 was almost impossible for anyone who did not participate in the program Ubuntu Phone Insiders. If people from the USA even managed to get it, then it did not connect to mobile networks at full speed.

For most of 2015 and 2016, if application developers wanted to get an officially supported device to test their applications, I did not know what to recommend.



On the other hand, the device that most were waiting for — the exceptionally high-performance Ubuntu Edge — turned out to be different. Devices bq were cheap, with a small amount of internal memory and only with support for 3G. The MX4 had a big screen, high speed and 4G, but nothing else, even the SD card slot was missing. The HDMI output required for convergence was missing on all official phones, and Miracast / Aethercast was not an equivalent replacement. Many thought that Ubuntu would reveal the full potential of their hardware, for example, FM radio on Aquaris E4.5 / E5, but this was not even in the plans, and without the source drivers for Android it was almost impossible to add such a feature.

Many also expected that their Ubuntu phones would initially be safer than Android, because there is open source and frequent updates. Obviously, this was not the case; Android drivers and software for a mobile transmitter still remained proprietary and insecure, while still having full access to the hardware. Few were aware of this.

4. Communication and marketing were rather chaotic and sometimes misleading.

I spent a huge amount of time every day trying to keep up with the development, but usually I didn’t even know what would appear in the next OTA and what would be removed. Mailing lists, IRC, Telegram channels, Launchpad, official websites, private conversations between developers, sprints, Ubuntu Online Summit - that was too much. And it's not even mentioning all the secret conversations in Canonical, when they wanted to keep the news secret, to ensure maximum media coverage at the time of the announcement.

Since many Canonical employees work from home and in different time zones, for me the situation only got worse. I remember trying to help with the bugs "When I press the power button, the phone wakes up only after a second" and "Fraudulent battery indicator." The only safe place was Launchpad, so people counted on it. But sometimes it's much more effective to just talk to someone for a minute before you actually publish something valuable to a bug report, or just to decide which side to approach the problem.

The person working on the source code of the kernel could be somewhere in Asia. The employee who is responsible for all Q & A could be somewhere in the USA. I was in Europe. Our working time did not really overlap. So some days I had to talk to a guy from Asia at 8:00 in the morning, before he left work, and then with a person from the United States at noon or at night, when the working day started there.


Advertised features bq Aquaris E4.5 Ubuntu Edition. Notice the lack of the words convergence, HDMI, FM radio, and many other things that people expected, but marketing did not give them that. Image: bq

I must say that I learned a lot from the marketing department, especially regarding “expectations and reality”. For example, many suggested that it goes without saying that Aquaris E4.5 / E5 and MX4 will have the convergence function with a later OTA update, but neither the manufacturers nor Canonical promised this when selling devices. Until the moment of the cancellation of the project, most people assumed that they could run the same applications as on the desktop (Firefox, SIP clients, etc.) and manage applications using apt-get, and it was here that marketing simply began to mislead . There were too many accents on the fact that “this is the same Ubuntu”, although in reality it is not. I can not remember how often I had to explain to random people on various support channels that Firefox would not start, and apt-get would break everything. Often people were very surprised to learn that Ubuntu for mobile is so different.

5. Too much emphasis on technical features that neither users nor application developers need.

I have a feeling that the announcement of a new and independent mobile operating system was a good reason for architects to say: “Yes, let's do it, but let's do it in the Right Way, and we will be better than the others.” Ubuntu should not just provide a graphical user interface, but an interface that will work on all devices and in all form factors. It does not just isolate applications from each other, as does the Linux and Android kernel, but implements a full-fledged sandbox with data and privacy protection. It will magically make operating applications not consume battery power. And so on. Whatever others do on the technical side, Ubuntu should do it in a better and more elegant way.


Not all of this made sense to me. The release of Unity 8 was necessary because Unity 7 depended on Compiz and was not very well suited to work on many form factors with swiveling displays, etc. But the only thing for Mir was to replace X.Org and SurfaceFlinger, so Unity 8 could use a single API on desktops and mobile devices. I am not an expert on graphics technologies and APIs, but it seems to me that at least from the point of view of “we do not have enough hands” to develop a completely new graphic server that no one else wants to use and that does not add anything special compared to existing alternatives , - this is what should be avoided at all costs. Especially if the user never sees the difference. Ubuntu Touch quietly used Android SurfaceFlinger until the end of 2013.

In my opinion, the same applies to the isolation model and the life cycle. If you complicate the design to save some battery power, and this complexity leads to a lot of extra work on implementing system services, but these services will never be implemented because your development team is too small, then your users and application developers will praise you for the fact that the devices work a little longer. Rather, they will endlessly complain about missing things. That is why the bug-report “Finish the implementation of high-priority background services” warmed up to 240 degrees — three years passed since the announcement of the project, but almost nothing was done in this regard.

Another good example is the planned message framework. You should have received one system application for all types of messages, be it Jabber / XMPP, SMS, Telegram or WhatsApp, and third-party services could produce plug-ins for their protocols. This framework was one of the main reasons why applications were not allowed to run in the background. So you could not just make a separate XMPP client that would receive messages in the background. But the message framework to which you were supposed to release the plugin was delayed and in the end never came out. Even Telegram client could not work in the background, it could only show pop-up messages, because Canonical convinced Telegram developers to change their server code (!) To support the Ubuntu Push Notification service.

Some key developers at Canonical really thought that Ubuntu is so important that all service providers change their server codes to support Ubuntu Push Notification, and this will solve the problem. No one except Telegram had even thought about it.

(By the way, I still think that the life-cycle model would not lead to a noticeable battery saving. If you think about it, the same and often more CPU cycles will always be used in it).

6. The life of application developers was too complicated.

A modern mobile OS is more than an OS. This is an ecosystem. And here, Ubuntu failed the most.

Ubuntu for mobile devices was fundamentally incompatible with any environment for program execution that existed before it. She could not run Android, Windows, X11 or iOS apps. You could not just recompile Android, Windows, X11 or iOS apps. Graphic system, system services, sandbox, a set of basic libraries - everything was different. It was even completely different from Ubuntu Desktop. You can say all day that “This is the same Ubuntu”, but if I can't even test my applications on the desktop, because Mir does not start there, then this is not the same Ubuntu and I need to develop for two different platforms.

Canonical came and made a whole SDK, an Integrated Development Environment based on Qt Creator, plus a cross-compiling environment, plus a completely new set of Ubuntu QML components. I really do not want to offend anyone here, but besides not using the existing code, the way it was done turned out to be extremely confusing and was a disappointment for application developers. Nothing worked, everything changed and broke constantly. Sometimes the SDK was broken for weeks. Versioning schemes appeared, but your application still didn’t work.

At some point, I had to rebuild and update my glmark2 application in the catalog, because I had OTA with updated Mir client libraries, although the OS claimed the same level of compatibility as before. Then it became clear that the versioning scheme simply ensures that the official method of writing the application is guaranteed to work, but the official method is just QML and HTML5. The glmark2 program interacted directly with Mir, like many others (for example, games using SDL). Applications in the directory could simply stop working if not checked and updated after each OTA. You can still run old Android applications on a modern Android smartphone, but here the Click application from last year may stop working after the next OTA, if you don’t keep track of it all the time. I remember a vivid IRC discussion at the end of 2015, during which several Canonical developers were perplexed by this fact and asked the SDK team how they believe application developers to work in such conditions.


Rendering bq Aquaris E4.5 with Panda Love, one of my apps on Click Store

I started as an application developer. Whatever I wanted to do, I had to start from scratch. Make a GUI? Only QML with Ubuntu QML components was supported, and QML cannot be called a well-established ecosystem with a lot of existing code and good tools. Just use one of the existing UI libraries? They are all designed to work with X11 or, perhaps, Wayland, and it took a long time until SDL and others received backends for Mir. Communication with hardware and system services? Because of the sandbox, it was difficult to directly access specialized Ubuntu services via D-Bus, most “standard” processes like NetworkManager could not be accessed from within the sandbox. Download anything in the background? To do this, contact the special Ubuntu download manager. Get event notifications off the phone? Only if you integrate everything with the Ubuntu Push Notification service.

That is why I had to start working with the most basic system. Then in January 2015 I wanted to make WiFi and Bluetooth scanners, but all the necessary API and system services just did not exist. Many of the missing API and system services never appeared.

All this made the platform extremely unattractive for most third-party developers. They did not see how their investment in creating another version of the application from scratch could pay off, especially given the small user base. I do not remember a single application that the “original” developer would download in the Click Store. Even Telegram client has developed Canonical itself.

So many of us created simple web applications or cloned existing applications. And immediately faced the problem that many applications rely on some kind of non-free online service with very unfavorable terms of use. Personally, I developed the BD Navigator, a clone of the Deutsche Bahn Navigator. I carried out the reverse development of their client-server protocol to the point where I could copy almost everything except the purchase of a real train ticket, but they embedded a small piece of cryptography, and the use of stolen cryptographic keys in Germany is illegal. I asked permission from Deutsche Bahn, they refused. In the end, the entire application has completely degraded to the level of a magnificent web container with bookmarks on their mobile web pages.

Exactly the same is true for WhatsApp, Twitter, Instagram, Google Plus, Google Drive and others. We can copy a lot of things, but service providers do not allow us to do that. It is said that WhatsApp requested a seven-digit amount just for accessing their API, and not for developing a full-fledged client. Instagram has closed its APIs so much that even the embedded Instagram Scope had to be removed. Google for many services does not have a public API.

(Also, a small number of open source tags would never be able to maintain the current collection of clones for all popular applications. It simply does not scale.)

7. It was not as open and did not enjoy such public support as intended.

Now I understand that this statement may be contradictory, and if you disagree, please keep in mind that these are exclusively my feelings. Maybe I'm in the minority. I have no idea.

Ubuntu for mobile devices should have become as open as the “normal” Ubuntu, but this did not happen.

• The source code of everything we developed was distributed somewhere over an unknown number of Launchpad projects.
• Kernel sources on GitHub were often outdated.
• The code for all proprietary Android drivers and other software was available only to Canonical employees.
• Canonical and its commercial partners had a completely private Launchpad zone with private bug reports. Quite often in public bug reports there were links to private pages, so you only had half the information.
• , paste.ubuntu.com .
• , , Launchpad Canonical . , Aethercast .
• Canonical, , , .
• , , Canonical. , , .
• I have never had the feeling that the wishes of users and the wider community have some influence on the list of future features or on what will be included in the next OTA. In many cases, the Launchpad bug reports and feature requests with the most support were delayed longer than others.

FAQ

Here are some questions that I sometimes get asked.

How many devices did you buy for development?

I think I bought two new Nexus 7, two used Nexus 4, three new bq Aquaris E4.5 and two cheap Chinese Mediatek phones (for reverse engineering) specifically for working with Ubuntu. I also had a MX4 from Canonical. I think I spent somewhere over a thousand euros on seven phones and two tablets.

Have you ever estimated your time spent?

Yes. After all the calculations, I had six man-months for a period of one and a half years or a workload similar to that if Canonical hired me at the 30% rate.

Do you regret spending so much time and effort?

Not

When did you first begin to doubt that everything will work out?

If I am not mistaken, it was in the area of ​​Christmas 2015. The big hype seems to have ended, and it became clear that we would never get full-fledged WhatsApp, Twitter and other applications, and for many phone owners there are not so many “really” important functions planned. There was work on convergence for the tablet, but few managed to get bq Aquaris M10.

Remember that I started working on the base operating system only because I could not create half of the applications I wanted. After a year of work, no API or system service needed by me was completed, and we still argued with some system architects about the necessity of all these things in principle. When your application developers say that you need something to build great applications, just give them something that you as an architect can get along with in the foreseeable future. It doesn't have to be perfect, but you need application developers more than you need them.

You left in the middle of 2016, long before the closure of the project. Why?

On the one hand, software development has become less interested in me. Most of the time now I travel the world , photograph, make bad card games, bad comics and bad games.

On the other hand, I no longer felt that working on a project brings me joy. Sometimes I thought, “I’m not doing enough, it’s my fault,” sitting on the device for eight hours in a row. Not such feelings should be in a person who is working on a project in his spare time for pleasure.