“I became a therapist or psychologist” - Vitaly Friedman about UX testing and mobile interfaces

If front-fighters have long known and highly valued Vitaly Friedman (co-founder of Smashing Magazine ), then mobile developers and testers can be far less aware of this name. But Vitali, who recently managed to take the first two places at the top of the HolyJS 2018 Piter reports , now got to the mobile and testing conferences with a new report “Testing Extremes: Tricks from Dark Corners of Mobile Interfaces”.

Therefore, on the eve of Heisenbug and Mobius, we asked him questions about UX testing and mobile interfaces: how this testing is carried out specifically in Smashing Magazine, and right up to smart phone users.

- The question is for those who are not deeply immersed in UX testing: we understand correctly that although it relates to testing, this is a rather peculiar and isolated area? The unit test is either green or red, and you have high matter here.
')
- Exactly. In the tests that I do, you cannot say "green" or "red." There is a lot of subjective things here: different users come with a certain amount of prejudice, and with skills learned on other sites. For example, a person can only go to the website of the exchange or to the news site. The way he sees the Internet and the interfaces is very different from us and from other people. Therefore, in such a context, it is very important to study the target audience of a particular site, see what other sites it visits, how it uses them, and what habits it has.

The most difficult thing is that from everything subjective that someone may like, but not others, select something objective in the interface, so that the idea reaches different users, everyone understands what and how to do. You have to search rather not what is done “well”, but what has a good recovery experience.

That is, even if a person comes with a different user experience and makes a mistake on the site, it is necessary that he can quickly return to a normal state and continue on his way, wherever he goes. This is the most important part that is difficult to fully provide with code coverage, for example.

So now I have a slightly different problem. I look at how you can change small details, for example, the text of adaptive errors. Suppose, if the payment with a card has not passed, it is necessary not only to inform about it, but also to say what exactly happened and how it can be corrected.

My task is to look at all these details, and automatic testing of such processes is impossible. You need to watch people. Therefore, I think that to some extent I became a therapist or a psychologist. I constantly look at people, watch them and draw conclusions based on how they look at the screen, where they click, where they don’t click, how their finger or mouse cursor moves. This is a slightly different side, more abstract, human.

- In the "usual" testing, humanity has already come to the fact that it is important and without tests anywhere. And in the case of UX-testing, where everything is more abstract, they often wave at it and do not pay enough attention?

- Very often. The problem is that for me, UX is always a large number of experiments. There is an idea - you have to try it and see how it will change the conversion or engagement. Often the problem is that we assume that something can be changed and immediately brought to light. But in UX the effect is often very long, long-term effect.

If you hang a pop-up and see how many people click on it - this is a short-term effect, it will be easy to measure. Of course, if there are serious mistakes, for example, when buying (a person does not understand what is required of him to make an order), this is noticed quickly. But some things only appear for a long time.

There are some things related to texts, for example, where it is not clear what a feature means, a product or its purpose: you need to give some time to see if people use this feature, come back to it or drop it, because initially they did not understand what she was doing. This can be measured by the metrics of click rates and engagement, but the problem may well not be that people do not need this feature, but that it is not clear how it helps the user or when to use it. Therefore, the only thing we do to systematize this process in some way is to regularly conduct user interviews with at least six users every Friday.

There is some version of the product, we have a continuous deployment, some features - we look, if a person understands what is happening in the interface. Is it faster for him to switch to paying for the order or to the information he needs? We look at the conversion and how a person understands what is going on in the interface in general, whether it is faster for him to move around in the checkout to the information he is looking for. This is the only way to carry out some kind of automation, which is in classical testing.

- IT giants are thoroughly suited to UX testing, using eye tracking and the like. But small companies may think: “Google is ready to set aside a special laboratory, but for us it’s something that goes into UX testing”. Without great resources here you can achieve results?

- Of course. There are a lot of resources, including free ones, which allow testing. Eye tracking can help, but in difficult situations. It can be used, for example, in the case of routes where people cannot find how to get to some rooms in a museum. In this case, it is useful to know where people look, when they go to the museum, and why they do not find what it would seem to be very simple to find. You can check in the interface where people look, what they pay attention to and what they see.

Heat maps are a fairly simple method, there are many tools for creating them. They analyze traffic and generate flow, you can record video for each user - where he went, where his mouse was, where it moved, where he clicked, where he returned, and so on.

On the other hand, there are funds that allow you to hire people from the pool of testers who receive money for each test and give them a task, for example, the site usertesting.com. This is much more expensive ($ 49 per video test), but you can choose the demographics of interest, give the person a task, and use the so-called “think-aloud protocol”. This means that when a person goes to the main page of the interface and goes from one to another, completing the task, he should say if he understands everything, what he sees, what this site is about, what he expects when he clicks the button. If you collect 6-7 such videos for mobile devices and 6-7 for the desktop, you can see a lot of patterns. Moreover, it is necessary to closely monitor where people are looking and how they behave with the mouse or with the touchpad.

Very often there are situations when someone does not understand something in the interface. People love buttons and any interactive elements. They begin to move in their direction, but do not click on it, but twist around the button with the mouse. And only then they click. This is one of the signs that a person is not entirely clear what will happen if you press a button. Such things can be easily detected without very expensive devices.

- In the case of other types of testing, there is an idea of ​​separation of duties: “here we will write the developers themselves with the unit-code, but the manual testers will take care of it”. And UX testing is whose concern?

- I think it depends on the structure of the company. In Western firms, there is often a UX-designer or even a usability tester, it is clear that they are doing it all day. But if the company is small, the responsibility will be with another person. If there is a front-end designer (who is closer to UX, not JavaScript), then he has. And any person who is involved in prototypes will have to carry out such tests, if there is no separate usability tester.

- And how is UX testing conducted specifically in the case of Smashing Magazine?

“Everything is very simple with us: as soon as new ideas appear, we collect 5-7 people, to whom I personally give assignments and conduct interviews.

Task - to find information or try to make a payment. I watch how they do it, with their consent, write down the behavior and face, and very, very slowly, I watch each video. It takes me an hour and a half to one video to see in detail how people move, use the mouse, where they look, what they notice and what they don't.

We also automatically generate a heat map: where people click, is there something they don’t notice. Based on this data, we are improving the interface and doing the same a week later. We have a small company, less than 20 people. In firms larger, it will look about the same, only the difference will be that in parallel there will be several tests going on at once.

And we follow not only what happens if you change a button, its location or text on a page. There are two types of tests: on the one hand, A / B testing - buttons and other small things. On the other hand, you can try to change something completely, it is called A / Z testing. For example, when we completely replace the design and test it.

It may have similar colors, typography, but it will be completely different in terms of organizing information, layout, with a completely different arrangement of all components. If we simply change the position of the button, we will most likely reach a local maximum, a local conversion, whatever the KPI would be. A completely different design can allow one to break out and achieve a completely different level of conversion, which is difficult to do with local small dimensions.

- Can you give a specific example, when you first did something in Smashing Magazine, and then, following the results of testing, you noticeably changed it?

- For example, when you enter a paid subscription with different capabilities. We had three tariff plans, we thought for a long time how much people could pay for each of them.

And then we tested and found that for many of our readers, these plans are completely unsuitable. They, especially people in large corporations, needed an annual plan: in order not to report each month, where they went $ 7– $ 9. People could not find the annual plan - although it was on request, but it was simply not found. As a result, we added the appropriate button, changed the interface a bit, and people began to notice it.



- Let us turn to the fate of mobile interfaces. Previously, you could often hear about how the share of mobile Internet is growing - and what about this in 2018, when smartphone sales stopped growing?

- In 2018, this continues. Moreover, I would say that this is a very important year, because this year we saw a lot of turning points.

No matter what we take - retail, consumption of music and video content, even radio - we see almost everywhere that mobile traffic has crossed desktop traffic. I would not say that desktop traffic drops dramatically, it just does not grow, but mobile traffic grows and very much.

Mobile traffic only now comes to India, Asia, Africa and South America - there is tremendous growth, much higher than in Europe or America.

And as a result, mobile traffic is very important, because for the first time this year mobile conversion exceeded desktop traffic. For many years, marketers have said that it is not worth investing in the mobile market, because there is little conversion. This year everything has changed - now there is nothing more important than mobile traffic.

It is not only about such things as phones, but also Apple Watch, any other gadgets, IoT. Refrigerators with internet connection can also show some information. Of course, it also deals with such things as a conversational interface and chat bots, which are primarily optimized for mobile use.

- Even years ago, a buzzword in the UI became “mobile first” (creating interfaces primarily for mobile devices) - does the current “turning point” mean that for many companies this is now becoming the main practice from buzzword?

- And there is. I am currently working with a firm that seriously thought about how they organize the entire corporation. Initially, they ignored the mobile market, and the desktop has existed for 15 years - this is a great e-commerce retailer. And suddenly there was a mobile market.

Of course, they did an optimization, responsive design. This is already good, but there are still Android and iOS applications. The result was that the absolute majority of developers and designers worked on the desktop site, while a small group worked on the mobile site. Now they are very much thinking about what they need to change, because they notice this trend.

The desktop is stable, in their case it does not grow, but the mobile market and traffic are growing very strongly. Therefore, they plan to change all of their architecture so that they will have everything to push off from the mobile first, some features will fall first only on mobile devices, and then on the desktop.

They had to think a lot about what it means for organizing a team, since a small team now has a very big responsibility. And they also thought about how to go to the mobile first, not only in terms of interfaces, but also how to make a “digital transformation” of the entire organization.

For them, as for many, this is rather not even mobile, but content first. It comes about how it is available to organize information on any form factor: watches, phones, tablets and so on. More attention is paid to the organization, filters, display of information in an accessible way, so that it is not too much for a person to use. Again, this is the moment when appropriate testing becomes very, very important.

- For the mobile version of the site and the desktop, is it worth doing significantly different interfaces, or rather something adaptive with minimal changes?

- I think that there are situations when you need to be very different. For example, there was an IKEA project that allows you to design the "kitchen of your dreams." Such a project cannot simply be “transferred” between the desktop and mobile devices, they have different capabilities. On the desktop, it looks like a regular desktop application, and on a mobile one, like an AR / VR application, where you can see the object right in front of you in 3D. This is a completely different scenario using the advantages of the device.

And for classic sites, I would say that the simplest thing is to create one site with minimal changes. This, of course, does not exclude the existence of applications. Moreover, now with the help of PWA (Progressive Web Apps) you can make the site an application that can be installed at the system level. But, in general, for most it would be most reasonable to have one site and code base, and use it as much as possible on different devices.

“The Smashing Magazine website is also a Progressive Web App, right?

- Yes, the site can be used offline, add yourself as an application, there is caching. But this is nothing special. And there are companies that are trying to create something like “offline e-commerce”, when a person can make a purchase offline. It's not easy to do, but the idea is good, because if you use PWA, you will have very good performance. This means that you can quickly place (or at least prepare) an order, without errors and constant access to the server.

How do we do this, for example, in the case of git, when I wrote a commit, and then it goes to the server, but all the changes happen first locally. The idea is similar, and I think it can work quite well. This is a big advantage of PWA.

- Smashing Magazine has atypical user statistics (for example, the share of browsers is not at all distributed over the Internet as a whole), so I want to clarify: what is your ratio of mobile visitors and desktop users?

- Three years ago we had 11% of mobile traffic, now about 22%. Honestly, it is very small. The fact is that we have very long articles. You need 40-50 minutes to read them. Read so much from the phone is quite difficult On average, the sites are now much more mobile traffic, but because of our features, we have not yet reached.

- Artemy Lebedev criticized mobile interfaces, saying "the bigger the screen, the better the interface." Do you have a feeling that with the transition to small screens it has become more difficult to make a good design?

- I think that making a beautiful design, on the contrary, has become more convenient. I am a big fan of any kind of restrictions. Each project needs some kind of restriction. When developing a concept, we often try to force ourselves to think outside the usual forms. We begin to design and invent design, setting ourselves limits.

For example, "in this project we can not use more than two colors." This does not mean that we will use only two colors, but it will lead us to an unusual solution. Or we say that we will not use triangles, circles and squares. And what can we use besides this? The size of mobile devices is a big limitation in itself. And it makes us come up with interesting solutions.

I think you can draw a beautiful interface anywhere. Of course, at some point there are difficulties. For example, chat bots do not have visual components in relation to colors or typography, you simply send text. But there can also be beauty in it.

The farther we go in the direction of the clock or to small devices or a conversational interface, the more visibility is lost, only the essence remains.It is important that the site, the company had its own core, which will remain, if you remove all the pictures, colors and typography. It makes us think about what is important and necessary for the brand, and focus its attention on this, let this core show itself. And now we are hiding behind good typography and design.

- And which of the interfaces, in principle, do not fit the mobile format? For example, tables that do not fit on the screen come to mind. Are there any other stumbling blocks?

- I even think that the tables do not fit, I doubt it. There are large complex tables where it is difficult to show everything at the same time, because there is no space. But the question is whether to show everything. Perhaps, our task as designers is to find an interface that does not display everything at once, but gives a person the necessary information in parts. We are accustomed to, that everything should be visible everywhere. Perhaps it would be worthwhile to find an opportunity to show a person what he needs, and not just to show the entire table.

It is quite difficult to work with maps, for example, and any kind of visualization. The places are quite small, and all the time you have to jump deep into the map and go out from there. By the way, at conferences I will tell you a little about how to work with this.

- And what can be said, on the contrary, about mobile interfaces that were unrepresentable on the desktop?

- Yes, such appear. I have already given an example with IKEA. Of course, there is WebVR, but to use it, you need some kind of device that you need to wear. I think one day there will be glasses that we will wear and see all the objects in front of us. But so far it has not come to this, we have huge devices that need to be pulled over the head.

The problem is that very often when we create an interface, we think mobile first or content first. We forget about the desktop, and it turns out desktop last. Therefore, some interfaces look rather strange on very large screens. Let's be honest, with Smashing Magazine such a problem. We optimized for mobile first, up to a maximum of 1800 pixels, and everything dissolves on large screens.

Usually this is just one central column for articles. Maybe the ads are left and right. And you can do much more. Very often I see interfaces where the mobile first is really, they look pretty good and are very comfortable on the mobile, but are completely lost on the desktop. Of course, you need to take into account that mobile is developing and conversion has exceeded the desktop, but the desktop does not go anywhere. All the same, it is significant. Therefore it is necessary to optimize and move in the direction of mobile first and desktop first.

- And the "HYIP" question finally. The theme of “bangs” on smartphones inevitably arises the last year and a half, and around this is a lot of excitement. What do you think, is it worth it to pay attention to it when you think about UX?

- We love to worry from scratch. Notice how people use the phone. Only 2% of people read the text in landscape mode, where the "bang" really hinders. And 98% read in portrait mode, where it does not interfere with reading at all.

It is important to know how people behave with devices and specific sites and on the basis of this information to correct the design. We use landscape mode, for example, when we play or watch a video. Therefore, game developers need to ensure that the "bang" does not interfere. But in general, most sites are completely unnecessary and not important.

: Heisenbug ( 6-7 ) Mobius (8-9 ). , , , — .