Two interviews with Rolf Molik

I present to you my translation of an interview with Rolf Molik, a renowned expert in the field of user interface quality assessment, published in UPA Voice. Link to the second interview at the bottom of the article.

Rolf Molick is extremely curious. He says that precisely curiosity led to the idea of ​​CUE - this is an English abbreviation of Comparative Usability Evaluation. This fairly well-known study examines the options for usability testing and expert assessments and the reproducibility of their results. However, Rolf’s activities are not limited to CUE.

early years

Rolf was always curious. Despite his technical education (he has a degree in software engineering), when Rolf met usability in the 80s, he immediately realized that what he wanted to do. Like a number of other pioneer practitioners, he recalls that he was actually doing the same thing, not knowing that this was the name: “I just talked to users. Nobody knew about usability testing then. ”

The union with Jacob Nielsen was fruitful. They both worked on a project for the Danish government. As a result of this union, a course of study, a book and a long friendship will be born.
')
Among those with whom Rolf collaborated in those years were Clayton Lewis, Jeff Raskin, Ben Schneiderman and Patricia Wright. Rolf was then an avid visitor to all conferences, especially when "the IT industry was so small that one conference could cover all the topics." Rolf then brought some speakers to Denmark.

After many years of cooperation with a number of different companies, Rolf "dared to become an independent specialist." In 1997, he founded Dialog Design. In addition to the “desire to focus fully on usability,” Rolf was also encouraged by the opportunity to have free time and independence, which can be gained by owning his own company. And this, as well as the desire to satisfy his own curiosity, led him to the first comparative usability assessment (CUE).

Cue

Rolf points out: “A lot of people were talking about how amazing usability testing is. I was inclined to agree with them, as I myself observed how powerful a political influence it has. But curiosity pushed me to the next question: "How reproducible are test results?"

He shared his thoughts with mailing list members and received the first four volunteer specialists who helped launch the first phase of CUE. In this study, the experts took a Windows application and each of them tested it as he deemed necessary, trying to find significant usability problems.

The results were presented at the UPA 1998 conference. In spite of the fact that “everyone found it interesting,” many specialists had some resistance, which from that time had become habitual for Rolf. “People said it simply could not be true. There can be no such significant difference in the results, ”says Rolf.

But there were differences. In the second study, Rolf found that of all the problems that nine independent teams found when conducting usability testing of the Hotmail mail, 75% did not repeat. Approximately the same result was in the first study.

In the third study, a different approach was taken. Instead of usability testing, twelve teams made an expert assessment of the car rental website (avis.com). Despite the fact that “the results of the third study were not published anywhere in detail”, it was used as a launching pad for the fourth.

The fourth study (the largest of all) was conducted in the form of a seminar at the CHI 2003 conference. There were 17 teams in total, half of which conducted usability testing, and the other half - peer review. Rolf says: “One of our goals was to check if there was a difference between these methods? Testing is much more expensive. But does it really give better results — better, more reliable, and more reproducible? We have found that this is not the case. ”

Rolf continues: “It was practically impossible to determine the difference in the results of the work of the teams. There was only one difference - the teams that applied peer review were a bit more productive. They spent less resources. For the rest, there was practically no difference. ”

The CUE-5 was held at the 2005 User Interface Conference. 13 teams explored a special tool IKEA, allowing users to configure the furniture themselves. This time, not all teams consisted of experts (on which Rolf relied in past research). However, "with the exception of some who simply did not have time to prepare a thorough report, some non-professionals worked as well as the experts from the fourth study."

The last, sixth study was also carried out at the UIC conference in October 2006. 13 teams explored another car rental website (enterprise.com). Rolf then discussed the results with the participants in a seminar held at the conference.

Preventive strike

Without a doubt, the most important discovery of these studies is that there is a clear discrepancy in the results of usability assessments. Rolf explains this with simple numbers: “If a non-trivial website contains 500 problems, then the usability test usually reveals about 40 more or less random problems, and it’s not surprising that you get new results every time.”

What is the way out? Oddly enough, Rolf sees no point in increasing the number of users or other methodological changes. “The solution is more radical,” he firmly states: “It consists in preventing problems. Most usability issues should not have appeared on the website right from the start. ”

But how to prevent mistakes? “Developers should own basic heuristics like the ones that Jacob Nielsen and I have been promoting for 30 years,” recommends Rolf. In fact, although many associate these famous heuristics with the name of Jacob Nielsen, they were first published in their joint work at the CHI 90 conference.

Usability testing

So why do we need testing? “I still think that usability testing is a very important method, but not for detecting usability problems. It is much more important that people realize the need to prevent usability problems. Usability testing is a very, very important political tool. This is an absolutely unique method for developers to realize that problems exist in their product. Your child is repulsively ugly, and something needs to be done. But the method is too expensive to catch absolutely all usability problems or even to catch the most serious of them. ”

Rolf points out another potential testing issue. Too often, developers do not implement the recommendations that have been agreed with them. And here again Rolf's curiosity worked: “Why is it often ignored what the usability experts said and demonstrated? Immediately after testing, everyone is enthusiastic, but then they [the developers] switch to other things, and the test results are abandoned. After all, it is very important not to forget that the problems of usability are not corrected by themselves. ”

Rolf has a hunch about the cause of this problem — it’s worth looking for in the testing reports. “At times, I’m shocked by what I see in professional test reports,” Rolf says in secret, “They are involved in usability, but the usability of their reports is completely unsuitable, which I think is not good at all.”

Rolf advises: “Make your presentations as intelligible as possible so that developers have no reason to ignore them.” In particular, Rolf recommends including a list of problems sorted by severity (from 30 to 40 points) in the report, as well as including a one-page output for decision makers in the report (“I saw a 10-page conclusion that, in my opinion, ridiculous "), and also do not forget to write about positive results (" By this you show that you not only criticize, but also praise ").

Future

Rolf goes to the main problem: “Our profession suffers from the fact that there are a number of people selling it in the guise of magic. Quality is a very important parameter in our profession. ” As a means of improving quality, Rolf calls “broad knowledge, experience, and inclination to practice in this subject area. However, I repeat once again that experience is very important. ”

Rolf continues: “Today, usability professionals must position themselves as professionals with knowledge. Our work is very specific. It is necessary to spend a lot of time on keeping up with the development of the industry as a whole. We need to develop certain qualities in ourselves and constantly improve them. ” Probably a special curiosity here also does not hurt.

Rolf also kindly answered my questions. The second interview is here: Questions for Rolf Molik for publishing GUI.RU

Article author: Cliff Anderson
Article translation: Alexey Kopylov

Original article: Thumbnail: Rolf Molich