Jacob Nielsen: usability, expensive and cheap

Summary:

The calculation of the net current value (ITC, net present value, NPV) allows you to make an assessment of the most profitable level of investment in usability. The use of expensive usability methods can be economically justified in the case of large projects.

I usually advocate “ cheap usability ” - quick and cheap methods, the use of which leads to the immediate improvement of your user interface. But, sometimes, it makes sense to spend more to get more .



When evaluating possible options for using usability methods, it is very often necessary to choose from two alternatives, the cost of which can vary considerably. For example:

• Testing the interface with local users only or testing with the involvement of users from one or several foreign countries to assess the usability of problems with international use ;
• Laboratory user testing or field research at the customer;
• Testing only your own design or additional testing of 2–3 designs of competing companies (28 or 45 thousand dollars at the Norman Nielsen Group rates );
• Paper prototyping or development of a more detailed prototype;
• The choice of the level of knowledge and experience for employed usability specialists ( salaries vary from 55 to 80 thousand dollars a year for beginners. Experienced specialists, respectively, receive more).

To select the appropriate option, we use a combination of 4 parameters :

1. The expected value of the project;
2. The cost of each of the usability alternatives;
3. The estimated difference in the results obtained for alternatives;
4. The percentage depreciation of future cash flows, to account for the uncertainty of the discount rate). (

Calculation example

To illustrate my idea, I will give the following calculation example. First, suppose that the expected value (expected value) of our project is $ 1 million . For example, it can be an electronic store with an expected sales volume of $ 5 million per year and a profitability of 20%, or a redesign of an organization’s intranet with 10,000 employees with an expected productivity increase of $ 100 per employee per year.



Note: Now I am only considering the part of the value that will be realized only during the first year after the product launch. You can consider this as a first approximation, since the life cycle of a project is often longer. For example, the average time between intranet redesign is 3 years . If you expect your project to be relevant for more than one year, then of course you must also take into account the value introduced over the following years (do not forget to take into account the added uncertainty of future periods of time, reducing the current expected value by the chosen percentage of cash flow devaluation). For simplicity, in this article I will consider only the first year after launch.



Now, let's estimate the difference in costs and the expected gains from using two usability alternatives. Let, for example, a cheap approach costs 10 thousand dollars , and expensive - 40 thousand .

')

Let us also take such a very low-profile assessment that the results of the expensive approach will be only 20% better than the results of the cheap one. Since the use of usability tends to double the value of design projects , we will assume that the cheap approach will provide an increase in the expected results by 90%, and expensive - by 110%.



The increase in results by 20% is quite typical, for example, for the situation of transition from testing the interface with local users to testing with international ones. Even if you learn more by testing in more countries, most of your results will be the same for each country. It turns out that the increase in usability by 20% will cost you an increase in cost by 300%. Probably you will now decide that without any doubt you should choose a cheap approach, but wait - you should read further.



Finally, we set the percentage depreciation equal to 100% . Thus, everything that we will receive a year after the launch of the project will be worth exactly twice as much in today's money: a gain of $ 2 in a year will cost $ 1 in today's money. Such a huge percentage of depreciation can be justified by the fact that design projects have a high degree of uncertainty :

• First, the project may be canceled or fail for reasons beyond our control. The best design costs nothing if the application is never launched;
• Secondly, all numbers used by us in calculations are only estimates . We cannot say how successful the project will be in reality, and we do not know what results the user research will give us until we spend money on it. Some usability consultants offer firm pricing, but if you do your own research or you have an hourly consultant’s fee, then you cannot even be sure what the costs of implementing each of the alternatives will be.

Thus, we have 3 choices: do without usability, cheap usability, expensive usability:

• Option: Without usability . It is easiest to calculate, since we do not consider usability costs here. Take our expected value of 1 million and multiply by 100% depreciation. Thus we get the current value from the project: 500 thousand dollars, excluding the cost of the project itself. Thus, if the cost of design and development (excluding usability) is, say, 200 thousand, then the net current value (CTC) of the project will be 300 thousand dollars ;
• Cheap usability . Increases the value of the project by 90%, which eventually gives 1.9 million dollars, or 950 thousand, taking into account the depreciation. Now let's subtract 10 thousand usability costs and get the current value of 940 thousand. Thus, the use of usability has added 440 thousand dollars to the expected current value of the project. After deducting 200 thousand development costs, the total ITC of the project will be 740 thousand dollars ;
• Expensive usability . Increases the value by 100%, which gives 2.1 million or 1 million 50 thousand taking into account the depreciation. Now subtract 40 thousand and get the current value equal to 1 million 10 thousand dollars. This usability alternative has added 510,000 to the current value of the project. With deduction of development costs, the total ITC is equal to 810 thousand dollars .

The use of an expensive usability approach instead of a more thorough approach, taking into account all our assumptions, has increased the CTC by 70 thousand dollars , so in this case it makes sense to apply a more expensive approach.



(Our calculations assume that development costs are unchanged, regardless of whether or not usability methods are used. Simply put, I assume that we have correctly chosen usability methods and set the design direction from the very beginning of the project, so we don’t have to redesign developed functionality. This assumption is incorrect if usability is involved only closer to the completion of the project, in this case it is necessary to add the cost of correcting the detected usability defects.) Pus s, for example, the cost of correcting defects constitute 50 tysyach usability for cheap and expensive for 60 tysyach (Detected 20% more defects). Deduction of these costs does not change our final decision to use a more expensive approach, it only reduces the benefit from its use to 10 thousand dollars.)

Expensive or cheap: what to choose?

All these rather complicated calculations can be avoided if you use the five simple rules for choosing between expensive and cheap usability:

1. The higher the expected value of the project , the more should be invested in usability. Usability pays off by increasing the value of the project due to more frequent use (or more efficient use in the case of intranet). If the value of your project is small, then you will not earn much by doubling it.
   • For example, if the expected value of the project is 5 thousand dollars, then in this case they should not even resort to cheap usability, because a gain of 5 thousand will not justify 10 thousand costs. (Although, there are ultra-cheap usability methods that can be applied to such tiny projects.)
2. The higher the degree of uncertainty of the project , the higher the percentage of depreciation. In this case, it makes sense to choose usability on the cheap. Yes, a more expensive option may lead to a greater gain if the project succeeds, but the probability of total failure may be too high to justify such investments.
3. The earlier in the project , you begin to apply usability, the more you will have to invest, because you will be able to implement additional research results without significant rework of the product. If you start research in the later stages of the project, then make it cheap, since you will still not be able to make any significant changes.
   • Notice the contradiction between the phases of the project and the degree of uncertainty : at the very beginning of the project, the degree of uncertainty is higher (which speaks against the high costs), but the potential for implementing the changes is greater (which speaks in favor of more thorough research). Therefore, you should save more money in your budget for future research when you have the opportunity to test a more advanced design.
4. The higher the expected difference in results , the more you should be inclined to use expensive methods. If the expected payoff is small, then save your money by choosing usability on the cheap. A great example of this is quantitative research , they are very expensive and are usually justified only for giant projects, because otherwise they give a very small gain in results.
5. Finally, the most obvious: the more expensive the methods, the less often you should use them . In our example, the ETC of both alternatives will be equal if the cost of the expensive method is 110 thousand dollars. Thus, if the expensive method gives a gain of 20%, then it can be used if the cost of it is not more than 10 times higher than the cost of the cheap method. If the excess of costs is 11 or more times, then the use of the expensive method does not give any gain.

To select the appropriate usability method, it is necessary to take into account many different parameters, and many different scenarios. It makes sense to use both expensive and cheap usability methods in a suitable environment.



Original article