Three years ago, I came across an interesting research paper by Microsoft employee Kent Sullivan about the process and results of designing a new user interface for Windows 95. Since then, the web page has disappeared - one of the reasons why I am such a digital Plyushkin.

This article describes some of the common problems of the Program Manager shell in Windows 3.1 and discusses options for developing a separate shell for "newbies." I am inclined to believe that it was supposedly created in the spirit of Apple's At Ease program, which was quite popular at the time of System 7. I remember well how we ran At Ease in elementary school, so the kids did not have to mess around with the hard drive in Finder.

So, here is what Kent literally wrote in his article titled “Windows 95 User Interface: A Concrete Example of Designing Functionality” (The Windows 95 User Interface: A Case Study in Usability Engineering). We publish it so that the document is never lost.

---

Summary

A lot of people are involved in developing a user interface for a large commercial software like Microsoft Windows 95. This project has extensive tasks and an aggressive schedule. The project summary here describes the experience of successfully applying the principles of usability design, iterative development, and problem tracking to improve the manageability of the UI development process. It also discusses specific design problems and their solutions.
')

Introduction

Windows 95 is an extensive update of Windows 3.1 and Windows for Workgroups 3.11 products. Many changes have occurred in almost all parts of Windows. No exception and user interface. This article discusses the design team, its tasks and work process. It then explains how the project applied usability design principles, such as iterative development and problem tracking. The specific problems of design and their solutions are given as examples.

Design department

Windows 95 User Interface Development Team was formed in October 1992 at an early stage of the project. I joined the group in December 1992 as an assistant to provide usability services. The team was truly interdisciplinary, with specialists in design, graphic design, usability testing, and computer science. The number of employees fluctuated during the project, but on average we were twelve. Just as many programmers to implement the user interface.

Design goals

The design team worked on two very broad tasks:

• Make it easier to study Windows for people who are just starting to use computers and Windows.
• Make it easier to use Windows for people who have already worked with computers - both for ordinary users of Windows 3.1, and for advanced, experienced users of Windows 3.1.

With more than 50 million installations of Windows 3.1 and 3.11 plus the virtually untouched home PC market, it became clear from the outset that the task of releasing the best product would be a nontrivial exercise. Without careful design and testing, we will most likely make a product that will improve usability only for a certain category of users, but worsen it for millions of others (existing or potential). We were well aware of the problems of medium and advanced users, but we knew little about the problems experienced by beginners.

Design process

Considering the very broad tasks and aggressive work schedule before the release of the product (the user interface was designed and programmed for about 18 months), we knew from the very beginning that development using the traditional cascade model (“Waterfall”) would not provide us with enough flexibility to achieve the best solution. . In fact, we were worried that the traditional approach would lead to the creation of an unusable system.

In the cascade model, the design of the system is divided into parts (usually limited to the phase of writing specifications), and usability testing, as a rule, occurs closer to the end of the process, during quality control measures. We realized that this is not enough to create a design, check it on users (perhaps in comparison with other designs), make changes and collect feedback. Fortunately, our desire to abandon the cascade model in favor of iterative development coincided with similar attempts in other departments of the company, so we had specific examples of its advantages and applicability.

Iterative development in practice

In fig. Figure 1 outlines the process of our development. It is typical of most products that are developed iteratively: design ideas were tested in the form of prototypes on paper or in digital form to collect laboratory data on usability. After programming the design, he clarified in the laboratory. When a sufficient level of coding and refinement was achieved, the project was studied more widely, over time, in a combat situation. Minor problems in usability immediately corrected before the release of the finished product. More importantly, field data is used to direct work on the next version.

Our iterative development process took place in three main phases: learning, rapid prototyping, and fine tuning.


Fig. 1: Windows 95 iterative development process

Study phase

At this first stage, we experimented with different areas of design and collected the first user reviews. We started with a solid foundation of visual UI design, using the work done by the Cairo group. We inherited from them a significant part of the fundamental UI and interaction interfaces: desktop, tray (notification area), context menus, three-dimensional view, and so on.). We also received data from the support service about 20 major problems for Windows 3.1 users.

In fig. Figure 2 shows a prototype of the Windows 95 desktop design, the usability of which we tested in January 1993. The design is based on Cairo and includes the first pass fixes some of the known issues of Windows 3.1 (in particular, window management).


Fig. 2: Early version of the Windows 95 desktop (with callouts for clarity)

The top File Cabinet icon opened the look in the style of the Windows 3.1 file manager (on the left is hierarchy, on the right is content). The second World icon showed the elements in the network. The third Programs icon is a folder with other folders full of links to programs installed in the system. Along the bottom of the screen is a system tray with three buttons (System, Find and Help) and a file storage area. Another Wastebasket icon is a container of deleted files.

The usability studies of this desktop prototype were carried out in the Microsoft usability laboratory, as well as the subsequent tests. We conducted typical iterative usability studies. From three to four users, each representing a separate group of interest (usually beginners and average users of Windows 3.1) performed tasks on the prototype. During testing, we wanted to get answers to very broad questions (for example, whether users like the interface), and very specific questions (for example, whether the user will find it possible to copy the file within 10 minutes by dragging and dropping the mouse). We collected typical data for iterative research: verbal protocols, task time, number of errors, types of errors, and estimates.

First results

Usability testing of this prototype has brought many results, including some unexpected ones:

• Beginners and many average users confused the two-pane file manager interface (Fig. 3). They were not sure about the connection between the panels and how to navigate to other folders. Beginners are often struck by the visual complexity of the file manager - and they have experienced more basic problems, such as a lack of understanding of the possibility of the existence of some folders inside other folders. Many users were confused by the Parent Folder icon. It appeared in each folder and looked like a file, although in fact it is a navigation element to exit to the previous level of the hierarchy.
  
  
  Fig. 3: File Cabinet, an early version of the file system manager
• Users of all skill levels were confused by the Programs folder. We thought that having such a folder on the desktop with other folders and links to programs inside would become natural for Windows 3.1 users who are used to the program manager, and at the same time it will be relatively easy for beginners to master. We were wrong! Newbies quickly got lost in all folders (unlike File Cabinet, each folder opened in a new window), and other users had a lot of problems with trying to understand, they see the real file system and its files or just links to real files.
• Users had considerable difficulty in determining what each of the three buttons in the tray was for, and later they had difficulty remembering where to go to execute a particular command, because in certain contexts their functions overlapped (for example, to search for something in the help section). Help click Find or Help?)

Comparison with Windows 3.1

From the first laboratory experiments, it became clear that we need the Windows 3.1 base to better understand what problems existed before Windows 95, and which problems are unique to the new design. First, we collected market research data on the 20 most frequent tasks that Windows 3.1 users perform. Then we carried out several laboratory studies comparing Windows 3.1 and Windows 95 with regard to these 20 tasks. We also interviewed Windows 3.1 professional teachers (and Macintosh, for comparison) to understand which aspects of the operating system they consider simple and difficult to train users.

Key results:

• In Windows 3.1, newcomers needed an average of 9.5 minutes to search and open a program that is not immediately visible on the screen. In our prototype of Windows 95, the results were not much better. Such results are clearly unacceptable given the fact that market research data (and common sense) suggested that the launch of programs for users is the number one task.
• New and some average users experienced big problems using the mouse, especially double-clicking. As a result, they often could not find items in containers, access to which was opened only by a double click.
• Beginners and many average users relied almost exclusively on visual information to find teams. They relied on the menu bars and toolbars, but did not use pop-up (contextual) menus, even after learning.
• All but the most advanced users did not understand how to effectively manage the multitude of overlapping windows. Most of the problems have arisen for newbies - when they minimized the window, they thought that it “went away” if it was closed by another window. We heard a lot of stories from teachers (and watched it in the laboratory), as users exhausted all the RAM on their computer, running numerous copies of the program instead of switching to the first running copy. Average users showed more experience, but also experienced problems, especially with Multiple-Document-Interface (MDI) applications, such as Program Manager and Microsoft Word. Market research data confirmed the problem: it turned out that 40% of average Windows users did not run more than one program at the same time, because they experienced some difficulties with this.
• Novice users were confused by the hierarchical file system. Average users usually coped with the hierarchy, but often did it with difficulty and saved all their documents in the default directory for the program they used. This problem (especially for beginners) was also observed among Macintosh users.

Change of direction

The results of these studies and interviews greatly changed the design of the Windows 95 user interface. In the early prototype of Windows 95, we specifically changed some elements of Windows 3.1 (for example, the Desktop element has now become a real container), while others have remained unchanged (for example, the file manager and program manager icons on the desktop), because they were afraid to change the design too much. We knew that a product with radical differences from Windows 3.1 could confuse and disappoint millions of existing users, which is clearly unacceptable.

However, the data collected in the prototype of Windows 95 and in Windows 3.1, showed the impossibility of saving the current design. The results of novice users in performing basic tasks turned out to be unacceptably bad, and many average users expressed the opinion that Windows 95 is just another, but not the best system.

We decided to retreat and think over the situation for several days. The design team conducted a field meeting outside the office and reviewed all the data collected at that time: basic usability studies, interviews, market research, and information from the support service. During the discussion of the data, we realized that we need to concentrate on the most frequently performed tasks. We also realized that too much attention was paid to connectivity with Windows 3.1.

In fact, we realized that a viable solution does not have to look like Windows 3.1, but definitely should be attractive for users of any level, potentially for different reasons. We realized that a truly user-friendly system will scale in accordance with the needs of different users: it is easy to learn and learn, and at the same time it will provide efficiency (through shortcuts or alternative methods) for more experienced users.

Fast iteration stage

When we started working on a new design, we hoped to avoid the classic “easy to master, but difficult to use” paradox. We constantly kept in mind that the basic functions of the UI should be scaled. We knew that in order to achieve this goal, we needed to quickly test many different ideas, compare them and perform the repetition of those that seem most promising. This required very efficient design and evaluation processes.

Evolution of the UI specification process

Although from the very beginning we chose an iterative development process, we still have one element of the cascade method: the monolithic construction of design specifications (“specs”). In the first few months, specs grew rapidly and reflected hundreds of man-hours of work. But because of the problems found during user testing, the design documented in the specs is suddenly out of date. An important decision had to be made: spend weeks changing specifications to reflect new ideas and lose the valuable time required for iterations, or stop updating specifications and let prototypes and code act as live specs.

After some disputes, the group chose the second option. Although such a change in something made it difficult for outside groups to follow our work, it allowed us to carry out iterations at maximum speed. The changes also led to an unexpected effect: they consolidated the team more strongly, because most of the specifications existed verbally and were discussed in conversations and on a white board in the offices of the employees. A lot of “corridor” conversations followed, which continued throughout the project.

So that all interested parties are always aware of the current design, we organized the following events:

1. Regular meetings of the design department staff . At weekly (sometimes more often) meetings, everyone checked their work with a common project and everyone discussed how the work of each employee could affect others.
2. Submission of graphs and test results usability by e-mail . Design team members received regular notifications about upcoming usability tests and completed test results in order to keep abreast of how the design is progressing.
3. Formal tracking of usability issues . With the Windows 95 scale project, we understood that a standard way of recording identified usability problems was required, when and how they should be fixed - and then close the tickets after fixing the problem and successfully checking on users. This process is described in more detail in the “Tracking Open Ticket” chapter.
4. Regular design presentations for external groups . As the project progressed, more and more groups (inside and outside Microsoft) wanted to see what we were doing, so we held such presentations. They are more efficient than sending documents, because presentations are easier to keep up to date and they allow for timely discussion of the design.

Separate UI for beginners

The first major design change we reviewed was a separate UI (“shell”) for novice users. The design was quickly sketched in Visual Basic and tested in a usability lab (Figure 4). Testing showed a good result, because the design successfully limited the possible choice of user actions to a very small set of actions, but the more users participated in the testing, the clearer the limitations were manifested:

1. If only one necessary function was not supported in the shell for beginners, then the user had to refuse to use the shell (at least temporarily).
2. In theory, most users, after gaining experience, should leave the shell for beginners and go to the standard interface. But the experience that they received in the shell for beginners is not necessarily transferred to the standard shell.
3. The shell for beginners is not at all like all other programs launched by the user (text editors, spreadsheets, etc.). As a result, users had to learn two ways to interact with the computer, which caused confusion.

Fig. 4. Partial view of a separate shell for beginners.

For these and other reasons, we abandoned the idea. It is important to note that, thanks to the prototyping tool and immediate testing in the usability laboratory, we still had plenty of time to test other ideas.

Quick iteration examples

Below is an overview of the five areas for which three or more large iterations in the design were designed and tested. Iterations were used in many other areas, but there is not enough space to discuss them.

1. Launch programs: Start menu . Although we abandoned the idea of ​​a separate shell for beginners, we have retained its most useful functions: one-click access, good visibility, menu interaction. We outlined a lot of options in Visual Basic and checked them on users of all levels, not only on beginners, because this design solution should have been well received by users of any level. Fig. 5 shows the final version of the Start menu and the Programs submenu. This menu serves not only to launch programs, but also combines other functions. All of them are opened by pressing a single button.


Fig. 5. Menu "Start" and submenu "Programs"

2. Window management: taskbar . Our first idea to improve window management was not very ambitious, but we didn’t know how much work would be needed to solve the problem. The first idea was to change the appearance of minimized windows from icons to "tiles". (Fig. 6).


Fig. 6. Rolled windows in the form of "tiles"

We hoped that the problem could be solved if the minimized windows would look different and have a larger size. We were wrong! Users experienced almost the same difficulty as in the case of Windows 3.1. The test results showed that the main problem is that the windows are not constantly displayed, so that users do not see which windows are open and cannot quickly access them. Having understood this, we quickly came to the idea of ​​the taskbar shown in Figure. 7. Each task has its own place on the panel, which is displayed on top of all windows. Testing on users has shown that this is an acceptable solution to the problem.


Fig. 7. Taskbar with “Start” button, programs and clock

3. Working with files: the dialogs "Open" and "Save As ..." . Information from the support service and laboratory test results showed that novices and average users experience many problems with the system dialogs for opening and saving files (Fig. 8).


Fig. 8. File Open Dialog Box in Windows 3.1

The problems are caused by the fact that the dialog box fields are not in logical order and have a complicated selection methodology. The Cairo team took the lead in solving this problem and developed a comprehensive Visual Basic prototype, including a file system layout. We tested several options until we stopped at the final version, shown in Fig. 9.


Fig. 9. File Open Dialog Box in Windows 95

4. Printing: installation wizard . Information from the support service said that installing and configuring the printer is the main reason for calls from Windows 3.1 users. Many problems stem from the printer setup interface (Figure 10).


Fig. 10. Main printer installation dialog box in Windows 3.1

Finding the right printer is difficult, because they are all on the same long list. To select a port, especially in a networked environment, it was necessary to descend 4-5 levels with non-standard and complex choices. Around the time when we began to solve this problem, the design department began to consider the master (wizard) as a solution for multistage, infrequently performed tasks. The installation of the printer fits perfectly into this definition, and the wizard created showed good results in user testing. In fig. Figure 11 shows the printer selection screen from the final wizard.


Fig. 11. Add Printer Wizard Screen in Windows 95

5. Help: search dialog and index tab . Laboratory testing of Windows 3.1 showed that users are experiencing problems with the search dialog in the help section (Fig. 12).


Fig. 12. Search Help Dialog in Windows 3.1

People hardly understood that the dialog box essentially consists of two parts and that you must first select something from the first list, and then from the second, using different buttons. We checked a few ideas before we came to the final version of the tab with the index (Fig. 13). There is only one list on this tab, and keywords with more than one topic cause a pop-up window with a dialogue that is hard to miss.


Fig. 13. Index Tab in the Windows 95 Reference Section

Fine tuning stage

When we designed all the main product areas, we realized that we need to take a step back and see how all the pieces fit together.For this, final laboratory tests and a long-term study with real users were conducted.

• . 20 UI. . Windows 3.1. , [3]. — , Windows 3.1 20 21 Windows 95.
• . - Windows 95. , Windows 3.1, Windows 95. . , UI . Windows, , , .

In developing and testing the user interface of Windows 95, we applied various usability principles and practices [2] [4]. With the Windows 95 scale project, we understood that a standard way of recording identified usability problems was required, when and how they should be fixed - and then close the tickets after fixing the problem and successfully checking on users.

For this we have developed a relational database (Fig. 14).


Fig. 14. Sample records in the database with tickets

After each stage of laboratory testing, I introduced there new problems and positive results and assigned to each ticket the appropriate person in charge - usually both a designer and a teacher who trains users. The status of current problems was updated - the ticket was either left open if additional work was required, or it was closed if a problem was resolved. Every few weeks I issued a series of reports with a printout of all the remaining problems, for each responsible person, and distributed them to the staff (Fig. 15). We met to discuss current progress and decide when a redesign would be ready for user testing.


Fig. 15. A sample report in the database with tickets

Report

As in any project, practice is a criterion of truth, so I’ll give some summary statistics.

Laboratory tests

We conducted 64 stages of laboratory testing with 560 users. 50% of them had average experience with Windows 3.1; the rest are novices, advanced users and users of other operating systems. These figures do not include testing of components received from other teams (Exchange mail client, fax program, etc.). Testing of these components took place in about 25 stages with the participation of 175 people.

Troubleshooting

For the key components of the shell during the project, 699 “usability reports” were entered into the database. Of these, 148 positive results and 551 problems. The problems were assigned one of three levels of severity:

• Level 1: Users cannot continue a task or a series of tasks.
• Level 2: Users experience significant difficulties in completing a task or a series of tasks, but they are still able to continue its execution.
• Level 3: Users experience minor difficulty completing a task or a series of tasks.

Of the 551 problems identified, 15% received level 1, 43% level 2, and 42% level 3.

Resolutions on issues

During the project, five resolutions on problems were used:

• Solved. The team fixed the problem and successfully tested the solution on users.
• Scheduled. The team has developed a fix for the problem and we are awaiting its implementation.
• Questionable. The team is not sure whether to solve the problem, or does not know whether its solution is possible.
• Partially. The team developed a solution and it was tested on users with satisfactory results, but still some questions remain.
• Not decided The team is not going to solve the problem.

By the end of the project, all problems with resolutions “planned” or “at issue” went into one of the other categories. 81% of the problems ended with a successful solution, 8% remained “partially” with the resolution, and 11% remained unresolved. In most cases, the lack of a solution was caused by technical constraints, and sometimes work schedule limitations.

findings

For many people in the department, the Windows 95 project was the first experience with iterative development, usability testing, and problem tracking.

Iterative development

Perhaps the best evidence of our commitment to iterative development was that literally no part of the original UI design for Windows 95 was left unchanged in the final product. At the beginning of the design process, we did not anticipate the scale and amount of changes that would have to be made. Iterative development using prototypes and product as a specification, as well as continuous user testing, allowed us to quickly explore many different ways to solve problems.

The group was so accustomed to design iterations that we felt a rush when we had to perform some recent design work towards the end of the project. There was no time for iterations. We felt disappointed that there was no time to fine-tune and re-test the design.

Specification process

The “prototype or code are specifications” approach generally worked well, although over time we naturally refined the procedure. For example, all the prototypes of a specific release began to be shared with open access (for all employees) with instructions for installing and launching them.

The design team edited documents from the original specification and distributed them for early feedback. However, when prototyping and usability tests began, specs often sent readers to prototypes for up-to-date information. In fact, we found that the prototype is a richer form of specification, requiring less time to create. At the same time, he has other useful applications (usability testing, demo versions, etc.). The prototype stimulates more meaningful reviews, since the reviewer does not need to connect his imagination so much to understand the operation of the system.

Usability testing

Although the design and iterative tests allowed us to create separate functions, it was the usability testing of the entire product that became the key for accurate fitting of individual parts to each other. As already mentioned, on the basis of the collected data, we made changes to the wording of the UI and the topics of the reference section. If we hadn’t done that testing, then the product would not have looked so effective and enjoyable at work.

Problem tracking

A high percentage of corrected usability problems would not have been possible without the intense commitment of all team members. The base for problem tracking increased the process controllability and ensured that the problems did not get out of sight. However, corrections would not have been made if the team did not believe in creating a product of the highest possible quality. In this belief, the main thing was our understanding that we probably would not get the right result the first time - and that the wrong result is just as useful and important for creating a product as the right one.

All tickets marked “partially” and “not decided” were transferred to the new base. They became the starting point for designing the next version of Windows. Planning specialists and designers worked daily with this information, as well as analyzed reports from the support service.

Bibliography

1. Dumas, JS, Redish, JC (1993). A Practical Guide to Usability Testing (pp. 324-325). Norwood, NJ: Ablex Publishing Company.

2. Nielsen, J. (1993). Usability engineering . San Diego, CA: Academic Press, Inc.

3. Usability Sciences Corporation. (1994). Windows 3.1 and Windows 95 Quantification of Learning Time & Productivity .

4. Whiteside, JL, Bennett, J, & Holtzblatt, K. (1988). Usability Engineering: Our Experience and Evolution. In M. Helander (Ed.), Handbook of Human-Computer Interaction (pp. 791-817). Amsterdam: Elsevier Science Publishers, BV

5. Wiklund, ME (1994). Usability in Practice: How Companies Develop a User-Friendly Products . Cambridge, MA: Academic Press, Inc.