PixelTone: a prototype of a graphical editor with voice control
The University of Michigan, together with Adobe Research, created a prototype of a graphic editor, controlled by voice and gestures. The prototype, called PixelTone , is still able to do only the simplest operations and gestures. He understands simple commands and is oriented in descriptive terms like “top”, “bottom”, “brighter”, “darker”. In addition, he can memorize the names of objects and the names of people in photographs, which allows the use of commands like "Make Vasya a little more contrast."
Such a combined interface can be useful on tablets and phones, where there is simply no room for numerous palettes and menus, and there is no keyboard to use hot keys. The prototype will be presented at the Computer-Human Interaction conference which will be held in Paris from April 27 to May 2. In the meantime, you can download a PDF c technology description.
This document, prepared according to all the rules of a serious scientific publication, is perhaps more interesting than the prototype itself. When creating the interface, scientists analyzed the system of generally accepted abbreviations and annotations for photographs developed among professional photographers and retouchers, and then using Amazon Mechanical Turk, they collected statistics on words and expressions that non-professional users use to describe the desired changes in the photograph.
')
Usability studies conducted after prototyping showed that the voice interface allows you to perform basic photo editing operations just as well as traditional ones. The combined use of voice and touch makes it possible to accurately and confidently interpret fuzzy formulations of a natural language like “here,” “from here and above.” As in the case of keyboard shortcuts, it takes some time to study and memorize the phrases that the program understands, which may require graphical hints. Having mastered, the user can control the editor almost exclusively with voice and direct manipulation on the touchscreen.
Such a combined interface can be useful on tablets and phones, where there is simply no room for numerous palettes and menus, and there is no keyboard to use hot keys. The prototype will be presented at the Computer-Human Interaction conference which will be held in Paris from April 27 to May 2. In the meantime, you can download a PDF c technology description.
This document, prepared according to all the rules of a serious scientific publication, is perhaps more interesting than the prototype itself. When creating the interface, scientists analyzed the system of generally accepted abbreviations and annotations for photographs developed among professional photographers and retouchers, and then using Amazon Mechanical Turk, they collected statistics on words and expressions that non-professional users use to describe the desired changes in the photograph.
')
Usability studies conducted after prototyping showed that the voice interface allows you to perform basic photo editing operations just as well as traditional ones. The combined use of voice and touch makes it possible to accurately and confidently interpret fuzzy formulations of a natural language like “here,” “from here and above.” As in the case of keyboard shortcuts, it takes some time to study and memorize the phrases that the program understands, which may require graphical hints. Having mastered, the user can control the editor almost exclusively with voice and direct manipulation on the touchscreen.
Source: https://habr.com/ru/post/170075/
All Articles