Luwrain - a working environment for people with visual impairments
On the pages of Habr, the Luwrain project has already been mentioned several times ( one and two times ), but it was a glimpse and without technical details. Since today is the Day of Universal Awareness of Accessibility , this is a good reason to fill this gap and tell in detail about what Luwrain is, what goals it pursues, what it is technically, and most importantly, what might be of interest to ordinary developers who have no with eyesight.
In this article, we will introduce the general concept of an accessible Luwrain environment, and also consider the task of creating applications based on it that are adapted for people with visual impairments. One of the authors of this article is none other than the coordinator of the Luwrain project, Mikhail Pozhidaev , so everyone has the opportunity to get information literally at first hand. Another author has already managed to become known as an opponent of Luwrain, so we hope that this mixture will provide a sufficient level of objectivity in the presentation.
Well, right away, so that readers are not distracted by unnecessary thoughts, we note that Luwrain is pronounced “lurein” and is a boy, that is, it is necessary to speak in the third person “he”.
Within the framework of the Luwrain project, work is underway to create a specialized working environment, mainly adapted to the specific needs of users with serious visual impairments. Serious visual impairment is understood as the inability to perceive information in a visual form from the screen, that is, it is either about total blindness or very severely impaired vision, but in the future, for simplicity, we will combine such users with the general term blind.
')
The project has three main objectives:
It is important to note that the fundamental problem of the availability of computer technology for blind people was solved a few decades ago, so Luwrain does not claim to be the discoverer's laurels. Today, a large number of absolutely blind people work at the computer and use it both at home and at work. Moreover, Luwrain is also not positioned as a complete replacement for the already existing technologies of non-visual access to a computer. His idea is that a number of accessibility issues can be solved differently and this may be more convenient for many visually impaired users, so they need to be given the opportunity to make this choice.
Ultimately, Luwrain does not supplant alternative technologies for non-visual availability of PCs. The meaning of the project is to blind users to offer an affordable interface to interact with the computer, and developers to provide the opportunity to implement available applications without any competence in the field of assistive technology. In the modern world, when most services go to the Internet, Luwrain can offer a simple, flexible and functional platform for implementing non-visual access interfaces to network services.
Luwrain develops the idea of ​​self-sounding environments, complementing the capabilities of traditional screen reading programs, which, in turn, work in the background and voice the native interface of the operating system and applications. That is, Luwrain is a separate application that does not provide accessibility of the operating system interface, but simply strive to solve the maximum number of user tasks within itself.
Emacspeak (a subsystem for the Emacs environment, which voiced its interface) had a significant impact on the development of the Luwrain concept, the strengths of which were given particular attention. In fact, it is Emacs that is the most vivid example of the viability of the text interface concept, and not only for the blind. Also among the projects close to Luwrain by the general idea - a special working environment for the blind, for example, such as:
That is, the basic principles underlying Luwrain, although they diverge from the mainstream assistive technologies for the blind, who today presumably presuppose the dubbing of a regular graphical interface, but are in themselves quite viable, as evidenced by the existence of a number of similar products, including commercially successful ones. . Nevertheless, Luwrain strives not only to repeat the path already passed, but to suggest further development of the idea of ​​self-sounded working environments, and here it begins to differ from the abovementioned analogues:
Firstly, Luwrain is written in Java, which immediately translates it from platform-level to cross-platform solutions. Potentially, Luwrain can be run wherever there is a JVM, except with some reservations, such as the need to meet certain minimum system requirements and the presence of information output interfaces that can be used to play speech synthesis or connect tactile displays. In particular, the work of Luwrain on computers running Linux, OS X and Windows is a reality today.
Secondly, the use of Java provides access to a huge amount of ready-made industrial-level code, which makes it possible to quickly and simply expand the functionality of the work environment by simply connecting to it the corresponding Java libraries that implement certain functions.
Thirdly, one of the main components of the Luwrain concept is the extensibility of the working environment. That is, in fact, it can act as a framework for creating accessible applications for blind users. This means that, unlike other specialized work environments, Luwrain does not lock the user into the rigid framework of ultimate functionality. Moreover, to create an accessible application inside Luwrain does not require any specific knowledge, so any private developer or company can easily prepare a Luwrain version of its application or a client for its web service.
Fourth, Luwrain comes in two forms: as a cross-platform working environment that can be run on top of all common operating systems, and as a standalone distribution based on Linux, which allows you to make a computer accessible to the blind From and To (including another stage of installation of the system).
Well, since with all this Luwrain is a non-commercial open source product that lives under the laws of the GPL, in total it guarantees three types of PC accessibility for people with visual impairments: financial, technological, and accessibility in terms of minimum user knowledge requirements.
In this article we will not delve into the deeply theoretical aspects of the non-visual presentation of information, although conceptually the ideas of Luwrain and traditional screen reading programs diverge quite strongly, but this is a separate conversation. This time we will focus on reviewing the opportunities Luwrain provides for developers, acting as a framework for creating applications that are accessible to users with visual impairments.
As already mentioned, the main development language of Luwrain is Java, although it is possible to write extensions in another language, you just need to implement support for a universal API. The basis of the interface is the search for methods of presenting and managing information in the form that is extremely adapted for the perception of blind people. However, the developer does not need any deep knowledge in the field of non-visual accessibility, as he constructs only the necessary functionality based on the Luwrain classes in almost the same way as he would do on the basis of Swing. All the specifics are limited by the need to know some rules.
To begin with, we will define a couple of basic concepts specific to the development under Luwrain.
An application is a software extension that runs in the Luwrain common platform environment. It simultaneously generates both speech output and text for visual presentation on the screen. In the future, there will be a third channel here - the Braille output, that is, the presentation of data on the connected tactile display, but at the current stage the Braille output in Luwrain has not yet been made. The developer must provide the implementation of the Application interface independently.
Application interface documentation.
An area is an abstraction of a control in an application's workspace. Except for the fact that the controls in the Luwrain environment have a textual appearance, there are no other fundamental differences from the GUI. The region acts both as a source of textual information and as a recipient of events from the system. An application workspace can consist of one or more areas. Unlike Application, for Area it is not necessary to prepare your own implementation, because Luwrain means a library of controls suitable for expansion and customization.
Interface Documentation Area.
When building the Luwrain application workspace, we essentially use textual equivalents of the usual GUI elements. It can be a list, editable text, a form with a set of fields, a table, a tree, and so on. For all this, the org.luwrain.controls package is responsible.
Documentation on the org.luwrain.controls package.
Here, for example, consider in detail the creation and customization of the list of elements. In the controls, the model (model) for storing content and the presentation (uppearance) are rendered into separate interfaces to refine the display of information to the user.
ListModel interface documentation.
Documentation for the ListItemAppearance interface.
In the list of controls (list), in essence, the following occurs:
Inside Luwrain, there are two groups of events: Keyboard Events (KeyboardEvent) and Environment Events (EnvironmentEvent).
Keyboards are keys. The mouse and other positional input devices are not used. This is due to the fundamental problems of the impossibility of convenient non-visual presentation of data on the position of the mouse cursor. The Touchpad can potentially be broadcast to events similar to pressing a keyboard shortcut, as, for example, implemented in VoiceOver on OS X.
Environment events pass general commands indicating their semantics, for example, closing, canceling, OK, clipboard operations, updates, and so on. There is also a special type of event that is used to organize the interaction of multi-threaded applications. You can work with the environment only in one stream, in which the environment itself works, all the other flows work in the background, but the system has mechanisms for their safe interaction.
Documentation on the KeyboardEvent class.
EnvironmentEvent class documentation.
Since a blind user is unable to cover the entire screen, but actually reads only the information under the cursor, perceiving it in a linear fashion, it is extremely important to ensure 100% availability of the entire content, making it focused from the keyboard. In traditional GUIs, this is achieved by special tricks, and in Luwrain, this is initially incorporated, so the developer doesn’t even have to think about it. Using the keyboard you can get to any part of the screen, that is, KeyboardEvent provides maximum control.
Currently, the Luwrain environment is in beta status. There are workable builds for Linux , OS X, and Windows , as well as an option in the form of a bootable ISO image . Over the past six months, the core of the medium has been substantially rewritten and is now close to stabilization. Version, current at the time of writing, 0.4.
Currently, active work is underway to create and develop internal Luwrain applications that provide basic functionality: a file manager, an RSS aggregator, an email client, and so on. In addition, platform-dependent blocks of the environment are being developed that are related to assemblies for specific platforms and ensure the launch and operation of Luwrain in the environment of one or another operating system. For example, this concerns speech synthesis, which is implemented on each platform in a special way through the use of its characteristic TTS API (for Linux, the VoiceMan speech server, for OS X — Speech Synthesis, for Windows — SAPI5).
Upcoming plans include bringing the Luwrain version with basic functionality to production ready status in order to release the first stable release. This will allow to run in the product not only on experienced users who are testing the beta now, but also on ordinary people, which will give additional feedback and understanding of what other aspects need to be refined to take into account the interests of the target audience.
Unfortunately, the very concept of a specialized environment, which Luwrain adheres to, has a number of difficulties in providing the user with some basic functionality. This is mainly due to those tasks that for some reason are difficult to implement inside Luwrain. For example, this applies to a full browser or things like Skype. And if the creation of a browser inside Luwrain is quite realistic due to the existence of a Java implementation of WebKit, then with the same Skype, due to the recent closure of SkypeKit, an internal implementation is impossible.
That is, the solution of some tasks in any case implies that the user exits the Luwrain limits and the corresponding functions inside the environment will not be implemented. However, since Luwrain, as noted above, is not positioned as a complete replacement for existing screen reading programs, this is not in the full sense of its drawback, but just a feature.
The Luwrain project develops in the tradition of free software and is completely open to the community. Anyone wishing to join the development project are invited to become part of the community. Work can be done in several directions: the development of the environment itself, the development of applications, the implementation of national standards of different countries (localization), the preparation of musical materials for registration, as well as any other constructive contribution that a person or organization will be prepared to make.
Luwrain source code is stored and constantly updated in several Git repositories located on GitHub . Their complete list can be found on the corresponding page of the official website of Luwrain .
If someone has a desire to participate in the project in one capacity or another, then get in touch!
In this article, we will introduce the general concept of an accessible Luwrain environment, and also consider the task of creating applications based on it that are adapted for people with visual impairments. One of the authors of this article is none other than the coordinator of the Luwrain project, Mikhail Pozhidaev , so everyone has the opportunity to get information literally at first hand. Another author has already managed to become known as an opponent of Luwrain, so we hope that this mixture will provide a sufficient level of objectivity in the presentation.
Well, right away, so that readers are not distracted by unnecessary thoughts, we note that Luwrain is pronounced “lurein” and is a boy, that is, it is necessary to speak in the third person “he”.
Project Goals
Within the framework of the Luwrain project, work is underway to create a specialized working environment, mainly adapted to the specific needs of users with serious visual impairments. Serious visual impairment is understood as the inability to perceive information in a visual form from the screen, that is, it is either about total blindness or very severely impaired vision, but in the future, for simplicity, we will combine such users with the general term blind.
')
The project has three main objectives:
- Provide the ability to work with the computer of those users with visual impairments who for various reasons could not do it before.
- Offer a reliable and productive tool for experienced users in solving a number of basic and specialized tasks.
- Provide a platform for creating and distributing applications adapted for people with visual impairments.
It is important to note that the fundamental problem of the availability of computer technology for blind people was solved a few decades ago, so Luwrain does not claim to be the discoverer's laurels. Today, a large number of absolutely blind people work at the computer and use it both at home and at work. Moreover, Luwrain is also not positioned as a complete replacement for the already existing technologies of non-visual access to a computer. His idea is that a number of accessibility issues can be solved differently and this may be more convenient for many visually impaired users, so they need to be given the opportunity to make this choice.
Ultimately, Luwrain does not supplant alternative technologies for non-visual availability of PCs. The meaning of the project is to blind users to offer an affordable interface to interact with the computer, and developers to provide the opportunity to implement available applications without any competence in the field of assistive technology. In the modern world, when most services go to the Internet, Luwrain can offer a simple, flexible and functional platform for implementing non-visual access interfaces to network services.
Luwrain concept
Luwrain develops the idea of ​​self-sounding environments, complementing the capabilities of traditional screen reading programs, which, in turn, work in the background and voice the native interface of the operating system and applications. That is, Luwrain is a separate application that does not provide accessibility of the operating system interface, but simply strive to solve the maximum number of user tasks within itself.
Emacspeak (a subsystem for the Emacs environment, which voiced its interface) had a significant impact on the development of the Luwrain concept, the strengths of which were given particular attention. In fact, it is Emacs that is the most vivid example of the viability of the text interface concept, and not only for the blind. Also among the projects close to Luwrain by the general idea - a special working environment for the blind, for example, such as:
- Adriane Knoppix is a non-commercial Linux distribution based on Debian, intended for blind and visually impaired users due to the rejection of the mandatory use of visually oriented output media. Created specially by Klaus Knopper for his blind wife, Adrian.
- Dolphin Guide is a commercial working environment for OS Windows, which has a simplified and articulated interface. Mainly positioned as a solution for older users.
- Mobile Accessibility is a commercial dubbed shell for Android OS, providing blind users with guaranteed accessibility to the basic functions of a smartphone. It was particularly relevant in the days of the emergence of special features of Android (up to about 4.1), when in fact it was the only tool to ensure the simple availability of a number of device functions.
That is, the basic principles underlying Luwrain, although they diverge from the mainstream assistive technologies for the blind, who today presumably presuppose the dubbing of a regular graphical interface, but are in themselves quite viable, as evidenced by the existence of a number of similar products, including commercially successful ones. . Nevertheless, Luwrain strives not only to repeat the path already passed, but to suggest further development of the idea of ​​self-sounded working environments, and here it begins to differ from the abovementioned analogues:
Firstly, Luwrain is written in Java, which immediately translates it from platform-level to cross-platform solutions. Potentially, Luwrain can be run wherever there is a JVM, except with some reservations, such as the need to meet certain minimum system requirements and the presence of information output interfaces that can be used to play speech synthesis or connect tactile displays. In particular, the work of Luwrain on computers running Linux, OS X and Windows is a reality today.
Secondly, the use of Java provides access to a huge amount of ready-made industrial-level code, which makes it possible to quickly and simply expand the functionality of the work environment by simply connecting to it the corresponding Java libraries that implement certain functions.
Thirdly, one of the main components of the Luwrain concept is the extensibility of the working environment. That is, in fact, it can act as a framework for creating accessible applications for blind users. This means that, unlike other specialized work environments, Luwrain does not lock the user into the rigid framework of ultimate functionality. Moreover, to create an accessible application inside Luwrain does not require any specific knowledge, so any private developer or company can easily prepare a Luwrain version of its application or a client for its web service.
Fourth, Luwrain comes in two forms: as a cross-platform working environment that can be run on top of all common operating systems, and as a standalone distribution based on Linux, which allows you to make a computer accessible to the blind From and To (including another stage of installation of the system).
Well, since with all this Luwrain is a non-commercial open source product that lives under the laws of the GPL, in total it guarantees three types of PC accessibility for people with visual impairments: financial, technological, and accessibility in terms of minimum user knowledge requirements.
Build Luwrain Applications
In this article we will not delve into the deeply theoretical aspects of the non-visual presentation of information, although conceptually the ideas of Luwrain and traditional screen reading programs diverge quite strongly, but this is a separate conversation. This time we will focus on reviewing the opportunities Luwrain provides for developers, acting as a framework for creating applications that are accessible to users with visual impairments.
As already mentioned, the main development language of Luwrain is Java, although it is possible to write extensions in another language, you just need to implement support for a universal API. The basis of the interface is the search for methods of presenting and managing information in the form that is extremely adapted for the perception of blind people. However, the developer does not need any deep knowledge in the field of non-visual accessibility, as he constructs only the necessary functionality based on the Luwrain classes in almost the same way as he would do on the basis of Swing. All the specifics are limited by the need to know some rules.
Basic concepts
To begin with, we will define a couple of basic concepts specific to the development under Luwrain.
An application is a software extension that runs in the Luwrain common platform environment. It simultaneously generates both speech output and text for visual presentation on the screen. In the future, there will be a third channel here - the Braille output, that is, the presentation of data on the connected tactile display, but at the current stage the Braille output in Luwrain has not yet been made. The developer must provide the implementation of the Application interface independently.
Application interface documentation.
An area is an abstraction of a control in an application's workspace. Except for the fact that the controls in the Luwrain environment have a textual appearance, there are no other fundamental differences from the GUI. The region acts both as a source of textual information and as a recipient of events from the system. An application workspace can consist of one or more areas. Unlike Application, for Area it is not necessary to prepare your own implementation, because Luwrain means a library of controls suitable for expansion and customization.
Interface Documentation Area.
Control Library
When building the Luwrain application workspace, we essentially use textual equivalents of the usual GUI elements. It can be a list, editable text, a form with a set of fields, a table, a tree, and so on. For all this, the org.luwrain.controls package is responsible.
Documentation on the org.luwrain.controls package.
Here, for example, consider in detail the creation and customization of the list of elements. In the controls, the model (model) for storing content and the presentation (uppearance) are rendered into separate interfaces to refine the display of information to the user.
ListModel interface documentation.
Documentation for the ListItemAppearance interface.
In the list of controls (list), in essence, the following occurs:
- The model stores a set of elements and does not care about the details of their presentation to the user.
- The getItemCount () and getItem () methods give access to a variety of items.
- The refresh () method notifies the model that the user has requested to refresh the contents of the list.
- The toggleMark () method requests a change in the selection state of an element and may return false if the model does not imply multiple selection.
- In the ListItemAppearance interface, the createItem () method must perform some sound action to inform the user that the cursor is positioned on the new list item. Luwrain is also actively using the accompaniment of sound signals, which carry a certain semantic load and allow you to convey information to a blind user faster than this can be done by speaking.
- The getScreenAppearance () method returns a string that should be displayed on the screen for visual presentation.
- The getObservableLeftBound () and getObservableRightBound () methods allow you to define a fragment of a line on the screen that the user can spell. This area is sometimes limited so as not to force the user to parse characters intended only for visual presentation.
Event handling
Inside Luwrain, there are two groups of events: Keyboard Events (KeyboardEvent) and Environment Events (EnvironmentEvent).
Keyboards are keys. The mouse and other positional input devices are not used. This is due to the fundamental problems of the impossibility of convenient non-visual presentation of data on the position of the mouse cursor. The Touchpad can potentially be broadcast to events similar to pressing a keyboard shortcut, as, for example, implemented in VoiceOver on OS X.
Environment events pass general commands indicating their semantics, for example, closing, canceling, OK, clipboard operations, updates, and so on. There is also a special type of event that is used to organize the interaction of multi-threaded applications. You can work with the environment only in one stream, in which the environment itself works, all the other flows work in the background, but the system has mechanisms for their safe interaction.
Documentation on the KeyboardEvent class.
EnvironmentEvent class documentation.
Since a blind user is unable to cover the entire screen, but actually reads only the information under the cursor, perceiving it in a linear fashion, it is extremely important to ensure 100% availability of the entire content, making it focused from the keyboard. In traditional GUIs, this is achieved by special tricks, and in Luwrain, this is initially incorporated, so the developer doesn’t even have to think about it. Using the keyboard you can get to any part of the screen, that is, KeyboardEvent provides maximum control.
Current project status
Currently, the Luwrain environment is in beta status. There are workable builds for Linux , OS X, and Windows , as well as an option in the form of a bootable ISO image . Over the past six months, the core of the medium has been substantially rewritten and is now close to stabilization. Version, current at the time of writing, 0.4.
Currently, active work is underway to create and develop internal Luwrain applications that provide basic functionality: a file manager, an RSS aggregator, an email client, and so on. In addition, platform-dependent blocks of the environment are being developed that are related to assemblies for specific platforms and ensure the launch and operation of Luwrain in the environment of one or another operating system. For example, this concerns speech synthesis, which is implemented on each platform in a special way through the use of its characteristic TTS API (for Linux, the VoiceMan speech server, for OS X — Speech Synthesis, for Windows — SAPI5).
Upcoming plans include bringing the Luwrain version with basic functionality to production ready status in order to release the first stable release. This will allow to run in the product not only on experienced users who are testing the beta now, but also on ordinary people, which will give additional feedback and understanding of what other aspects need to be refined to take into account the interests of the target audience.
Unfortunately, the very concept of a specialized environment, which Luwrain adheres to, has a number of difficulties in providing the user with some basic functionality. This is mainly due to those tasks that for some reason are difficult to implement inside Luwrain. For example, this applies to a full browser or things like Skype. And if the creation of a browser inside Luwrain is quite realistic due to the existence of a Java implementation of WebKit, then with the same Skype, due to the recent closure of SkypeKit, an internal implementation is impossible.
That is, the solution of some tasks in any case implies that the user exits the Luwrain limits and the corresponding functions inside the environment will not be implemented. However, since Luwrain, as noted above, is not positioned as a complete replacement for existing screen reading programs, this is not in the full sense of its drawback, but just a feature.
Participation in the project
The Luwrain project develops in the tradition of free software and is completely open to the community. Anyone wishing to join the development project are invited to become part of the community. Work can be done in several directions: the development of the environment itself, the development of applications, the implementation of national standards of different countries (localization), the preparation of musical materials for registration, as well as any other constructive contribution that a person or organization will be prepared to make.
Luwrain source code is stored and constantly updated in several Git repositories located on GitHub . Their complete list can be found on the corresponding page of the official website of Luwrain .
If someone has a desire to participate in the project in one capacity or another, then get in touch!
Source: https://habr.com/ru/post/258413/
All Articles