LUWRAIN - distribution and work environment, in which it is not necessary to look at the screen
Today is the next Universal Accessibility Awareness Day (GAAD), in which it is customary to popularize assistive technologies and the availability of IT products for users with disabilities. However, during GAAD, accessibility (or accessibility) is in most cases referred to as a property of any software or web service, which is certainly important and deserves all attention, but it is not the only possible direction of application of the developers. In addition to ensuring the availability of interfaces for assistive technologies, there is such a direction as the development of these assistive technologies themselves: screen magnifiers, screen readers, specialized work environments, and others. It is about one such project, developing a specialized working environment for the blind and visually impaired under the name LUWRAIN, and will be discussed in this article.
A year ago, at GAAD-2015, detailed information about LUWRAIN was published, which outlined the general concept of this work environment, and also addressed the task of creating applications based on it that are adapted for people with visual impairments, therefore, everyone who is afraid of losing context , may preview the publication " Luwrain - a working environment for people with visual impairment ." This time, the discussion will focus on the technical tasks that were solved within LUWRAIN over the past year, and the prospects for further development, as well as readers, waiting for a story about how the LUWRAIN team treated Armenian cognac and Russian vodka to their colleagues from Canonical.
Like the last time, the head of the LUWRAIN project, Mikhail Pozhidaev , was directly involved in writing the material, so the article may well be considered the official version of the project’s history, since the most scandalous facts were cynically censored by him.
')
Recall that the main goal of the LUWRAIN project is to create a specialized working environment, mainly adapted to the specific needs of users with serious visual impairments. The idea is that the work of a blind person with the native graphical interface of operating systems and applications with the help of existing screen access programs is far from always and convenient in everything. There is a class of tasks that are quite realistic to solve in a special working environment more efficiently, and some categories of users directly need such a solution, since the native interfaces for them are overly complex, for example, this applies to many older people and people with simultaneous visual and hearing impairment. . And, in the end, everyone has the right to simply dislike the GUI, which, between us, fully applies to the main ideologue of LUWRAIN.
LUWRAIN proposes to achieve this goal by creating a specialized working environment, initially adapted to the specifics of visually impaired users (text-only interface, full keyboard control without a mouse, which is not accessible to blind people, guaranteed accessibility of all on-screen information, etc.). Inside the LUWRAIN environment, applications are implemented for solving basic tasks (surfing, working with mail, reading news, etc.), and there is also an API for creating additional applications by third-party developers, which allows you to write extensions without thinking about the task of ensuring their availability, since already solved at the level of LUWRAIN, which is like a framework for writing such applications.
LUWRAIN itself is implemented in two versions: as a Java application for various operating systems and as a Linux distribution with LUWRAIN as a working environment. As a result, the whole development of LUWRAIN is divided into several types of tasks:
These are rather diverse tasks that require flexible approaches to their solution. In some cases, it turns out to use practically ready open source libraries and only implement support for the LUWRAIN API for them, such as, for example, in the case of the mail client, in others, you have to write your own implementations from scratch, such as the implementation of interaction with MS Speech API, well and in the third there is the need for a serious refinement of existing solutions, as, for example, in the case of a browser.
LUWRAIN is, to a certain extent, an escapist solution, but not enough to ignore the need for basic functionality, which, among other things, includes work on the Internet. Nevertheless, a year ago, the implementation of an accessible for the blind web-navigation within the framework of LUWRAIN seemed to be an almost unsolvable task. That is why the possibility of using a parallel running Chrome browser with the ChromeVox extension intended to ensure non-visual work with web pages was considered. However, it was rather a desperate step, since the elegance of such a solution, frankly, was lame, and ChromeVox itself at that time had a number of fundamental problems when working with non-ASCII characters that have not been solved yet.
In mid-2015, it was decided to try to take the browser engine WebKit, compiled as a JNI-module. His perspective was that in fact he is a fairly self-sufficient component. That is, being compiled in C ++, it gives the same result on all platforms and does not pull heavy dependencies. It also has a Java interface, which is almost only JNI call translators. Plus, the engine itself and the graphic display unit are spaced apart, that is, the visualization object can not be created at all, and the engine will still work, which makes it quite a convenient solution for implementing an alternative interface browser in the spirit of LUWRAIN.
Obviously, after finding all these details, the idea of running Chrome along with forwarding calls to ChromeVox seemed even more unattractive, and it was decided to implement its own browser inside LUWRAIN based on WebKit. However, this task turned out to be far from simple due to the specific nature of ensuring the non-visual accessibility of web interfaces.
The fact is that blind users have a number of specific requirements for working with the web:
Firstly, just a sheet of text of a web page is not very convenient for non-visual work. It is necessary to provide a number of additional features, which are options for navigating the page’s structural elements (titles, links, tables, etc.).
Secondly, pages that are a priori developed with an orientation to visual perception do not always and conveniently look like text, for example, the navigation or information side columns for textual representation are often absolutely redundant text in the beginning or middle of the main content ( depending on their order in the original structure of the document). That is, it is necessary to provide various additional functions for the blind to adapt the page for reading (reading mode with the removal of unnecessary elements, the ability to customize the page reading when loading from a certain place, not from the beginning, etc.).
Thirdly, it would also be wrong to completely abandon the original presentation of the page with all graphic elements, since the system can be used by a visually impaired person who is capable of limited visual perception, and, unfortunately, blind people often have to ask sighted people to help solve the graphic CAPTCHA on sites.
All this requires a fairly serious refinement of WebKit for its use in LUWRAIN. In particular, it is necessary to implement on top of its own structure, which provides navigation through the elements of hypertext. Moreover, it is necessary to disassemble not just the source HTML, but already the DOM tree, because many interface elements appear only after JavaScript is processed.
There is one problem WebKit, which consists in the fact that the engine does not fully understand when exactly the page is fully loaded.
That is, the presence of a large number of scripts makes the moment of getting the finished page very blurry, which obviously affects the ability to adequately display it in the LUWRAIN interface.
The main master of this trend in LUWRAIN was Roman Volovodov rPman , who made all these things a reality.
To date, the work on the adaptation of WebKit is not completed and the browser LUWRAIN still can not boast of stability. This is still one of the key areas for the development of the working environment. For the near future, the minimum task is to obtain a truly stable and user-friendly browser. After this, it will be possible to think about some modularity for extensions, in particular, to tie in support for spamming APIs for CAPTCHA recognition.
Frankly, there are some doubts about the ability of the LUWRAIN browser to catch up on the functionality of non-visual work of existing solutions on Windows in conjunction with advanced screen reading programs, which today are industry leaders in the field of web accessibility for the blind. However, all this is still the subject of speculation and friendly holivar within the auxiliary technology industry, and it will be premature to make any comparisons before the first stabilization of the LUWRAIN browser.
As already mentioned, one of the options for delivering LUWRAIN is to build a standalone Linux distribution, since despite the fact that the environment is designed as a cross-platform solution, it is on Linux that its capabilities can be fully disclosed. The choice of the base distribution was one of the questions that arose at the stage of the project's formation, but in the end it was decided that they would be Ubuntu.
Of course, the base distribution is always quite a flame-forming moment, often affecting almost religious feelings, so it's hard to please everyone here. In the case of LUWRAIN, the choice is
Ubuntu was made mainly for reasons of the greatest mass and documentation. By the way, up to this point, the main developer was sitting on ALT Linux, which probably can serve as some protection against bias charges.
Anyway, in 2015, serious work began on the preparation of the LUWRAIN distribution, which on the very new year ended with the release of images of the beta version of LUWRAIN 0.5.0 based on Ubuntu 15.10 (Wily Werewolf). To date, version 0.5.2 of March 15, 2016 is relevant. All bootable ISO images that can be used to run LUWRAIN in live CD mode without installing on a hard disk, or for installing on a PC as a standalone OS, are available on the official project website .
There is another distribution kit in which LUWRAIN is used as a working environment along with Emacspeak - Homeros, released in early April 2016 . It is also based on Ubuntu 15.10, but is more focused on advanced users and solving development problems, including LUWRAIN itself.
In general, the future of LUWRAIN as a distribution is closely associated with Ubuntu, including with respect to the use of the entire ecosystem and others. By the way, it was the interest of the main developer of LUWRAIN to the Snappy mechanism that led to unexpected consequences.
At the end of 2015, Mark Shuttleworth answered some of Mikhail Pozhidaev’s questions about Snappy by mail, after which Mikhail inquired about the legal side of using Ubuntu, showing an article about LUWRAIN in the Linux Journal . In the end, it all ended with an invitation to the Canonical London office, where the LUWRAIN team did not come together empty-handed.
There is a legend , including a famous one in Great Britain, that at the Yalta conference, Stalin offered Churchill a glass of Armenian brandy, which the British Prime Minister liked so much that he became a fan of this drink. In subsequent years, deliveries of this cognac to Sir Churchill were controlled by Stalin personally, and when one day the quality of the product changed due to the arrest of one of the masters, Joseph Vissarionovich ordered it to be released and returned to work. Of course, after this, the LUWRAIN team could not fail to bring a bottle of Armenian brandy to British colleagues, which they handed over to John Lee and Richard Collins, saying that Churchill respected, and Mark, who was not in London at that time, was given a bottle of real Russian vodka.
Communication with these Canonical employees was not accidental. The fact is that Richard Collins oversees the telephone direction, and John Lee is responsible for its design.
Today, all major mobile operating systems (Android, iOS, Tizen, BlackBerry, Windows 10 Mobile) are accessible to blind users to varying degrees, but unfortunately, Ubuntu Touch remains completely unadapted for non-visual use. Ubuntu's desctop versions try to accommodate the needs of the blind, including screen-reading devices, but Canonical cannot offer anything in terms of accessibility in the mobile direction. Potential cooperation between Canonical and LUWRAIN in this area could offer the blind another option of affordable smartphones and tablet computers. Moreover, not in the form of repetition of already existing solutions, as it mostly happens in the already available mobile OS, but within the framework of its concept of adapted interfaces.
The international contacts of the LUWRAIN project also contributed to the beginning of the movement towards greater unification and standardization. In particular, work began on the implementation of support for voice servers Emacspeak and Speech Dispatcher . Earlier, in LUWRAIN, synthesized speech was output via the VoiceMan speech server, which is a separate development. This gave some advantages, but also created certain inconveniences for blind users, namely the impossibility of using those Emacspeak and Speech Dispatcher, which are more common and even de facto standards for many distros, including Ubuntu.
Many alternative assistive technologies, mainly the Orca screen reader, do not work with VoiceMan, so users had problems combining LUWRAIN simultaneously with other adaptive software on Linux, which was one of the criticisms of this working environment. Now, thanks to the implementation of support and other speech servers, LUWRAIN is becoming more flexible and capable of combined use.
In the near future, the plans of the LUWRAIN project include:
The Luwrain project develops in the tradition of free software and is completely open to the community. More information can be obtained on the official website . Existing LUWRAIN assemblies are also available for download, both as an ISO image and as a separate application for Windows and Linux . Those interested in the direct development of the project can familiarize themselves with the LUWRAIN SDK , with a complete list of Git repositories , as well as with nightly builds .
Additionally, you can familiarize yourself with a number of other materials about LUWRAIN, published last year and containing a more detailed description of certain aspects of the project:
A year ago, at GAAD-2015, detailed information about LUWRAIN was published, which outlined the general concept of this work environment, and also addressed the task of creating applications based on it that are adapted for people with visual impairments, therefore, everyone who is afraid of losing context , may preview the publication " Luwrain - a working environment for people with visual impairment ." This time, the discussion will focus on the technical tasks that were solved within LUWRAIN over the past year, and the prospects for further development, as well as readers, waiting for a story about how the LUWRAIN team treated Armenian cognac and Russian vodka to their colleagues from Canonical.
Like the last time, the head of the LUWRAIN project, Mikhail Pozhidaev , was directly involved in writing the material, so the article may well be considered the official version of the project’s history, since the most scandalous facts were cynically censored by him.
')
Main tasks
Recall that the main goal of the LUWRAIN project is to create a specialized working environment, mainly adapted to the specific needs of users with serious visual impairments. The idea is that the work of a blind person with the native graphical interface of operating systems and applications with the help of existing screen access programs is far from always and convenient in everything. There is a class of tasks that are quite realistic to solve in a special working environment more efficiently, and some categories of users directly need such a solution, since the native interfaces for them are overly complex, for example, this applies to many older people and people with simultaneous visual and hearing impairment. . And, in the end, everyone has the right to simply dislike the GUI, which, between us, fully applies to the main ideologue of LUWRAIN.
LUWRAIN proposes to achieve this goal by creating a specialized working environment, initially adapted to the specifics of visually impaired users (text-only interface, full keyboard control without a mouse, which is not accessible to blind people, guaranteed accessibility of all on-screen information, etc.). Inside the LUWRAIN environment, applications are implemented for solving basic tasks (surfing, working with mail, reading news, etc.), and there is also an API for creating additional applications by third-party developers, which allows you to write extensions without thinking about the task of ensuring their availability, since already solved at the level of LUWRAIN, which is like a framework for writing such applications.
LUWRAIN itself is implemented in two versions: as a Java application for various operating systems and as a Linux distribution with LUWRAIN as a working environment. As a result, the whole development of LUWRAIN is divided into several types of tasks:
- Creating a cross-platform kernel for LUWRAIN in Java;
- Development of internal applications based on the existing API;
- Writing platform-specific elements to run LUWRAIN on different systems;
- Build a Linux distribution with the highest integration of LUWRAIN into the system.
These are rather diverse tasks that require flexible approaches to their solution. In some cases, it turns out to use practically ready open source libraries and only implement support for the LUWRAIN API for them, such as, for example, in the case of the mail client, in others, you have to write your own implementations from scratch, such as the implementation of interaction with MS Speech API, well and in the third there is the need for a serious refinement of existing solutions, as, for example, in the case of a browser.
Adapt WebKit
LUWRAIN is, to a certain extent, an escapist solution, but not enough to ignore the need for basic functionality, which, among other things, includes work on the Internet. Nevertheless, a year ago, the implementation of an accessible for the blind web-navigation within the framework of LUWRAIN seemed to be an almost unsolvable task. That is why the possibility of using a parallel running Chrome browser with the ChromeVox extension intended to ensure non-visual work with web pages was considered. However, it was rather a desperate step, since the elegance of such a solution, frankly, was lame, and ChromeVox itself at that time had a number of fundamental problems when working with non-ASCII characters that have not been solved yet.
In mid-2015, it was decided to try to take the browser engine WebKit, compiled as a JNI-module. His perspective was that in fact he is a fairly self-sufficient component. That is, being compiled in C ++, it gives the same result on all platforms and does not pull heavy dependencies. It also has a Java interface, which is almost only JNI call translators. Plus, the engine itself and the graphic display unit are spaced apart, that is, the visualization object can not be created at all, and the engine will still work, which makes it quite a convenient solution for implementing an alternative interface browser in the spirit of LUWRAIN.
Obviously, after finding all these details, the idea of running Chrome along with forwarding calls to ChromeVox seemed even more unattractive, and it was decided to implement its own browser inside LUWRAIN based on WebKit. However, this task turned out to be far from simple due to the specific nature of ensuring the non-visual accessibility of web interfaces.
The fact is that blind users have a number of specific requirements for working with the web:
Firstly, just a sheet of text of a web page is not very convenient for non-visual work. It is necessary to provide a number of additional features, which are options for navigating the page’s structural elements (titles, links, tables, etc.).
Secondly, pages that are a priori developed with an orientation to visual perception do not always and conveniently look like text, for example, the navigation or information side columns for textual representation are often absolutely redundant text in the beginning or middle of the main content ( depending on their order in the original structure of the document). That is, it is necessary to provide various additional functions for the blind to adapt the page for reading (reading mode with the removal of unnecessary elements, the ability to customize the page reading when loading from a certain place, not from the beginning, etc.).
Thirdly, it would also be wrong to completely abandon the original presentation of the page with all graphic elements, since the system can be used by a visually impaired person who is capable of limited visual perception, and, unfortunately, blind people often have to ask sighted people to help solve the graphic CAPTCHA on sites.
All this requires a fairly serious refinement of WebKit for its use in LUWRAIN. In particular, it is necessary to implement on top of its own structure, which provides navigation through the elements of hypertext. Moreover, it is necessary to disassemble not just the source HTML, but already the DOM tree, because many interface elements appear only after JavaScript is processed.
There is one problem WebKit, which consists in the fact that the engine does not fully understand when exactly the page is fully loaded.
That is, the presence of a large number of scripts makes the moment of getting the finished page very blurry, which obviously affects the ability to adequately display it in the LUWRAIN interface.
The main master of this trend in LUWRAIN was Roman Volovodov rPman , who made all these things a reality.
To date, the work on the adaptation of WebKit is not completed and the browser LUWRAIN still can not boast of stability. This is still one of the key areas for the development of the working environment. For the near future, the minimum task is to obtain a truly stable and user-friendly browser. After this, it will be possible to think about some modularity for extensions, in particular, to tie in support for spamming APIs for CAPTCHA recognition.
Frankly, there are some doubts about the ability of the LUWRAIN browser to catch up on the functionality of non-visual work of existing solutions on Windows in conjunction with advanced screen reading programs, which today are industry leaders in the field of web accessibility for the blind. However, all this is still the subject of speculation and friendly holivar within the auxiliary technology industry, and it will be premature to make any comparisons before the first stabilization of the LUWRAIN browser.
LUWRAIN distributions
As already mentioned, one of the options for delivering LUWRAIN is to build a standalone Linux distribution, since despite the fact that the environment is designed as a cross-platform solution, it is on Linux that its capabilities can be fully disclosed. The choice of the base distribution was one of the questions that arose at the stage of the project's formation, but in the end it was decided that they would be Ubuntu.
Of course, the base distribution is always quite a flame-forming moment, often affecting almost religious feelings, so it's hard to please everyone here. In the case of LUWRAIN, the choice is
Ubuntu was made mainly for reasons of the greatest mass and documentation. By the way, up to this point, the main developer was sitting on ALT Linux, which probably can serve as some protection against bias charges.
Anyway, in 2015, serious work began on the preparation of the LUWRAIN distribution, which on the very new year ended with the release of images of the beta version of LUWRAIN 0.5.0 based on Ubuntu 15.10 (Wily Werewolf). To date, version 0.5.2 of March 15, 2016 is relevant. All bootable ISO images that can be used to run LUWRAIN in live CD mode without installing on a hard disk, or for installing on a PC as a standalone OS, are available on the official project website .
There is another distribution kit in which LUWRAIN is used as a working environment along with Emacspeak - Homeros, released in early April 2016 . It is also based on Ubuntu 15.10, but is more focused on advanced users and solving development problems, including LUWRAIN itself.
In general, the future of LUWRAIN as a distribution is closely associated with Ubuntu, including with respect to the use of the entire ecosystem and others. By the way, it was the interest of the main developer of LUWRAIN to the Snappy mechanism that led to unexpected consequences.
Contact with Canonical
At the end of 2015, Mark Shuttleworth answered some of Mikhail Pozhidaev’s questions about Snappy by mail, after which Mikhail inquired about the legal side of using Ubuntu, showing an article about LUWRAIN in the Linux Journal . In the end, it all ended with an invitation to the Canonical London office, where the LUWRAIN team did not come together empty-handed.
There is a legend , including a famous one in Great Britain, that at the Yalta conference, Stalin offered Churchill a glass of Armenian brandy, which the British Prime Minister liked so much that he became a fan of this drink. In subsequent years, deliveries of this cognac to Sir Churchill were controlled by Stalin personally, and when one day the quality of the product changed due to the arrest of one of the masters, Joseph Vissarionovich ordered it to be released and returned to work. Of course, after this, the LUWRAIN team could not fail to bring a bottle of Armenian brandy to British colleagues, which they handed over to John Lee and Richard Collins, saying that Churchill respected, and Mark, who was not in London at that time, was given a bottle of real Russian vodka.
Communication with these Canonical employees was not accidental. The fact is that Richard Collins oversees the telephone direction, and John Lee is responsible for its design.
Today, all major mobile operating systems (Android, iOS, Tizen, BlackBerry, Windows 10 Mobile) are accessible to blind users to varying degrees, but unfortunately, Ubuntu Touch remains completely unadapted for non-visual use. Ubuntu's desctop versions try to accommodate the needs of the blind, including screen-reading devices, but Canonical cannot offer anything in terms of accessibility in the mobile direction. Potential cooperation between Canonical and LUWRAIN in this area could offer the blind another option of affordable smartphones and tablet computers. Moreover, not in the form of repetition of already existing solutions, as it mostly happens in the already available mobile OS, but within the framework of its concept of adapted interfaces.
The international contacts of the LUWRAIN project also contributed to the beginning of the movement towards greater unification and standardization. In particular, work began on the implementation of support for voice servers Emacspeak and Speech Dispatcher . Earlier, in LUWRAIN, synthesized speech was output via the VoiceMan speech server, which is a separate development. This gave some advantages, but also created certain inconveniences for blind users, namely the impossibility of using those Emacspeak and Speech Dispatcher, which are more common and even de facto standards for many distros, including Ubuntu.
Many alternative assistive technologies, mainly the Orca screen reader, do not work with VoiceMan, so users had problems combining LUWRAIN simultaneously with other adaptive software on Linux, which was one of the criticisms of this working environment. Now, thanks to the implementation of support and other speech servers, LUWRAIN is becoming more flexible and capable of combined use.
Immediate plans
In the near future, the plans of the LUWRAIN project include:
- By the end of 2016, complete the development of the first stable version of the browser.
- By the beginning of the next academic year, create conditions for working within LUWRAIN with educational materials in a special DAISY format adapted to the needs of the blind. Of course, in the framework of the LUWRAIN project, we are talking only about creating tools for reading and possibly creating, and the relevant organizations should be responsible for preparing the content (correctional schools, libraries for the blind, etc.).
- Implement a mobile device accessible via the LUWRAIN environment with Canonical, which can be either a smartphone or a tablet.
- Move from the dead center the issue of integration with payment systems, that is, to realize the possibility of working with them in the LUWRAIN interface, since the use of web services is less convenient, and often frankly less accessible to blind users. Obviously, this is an extremely difficult area of work, which is caused by certain security requirements, but it is hoped to find some points of contact with relevant organizations.
Additional Information
The Luwrain project develops in the tradition of free software and is completely open to the community. More information can be obtained on the official website . Existing LUWRAIN assemblies are also available for download, both as an ISO image and as a separate application for Windows and Linux . Those interested in the direct development of the project can familiarize themselves with the LUWRAIN SDK , with a complete list of Git repositories , as well as with nightly builds .
Additionally, you can familiarize yourself with a number of other materials about LUWRAIN, published last year and containing a more detailed description of certain aspects of the project:
Source: https://habr.com/ru/post/283584/
All Articles