If you look at the press, the phrase "digital economy" is expected to be one of the most popular in the next few years.

But in order to go from words to deeds and really make a digital leap, it is necessary to revise the approaches and tools used. As part of this publication, which is a continuation of previous publications , I plan to briefly illustrate the thesis that the use of ecosystems in business R fits in perfectly with the task of transition to a digital economy.

Why are new approaches and tools needed?

• Repeated increase in data volumes
• Repeated increase in data sources
• Repeated increase in exchange formats
• Work with unstructured data
• Shift focus from historical analysis to scientific prediction
• Focus on visualization and perception
• Repeated decrease in time for making a decision up to work in the “real time” mode

High Level Design (HLD) analytical system based on R

In the evolutionary development of various tasks, various techniques and modern open-source tools were revised. As a result, a fairly universal general-purpose stack has been formed, the general architecture of which is as follows:

Key components of the solution

• RStudio - analytical ecosystem (import, processing, visualization) based on the platform R
• Yandex ClickHouse - ultra-fast column-oriented database optimized for working with time data
• Appache Drill - a platform for providing unified SQL access to BigData & NoSQL data
• Appache Airflow - orchestrator
• "ETL" - a platform for receiving a variety of structured information in a relatively "clean" form using the Go language

Depending on the subject area, data types and scales, not all stack elements can be used. But whatever the task, the analytical core, as well as the face of the system from the user's point of view, remains R & Shiny, respectively.

Expected business exits from analytical system

As a rule, most people expect to see "reports", without specifying what they are investing in this word. Ecosystem R allows you to get a lot more typical expectations:

• generation of standard reports in the form of HTML (with interactive elements in the form of embedded htmlWidgets);
• generation of regular reports in the form of PDF;
• generation of various uploads in various formats for M2M interaction;
• interactive analytical applications (dashboards);
• elements of operational analytics (automatic changes to other IT systems based on the obtained calculations).

The environment of existence of all mentioned types of reports and AWS is Shiny Server \ Connect Server. In paid or free edition - depends on the requirements that are beyond the scope of analytics and are determined by the requirements for load, security, centralized management.

5 business arguments for the above HLD

1. Fast commissioning dates and minimum cost of ownership through the use of advanced proven open-source tools.
2. The widest range of functionality for import, processing and visualization.
3. Unified high-performance technologies for data of various data scales (millions - hundreds of trillions of lines \ gigabytes - petabytes of data).
4. The use of open public packages (> 10 thousand pieces), including in part:
   
   • algorithmic processing, including methods of machine learning;
   • visualization and creation of interactive analytical dashboards based on HTML5 + CSS + JS technologies.
5. Availability of “enterprise compliant” commercial versions available by subscription model for key open-source components.

PS
Practice over and over again shows that digital transformations do not rest against the possibilities of tools (open-source), but against the unwillingness of people to change their perception, learn new things, think strategically, or simply fear change.

An example of such a typical wish is the presence of a “visual” constructor, so that only a mouse, without any programming, can get the result of unlimited complexity. However, this is a beautiful requirement, cultivated by representatives of BI visualization, it is very poorly combined with the very content of the digital changes that humanity is expecting.

The paradox of this requirement is quite transparent. Everywhere using machines as assistants it is extremely difficult to communicate with them using a limited sign language or the dictionary of Ellochka-cannibals. Even from the theory of information it follows that with two or three clicks very little can be transferred, if only it is not a code of a thoroughly coordinated action in advance.

In the digital world, a programming language is becoming as important as an international language. Interestingly, in some Western companies, previously perceived as a classic production, programming becomes an important skill even for managers. A great example of this transformation is GE, a division of GE Digital . Video - Discover GE Digital: The Digital Industrial Company

Previous Post: RStudio Connect - Corporate Shiny Facelift