“Very handles”: make Tableau / Qlik from R and “blue tape”

It is a continuation of previous publications .

Naturally, the name is amusing, but, as is well known, in every joke there is some truth. The topic itself arose when, for the hundredth time, it was necessary to hear the insistent wish that a “flexible report / chart designer” is needed. After a certain point, it is easier to take and do than to once again explain that the tidyverse covers all necessary needs.

The very formulation of the problem is very simple: to provide a graphical interface for drawing various graphical representations from arbitrary tabular data. The classic solution consists of two related entities:

• interface with a large number of menus and buttons, with multiple behind-the-scenes IF for managing the mutual states of these elements;
• “Flexible plotter” with a large number of nested IF for drawing graphs in accordance with the fed data and the position of the slider buttons set in the UI.

On the one hand, doing “Yet Another Tableau” is completely uninteresting. On the other hand, staging in the style of “making it so that everything is, but nothing needs to be done” is a typical task for TRIZ.

In general, after a brief reflection, a solution was developed that almost satisfies the last formulation. The Shiny application itself is still under the NDA, a freely published prototype is shown in the picture.

Two key ideas for simplifying the tasks are the following (nothing new, everything has already been thought out before us):

1. instead of a statically defined UI, go to the dynamically generated UI;
2. We use the R interpreter not only for the source code, but also inside the code itself.

Idea 1. Dynamic web-interface

The option when all control elements are statically defined and only their parameterization changes (name, status, lists, selected elements ...) is convenient at the design stage. Everything is clear, everything is obvious, you can touch it with pens. But if the admissible states of these elements are very strongly connected both with the initial data for analysis ( data.frame ) and with the state of each other, we find ourselves in a situation of a very large number of non-trivial event handlers for each element. Lots and lots of confusing code.

Let's do it differently. Instead of UI elements with complex behavior, we uiOutput with uiOutput , which we dynamically calculate and generate with the help of the shiny::renderUI representation of this element. All external parameters required for element generation are treated as reactive elements (reactive). At the same time, all such interactive elements act as “autonomous agents” who look at the environment and adapt to it. The user changed the state of one element - all dependent began to recalculate their status in turn (we obviously do not process events, but use the shiny shiny approach). When their states change, new induced changes may occur. And so, until everything stabilizes.

  observeEvent(input$gen_plot, { #    escname <- function(x){ #     # ..... } point_code <- "" if(input$shape_type!="__NO_MAPPING__") { aes <- c("shape"=escname(input$aes_shape_col), "color"=escname(input$aes_color_col)) point_code <- buildPointCode(fixed=c("shape"=input$shape_type, "color"=glue("'{input$plot_color}'")), aes=aes) } line_code <- "" if(input$line_type!="__NO_MAPPING__") { aes <- c("linetype"=escname(input$aes_linetype_col), "color"=escname(input$aes_color_col)) line_code <- buildLineCode(fixed=c("linetype"=input$line_type, "color"=glue("'{input$plot_color}'")), aes=aes) } gcode <- glue("ggplot(data_df(), aes(x=`{input$x_axis_value}`, y=`{input$y_axis_value}`))\\ {point_code} {line_code} + xlab('{input$x_axis_label}')") %>% style_text(scope="spaces") plot_Rcode(gcode) }) 

Dependencies of elements can be very difficult, multi-step, but the equilibrium state is fast, the user does not notice all this. For an analyst-developer, this is all also hidden under the hood. To simplify life, we give the end user pictures of points and lines instead of numbers appearing in ggplot .

Idea 2. The reuse of the interpreter R

Many people like to poke into the fact that R is "slow because he is an interpreter." Omitting the groundlessness and unsubstantiatedness of such an assertion (in fact, no one wishes to go down in detail), we use this “weakness” as a force.

Instead of writing a complicated “flexible plotter” which, when generating graphics, will parameterize the process (welcome to Non-Standard-Evaluation!) Initial data and take into account all the nuances of the UI state, we will instead make the tidyverse generator of the R code dialect ( which during subsequent program execution (eval) will generate the required schedule:

  output$staticPlot <- renderPlot({ base::eval(parse(text=req(plot_Rcode))) }) 

Fin

Shiny prototype "a la Tableau" fits into 250 lines of code, including the UI part, comments and multiple validation (assertions) and costs 0 rubles 0 kopecks for licenses.

Happy 2018 year!

Previous publication - “R and Information Security. How to eliminate the contradiction of interests and run R on Linux in offline mode . ”