How to connect Java, Js and graphs with art, or the story of how to create an interactive theater
This year in Estonia is a series of theatrical performances, united in a series of "History of the Century." During the year, 22 theaters in the country presented to the public in their performances the centenary history of Estonia. In the course of the draw, the Russian Theater of Estonia was given the topic of the future of Estonia.
Under the cut - how to combine technology and art.
Each of us has an idea of ​​what the future could be: what we fear, what we dream of, what it may actually be. In order to avoid a performance that tells only a narrow circle of theater people who think about the future, we addressed this question to those who will live in the future - children, Estonian youth aged 3-19 years. There were hundreds of respondents, and we are very grateful to them.
')
Each answer received is unique, but, sorting the materials, we saw noticeable trends and patterns. From the children's answers loomed an incredible number of possible worlds of the future - and attractive, and terrible. But these worlds also had common features, on the basis of which the plot was built.
The performance takes us to the year 2118. Estonia is under a protective dome. We have learned a lot, including life extension. The main characters - an elderly couple Linda and Timo. A hundred years ago, being children, they fantasized about the future in which they now live. This is one of those very children, thanks to whom this performance turned out. Heroes fall into different worlds, in different Estonia. During the performance, it is the spectators who, by their decision, influence what their 2118 year will be.
The theater does not start with a hanger, but much earlier. After processing all the material, the main paths for the development of the future were identified according to the children, from the information technology path to the ecological state. Since the future is not predetermined, but depends on the choice of each of us, the possible paths of development were divided into stories, united by the history of the main characters and their families.
Artyom Gareev (director of the play) shows the main ideas on a model
The idea of ​​scenery is a constructor, which in the hands of children can take any form and function. He also has the form of a graph. A graph is a special abstract mathematical object that allows you to describe and model many real-world phenomena: the road network, the organizational structure, the Internet, and various choices that people can make. Elections that lead on different lines of the plot.
In addition to the dramatic task, there were considerable challenges from the technical side: stage lighting, a complex system of controllers that controlled hundreds of LEDs on the scenery, a variety of costumes for each variant of the future, skillfully performed by one of Estonia’s best theater artists Rosita Raud. The actors needed to change quickly after each vote, which was especially difficult when the intrigue remained until the very end of the vote and the next scene was unknown.
All music is played live by the MODULSHTEIN project.
We found ourselves in a multi-cultural society, where languages ​​and genders are intermingled. Everything is allowed.
A news release immerses us in the proposed circumstances.
Ironically adds that Elena Solomina - leading on national television and well known to many viewers.
Multicultural society is sheer fun, the younger Semenova leads to get acquainted with the parents of a huge boyfriend dressed in arafatka and a tutu.
After all, they go to a party.
At the end of the scene, a vote opens, offering a choice of whether Estonia will take the proposed path of multicultural development.
And if the answer is positive, then ...
Utopia falls into dystopia and freed freed up with a knot. There are new bans, and now the special forces break into the party and sort people. Some are allowed to pass to the right, towards the future, others are stripped and sent to the left, where their fate is decided by the machine of destruction. (based on articles by Nikolai Karaev and Boris Tukh)
In fact, it was another EMA actor (Est. Mama), artificial intelligence, which became a reality in the future. EMA leads the audience through the story, manifests itself in different variants of the future, and also announces and launches the vote, counts and presents the results, moves along the subject lines. The hall could accommodate 600 people. And each viewer should have the opportunity to vote, and the system correctly calculate his voice.
Possible options for the development of the plot, presented and modeled as a graph, formed a complex and branched structure.
Utopia tends to flow into a dystopia, and the road to hell is paved with good intentions. The plots are interconnected by the logic of drama. Given the number of options for the future that the children offered, a large number of branches came out. It is important to avoid repetitions - what was played should not be put to the vote again as a possible option. For example, if after viewing the Information and Technological Utopia, viewers confirm their choice of such a future, then we end up in a technological dystopia. If the audience rejects the proposed option, then they must again make a choice between the proposed options - for example, the Ecological and Cosmic-Colonial future. Very often there were situations when the hall struggled, the votes were divided almost 50% to 50% and everything was decided by a difference of several votes.
All this complexity required an appropriate structure, which is the graph.
What other functional requirements were there for the technical solution for the performance? We needed:
So we have 600 people who must vote in less than a minute. People should be able to cast a voice from their phone. The mobile application option disappeared immediately, because downloading the application is an extra effort. No one is downloading apps if you are not on Facebook or Google. Different models and phone types require different designs, which is unrealistic in the conditions of 1.5 months and one developer. Consequently, we need a Web application for a mobile phone.
Another Web application is required for system administration, startup, activation of states, and other technical issues.
And finally, the most important thing is how the EMA should interact with the hall.
And here it is necessary to introduce the concept of "super", which was new and for me. Super is a screen that goes down and closes the stage.
Lift curtain
Also known as “guillotine curtain” or “German” curtain. More often used in musical theaters. The curtain is a whole canvas, straight or pleated, fixed on the upper edge to the carrier beam. Lifting the curtain occurs simultaneously with the lifting of the carrier beam. Such a curtain requires a great height of space above the stage.
(Wikipedia)
The curtain is reflective and allows you to display various projections, such as news, about which will be below, or the EMA screen, which displays information related to voting, its results and the choice made.
So, we have a complicated graph, a voting system and three FrontEnd systems that must interact with the audience and the administrator (an incredible assistant director Svetlana Shushina).
So let's start with the graph. The data structure, in this case variants of the history of Estonia, must be stored somewhere. Standard relational databases are not suitable for graphing for several reasons. For us, the most essential was the flexibility of the stored data, transitions, links to audio files depending on the choice, and so on.
The definitive choice was Neo4j, the world's leading graph database system. 7 of the top 10 technology companies in the world use it. This is a very mature technology. Speed, convenience and excellent community leave no other choice in this kind of solutions.
We modeled the plot, built by dramatists, in the form of a graph on Neo4j. And here is a piece of plot twists and turns, as it looks from the inside Neo4j.
One of the advantages that Neo4j gave was flexibility and the lack of a scheme, which allowed to change the plot lines without any serious consequences, add new audio options, link various novels to each other. This is the native representation of the graph in the native environment.
In terms of the server side there were some doubts - Python vs Java. The choice leaned in favor of Java. The reasons are speed, less risk to break something with clear requirements and straight arms, and finally Spring Boot, which supports Neo4j out of the box.
The server part is implemented on Spring Boot. In fact, the server collects votes, switches from “super” to the following states when the voting is completed, accepts commands from the admin, gives statistics and so on. Plus gives all the options FrontEnd-am, and also registers the audience.
Now begins the most non-trivial part of the task. The fact is that according to the idea of ​​the component that is displayed on the "super" should talk with the audience: prepare to vote, suggest options, start and end voting, announce the results and, depending on the choice, suggest other options, or go to the choice and tell the introduction to the next scene. Thus, the audio must be dynamic and prepared on the fly. Given that the number of options is enormous, and the length of each segment of audio is different, synchronization is important (for example, voting should open only after the announcement “Voting is open”) and this was not a completely trivial task.
The system has two control centers.
The first is the server. It stores the current state of the plot, collects votes, gives statistics data, controls the ability to give only one vote from one person.
The second is “super”, which turns the voting on and off, shows the voting process, and so on.
"Super" is implemented on Vue. Initially, we wanted to call audio with the Google Speech API, but we could not guarantee the absence of network delays. Therefore, another option was chosen - we recorded the response of the Google Speech API piece by piece.
In total, we got 55 pieces of audio. Information about which audio files should be included depending on the current state, we stored on the graph. Since we are dealing with JavaScript, we stringed audio files and command calls at each other, like beads. Thus, we did not depend on the fact that audio files of various lengths, and got a fairly universal solution.
Initially, the mobile client (web page) was written on Vue, but we got partial incompatibility on some phone models. We tried the babel transpilers, but because of the heap of settings, the volume of the page grew, and although voting began to work on some telephones, it broke down on others. As a result, the mobile client was rewritten on Vanilla JS.
In essence, this was a distributed system.
The final volume of the system (jar file) was 146 megabytes. The telecommunication company Telia provided us with a virtual server at a symbolic price of 1 euro per month, for which she is very grateful.
The pace of work before the premiere all accelerated. At the premiere of the expected president. The last bug was detected in half an hour, and there was no time for testing, so the first performances went with parallel testing locally on the machine behind the scenes, so that in case of an error it was possible to draw up a plan of action.
Of the interesting moments, we can highlight the fact that we showed the names of the last voters on the screen and used the Vue template animation for this. At the first submission, we were given a killer character, which broke the animation and display of Vue templates. Subsequently, we abandoned this solution and switched to the good old jQuery.
But a piece of voting taken during the performance.
The performance has received a huge amount of positive reviews in the press.
https://rus.postimees.ee/6141942/budet-ne-budet-net-nichego-opasnee-budushchego
https://rus.postimees.ee/6147704/nashe-zavtra-v-zerkalah-utopii-i-antiutopii
At the Drama Festival in Tartu, the standing ovation lasted almost 5 minutes.
It was an incredible adventure due to the crazy energy and dedication of the people involved in the project. A huge bow to all the people who participated in the creation of this show.
Director and set designer - Artyom Gareev
Composer, producer of the project - Alexander Zhedelev
Visual - Alena Movko
Costume Designer - Rosita Raud
Lighting Designer - Anton Andreiuk
Choreographer - Olga Privis
Playwrights: Karin Lamson, Marie-Liis Lille, Elena Chicherina and Laura Kalle
Video - Nikolay Alkhazov
Programming and creating applications - Alexander Tavgen, Anna Agafonova
Technical solution ice shine - Aleksander Sprohgis
Animation - Martin Yakush
Band - Modulshtein
Martin Altrov - clarinet, bass clarinet
Alexey Semenikhin– work with samples, effects
Alexander Zhedelev - guitar, synthesizers, ableton
Vocal teacher - Anna Dydina
Assistant Director Svetlana Shushina
In the play are busy:
Natalia Dymchenko, Alexander Zhilenko, Daniil Zandberg, Dmitry Kordas, Ekaterina Kordas, Alexander Kuchmezov, Victor Marvin, Natalia Murina, Elena Tarasenko, Eduard Tee
Sergey Furmanyuk, Leonid Shevtsov, Elena Yakovleva
Russian Theater Studio: Denis Volkov, Pauline Grinyova Nina Zagvozdkin Anastasia Kalyada Natasha Christensen, Anastasia Masalova Sandra Minosyan Sofia Mikhalev, Catherine Magi, Michael Pashchuk, Catherine Selyugina Christina Sorokina, Sofia Strömberg, Ilya Sutta.
Under the cut - how to combine technology and art.
Each of us has an idea of ​​what the future could be: what we fear, what we dream of, what it may actually be. In order to avoid a performance that tells only a narrow circle of theater people who think about the future, we addressed this question to those who will live in the future - children, Estonian youth aged 3-19 years. There were hundreds of respondents, and we are very grateful to them.
')
Each answer received is unique, but, sorting the materials, we saw noticeable trends and patterns. From the children's answers loomed an incredible number of possible worlds of the future - and attractive, and terrible. But these worlds also had common features, on the basis of which the plot was built.
The performance takes us to the year 2118. Estonia is under a protective dome. We have learned a lot, including life extension. The main characters - an elderly couple Linda and Timo. A hundred years ago, being children, they fantasized about the future in which they now live. This is one of those very children, thanks to whom this performance turned out. Heroes fall into different worlds, in different Estonia. During the performance, it is the spectators who, by their decision, influence what their 2118 year will be.
The theater does not start with a hanger, but much earlier. After processing all the material, the main paths for the development of the future were identified according to the children, from the information technology path to the ecological state. Since the future is not predetermined, but depends on the choice of each of us, the possible paths of development were divided into stories, united by the history of the main characters and their families.
Artyom Gareev (director of the play) shows the main ideas on a model
The idea of ​​scenery is a constructor, which in the hands of children can take any form and function. He also has the form of a graph. A graph is a special abstract mathematical object that allows you to describe and model many real-world phenomena: the road network, the organizational structure, the Internet, and various choices that people can make. Elections that lead on different lines of the plot.
In addition to the dramatic task, there were considerable challenges from the technical side: stage lighting, a complex system of controllers that controlled hundreds of LEDs on the scenery, a variety of costumes for each variant of the future, skillfully performed by one of Estonia’s best theater artists Rosita Raud. The actors needed to change quickly after each vote, which was especially difficult when the intrigue remained until the very end of the vote and the next scene was unknown.
All music is played live by the MODULSHTEIN project.
To make it a little clearer, let's get into the context of the play a little.
We found ourselves in a multi-cultural society, where languages ​​and genders are intermingled. Everything is allowed.
A news release immerses us in the proposed circumstances.
Ironically adds that Elena Solomina - leading on national television and well known to many viewers.
Multicultural society is sheer fun, the younger Semenova leads to get acquainted with the parents of a huge boyfriend dressed in arafatka and a tutu.
After all, they go to a party.
At the end of the scene, a vote opens, offering a choice of whether Estonia will take the proposed path of multicultural development.
And if the answer is positive, then ...
Utopia falls into dystopia and freed freed up with a knot. There are new bans, and now the special forces break into the party and sort people. Some are allowed to pass to the right, towards the future, others are stripped and sent to the left, where their fate is decided by the machine of destruction. (based on articles by Nikolai Karaev and Boris Tukh)
What technical solution did we need?
In fact, it was another EMA actor (Est. Mama), artificial intelligence, which became a reality in the future. EMA leads the audience through the story, manifests itself in different variants of the future, and also announces and launches the vote, counts and presents the results, moves along the subject lines. The hall could accommodate 600 people. And each viewer should have the opportunity to vote, and the system correctly calculate his voice.
Possible options for the development of the plot, presented and modeled as a graph, formed a complex and branched structure.
Utopia tends to flow into a dystopia, and the road to hell is paved with good intentions. The plots are interconnected by the logic of drama. Given the number of options for the future that the children offered, a large number of branches came out. It is important to avoid repetitions - what was played should not be put to the vote again as a possible option. For example, if after viewing the Information and Technological Utopia, viewers confirm their choice of such a future, then we end up in a technological dystopia. If the audience rejects the proposed option, then they must again make a choice between the proposed options - for example, the Ecological and Cosmic-Colonial future. Very often there were situations when the hall struggled, the votes were divided almost 50% to 50% and everything was decided by a difference of several votes.
All this complexity required an appropriate structure, which is the graph.
What other functional requirements were there for the technical solution for the performance? We needed:
- The voting system, which collects votes during the voting period, shows the voting process and their results.
- Automatic selection of actions - depending on the results of the vote, the opening of a new vote, or the announcement of the beginning of the next storyline, in accordance with the logic of the graph.
- The system should communicate with viewers, offer choices and announce results in natural language.
- The system should be managed by an assistant director and display complete statistics on the progress of the vote.
So we have 600 people who must vote in less than a minute. People should be able to cast a voice from their phone. The mobile application option disappeared immediately, because downloading the application is an extra effort. No one is downloading apps if you are not on Facebook or Google. Different models and phone types require different designs, which is unrealistic in the conditions of 1.5 months and one developer. Consequently, we need a Web application for a mobile phone.
Another Web application is required for system administration, startup, activation of states, and other technical issues.
And finally, the most important thing is how the EMA should interact with the hall.
And here it is necessary to introduce the concept of "super", which was new and for me. Super is a screen that goes down and closes the stage.
Lift curtain
Also known as “guillotine curtain” or “German” curtain. More often used in musical theaters. The curtain is a whole canvas, straight or pleated, fixed on the upper edge to the carrier beam. Lifting the curtain occurs simultaneously with the lifting of the carrier beam. Such a curtain requires a great height of space above the stage.
(Wikipedia)
The curtain is reflective and allows you to display various projections, such as news, about which will be below, or the EMA screen, which displays information related to voting, its results and the choice made.
So, we have a complicated graph, a voting system and three FrontEnd systems that must interact with the audience and the administrator (an incredible assistant director Svetlana Shushina).
Decision
So let's start with the graph. The data structure, in this case variants of the history of Estonia, must be stored somewhere. Standard relational databases are not suitable for graphing for several reasons. For us, the most essential was the flexibility of the stored data, transitions, links to audio files depending on the choice, and so on.
The definitive choice was Neo4j, the world's leading graph database system. 7 of the top 10 technology companies in the world use it. This is a very mature technology. Speed, convenience and excellent community leave no other choice in this kind of solutions.
We modeled the plot, built by dramatists, in the form of a graph on Neo4j. And here is a piece of plot twists and turns, as it looks from the inside Neo4j.
One of the advantages that Neo4j gave was flexibility and the lack of a scheme, which allowed to change the plot lines without any serious consequences, add new audio options, link various novels to each other. This is the native representation of the graph in the native environment.
In terms of the server side there were some doubts - Python vs Java. The choice leaned in favor of Java. The reasons are speed, less risk to break something with clear requirements and straight arms, and finally Spring Boot, which supports Neo4j out of the box.
The server part is implemented on Spring Boot. In fact, the server collects votes, switches from “super” to the following states when the voting is completed, accepts commands from the admin, gives statistics and so on. Plus gives all the options FrontEnd-am, and also registers the audience.
Now begins the most non-trivial part of the task. The fact is that according to the idea of ​​the component that is displayed on the "super" should talk with the audience: prepare to vote, suggest options, start and end voting, announce the results and, depending on the choice, suggest other options, or go to the choice and tell the introduction to the next scene. Thus, the audio must be dynamic and prepared on the fly. Given that the number of options is enormous, and the length of each segment of audio is different, synchronization is important (for example, voting should open only after the announcement “Voting is open”) and this was not a completely trivial task.
The system has two control centers.
The first is the server. It stores the current state of the plot, collects votes, gives statistics data, controls the ability to give only one vote from one person.
The second is “super”, which turns the voting on and off, shows the voting process, and so on.
"Super" is implemented on Vue. Initially, we wanted to call audio with the Google Speech API, but we could not guarantee the absence of network delays. Therefore, another option was chosen - we recorded the response of the Google Speech API piece by piece.
In total, we got 55 pieces of audio. Information about which audio files should be included depending on the current state, we stored on the graph. Since we are dealing with JavaScript, we stringed audio files and command calls at each other, like beads. Thus, we did not depend on the fact that audio files of various lengths, and got a fairly universal solution.
Initially, the mobile client (web page) was written on Vue, but we got partial incompatibility on some phone models. We tried the babel transpilers, but because of the heap of settings, the volume of the page grew, and although voting began to work on some telephones, it broke down on others. As a result, the mobile client was rewritten on Vanilla JS.
In essence, this was a distributed system.
The final volume of the system (jar file) was 146 megabytes. The telecommunication company Telia provided us with a virtual server at a symbolic price of 1 euro per month, for which she is very grateful.
The pace of work before the premiere all accelerated. At the premiere of the expected president. The last bug was detected in half an hour, and there was no time for testing, so the first performances went with parallel testing locally on the machine behind the scenes, so that in case of an error it was possible to draw up a plan of action.
Of the interesting moments, we can highlight the fact that we showed the names of the last voters on the screen and used the Vue template animation for this. At the first submission, we were given a killer character, which broke the animation and display of Vue templates. Subsequently, we abandoned this solution and switched to the good old jQuery.
But a piece of voting taken during the performance.
The performance has received a huge amount of positive reviews in the press.
https://rus.postimees.ee/6141942/budet-ne-budet-net-nichego-opasnee-budushchego
https://rus.postimees.ee/6147704/nashe-zavtra-v-zerkalah-utopii-i-antiutopii
At the Drama Festival in Tartu, the standing ovation lasted almost 5 minutes.
It was an incredible adventure due to the crazy energy and dedication of the people involved in the project. A huge bow to all the people who participated in the creation of this show.
Director and set designer - Artyom Gareev
Composer, producer of the project - Alexander Zhedelev
Visual - Alena Movko
Costume Designer - Rosita Raud
Lighting Designer - Anton Andreiuk
Choreographer - Olga Privis
Playwrights: Karin Lamson, Marie-Liis Lille, Elena Chicherina and Laura Kalle
Video - Nikolay Alkhazov
Programming and creating applications - Alexander Tavgen, Anna Agafonova
Technical solution ice shine - Aleksander Sprohgis
Animation - Martin Yakush
Band - Modulshtein
Martin Altrov - clarinet, bass clarinet
Alexey Semenikhin– work with samples, effects
Alexander Zhedelev - guitar, synthesizers, ableton
Vocal teacher - Anna Dydina
Assistant Director Svetlana Shushina
In the play are busy:
Natalia Dymchenko, Alexander Zhilenko, Daniil Zandberg, Dmitry Kordas, Ekaterina Kordas, Alexander Kuchmezov, Victor Marvin, Natalia Murina, Elena Tarasenko, Eduard Tee
Sergey Furmanyuk, Leonid Shevtsov, Elena Yakovleva
Russian Theater Studio: Denis Volkov, Pauline Grinyova Nina Zagvozdkin Anastasia Kalyada Natasha Christensen, Anastasia Masalova Sandra Minosyan Sofia Mikhalev, Catherine Magi, Michael Pashchuk, Catherine Selyugina Christina Sorokina, Sofia Strömberg, Ilya Sutta.
Source: https://habr.com/ru/post/425537/
All Articles