IBM Watson: the cognitive system in cinema

The IBM Watson system is capable of much - from solving problems in the field of oncology to predicting possible natural disasters in a particular region. Now IBM Watson is trying himself in the cinema, together with the studio 20th Century Fox. The cognitive system was commissioned to create a trailer for the film "Morgan". The plot of the film is tied to the technology that allows people to improve, and the person created using this technology. In order to attract an audience, IBM Watson had to collect the most exciting moments of the picture in a short trailer.

Even for a professional director it’s not so easy to determine the most subtle psychological moments of the film. What to say about IBM Watson, a cognitive system that has just begun its journey into the world of cinema. But, it seems, the service coped with its task.

In order to create the trailer, IBM Watson analyzed a number of man’s innermost fears, including the fear of death, xenophobia, pain, and more. Moreover, in order for the viewer to go to the film, moments must be shown in the trailer that lead the viewer to the storyline and the interchange. Fail, but do not reveal all the secrets. The task of the trailer is to interest the person, make him think about the film and the possible development of the plot. Well, if the trailer about an artificial person is created by artificial intelligence - this can serve as an additional factor in attracting the attention of viewers to the picture.
')
It is clear that the computer is difficult to understand a number of moments of the human psyche. In order to help the cognitive system, it needs to be trained. IBM Watson developers conducted training using horror trailers as a visual aid. They used not one and not two trailers, but a whole hundred. To increase the effectiveness of training, each trailer was divided into separate fragments. These fragments were analyzed according to a number of criteria. There are three main criteria.



The first is the analysis of the video sequence, with the study of individual characters and related objects. Particular attention was paid to the emotions of people. The cognitive system highlighted 24 emotions. Additionally, the content was tagged. The second is audio analysis, where the system studied and tried to identify surrounding sounds, including human voice and music. The third criterion is frame composition. IBM Watson analyzed lighting dynamics and camera position.

The IBM Watson cognitive system can process huge amounts of data at a time. When creating the trailer, we used the analysis of each particular scene. We studied both individual frames and a bunch of frames.

As a result, after many days of training, IBMWatson was able to identify the 10 most emotional moments of the entire film. Of these, the trailer was made and lasting six minutes. Here is the trailer itself:


The developers of the cognitive system, who were given the task of teaching IBM Watson cinematic wisdom, could not influence the choice of computer. True, the system could only indicate the time of each individual section of the film, and not engage in the layout of the trailer video sequence. People had to carry out the installation work on the video for the trailer. But it also allowed to drastically reduce the time of preparation of the video. Usually it takes about a month to create a movie trailer. Thanks to IBM Watson, the work was done in 24 hours.

It is noteworthy that a trailer has already been created for this movie. But the previous version was created by employees of the studio 20th Century Fox:


It is clear that someone likes the version created by the machine, someone - the work done by man. Perhaps in the near future, cognitive systems will actively help people create both trailers for films and films themselves.