The CTC examines VR and AR in cinema

As a venue that has been showing film since 1896 when the theatre played a movie by the Lumière Brothers (regarded as the first motion picture shown to the public in the United Kingdom), the Regent Street Cinema, London, was an appropriate location for the IMIS seminar on Virtual Reality and Augmented Reality in Cinema. The evening had been arranged by the CTC, whose chairman Richard Mitchell brought together two world-class experts in these fields, one from each side of the Atlantic: Scott Grant, CEO of Soluis, one of the UK’s leading creative organisations, and industry guru Professor Harry Mathias from San Jose University. Over 70 people attended, with many others around the world participating in an on-line streamed session.

Cinema storytelling

Speaking via an excellent Skype link, Professor Harry Mathias (whose in-depth interview with CT appears on page 17) said he would talk exclusively about the potential for VR in cinemas, rather than gaming or other applications. Previous innovations in cinema have been driven by the need to tell stories more effectively. We are now in an era of cinema ‘ultras’ where all sorts of new technologies are being applied, but it is unclear whether these will contribute to the key element of storytelling.

Showing an audience picture with each member encased in a VR headset, he supposed that each would be having a VR experience but questioned whether they were sharing a communal experience, since each person has a different experience depending upon where they chose to look. He surprised many by suggesting that many new technologies aren’t actually fulfilling a need, saying that the last invention that truly revolutionised cinema was the Steadicam, which allowed picture taking whilst moving along with the actors, thus improving the way stories can be told.

Never afraid to provide his candid opinions, Harry told us that there are many things ‘not to like’ about VR and the cinema.

The way it is shot

Multi-camera setups, whether lots of cheap GoPros or expensive 360-degree camera rigs require many images to be stitched together, and we cannot yet do this acceptably. Developments in lens technologies will continue to be expensive. All round shooting means there is no room for the camera operator or crew — all those specialists who play their part on a film set.

The audience experience

Even if you move forward from today’s VR experience where goggles make movie viewing a solitary experience to some new 360 degree projection system, (as has been tried before in 35mm film days) your view of the ‘experience’ will be interfered with by all the rest of the audience.

Will Editing VR be impossible?

Editing is an essential part of storytelling, it structures a film and focuses the audience’s attention on what the director considers is important. Rapid cutting is a vital part of editing, and such cuts would be impossible to structure in VR — think of the effects on the viewers. Nausea is already a well-known side-effect of VR. The technology of VR is probably 10-15 years from being able to provide ‘visually correct’ unflawed images that will work for cinema.

Harry said that it is important to remember that film is not and never has been a substitute for reality — it is a carefully structured experience. We are only hearing about VR today because companies such as Apple see it as a way forward for using their technologies. When asked what he would do if asked to make a movie in VR, Harry explained that he had already done some shooting for VR games, describing the process as ‘the most tedious thing in the world’ — you have to take multiple shots to cope with all the different options that games provide at each point, and that most material will never be seen.

Questions came thick and fast on topics from space technology to HFR to Immersive cinema to human visual perception, with the professor providing instant and sometimes provocative answers to each.  When talking of wrap-around formats such as Barco Escape he said that, as a cinematographer, he wouldn’t want to shoot a movie that causes people to turn in their seats — he wants to compose a picture within the desired frame. “In a cinema we want a different experience from real life — if you want 360-degree viewing you can leave the cinema and join the outside world.”

a technological trinket?

Professor Mathias summed up his views by saying that current attempts at VR for cinema are little more than a nice novelty, and that since you can’t include a crew, you can’t light a 360 degree scene properly and you can’t create good sound, he isn’t convinced that this is a future genre for filmmakers. He ended as provocatively as he had begun by leaving the audience several key points to ponder: Are these new technologies trying to fix something that is wrong with cinema? What is it that we are trying to fix? Are any of these technologies good enough to take a bad story and make it good? If not, although there is always money available for new technology, we would be better off improving the storytelling.

Professor Mathias ended with the telling statement “I am not convinced that it is technology that is holding the cinema back!”

Just as the VR experience will be subtly different for each of the people in a shared cinema space, so every person who attended this seminar will have taken away something slightly different. Whatever we end up doing with VR — and the technologies make almost anything possible — the key factor for cinema is that we must ensure superb storytelling remains at the heart of what we experience on screen. 

Watch the seminar:



Augmenting reality in the cinema

Scott Grant explained the work of Soluis in VR and AR, producing a range of visual media and interactive environments that showcase projects to any audience. A leading provider of computer-generated imagery and digital media, Soluis develops VR and AR applications for use on many different devices. Much of their work involves explaining complex proposals and designs to people by bringing viewers into the virtual world of a design.

In film production their motion graphic design team provides a wide variety of video content, and understand how to evaluate the most effective camera paths for capturing the key elements of a design in the most visually engaging manner.

In gaming they have been working in interactive real-time environments and have pioneered adoption of real-time games engines technology for mainstream visualisation for over 10 years.

It is early days for VR applications in cinema exhibition, but AR has been used in the business for over four years, mainly linked to trailers, and TimePlay has shown how the technology can be used to provide an interactive experience. Many ideas are being trialled, one being interactive posters that use AR to trigger some experience as you go past — ‘Tom Cruise walked out of the poster and I took a selfie with him’ is one idea. VR is being used in pods outside the auditorium space, but there are still major queries about providing a communal VR experience in the auditorium, a totally different experience to watching a movie with add-on AR features. Filmmakers well understand how to direct a story that will be shown in a frame, but telling a story shot around 360 degrees requires a different set of skills, as yet undeveloped. Scott was confident that we will get to a stage where an audience member will be able to effectively participate in a movie, choosing which acting role you take up, and he felt that this will begin with movies like Star Wars, where many fans already feel an inherent connection with the story. He predicted that this could happen within five years, saying that we need higher resolution and image quality to provide the illusion of reality.