technology is important but story-telling is the future…

CT had a rare chance to talk with Harry Mathias — a doyen of cinema technology whose experience in all aspects of cinema, from cinematography, to directing, to the science of digital projection make his candid opinions on the latest technical developments well worth listening to….and something of a masterclass.

The Cinema Technology archive holds two memorable papers by Professor Harry Mathias from the late 1980s: “Image Quality From A Non-Engineering Viewpoint”, a topic still at the heart of work the Cinema Technology Committee (CTC) does today, and “Gamma and Dynamic Range Needs for an HDTV Electronic Cinematography System” which led discussions that resulted in development of international standards for electronic imaging at a time when much of the industry was conflicted as to what part HDTV would play in the future. Here is a man with his finger on the industry pulse.

Professor Mathias had agreed to be interviewed on the latest developments in our industry, so Cinema Technology caught up with him on a trip he made to London earlier this year while undertaking a series of master-classes at several universities, as well as to promoting his latest book, which has the somewhat apocalyptic title The Death and Rebirth of Cinema. So, is the cinema industry due for a resurrection?

CT: As one who has been in digital cinema since its earliest days, what state do you think the industry is in now, and what will the cinema experience look like in 2047?

HM: The industry is in an appalling state. We have forgotten we are in the storytelling business. Much technology is used purely for the sake of introducing new technology rather than for improving the quality of films and the cinema-going experience. Many cinematographers, often pushed by directors and producers to use the ‘next big thing’, have stopped using film and jumped straight to digital, often losing the benefit of carefully honed craft skills developed over years. As an example, nobody use light-meters any more, which as a cinema-tographer I always considered essential.

Correct exposure is critical to achieving full tonality and colour saturation, but good lighting is more than just correct exposure; properly ‘shaped’ light, incident light readings, reflected light readings and spot readings all have their place, together with precise camera settings. With digital cinematography much of this has been forgotten, with cinematographers often content to take what appears on the monitor. Properly done, digital cinema images do not render correctly on HDTV (REC 709) monitors and using an on-set monitor is the worst way to judge image quality.

Proper lighting sets the mood and involves  the viewer with the storytelling. To be more charitable, I suppose it is possible that modern cinematographers are ‘behind the curve’ with use of new technologies and may eventually catch up — let us hope so!

Wider colour gamuts present an interesting challenge and have some potential advantages, but I am not sure that current colour gamuts are a real issue — just because the technology allows us to use WCG, it doesn’t mean our cinematography will benefit. What we must do is to consider and work on colour gamuts in relation to different types of lighting, trying to optimise the cinematographic results we can get from LED lighting, for example — there is much work to be done and much to be learned.

Higher frame rates also seem to be a solution looking for a problem. It will be fine in instances where you can see there is a real improvement, but that is far from the norm. I am a believer in the adage that ‘film sees things the way we do’ — some blur is a fact of life, our eyes and brains are used to it. 

Higher dynamic range is currently being regarded as the next big thing, but cinema has always managed to provide magnificent images without it. I regard HDR as just another improvement, something that we must learn to use to best advantage. What I don’t like is the current tendency to use HDR as another selling point; spectacularly high-contrast images may have their place in cinema, but certainly not everywhere.

So you asked me what I thought about developments in the past 30 years. My conclusion is that we are in danger of using technologies for their own sake rather than to provide improvements in the cinema experience. There is a grave danger from the currently fashionable attitude that ‘if it is new it must be great’. That really isn’t true and we must take care not to lose the skills and experience that have served us well over generations.

As for what things will be like in a further 30 years  time I can only agree with your CTC members’ own predictions that people will still be enjoying going out to cinemas to share magnificent images and sound with each other, but there is little point in speculating as to the technologies that will be in use ­— things move so quickly these days. But, always remember: cinematography is about making visual art, it is not about mastering new technology, but sometimes mastering daunting technology is what you have to do to make art. Just keep the art foremost in your mind, not the technology.

Projection technologies

CT: Recently alternatives to xenon projection have appeared in the form of laser projectors, and these are already being challenged by ‘active displays’. What are your thoughts?

HM: Current xenon projection technologies have served us well for years and survived the changeover from film to digital. Those who are smart enough to ignore the ‘if it is new it must be better’ messages understand that we still have much to learn about new projection technologies. There are real problems with the narrower colour spectra used by different forms of laser projection ­— some cinematographers and industry scientists claim that the images shown are not the ones that they so carefully crafted.

The danger with the use of the narrow-band primaries in laser-illuminated projectors, is that every audience member will see a different colour image based on the peculiar colour sensitivities of their own eyesight. This means everyone sees a different colour rendition of the film, and not the one that the cinematographer intended them to see. So we still have much to learn about this area and much research and development is continuing.

As far as LED ‘active screens’ are concerned, these are brand new to cinema, and it will be interesting to see how they develop, but it is too early to speculate. Active screen technologies will continue to improve. There are, of course, other considerations relating to their use, such as the power consumption and ways in which sound can be made to appear to come directly from the screen — some experts in cinema sound are convinced that this will be a major problem.

Entering the Virtual world

CT: On to Virtual Reality. What are have your thoughts on VR and where you see it fitting into the exhibition industry?

HM: Where to begin? So many factors, so many potential problems, but it does seem that VR won’t be going away. The wider interest in artificial intelligence is pushing virtual reality forward, but I am more concerned at the underlying suggestions that we need VR because there is something fundamentally wrong with cinema (there isn’t!) and that the more technology we have the better cinema will be.

If there are things wrong with cinema it is not because of a lack of technology, but far more because we need better content. Although sequels have traditionally done well (they are safer) there are now signs that this is coming to an end, and occasional commercial flops such as Tomorrowland and The Lone Ranger show that the industry still doesn’t fully understand what will make a commercially successful movie. We can look now at what VR will bring to the business, and it may help, but it will certainly not overcome such content-related problems. As far as I am concerned, in spite of the huge number of start-up companies and the billions being invested, VR to date just isn’t film-making. It is much closer to competitive computer gaming. And I speak with some knowledge — I have shot a VR game, and it was a painfully tedious experience, a million miles from the creative joy of working with a director to create a work of art that a good movie can be.

So what makes me say that, and why don’t I like VR for film-making? Where do I start the diatribe? I can see problems in all areas of cinema: shooting; storytelling; editing; displays; can it be a shared cinema experience?; will the audience feel sick, dizzy or uncomfortable? And that is before we consider a multitude of related technical factors like resolution, frame rate, colours.

The VR Shooting experience

VR isn’t shot like movie cinematography. In order to keep humans out of the 360 degree sight-lines, the images are shot with large numbers of relatively low-quality cameras mounted on a rig, and no human camera crews can operate the equipment, without being in the movie. The clever images that result are a world away from cinematography, where people make movies, and use their skills to tell the emotionally moving stories that keep audiences coming back. Movie-making is a people business, and all those people around the set are there to contribute their own individual skills to create the movie and to ensure that the all-important storytelling isn’t forgotten.

VR at the other end of the chain

Moving to the far end of the cinema chain, much of the joy of watching a movie comes from sharing the social experience in the auditorium. How can this possibly happen when the members of a VR audience are shielded from their neighbours by all-enclosing headsets? VR doesn’t contribute to the storytelling, it merely provides you with different surroundings, and whatever the audience is experiencing, VR isn’t contributing to the social group experience, and I do think that this matters.

VR: It’s all about the editing

It is in the editing phase that a film’s story takes shape. The great films of history were well-paced and moving because of their carefully crafted images that were combined by skilled editing. Cutting within scenes is needed to tell an engrossing story on film. But I have found by experience that when an audience is watching in a VR environment, editing is practically impossible. Switching between virtual image sequences is disorienting, even to the point of causing audience disorientation and nausea, and cutting from one 360 degree landscape to another is painful to watch.

Without the sophisticated editing we have become used to in our film-making, complex storytelling just won’t work in VR productions. It is important to remember that any film is a dramatic or comedic interpretation of reality, not a substitute for it, so VR isn’t likely to help create the mood. If there is to be a market for VR films they will have to be shot as totally different versions from the films we see in cinemas.

And I am not over-emphasising the problems of nausea or ‘VR sickness’ — they are uncomfortably real. As far back as 1995, Nintendo’s ‘VR Gameboy’ console was withdrawn due to complaints of customer nausea. NASA and the USAF have also extensively studied ‘VR sickness’ in pilots which occurs after training on VR flight simulators. It was found that some pilots weren’t in a fit state to drive their cars home after a long VR working session. A study by Dr Michael Korpi of Baylor University called: How Far to “Perfect”?, concluded that we are at least 15 years from visually correct (unflawed) VR, and listed the many psychophysical issues of the imperfect VR systems that are available today.

Augmented Reality

CT: What about your views on augmented reality — are they the same? Augmented reality is perhaps even more difficult to define, because its possibilities are so wide: adding graphics, sounds, haptic feedback and smell to the natural world around us. As with VR, video games are driving much of the development of augmented reality, but there is obviously far more to it than that, with extra information popping up in our line of vision or in our ear whether we are at an art gallery, riding a bike in town or along a country lane, automatically seeing more information about a product that we are looking at in a shop window, etc.  Can you foresee ways in which AR could enhance the cinema-going experience, both in the auditorium and beyond?.

HM: I don’t have a problem with cinema and augmented reality — just so long as AR is used genuinely to enhance the cinema-going experience. In many ways you could consider that the special effects that are so much a part of our regular cinema productions are forms of AR. But, there is a lot of time, expertise, and equipment poured into film FX. The idea that this money and expertise would be unnecessary in the “brave new world of AR” is naive. Expensive AR already exists in film production, the cheap AR of the future is a fantasy. The idea of overlaying information on your view of the real world is familiar to everyone, and many movies, particularly those in the sci-fi genre, have given us all a pretty good idea of how useful AR techniques could be, so I consider AR to be a valuable adjunct to movie-making, so long as it never interferes with the storytelling. I feel much the same about systems such as 4DX — they add something extra to some movies, but must be used carefully so as not to detract from experiences the director wanted to convey.

HDR and HFR: are they relevant to VR?

CT: Do HDR, HFR and WCG have any relevance to VR, and if so, how? Do these factors affect your views of where VR will fit (or not!) into the cinema?

HM: Let’s concentrate on the relevance of HDR and HFR to film-making in general. Both are useful tools, but I really don’t believe that they are going to change the future of cinema. As a cinematographer I have shot lots of movies in all different kinds of lighting environments, and I well understand the importance of using contrast and dynamic range to achieve the particular ‘look’ that the director wants.

Some of the classic movies by the world’s most respected directors would, by today’s imaging standards, be considered ‘low dynamic range’, shot on black and white film with only 10 stops of contrast latitude, and yet it would be a brave movie critic who dared to say that these needed more contrast. But we can’t stop progress, and we are now in a situation where film and digital cameras can provide a least 14 stops of dynamic range. Film-makers are currently making good use of these capabilities, but whether we need even more dynamic range is debatable. It is important to remember that the real world is far more contrasty than any film or digital camera can reproduce — film lighting units don’t try to emulate nature’s excessive contrast, they are used to create a mood and to produce powerful and memorable film images.

HDR must be appropriate for the mood of the story, and not a gimmick merely used to attract bigger audiences. The examples of two of Rembrandt’s paintings shown on the opposite page make the key point that HDR only sometimes improves the image. HDR can be a good thing, only if it is controllable and consistently reproducible, and from what I see today this is not yet the case. We are even being told that HDR could save money by solving the ‘problem’ of a cinematographer needing ‘expensive’ lighting on a film — something that we should never allow to happen.

Projecting images with HDR on a cinema screen has different issues from using it for cinematography. In this case there is a limit to the amount of light that can be projected on the screen in the bright areas without destroying the contrast of the rest of the screen image, so higher-contrast HDR projection could represent an improvement, but again only if it is controllable and consistent. I would caution against the use of HDR as a ‘sales gimmick’. We really don’t want to end up with all films providing spectacularly high-contrast images. It is important to ask if HDR will bring more people into our cinemas. I believe it will, if it is used artfully, but not if it just provides brighter, more contrasty screen images.

Higher frame rates can raise even more contentious issues. In film we have traditionally used a high-resolution 35mm medium at a ‘slow’ 24 frames per second. Higher frame rates can change the complete ‘look’ of a film, since they result in less motion blur in the image. At 24 fps (with a standard 180 degree shutter) the shutter speed is 1/48th of a second, while at 48 fps it is 1/100th of a second, resulting in virtually no motion blur.

The Hobbit, at a frame rate of 48 frames per second gave rise to mixed reviews. In general, film critics and cinematography experts were not impressed by 48 frames per second, but many others thought the ‘sharper, clearer’ results were great. Many have seen Douglas Trumbull’s even higher frame rate experiments, that can provide some stunning images, but I have to say that as far as cinema films are concerned, motion blur is a good thing, which contributes to a realistic portrayal of motion. Whatever the HFR enthusiasts tell you, motion blur exists in the real world and any moving object leaves an impression of its path in our visual system — a smear, if you like. In modern animation, the motion blur rule of thumb is that if an object moves more than half its size between any two frames, motion blur must be added. So I have to tell you that removing motion blur, with HFR, is a bug, not a feature!

Wider and wider Colour Gamuts

The key question is whether moving to a wider colour gamut is a major breakthrough in the audience viewing experience or simply another over-hyped technology? It is undoubtedly true that there are colours that exist in nature that even the latest digital cinema projectors can’t display, and the long-promised CIE/ITU Rec.2020 standard could provide many of these colours. But although I agree that this development could offer a breakthrough in colour rendering, we need to consider the creative implications of such a change from a storytelling point of view. When seeing side-by-side comparisons you can readily discern that moving from Rec 709 (HDTV colour) to P3 (DCI cinema colour) to CIE 2020 colour provides colours that are missing from the previous versions.

In reality, we simply aren’t aware of what we are missing when we view a cinema picture. Our human visual system can see far more colours than even CIE 2020 provides, so since neither 2020 nor any future colour standard will be likely to allow cinema screens truly to display the entire range of colours the human eye can perceive in nature, any change to a wider colour gamut would be unlikely to make a huge difference in terms of creative storytelling.

Everyone sees ultra-wide colour gamut with their eyes every time they go for a walk. Now, think of all of the great films that you have seen in your lifetime. Did you ever think to yourself while watching them: “I am concerned about all of the colours that aren’t visible!?”.

At the end of the day, it is an interesting point to ponder. I’m not arguing that wider color gamut isn’t a good thing, just as I’m not arguing increased dynamic range isn’t good. I’m trying to put it in perspective from a creative point of view. These are tools to be used carefully — the mere act of expanding the range, whether it is colour or dynamic range, does not necessarily do anything to make it a better movie or bring in a larger audience. That depends entirely on the story and its visual representation.

Jim Slater

See author's posts