The promise of light field cinematography is a radical disruption of the filmmaking process — one that could transform the creative process for films on the big screen. The technique is still in its infancy, though its roots go back decades. Bryan Cook brings depth and focus to the subject.
As with any technology and artform, filmmaking has come a long way. The fundamental way in which images have been captured through the years has not changed that significantly —light is reflected off a 3D object which in turn is then transmitted through a lens series and focused onto a 2D surface where it is recorded. The film plane records the intensity of the light rays of the image formed. However, starting at the turn of the 20th century, alternative techniques were developed that have major implications on film production and exhibition today.
The History of Light Field Photography
Modern light field photography evolved out of a desire in the early 20th century to be able to see 3D images without the need of spectacles. At the time it was necessary not only to hold glasses to your face, but also to have two separate images — a stereogram — mounted on the device at the correct distance from your eyes in order for a 3D image to be viewed. Frederic Ives used what was known as a ‘line screen’ — a vertical barrier created with strips mounted in front of the film plane — to record a different set of images for each eye that, when shown via a similar apparatus, would allow the user to see two separate images simultaneously. The brain would piece together the two images to construct a 3D image without glasses. The issue with this method of image capture was that it was extremely difficult to modify existing cameras with this line screen —therefore it never really took off. Years later, advertisers have adapted the method in lenticular printing, seen on the side of bus shelters and the like, to create different images depending on where the viewer is standing.
Other experiments through the century picked up Ive’s torch for interactivity without glasses. Gabriel Lippmann sought to create images that would appear and act more like a window so that when the viewer moved positions new perspectives could be seen.
Lippmann and other photographers experimented with putting a series of micro converging lenses directly to the film in the camera. In order to be viewed correctly, a matching set of microlenses was needed to render the images into something viewable. Again, the technique, proved tedious to manufacture, but the application of microlenses became a lasting concept.
The pinhole camera comes into play
Fast-forward to the end of the 20th century when a few mathematicians — Edward Adelson, James Bergen and John Wang — proposed that if you create a pinhole camera with finite apertures at a fixed distance to the film plane, it becomes possible to create subsections of the image. The outcome of these photographs is similar to the perspective of a fly with their thousands of lenses on their compound eyes — each one seeing a portion of the image simultaneously from different angles.
So, what is Light Field Photography?
If we look inside a traditional camera, we can envisage light rays traveling from the primary lens through this space to 
the film plane (or digital sensor) where the intensity of the rays are recorded. The difference between traditional photography and light field photography lies in the understanding of each of the subsections that were proposed from Adelson, Bergen and Wang. These subsections of the image allow us the possibility to trace precisely where each light ray strikes the film plane and the direction from which it came.
This concept is easier to imagine if you think of a ray traveling through two pieces of paper. It enters one sheet of 
paper at a particular angle and if we know the distance between the next sheet we can trace it precisely to the next point of entry on that sheet through basic geometry.
Now visualise thousands of sheets in the camera between the primary lens and the camera’s sensor — this is the light field — knowing the direction as well as the intensity of each light ray.
Using some mathematics we can calculate where the light at any point in space will fall — and this allows us the ability to reform a traditional image.
Light Field Filmmaking
It is easy to imagine how light fields could be adapted to filmmaking. Given the often rapid pace of today’s film productions, time is becoming more and more a commodity. Unfortunately the digital era has erased the discipline and practice of having rehearsals. Without the benefit of rehearsals, focus pullers are put under increasing pressure to maintain sharp focus successfully. Compounding this problem, larger sensor sizes, higher resolution sensors and more efficient lenses all make capturing sharp focus on set very difficult. If the focus puller is unable to maintain sharp focus throughout a scene, it is possible to correct the issue in post with light fields. It is important to note that while the technology can adjust the focus, it has its limitations — in other words, the technology is far from replacing people’s jobs on set.
Lytro brought the first commercial handheld light field camera to the market in 2012 and the second generation in 2014. More recently, Lytro has incorporated the technology in its Cinema Camera that captures a 755 megapixel image (equivalent to a 40k image). This camera has been specifically designed to help visual effects artists determine the distance of the light rays outside the camera. By knowing the distance, it becomes possible to remove some of these rays behind characters or objects — in effect creating a virtual greenscreen without the need of a physical one.
Angular vs spatial
There is a trade-off in what is termed the angular resolution versus spatial resolution. In other words, the more microlenses that are introduced to the camera causes a decrease in the number of pixels on the final image. This is why Lytro developed a camera with a sensor 1½ft wide (versus a standard Super 35mm on a traditional video camera) in order to produce a decent 8k final image.
The Lytro Cinema Camera produces data output of up to 400 gigabytes per second. The issue here becomes how to handle storing so much data, let alone processing it. This is well beyond what a typical film production would sustain currently. The Lytro Cinema Camera is over 8 feet in length which is too large to mount to a crane or dolly, let alone a Steadicam operated by a single cameraman.
Currently, light field cameras need to be custom-made as mounting the microlens array precisely inside it is extremely cumbersome — at the moment it is not technically possible to retrofit a digital cinema camera into a light field cinema camera.
The Impact on Exhibition
While the technology is still very much in its infancy, there are some developments on the horizon that may have a great deal of impact on cinema exhibition.
If a light field camera is used and then a light field display is used, viewers would be able to see 3D images on a screen without the requirement for 3D glasses. Light Field Labs, a company started by 
former Lytro employees, aims to make such panels similar to those being used in the latest LED screens in cinemas today. The other unique use of these panels, is that a virtual “holodeck” as seen in Star Trek could be created at amusement parks for an experience that truly deserves the oft over-used term “immersive”.
The Future
The real challenge to light field cameras being used is their development in three areas: they need to show that their capabilities outweigh traditional cameras, they need to become a critical piece of the production pipeline and they need to address all of the downfalls of traditional filmmaking without creating new ones.
Currently there are not many companies implementing light fields into filmmaking. Just this March, Lytro was acquired by Google for $40M for its patents in a move that will aid its development of its VR and AR headsets. The market seems to have shifted away from commercial to personal use. Right now virtual reality seems to be the ideal development ground for the technology as the ability to refocus a scene when combined with eye-tracking headsets could create a more realistic experience for the user.
The question looming down the road is whether we will see the technology implemented effectively into the production and exhibition pipeline or if light-field cinematography is simply another gimmick. Time will tell.
