Reinventing the photograph (Image: Michael Glenwood)
LIKE a lot of professionals, isn鈥檛 crazy about modern cameras. Granted, the London-based photographer keeps an iPhone in her pocket for the occasional snapshot, but really she likes nothing more than to take out her well-worn Hasselblad and hear the satisfying pah-clunk of a mechanical shutter. 鈥淎s an artist and a photographer, I do believe in embracing new mediums,鈥 she says. 鈥淏ut, to be honest, I use old technology more than new.鈥
With cameras, though, what counts as old is relative. Pannack鈥檚 camera phone and Hasselblad may be separated by several decades of innovation, but they have more in common than most gadgets spanning this timescale. Both focus light into an image using a series of glass lenses, like every other camera on the market. Whether that image is then captured on film or on the latest smartphone鈥檚 digital sensor, it is the result of a technique invented in the 1830s.
Advertisement
Photography has been the medium of the modern age, recording everything from momentous events like Neil Armstrong on the moon and the , to Marilyn Monroe on a subway grating and that selfie you took on New Year鈥檚 Eve, dressed as an orangutan and with a banana squashed in your friend鈥檚 ear. But all that could be just the start, because the way we take photographs and what we do with them afterwards is changing dramatically. The job performed by finely ground slivers of glass is being eclipsed by the work of finely crafted algorithms. As they take the lead, the 180-year reign of the lens might be over (Gallery: Click here to see the evolution of the camera in pictures). It is a fundamental reimagining of photography and, like an ageing Polaroid, the camera as we know it is fading from existence.
Digital photography has entered an age of extremes. There are gigapixel cameras that capture ultra-high-res images, and so-called single-pixel cameras that construct an image by taking multiple snaps with just one relatively cheap, low-powered detector. But the real battle for the future of photography is happening in our pockets. Although smartphone manufacturers have pushed camera technology massively in recent years, they are finding it increasingly difficult to cram a stack of lenses into a wafer-thin body. When Apple released promo shots of the iPhone 6 last year, it airbrushed out the protruding lens 鈥 the only component that could not be sufficiently miniaturised. But while the lens has reached its limit, computational photography is just getting going.
Most smartphone cameras give us the ability to take a panorama, essentially a super-wide-angle image. As you pivot on the spot, an algorithm records several images and then stitches them together, making slight adjustments for exposure, parallax and so on, so that the seams are invisible. Another common phone feature is high dynamic range (HDR) imaging, which allows you to photograph a very high-contrast scene 鈥 a person standing with the sun directly behind them, for instance 鈥 and still see details of their face rather than just a silhouette.
The HDR function works by merging several images of the same scene: at least one image with a long exposure to capture detail in the shadows, and one image with a short exposure to preserve highlight detail. 鈥淯nder the hood, a lot of these computational-imaging techniques are already taking place,鈥 says computer scientist at the King Abdullah University of Science and Technology in Thuwal, Saudi Arabia.
On the moon: A great photo stands the test of time (Image: NASA)
Computional trickery will soon be covering up even bigger imperfections. One shortcoming of smartphone photos, for example, is blurry edges 鈥 the natural product of cheap, miniaturised optics. Traditional cameras do not suffer from this because they usually use larger lenses that contain many individual elements. That is obviously a no-no for smartphones, so Heidrich is developing an algorithm that can correct optical aberrations. Since a bad lens typically affects the primary colours differently, the algorithm looks for areas where the red, green and blue layers that make up the image are misaligned. It then adjusts the image until they line up, .
Blurring can also come from camera shake, which is all too common when holding a phone at arm鈥檚 length. But an algorithm developed by Kari Pulli of photography tech start-up in Palo Alto, California, can help keep things steady. It takes readings on the fly from a smartphone鈥檚 built-in gyroscope, and uses them to adjust for motion blur without sacrificing genuine movement in the scene.
Risk and reward
Heidrich鈥檚 and Pulli鈥檚 ideas are set to improve smartphone image quality, but they will not do much to make up for our amateurish blunders. After all, photography 鈥 in the spirit of the pioneering French photojournalist Henri Cartier-Bresson 鈥 is all about the 鈥渄ecisive moment鈥, that crucial split second when you press the shutter. Pannack agrees. More goes into great photos than just a good-quality image, she says. Creativity is wrapped up with risk and what Pannack calls 鈥渂eautiful mistakes鈥. Get it right and you鈥檝e captured your friend鈥檚 expression the moment the banana finds his ear; get it wrong and you have accidentally focused on the background.
This is where we could see the biggest shift in how we take photos. What if we could simply take a shot and only later decide what its composition and focus should be? Hard though it might be to imagine this, computational photography is already showing us how it could be done.
鈥淲hat if we could simply take a shot and decide how to compose it afterwards?鈥
In 2011, the California-based company launched the first consumer 鈥渓ight-field鈥 camera. It still focuses light through a main lens towards a sensor, as in a normal camera. But the Lytro has an extra array of micro-lenses which capture bundles of rays and focus them on to individual pixels. The result is a device that captures the direction, as well as the intensity, of incoming light 鈥 known as the light field.
Using this information, algorithms can work out how the light would have behaved if the main lens had focused it differently. That leads to the key benefit of a light-field camera: the ability to refocus photos after they are taken. If your friend is out of focus, you can tap where they are on the screen and, as if you鈥檝e had an instant cataract removal, their face turns crystal clear. Even better, the algorithms can calculate how the scene would look if the light had entered the lens at a slightly different angle, giving the option of viewing the scene from more than one perspective. If you were photographing a football tackle, you might be able to adjust the image to show the point of contact between the players, for example.
Refocusing a picture long after you took it is within our grasp (Image: Lytro)
Similar tech is headed to smartphones. The PiCam replaces the single lens on a smartphone with an array of 16 smaller ones. These are a small distance apart, so can also record depth information. 鈥淚t kind of mimics the human eyes,鈥 says Kartik Venkataraman, CEO and co-founder of Pelican Imaging, also in California, which developed the technology. Next-gen selfies will come in 3D, ready to be printed.
This year PiCam is being released as a development kit to smartphone manufacturers and we can expect light-field photography to be widely available soon. But not everyone involved in computational photography believes Lytro or PiCam have got the right idea. Pulli points out that taking up sensor space with a lens array kills the resolution of the final shot, unless yet more algorithms are employed to merge several low-res images. 鈥淵ou pay with low resolution, and have to expend a lot of effort trying to get it back,鈥 he says.
Even so, there are other ideas for how to exploit light fields to great effect. One of these, says , a computer scientist at the University of Toronto in Canada, is relighting 鈥 the ability to impose light that was not there to begin with on a scene. Whereas traditional photographers have to laboriously set up a separate flash unit away from their camera to achieve certain lighting conditions, light-field cameras could add a spotlight effect automatically. Their depth information equates to a 3D map of the scene, which makes it possible to work out exactly where the shadows would naturally fall.
Kutulakos is developing yet another trick: separating light that has bounced directly off the subject from light that has bounced indirectly, scattered by different surface angles and textures. This technique involves placing an electronic, slit-shaped mask over a camera鈥檚 internal flash unit, limiting the outward light to one plane; another mask shields the camera鈥檚 sensor. When the masks match up in the same plane, only the direct light reaches the sensor.
Cameras 鈥 and our eyes 鈥 normally perceive a mixture of direct and indirect light. But things look quite different if . A glass turns black. Yellow foam turns grey. A latex glove becomes almost invisible. 鈥淲e are only just starting to realise what happens when you take pictures in this way,鈥 says Kutulanos.
Something particularly interesting happens when you photograph faces in direct light: they look stark and haggard, a consequence of masking out all the light scattered indirectly from beneath the skin. Indirect light, on the other hand, reveals perfectly smooth features and bright white teeth. You wouldn鈥檛 want this photo as your profile pic 鈥 it has an overall red cast 鈥 but Kutulanos believes a boost of the indirect channel could give anyone an instant makeover. More generally, if you can distinguish between the direct and indirect natural light in a scene, you will be better equipped to manipulate images, for example using Photoshop-style edits.
For Kutulanos, this is just another illustration of the level of control that could see computational photography supplanting traditional techniques. 鈥淚 have colleagues who have been announcing the death of the camera for a few years already,鈥 he says.
鈥淪ome people have been announcing the death of the camera for years鈥
And that death is on the cards. If the lens is the essence of traditional photography, then the final nail in the coffin will be imaging technology that has no need of one. Such a device has been designed by neuroscientist-turned-engineer Patrick Gill at Rambus, a technology company based in California. It forgoes a lens for a single spiral-shaped aperture, which diffracts incoming light into numerous mini-spiral 鈥減ixels鈥, each as unique as a snowflake. Although these mini-spirals arrive at the sensor in a blurry mess, their individual shapes allow an algorithm to unpack them (see diagram above). It is then just a matter of recording their intensity 鈥 as a digital camera does with normal pixels 鈥 to recreate the original scene.
For now, lensless technology is primitive. Gill shows an image of the Mona Lisa captured on his device. Although the enigmatic smile is there, the painting appears grey and blotchy. Nonetheless, it speaks volumes for the power of computational photography that, to preserve a moment in time, the most recognisable part of a camera is not even necessary. 鈥淚 shouldn鈥檛 even call them cameras,鈥 says Gill. 鈥淚f my Rambus colleagues were here right now they鈥檇 give me a slap on the wrist!鈥
Not much bigger than a breadcrumb, and incredibly cheap to manufacture, Gill鈥檚 lensless cameras 鈥 or 鈥渓ensless smart sensors鈥, as he is supposed to call them 鈥 could grace the surface of almost anything, and make photography even more ubiquitous than it already is. That raises questions about the effect incessant photography is having on our perception of the past (see 鈥Making memories鈥), as well as questions about the future of those who make a living from being in the right place at the right time. If cameras are everywhere, and capturing the decisive moment is simply a matter of trawling back through an endless photostream, what place is there for a pro?
Pannack stands by her trusty Hasselblad, despite having just returned from Romania to discover that security scanners had blanked 30 rolls of film. 鈥淚t鈥檚 kind of like saying, if we reinvented the paintbrush, would all painters be out of a job?鈥 she says. 鈥淚 don鈥檛 think so, because they would still need an eye 鈥 they need to know what works and what doesn鈥檛.鈥
But with new technologies making photography snappier than ever, perhaps the rest of us can work on cultivating that expert eye.
Competition: To be in with a chance to win a Lytro camera click here
Leader: 鈥Who said the camera never lies?鈥
Making memories
You either can鈥檛 stand them or are one yourself: people who choose to witness everything through their smartphone cameras. Those outstretched arms and selfie sticks have become a common sight. But is the habit affecting our ability to remember?
In 2013, psychologist at Fairfield University in Connecticut decided to find out, by . One group took no photos, another took photos of everything, and a third group took photos only of specific details. She then tested what they recalled.
Those who took photos indiscriminately remembered less than those who browsed the exhibits without a camera, possibly because the frequent snappers relied on the cameras to record what they were seeing, says Henkel. But those who used their cameras to zoom in on the details of exhibits recalled just as much as those without cameras. 鈥淚t focuses your attention,鈥 says Henkel. 鈥淎ll research tells us that focusing your attention is going to lead to better memory.鈥
Henkel鈥檚 study suggests that constant use of cameras could be free from ill effects, as long as we are selective and pay attention to what we are taking photos of. But photography can skew memory in another way.
In a classic 2002 experiment, , now at the University of Warwick, UK, and her colleagues gave a group of adults fake photos of a balloon ride and told them it took place when they were children. , or at least certain details.
For Henkel, this shows the dangers of the kind of photo-manipulation that computational photography will make all the easier (see main story).
This article appeared in print under the headline 鈥淧hoto finish鈥
