Now we are accustomed to the wonders of 3D, turning movies interactive might be the next step in cinema. But how will the technologists do it?
IT IS time for cinema to take its next step. 3D technology now fills our screens with beautifully rendered characters and virtual environments, but we could have so much more.
So says Dennis Del Favero, director of what he calls the world鈥檚 first 3D interactive film, Scenario. Rather than having audience members sit back and enjoy the action, the interactive narrative has them drive the story.
Advertisement
Undoubtedly, the ultimate synthetic interactive environment must be the virtual worlds generated by Star Trek鈥榮 鈥淗olodeck鈥�. To date, steps in this direction have been restricted because computer-generated characters cannot yet understand and speak in natural language. One solution is to sidestep the need for language and interact with audience members using physical markers, like movement.
鈥淔ilm-makers sidestep language and interact with audience members using movement鈥�
In Scenario, which is loosely based on the life of Elisabeth Fritzl, the Austrian woman imprisoned by her father in a cellar for 24 years, audience members鈥� movements around the cinema are tracked using 16 near-infrared cameras. Treading the line between a movie and a game, audience members are introduced to the plot, and assigned avatars and a mission: to collect virtual body parts and return them to an oversized baby. But they must work against artificially intelligent sentinel avatars, which use information from the cameras and the position of objects in the film鈥檚 virtual world to plan their actions. For example, the sentinels can only move the baby鈥檚 head if they are next to it, but might better achieve their objective by pushing an audience member鈥檚 avatar.
Makers of interactive films can also hook into physiological reactions. Earlier this year, Unsound debuted in Austin, Texas. In this horror film, the visuals, music score and sound effects change depending on the heart rate and skin response of its collective audience members. For example, something horrific happens to the lead character, but in one version of the scene the audience can see it happening, while in another they can only hear it. 鈥淚f the audience is highly reactive we will not show the graphic scene, and if the audience is bored to tears we would,鈥� says Ben Knapp of BioControl Systems, a technology firm based in San Francisco that collaborated on the film. According to Knapp, amalgamating the average emotional response of an audience overcomes any 鈥渘oise鈥� in the data 鈥� such as an audience member thinking 鈥淚 need to pee.鈥�
Approaching the hurdle of language recognition, Marc Cavazza at Teesside University, UK, has created a computer system that detects the emotional content of speech. While it takes no notice of the user鈥檚 words per se, it categorises speech according to a range of attributes in their voice including pitch, duration and pauses. Using this technology, you can enjoy a conversation of sorts with a virtual character from Gustave Flaubert鈥檚 Madame Bovary. The character鈥檚 responses depend on predispositions based on their personality in the novel.
The Holodeck is still some way off, then. According to Cavazza, for complex plots to be driven by the audience, virtual characters must understand words.
Even once this is achieved, another issue appears: confining the narrative. Computer-memory constraints mean that an interactive movie cannot allow for unlimited plot choices. One work-around is to build a movie like a choose-your-own-adventure book, where audiences only influence the plot at specific points. But Michael Mateas at the University of California, Santa Cruz, reckons it would be more interesting if interactive films are developed as a virtual world, where the plot is directed by characters and their relationships.
Along these lines, Mateas has created Prom Week, which follows a fictional group of high school students in the week before their final dance. To be released on Facebook in August, will immerse the player in hallway politics as they dictate the future of the virtual students through a range of options. Each interaction between characters is recorded into a database, the sum of these interactions driving the plot鈥檚 direction by evolving the characters鈥� sentiments towards each other. 鈥淲e want it to feel like these characters are alive,鈥� says Mateas.