ÐÓ°ÉÔ­´´

Hollywood has fallen head over heels in love with computer graphics.
Superficially, it’s been one of those whirlwind romances of which blockbusters
are made. In reality, the relationship has been on the cards for a long
time, well rehearsed by both sides over the years. For cinema audiences,
there is a growing fascination with spotting the latest computer effects
– even among those who claim to prefer the days of B-movie flying saucepan
lids to the brave new worlds of synthesised reality. But it’s not just what
you see on the screen that counts. Even more important is what you can’t
detect.

Film-makers always looked likely to benefit from developments in the
worlds of engineering and computer-aided design, but it was not until the
early 1980s that they directed their first simulated images on screen, in
Tron and The Last Starfighter. Though these movies were commercial flops,
the critics generally put this down to the scripts rather than the effects.
Film-makers persisted, encouraged by the increasing reliability and quality
of the technology – as well as its falling cost – and they have eventually
come good.

While the detailed processes of computer animation vary a great deal,
the basic stages are much the same in every case. First, animators must
build a model of the required shape. This is generally constructed from
a mesh of polygons, which appears on the computer screen as a three-dimensional
object that looks like a wire frame; it can be viewed from any angle, just
like a real object. The more polygons used, the more subtle and realistic
the surface detail will be. Animators derive information for constructing
the mesh in one of three ways: they build up a grid of geometric points
from scratch, assemble a number of precalculated primitive shapes stored
in the computer’s memory, or scan real objects with lasers.

Animation comes next, though it must be considered while the modelling
is being done – sculpting a statue is one thing, giving it joints that will
move naturally is another matter altogether. Human skills are still extremely
important in this respect. While machines can be programmed to ensure that
the motion portrayed follows natural laws, artists can tweak models so that
their movements, although unnatural, look more realistic on the cinema screen.

SPEEDY ANIMATION

Nevertheless, automation can ease the job of animators. For instance,
they need create only the key positions in a motion sequence, say four pictures
a second; the computer can then be used to fill in the other 20 pictures
a second. This technique is known as ‘inbetweening’. Another useful process
is ‘motion capture’, which does for animation what laser scanning does for
modelling. Patterns of live motion can be tracked with devices such as electromagnetic
coils and ‘datagloves’, whose movements trigger corresponding electrical
pulses that can be converted into binary code and stored. Such devices are
now common in virtual reality systems in up-market amusement arcades. More
usually, however, motion is captured simply by filming the real thing and
digitising the results, with the action shot from more than one angle to
produce the perspective required for a model moving in three dimensions.

This was the process used in the hugely successful film Terminator
2 – Judgement Day, released last year. The star of the film is the ‘T-1000’,
a perfect post-apocalyptic killing machine. Supposedly constructed from
an intelligent liquid metal, it is able to mimic the characteristics of
anything it touches, changing its shape and colour at will. Early in the
film the T-1000 adopts the shape of a police officer it has killed, subsequently
dissolving between the officer and its liquid metal form.

‘On Terminator 2, the heart of the matter was the character of the T-1000
and the way that it moved,’ recalls Mark Dippe, an animator with Industrial
Light And Magic, the best-known of all computer-generated special effects
companies. ‘We spent two days working with the actor to see how his body
worked, so that we would be able to replicate it. Two synchronised cameras
filmed him from different angles while he was running, with markings drawn
across his body as coordinates that could be fed into the computer. We then
built a skeleton that would walk and run with exactly the same characteristics
as the actor.’

The third stage of computer animation is applying a surface texture
or colour to the model on each frame of the animation, a process known as
‘rendering’. Aside from simple colours or textures, the effect could be
something more complex if the object has to merge realistically with live
action.

All the initial developments in computer graphics for films were possible
because of advances in software, with innovations coming just as much from
the engineering research laboratories as from the entertainments industry.
But, bit by bit, most of the facilities required by computer graphics specialists
found their way into hardware. This meant that most people in the industry
could be sufficiently well served with standard workstations designed with
them in mind. These machines could model and animate productions but, until
recently, were too slow to cope with the lengthy computations needed to
render images of the same quality as that of film and to integrate them
with live action. This is where the development of parallel processing has
been crucial, providing the kind of power required at a reasonable cost.
Equally important has been the development of powerful disc storage systems,
which allow designers to play back their work as they produce it, in real
time, rather than wait to see the results on film after processing. These
units enable directors to observe and control the special effects production
process as it happens.

The final piece of hardware in the chain is the film recorder, which
converts digital data to film. The recorder scans a beam of light across
the emulsion surface of the film through a series of coloured filters. The
standard scan rate is between 2000 and 4000 lines, which can produce images
of photographic quality.

COMPUTER POWER

Special effects that once relied on traditional optical methods now
depend increasingly on computers, says Alan Fetzer, chief operations officer
at Boss Film Studios, one of Hollywood’s leading special effects companies
whose recent credits include Batman Returns, Alien3 and Ghost. A new generation
of powerful visualisation systems brings together parallel processing, a
disc storage unit and a number of workstations in a single system. Boss
recently invested in the new IBM Power Visualisation System, which incorporates
eight Intel processors and a 21-gigabyte storage unit. ‘With previous workstations
we had access to only one frame at a time but now we have full motion in
real time,’ says Fetzer.

But what keeps the special effects people happy is not necessarily good
news for actors. Meryl Streep’s latest performance in Death Becomes Her,
due for a December release in Britain, includes a scene in which her head
faces backwards. Multiple takes of Streep wearing a variety of hoods and
body stockings were required to provide the raw material for the shot.

Computers can be used in film production to generate images (CGI), to
change live action images (digital manipulation) and to combine more than
one image onto a single frame (digital compositing or matting).

CGI is the area that is developing fastest. It can be used in three
different ways. The technique can animate three-dimensional computer images
in a fully computer-generated environment, as it did in the virtual reality
extravaganza The Lawnmower Man. It can assist the production of classic
animated films, whose characters have been drawn or modelled by hand, as
was done in FernGully: The Last Rainforest. These two approaches produce
films that are transparently artificial; but the third aims to fool the
audience that the unbelievable really can happen. This approach merges live
action, image processing and synthetic objects to give audiences an alternative
view on reality. The film that rewrote the textbooks in all these respects
was Terminator 2.

The technological theme of The Lawnmower Man proved to be an ideal vehicle
for a visual approach that was predominantly generated by computers. Much
of the action of this futuristic remake of the Frankenstein myth takes place
in virtual worlds, so it was important that material represented how the
characters would be seen if the audience were viewing them through head-mounted
displays.

VIRTUALLY REAL

But CGI gave the film’s makers the edge over the real-time virtual
reality systems that are currently available. These systems cannot produce
realistic worlds because the rate at which images appear before the eyes
(the frame rate) is too slow and the detail in them (the resolution) is
poor (‘Did reality move for you?’, New ÐÓ°ÉÔ­´´, 23 May). Film-makers are
able to take their time and render scenes a frame at a time to provide
images that have the quality of film – equivalent to as many as 4000 scanning
lines – and display them at a full film rate of 24 frames per second. The
result is something that makes virtual reality appear far more impressive
than it really is.

STYLISED BUT STUNNING

Most of the scenes in The Lawnmower Man did not mix with live action
and had to work in their own right as alternative realities. The dynamics
of these scenes therefore needed to be simultaneously convincing and different.
Most famous of all was cinema’s first cybersex sequence where the leading
characters’ virtual selves blended together in a swirl of passion – though
director Brett Leonard says that the techniques used for the sequence were
not so different from those used for more conventional scenes. ‘These are
fantastical worlds. They are very stylised, extremely visually stunning
worlds – this is what I set out to do. When you direct three-dimensional
computer graphic effects, it’s like you build a set in a computer, you build
people in a computer and you move a camera the same way you would on a set,’
he says.

A completely different use of much the same technology can be seen in
FernGully: The Last Rainforest, where computers were used to assist traditional
cartoon animation, speeding up the process and creating a more familiar
kind of fantasy world. All major studios specialising in animation features
now use computer graphics as part of their production process, but FernGully,
directed by Bill Kroyer, uses them more extensively than ever before. ‘In
this film computer graphics became an integral part of the animation process
– it wasn’t just a special effect,’ he says. ‘A vast amount of the animation
process hasn’t really changed much since Snow White (1937) – and it’s amazing
how many of the processes are still not ready to change.’

The main CGI applications are in drawing backgrounds that are three-dimensional
(and so have greater depth and perspective than can be achieved with hand-drawn
methods), building complex props such as mechanical vehicles, and generating
difficult effects such as water and fog. In FernGully, simple character
models were also built and animated using computers as guides to motion
and perspective. The artists would then draw over these guides, filling
in the detail a frame at time without needing to spend time calculating
movements.

Computer graphics allows models to be built and viewed from any angle
before the director makes a final decision, something that would be prohibitively
costly if the models were drawn by hand. When the creative decisions have
been taken, a film’s computer-generated elements are printed as outline
drawings. Just like the hand-drawn parts, they are photocopied onto sheets
of celluloid, known as cels, and painted by hand. The cels are then sandwiched
between sheets of glass, making assemblies that provide groups of foreground
and background figures. These are then shot a frame at a time under a multiplane
camera, which points down through them.

CHEAP LABOUR

Though painting the cels is the most labour-intensive process of all,
Kroyer says that it is not quite ready to be replaced by computer graphics.
This is not so much a technical decision as an economic one – indeed, some
parts of FernGully were coloured by computer and it is impossible to distinguish
them from the rest of the film. Kroyer says that as long as film-makers
can find a large low-cost labour force, it will be cheaper to colour by
hand. Once this was done in Japan and Korea; nowadays the Philippines,
China and former states of the Soviet Union provide pools of cheap labour.

But most animation producers agree that lack of direct control when
working across such large geographical distances is likely to hasten the
large-scale introduction of computer ink and paint systems. FernGully may
represent the biggest step so far in this direction – but there is a lot
more change on the way. Cambridge Animation Systems, a British company,
has already developed a software package for producing animated films from
beginning to end. The package runs on a network of up to 20 workstations.
This system models, animates and paints the images, as well as providing
facilities such as a simulated multiplane camera, simulated camera movements
and even automated lip synchronisation, which allows the computer to generate
lip movements directly from the soundtrack.

When it comes to digital manipulation to create the special effects
in live action films, some of the changes are invisible to the viewer, while
others are clearly put there to make an impact. But one thing is certain,
this traditional craft centre of the industry has undergone a revolution.
The optical techniques that have been evolving since the very birth of cinema
are being replaced by the greater flexibility, efficiency and accuracy
of digital effects.

Terminator 2 was a benchmark for the new approach, featuring much CGI,
image manipulation and digital compositing. The primary state of the T-1000
is that of the human being that it has chosen to replace. But it also transforms
into other human shapes, as well as a detailed human form apparently made
of mercury, a featureless human form, an amorphous mercury-like blob and
other inanimate objects. To feature such a character, the film’s makers
had to combine three distinct aspects of computer graphics.

First, computer models had to be animated to perform in the real world.
This meant adding detailed reflections to the T-1000’s reflective body
that mirrored what was happening around him, and generating synthetic shadows
to add to the scene. The two main techniques used to render objects with
such levels of realism are known as ray tracing and radiosity. Both involve
intensive computation to calculate the paths of direct and reflected light
in an environment. Where such levels of detail are not required, such as
when we get only a fleeting glimpse of an object, a third option is to apply
a two-dimensional ‘texture map’ to a surface and then distort this map to
give an impression of reflections.

The second aspect applied was the transformation between one computer
model and another, or between a computer model and an actor or an inanimate
object. This is a software process that is becoming increasingly popular
and is known as ‘morphing’, from the term ‘image metamorphosis’. Traditional
alternatives for this technique would have included clever cuts, such as
a character changing while running through a forest and passing behind a
number of trees, or a simple dissolve.

But such traditional approaches are limited, and early applications
of digital morphing have already demonstrated that this is a particularly
powerful tool for special effects directors, creating very natural-looking
results. Three-dimensional morphing is achieved by representing the objects
before and after transformation as sets of polygons. The vertices of the
pre-transformation object are then displaced over time to coincide with
the corresponding vertices of the post-transformation object, with colour
and other attributes interpolated in the same way.

NATURAL TRANSFORMATIONS

This three-dimensional technique has been used successfully, but it
does have inherent problems because of the difficulties in finding corresponding
points that do not make the transformation look unnatural. The preferred
option, particularly where human features are involved, is two-dimensional
morphing. In its simplest form, this involves drawing a grid over a pair
of images and transforming from key points on one to key points on the other.
But as transformations must take place over a period of time and as people
move over time, grids are set up for a series of frames of each image. The
morph sequence is made up of a progressive level of transformation selected
from each pair of frames over the chosen period.

Whether characters are morphed or computer-generated, they need to be
set in the real world among live action. With digital matting, a laser beam
is used to scan images from film; the signals are then converted into binary
code that can be stored on a computer. The information can then be manipulated
and combined with other images, before the result is transferred back onto
film as a new composite image.

But as the technology develops, public expectations increase. It would
be impossible today to get away with many of the effects we still happily
watch from the classics of a generation ago. Combining the real and synthetic
worlds is now an exact science, says Dippe. ‘We have to take account of
motion blur, camera shake, exposure, film grain, shadows and so on. We have
to make an exact match between the computer camera and the real camera.’
(The ‘computer camera’ is the term used to describe the way designers build
up three-dimensional images and scenes on a computer screen.)

One of the most remarkable and original recent uses of computer-generated
special effects was in the film Batman Returns. Boss and Video Image Associates,
another special effects company, created a cast of thousands of extras –
both bats and penguins. They programmed the models with information on how
they should move, where they should go and what they should avoid, incorporating
information on how real bats and penguins flock. The models were then released
into the background of a scene to act as they saw fit.

It may not be too long before human extras are programmed in just the
same way. And there could come a time when principal actors finally price
themselves out of the market. For several years, researchers at the University
of Geneva have been trying to develop synthetic actors. And John Lasseter,
one of the most respected of all computer animation specialists and an
Oscar winner for his short film Tin Toy, is currently working on a full-length
feature film that is entirely computer-generated – ‘real’ actors and all.

Bob Swain is a freelance journalist who specialises in computer graphics
for film and television.