Surface effects including 鈥渧irtual make-up鈥� can be automatically added to video clips using software developed by US researchers.
The software 鈥� called RotoTexture 鈥� provides a less complicated way to artificially modify a surface, including a person鈥檚 face. Normally, a 3D model of a scene must be built in order for it to be modified in this way.
鈥淥ur approach is quite original because it only needs the minimum input: a raw video clip,鈥� explains Hui Fang, the computer-vision researcher who developed the system with colleague John Hart while at the University of Illinois at Urbana-Champaign, US, and who at now works at Google.
Advertisement
Videos show RotoTexture being used to (mov format, 19.2MB) and (mov format, 6.5MB).
Deformed surface
RotoTexture first estimates the orientation of a target surface based on its shading, Fang told New 杏吧原创. Then, using this information, the software applies either an image or a textured pattern to the surface.
鈥淲e treat an image to paste as a large rubber band,鈥� explains Hui. 鈥淭hen we try to paste that rubber band [onto a surface] such that the stretching within the rubber band is minimised. As a result, the image will appear to follow the undulation of the surface.鈥�
To apply a texture, the software segments a surface into small overlapping patches, each of which is applied in the correct order and with the correct orientation to make a seamless surface.
Consistent tracking
Features of the original surface are compared with the modified picture to ensure that the texturing is consistent between frames. But Fang concedes that better motion-tracking software could improve the process, pointing out that, currently, the applied texture sometimes 鈥渟wims鈥� slightly on the underlying surface.
Geoff Hodbod, founder of UK computer animation firm 3D Imaging, says RotoTexture would be attractive to many animators. 鈥淚 could see this being used in children鈥檚 TV to make people鈥檚 faces look more interesting,鈥� he says. 鈥淚t would also have great potential in science-fiction productions.鈥�
Currently, skilled artists must build a 3D wire-frame computer model of an object or surface in order for it to be re-textured, Hodbod explains. 鈥淸RotoTexture] looks like a prospective alternative,鈥� he says, 鈥渂ut it does slip about a bit, so I don鈥檛 think it could immediately be used for a professional production.鈥� Providing this flaw can be fixed, Hodbod says the system could prove a big hit with animators.
