Soon, the model showcasing your online clothing purchases may not have actually made the pose in the picture. That鈥檚 because a neural network can now repose human beings and change their clothes in photographs without losing key details.
at Virginia Tech in Blacksburg and her colleagues developed an algorithm that breaks down a source image into constituent body parts, with a neural network identifying where key joints and limbs are. It is then fed the target pose of how the user wants the model to stand, before it identifies the new positions of relevant body parts.
Advertisement
At the same time, the model uses generative adversarial networks聽鈥� the technology behind deepfake videos and images 鈥� to reposition key elements, such as a model鈥檚 face or the clothes they are wearing, onto the new pose.
This requires flattening out the face and clothes into a 2D image and then wrapping it back round the reposed body using an ultraviolet colour-coded heat map to show where the relevant parts need to go. The same flattening and rewrapping technique allows the neural network to swap out clothes by cutting and pasting different items onto a body.
The neural network is largely successful in reposing people and swapping clothes, but struggles to accurately move hands because of a lack of detail around fingers in DensePose, one of the pre-existing technologies that the process uses. It also becomes less accurate when asked to repose people of colour, mutating their faces in a unnatural way.
鈥淲e use fashion data sets, and when you think about models, the data is not very diverse,鈥� says AlBahar. 鈥淚t鈥檚 very difficult for us to train on that data set and extend it to a more diverse set of people.鈥� She hopes to improve the model by training it on more diverse data sets.
鈥淭his is a mind-blowing use of style-guided and human-conditioned image generation,鈥� says at the University of Udine, Italy. 鈥淚t could be improved by fixing the biases. But this could give huge possibilities to fashion design and retailers.鈥�
Reference:
