PhotoCloth · A framework to synthesize real-time photorealistic cloth animation from video input.
Horizon 2020 — Marie Skłodowska-Curie Actions
- Duration
- 2016-10-01 → 2018-09-30
- EU contribution
- €170,122
- Participants
- 1
- Scheme
- MSCA-IF-EF-ST
Lines connect the coordinator with its partners.
Results in brief
PhotoCloth: A framework to synthesize real-time photorealistic cloth animation from video input.
Among all the potential contexts where computer graphics techniques can be used, cloth animation is a particularly interesting case since, in the real world, clothing is far more than just the physical objects that we wear; clothing is a key element to show someone’s expressiveness and motion, it even defines his or her identity. Consequently, the relevance of creating interactive digitally dressed characters with photorealistic clothing is enormous, and has a direct impact in a number of industries, including: video game industry, to create digital characters, edit apparel, etc.; and textile and fashion industries, for computational design tools, e-commerce, virtual try-out, etc. Considering the huge size of these industries –the worldwide revenue of the videogame industry alone is estimated to be over $60B in 2015, and expected to grow 10% yearly, while the global revenue of the fashion industry is estimated at $1,200B–, the interest in developing robust real-time techniques for photorealistic clothing is strongly motivated. Important social and economic aspects of our everyday life will directly benefit from investigating animable digital dressed characters. The aim of this project was to investigate fundamental cloth simulation methods to develop a novel framework to synthesize real-time photorealistic cloth animation. In particular, having a video containing a heterogeneous cloth motion, the project has researched methods to incorporates image information from the video into a physical model, enabling interactive animation of video cloth motion. Such algorithms pave the way towards a fully digitized fashion and clothing industries, enabling ground-breaking applications such as virtual try-on or digital design of garments.
Data: CORDIS, © European Union
Project objective
Computer Graphics is the area of computer science that studies methods for digitally synthesizing and animating visual content. Among all the potential contexts where computer graphics techniques can be used, cloth animation is a particularly interesting case since, in the real world, clothing is far more than just the physical objects that we wear; clothing is a key element to show someone’s expressiveness and motion, it even defines his or her identity. However, cloth animation is a complex and extremely high-dimensional problem. To digitally synthesize cloth animation, a large number of properties that affect the way cloth behaves need to be estimated: textures, deformations, collisions, materials, illumination, etc. Current approaches for cloth animation tried to overcome this challenging problem following two main trends: image-based methods use captured data to construct a low-dimensional model to digitally synthesize new animations, however they can only sample a small portion of the high-dimensional space of cloth and poses; the physics-based methods aim to simulate cloth only using mathematical equations that express physics laws, however, they are computationally expensive and have trouble replicating real-world behavior. This fellowship will investigate a new model for cloth simulation that combines a physical-based method with image-based infomation to generate real-time believable cloth animation. The new model will use a multi-scale framework to handle the dynamic geometry and appearance at different levels of detail. The most salient dynamic geometric properties of the animation will be handled by a low-resolution representation of the cloth using a physics-based model, which reduces the high-dimensionality of the pose space to a lower-dimensional subspace. Mid- and fine-scale details such as shading, wrinkles and appearance will be incorporated by an image-based approach, using the input imagery to learn to predict those properties.
Original text from CORDIS.
Participants
- UNIVERSIDAD REY JUAN CARLOS · MostolesCoordinatorSpain
Links
Data: CORDIS, © European Union