Computer Graphics
TU Braunschweig

Video Based Reconstruction of 3D People Models


Video Based Reconstruction of 3D People Models

This paper describes how to obtain accurate 3D body models and texture of arbitrary people from a single, monocular video in which a person is moving. Based on a parametric body model, we present a robust processing pipeline achieving 3D model fits with 5mm accuracy also for clothed people. Our main contribution is a method to nonrigidly deform the silhouette cones corresponding to the dynamic human silhouettes, resulting in a visual hull in a common reference frame that enables surface reconstruction. This enables efficient estimation of a consensus 3D shape, texture and implanted animation skeleton based on a large number of frames. We present evaluation results for a number of test subjects and analyze overall performance. Requiring only a smartphone or webcam, our method enables everyone to create their own fully animatable digital double, e.g., for social VR applications or virtual try-on for online fashion shopping.

 

Code & Dataset

Download code and dataset here.

 

In Press


Author(s): Thiemo Alldieck, Marcus Magnor, Weipeng Xu, Christian Theobalt, Gerard Pons-Moll
Published: June 2018
Type: Article in conference proceedings
Book: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE)
ISBN: 978-1-5386-6420-9
DOI: 10.1109/CVPR.2018.00875
Presented at: Conference on Computer Vision and Pattern Recognition (CVPR) 2018
Note: CVPR Spotlight Paper
Project(s): Comprehensive Human Performance Capture from Monocular Video Footage  Immersive Digital Reality 

BibTex PDF Link Suppl. Material Code Video arXiv 



@inproceedings{alldieck2018video,
  title = {Video Based Reconstruction of 3D People Models},
  author = {Alldieck, Thiemo and Magnor, Marcus and Xu, Weipeng and Theobalt, Christian and Pons-Moll, Gerard},
  booktitle = {{IEEE}/{CVF} Conference on Computer Vision and Pattern Recognition ({CVPR})},
  isbn = {978-1-5386-6420-9},
  doi = {10.1109/{CVPR}.2018.00875},
  note = {{CVPR} Spotlight Paper},
  pages = {8387--8397},
  month = {Jun},
  year = {2018}
}

Authors