Integrating multiple depth sensors into the virtual video camera

In this ongoing work, we present our efforts to incorporate Microsoft Kinect depth sensors into a multi camera system for free view-point video. Both the video cameras and the depth sensors are consumer grade. Our free-viewpoint system, the Virtual Video Camera, uses image-based rendering to create novel views between widely spaced (up to 15 degrees) cameras, using dense image correspondences. The introduction of multiple depth sensors into the system allows us to obtain approximate depth information for many pixels, thereby providing a valuable hint for estimating pixel correspondences between cameras.