Open Access

Joint Rendering and Segmentation of Free-Viewpoint Video

EURASIP Journal on Image and Video Processing20102010:763920

DOI: 10.1155/2010/763920

Received: 7 December 2009

Accepted: 18 August 2010

Published: 16 September 2010


This paper presents a method that jointly performs synthesis and object segmentation of free-viewpoint video using multiview video as the input. This method is designed to achieve robust segmentation from online video input without per-frame user interaction and precomputations. This method shares a calculation process between the synthesis and segmentation steps; the matching costs calculated through the synthesis step are adaptively fused with other cues depending on the reliability in the segmentation step. Since the segmentation is performed for arbitrary viewpoints directly, the extracted object can be superimposed onto another 3D scene with geometric consistency. We can observe that the object and new background move naturally along with the viewpoint change as if they existed together in the same space. In the experiments, our method can process online video input captured by a 25-camera array and show the result image at 4.55 fps.

Publisher note

To access the full article, please see PDF.

Authors’ Affiliations

Graduate School of Information Science and Technology, The University of Tokyo


© Masato Ishii et al. 2010

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.