Robust deformable shape reconstruction from monocular video with manifold forests

Article English OPEN
Tao, L. ; Matuszewski, B. (2016)
  • Publisher: Springer
  • Related identifiers: doi: 10.1007/s00138-016-0769-3, doi: 10.13039/100010044
  • Subject: Deformable shape reconstruction | Computer Vision and Pattern Recognition | Software | Manifold forests | Hardware and Architecture | Computer Science Applications | Missing data and outliers | Nonlinear manifold learning
    acm: ComputingMethodologies_COMPUTERGRAPHICS | ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION

<p>Existing approaches to recover structure of 3D deformable objects and camera motion parameters from an uncalibrated images assume the object’s shape could be modelled well by a linear subspace. These methods have been proven effective and well suited when the deformations are relatively small, but fail to reconstruct the objects with relatively large deformations. This paper describes a novel approach for 3D non-rigid shape reconstruction, based on manifold decision forest technique. The use of this technique can be justified by noting that a specific type of shape variations might be governed by only a small number of parameters, and therefore can be well represented in a low-dimensional manifold. The key contributions of this work are the use of random decision forests for the shape manifold learning and robust metric for calculation of the re-projection error. The learned manifold defines constraints imposed on the reconstructed shapes. Due to a nonlinear structure of the learned manifold, this approach is more suitable to deal with large and complex object deformations when compared to the linear constraints. The robust metric is applied to reduce the effect of measurement outliers on the quality of the reconstruction. In many practical applications outliers cannot be completely removed and therefore the use of robust techniques is of particular practical interest. The proposed method is validated on 2D points sequences projected from the 3D motion capture data for ground truth comparison and also on real 2D video sequences. Experiments show that the newly proposed method provides better performance compared to previously proposed ones, including the robustness with respect to measurement noise, missing measurements and outliers present in the data.</p>
  • References (45)
    45 references, page 1 of 5

    1. Akhter, I., Sheikh, Y., Khan, S.: In defense of orthonormality constraints for nonrigid structure from motion. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1534-1541 (2009)

    2. Akhter, I., Sheikh, Y., Khan, S., Kanade, T.: Trajectory space: a dual representation for nonrigid structure from motion. IEEE Trans. Pattern Anal. Mach. Intell. 33(7), 1442-1456 (2011)

    3. Arias, P., Randall, G., Sapiro, G.: Connecting the out-of sample and pre-image problems in kernel methods. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8 (2007)

    4. Bartoli, A., Gay-Bellile, V., Castellani, U., Peyras, J., Olsen, S., Sayd, P.: Coarse-to-fine low-rank structure-from-motion. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8 (2008)

    5. Berg, M., Cheong, O., Kreveld, M., Overmars, M.: Computational Geometry: Algorithms and Applications. Springer, Berlin (2008)

    6. Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 690-696 (2000)

    7. Brunet, F., Hartley, R., Bartoli, A., Navab, N., Malgouyres, R.: Monocular template-based reconstruction of smooth and inextensible surfaces. In: Asian Conference on Computer Vision, pp. 52-66. Springer (2011)

    8. Buchanan, A., Fitzgibbon, A.: Damped newton algorithms for matrix factorization with missing data. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 316-322 (2005)

    9. Coifman, R., Lafon, S.: Diffusion maps. Appl. Comput. Harmonic Anal. 21(1), 5-30 (2006)

    10. Coifman, R., Lafon, S., Lee, A.B., Maggioni, M., Nadler, B., Warner, F., Zucker, S.W.: Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps. Natl. Acad. Sci. 102(21), 7426-7431 (2005)

  • Metrics
    No metrics available
Share - Bookmark