4D (3D Dynamic) statistical models of conversational expressions and the synthesis of highly-realistic 4D facial expression sequences

Doctoral thesis English OPEN
Vandeventer, Jason
  • Subject: QA75

In this thesis, a novel approach for modelling 4D (3D Dynamic) conversational interactions and synthesising highly-realistic expression sequences is described.\ud To achieve these goals, a fully-automatic, fast, and robust pre-processing pipeline was developed, along with an approach for tracking and inter-subject registering 3D sequences (4D data). A method for modelling and representing sequences as single entities is also introduced. These sequences can be manipulated and used for synthesising new expression sequences. Classification experiments and perceptual studies were performed to validate the methods and models developed in this work.\ud To achieve the goals described above, a 4D database of natural, synced, dyadic conversations was captured. This database is the first of its kind in the world.\ud Another contribution of this thesis is the development of a novel method for modelling conversational interactions. Our approach takes into account the time-sequential nature of the interactions, and encompasses the characteristics of each expression in an interaction, as well as information about the interaction itself.\ud Classification experiments were performed to evaluate the quality of our tracking, inter-subject registration, and modelling methods. To evaluate our ability to model, manipulate, and synthesise new expression sequences, we conducted perceptual experiments. For these perceptual studies, we manipulated modelled sequences by modifying their amplitudes, and had human observers evaluate the level of expression realism and image quality.\ud To evaluate our coupled modelling approach for conversational facial expression interactions, we performed a classification experiment that differentiated predicted frontchannel and backchannel sequences, using the original sequences in the training set. We also used the predicted backchannel sequences in a perceptual study in which human observers rated the level of similarity of the predicted and original sequences. The results of these experiments help support our methods and our claim of our ability to produce 4D, highly-realistic expression sequences that compete with state-of-the-art methods.
  • References (135)
    135 references, page 1 of 14

    [64] M. Castelan. Face Shape Recovery from a Single Image View. PhD thesis, University of York, 2006.

    [65] C.-C. Chang and C.-J. Lin. LIBSVM: A library for support vector machines. Intelligent Systems and Technology (TIST), ACM Transactions on, 2(3):27:1{ 27, 2011.

    [66] G. Chen. Edge detection by regularized cubic B-spline tting. Systems, Man and Cybernetics, IEEE Transactions on, 25(4):636{643, 1995.

    [67] H. Chipman, T. J. Hastie, and R. Tibshirani. Clustering microarray data. In T. Speed, editor, Statistical Analysis of Gene Expression Microarray Data, chapter 4, pages 159{200. Chapman & Hall/CRC, 2003.

    [68] P. Cignoni, M. Corsini, and G. Ranzuglia. Meshlab: An open-source 3D mesh processing system. European Research Consortium for Informatics and Mathematics (ERCIM) News, 73:45{46, 2008.

    [69] J. Cockburn, M. Bartlett, J. Tanaka, J. Movellan, M. Pierce, and R. Schultz. SmileMaze: A tutoring system in real-time facial expression perception and production in children with Autism Spectrum Disorder. In Automatic Face & Gesture Recognition (FG 2008), Workshop on Facial and Bodily expressions for Control and Adaptation of Games (ECAG), IEEE International Conference on, pages 978{986, 2008.

    [70] T. Coleman. Estimating the correlation of non-contemporaneous time-series. Social Science Research Network (SSRN), 2007.

    [71] T. F. Cootes. Model-based methods in analysis of biomedical images. In R. Baldock and J. Graham, editors, Image Processing and Analysis: A Practical approach, chapter 7, pages 223{248. Oxford University Press, 1999.

    [72] T. F. Cootes, G. J. Edwards, and C. J. Taylor. Active appearance models. In Proceedings of the 5th European Conference on Computer Vision (ECCV 1998), volume 1407, pages 484{498. Springer, 1998.

    [73] T. F. Cootes, G. J. Edwards, and C. J. Taylor. Active appearance models. Pattern Analysis and Machine Intelligence (TPAMI), IEEE Transactions on, 23(6):681{685, 2001.

  • Metrics
    views in OpenAIRE
    views in local repository
    downloads in local repository

    The information is available from the following content providers:

    From Number Of Views Number Of Downloads
    Online Research @ Cardiff - IRUS-UK 0 214
Share - Bookmark