{"references": ["I. C. Yoo and D. Yook, \"Automatic sound recognition for the hearing\nimpaired,\" IEEE Trans. Consumer Electron., vol. 54, no. 4, pp. 2029-\n2036, Nov. 2008.", "http://www.cdpf.com.cn/ggtz/content/2008-05/04/content 25053452.htm\n(in Chinese)", "W. Gao, Y. Chen, G. Fang, C. Yang, D. Jiang, and et al., \"HandTalker II:\na Chinese sign language recognition and synthesis system,\" in Proc. The\n8th Int. Conf. on Control, Automation, Robotics and Vision (ICARCV\n2004), pp. 759-764, 2004.", "L. R. Rabiner, \"A tutorial on hidden Markov models and selected\napplications in speech recognition,\" Proceedings of the IEEE, vol. 77,\nno.2, pp. 257-286, 1989.", "J. L. Gauvain and C. H. Lee, \"Maximum a posteriori estimation for\nmultivariate Gaussian mixture observations of Markov chains,\" IEEE\nTrans. Speech Audio Process., vol. 2, no. 2, pp. 291-298, Apr. 1994.", "B. J. Frey and D. Dueck, \"Clustering by passing messages between data\npoints,\" Science, vol. 315, no. 5814, pp. 972-976, 2007.", "J. Takahashi and S. Sagayama, \"Vector-field-smoothed Bayesian learning\nfor incremental speaker adaptation,\" in International Conference on\nAcoustics, Speech, and Signal Processing, pp. 696-699, 1995.", "C. F. Li, M. H. Siu, and J. S. K. Au-Yeung, \"Recursive likelihood\nevaluation and fast search algorithm for polynomial segment model with\napplication to speech recognition,\" IEEE Trans. Audio Speech Lang.\nProcess., vol. 14, no.5, pp. 1704-1718, 2006.", "C. Wang, X. Chen, and W. Gao, \"Generating data for signer adaptation,\"\nin Proc. Gesture Workshop, pp. 114-121, 2007.\n[10] X. Zhu, \"Semi-supervised learning literature survey,\" Computer Sciences\nTechnical Reports 1530, University of Wisconsin Madison, 2008."]}

An adaptive Chinese hand-talking system is presented in this paper. By analyzing the 3 data collecting strategies for new users, the adaptation framework including supervised and unsupervised adaptation methods is proposed. For supervised adaptation, affinity propagation (AP) is used to extract exemplar subsets, and enhanced maximum a posteriori / vector field smoothing (eMAP/VFS) is proposed to pool the adaptation data among different models. For unsupervised adaptation, polynomial segment models (PSMs) are used to help hidden Markov models (HMMs) to accurately label the unlabeled data, then the "labeled" data together with signerindependent models are inputted to MAP algorithm to generate signer-adapted models. Experimental results show that the proposed framework can execute both supervised adaptation with small amount of labeled data and unsupervised adaptation with large amount of unlabeled data to tailor the original models, and both achieve improvements on the performance of recognition rate.