Latent variable model is a common probabilistic framework which aims to estimate the hidden states of observations. More specifically, the hidden states can be the position of a robot, the low dimensional representation of an observation. Meanwhile, various latent variable models have been explored, such as hidden Markov models (HMM), Gaussian mixture model (GMM), Bayesian Gaussian process latent variable model (BGPLVM), etc. Moreover, these latent variable models have been successfully applied to a wide range of fields, such as robotic navigation, image and video compression, natural language processing. So as to make the learning of latent variable more efficient and robust, some approaches seek to integrate latent variables with related priors. For instance, the dynamic prior can be incorporated so that the learned latent variables take into account the time sequence. Besides, some methods introduce inducing points as a small set representing the large size latent variable to enhance the optimization speed of the model. Though those priors are effective to facilitate the robustness of the latent variable models, the learned latent variables are inclined to be dense rather than diverse. This is to say that there are significant overlapping between the generated latent variables. Consequently, the latent variable model will be ambiguous after optimization. Clearly, a proper diversity prior play a pivotal role in having latent variables capture more diverse features of the observations data. In this thesis, we propose diversified latent variable models incorporated by different types of diversity priors, such as single/dual diversity encouraging prior, multi-layered DPP prior, shared diversity prior. Furthermore, we also illustrate how to formulate the diversity priors in different latent variable models and perform learning, inference on the reformulated latent variable models.