In Bayesian statistics, a general and widely used approach to extract information from (complex) posterior distributions relies on Markov chain Monte Carlo (MCMC) methods. Although MCMC samplers provide powerful tools for Bayesian inference in various applications, they are often computationally intensive due to their iterative nature. In this thesis, we develop a much faster alternative for approximate Bayesian inference called ‘‘Laplace-P-splines’’ (LPS) that combines Laplace approximations to selected posterior distributions and P-splines for flexible modeling of smooth model terms. The first part of the thesis starts with the implementation of LPS in the framework of survival analysis. First, the synergy between Laplace’s method and P-splines is exploited in the Cox proportional hazards model where the log baseline hazard is specified as a linear combination of cubic B-splines. The LPS approach is then extended to the promotion time cure model that intrinsically accounts for long-term survivors who will never experience the event of interest whatever the duration of the follow-up. The second part of the thesis expands the LPS methodology to (generalized) additive models. Efficient methods for exploring the posterior penalty space are proposed based on analytical formulas for the gradient and Hessian of the joint posterior of the penalty vector. Finally, the third part is devoted to a software package written in the R language that gathers several routines to implement the proposed methodology in latent Gaussian models. (SC - Sciences) -- UCL, 2020