Abstract Mixed logit models with normally distributed random coefficients are typically estimated under the extreme assumptions that either the random coefficients are completely independent or fully correlated. A factor structured covariance offers a range of alternatives between these two assumptions. However, because these models are more difficult to estimate they are not frequently used to model preference heterogeneity. This paper develops a simple expectation-maximization algorithm for estimating mixed logit models when preferences are generated from a factor structured covariance. The algorithm is easy to implement for both exploratory and confirmatory factor models. The estimator is applied to stated-preference survey data from residential energy customers (Train, 2007). Comparing the fit across five different models, which differed in their assumptions on the covariance of preferences, the results show that all three factor specifications produced a better fit of the data than the fully correlated model measured by BIC and two out of three performed better in terms of AIC.