Estimation of Distribution Algorithms (EDAs) are a set of techniques that belong to the field of Evolutionary Computation. They are similar to Genetic Algorithms (GAs), in the sense that, given a problem, they use a population of individuals to represent solutions, and this population is made to evolve towards the most promising solutions. However, instead of using the usual GA-operators such as mutation or crossover, EDAs learn a probabilistic model that tries to capture the main characteristics of the problem. Based on this idea, several EDAs have been introduced in the last years, showing a good performance and being able to solve problems of different complexity. One important drawback of EDAs is the significant computational effort required by the utilization of probabilistic models, when applied to real-world problems. This fact has led the research community to apply parallel schemes to EDAs, as a viable way to reduce execution times. Schemes already proposed for GAs have been used as the foundation for these parallel schemes. In this chapter, we make a review of parallel EDAs, with a main focus: identifying those parts that are susceptible of parallelization. Then we describe a collection of parallelization strategies proposed in the literature. Additionally, we provide some recommendations for those that are considering the implementation of parallel EDAs on state-of-the-art parallel computers.