This paper proposes a kind of evolutionary parallel local search technique (EPLS) that integrates the reproduction mechanisms from evolutionary algorithms and simplex method. The major aim is to explore the tradeoff between exploration and exploitation for optimizing multimodal functions. It has been cost-efficiently reached by means of parallel local search using simplex method. In each generation, EPLS partitions the population into a group of subpopulations, each of which consists of several individuals with adjacent space locations. EPLS independently locates multiple local optima in these disjoint neighborhoods, thus to reduce the probability of losing the global optimum. The local search in a neighborhood speeds up the convergence rate of simplex method. Recombination, adaptive Gaussian mutation and selection are incorporated into EPLS to further enhance the ability of global exploration and exploitation. The experimental observations and the extensive comparisons show that EPLS remarkably outperforms the standard evolutionary algorithms (EA) and some hybrid ones for almost all the problems tested, thus justifying the rationality and the competitive potential of EPLS for optimizing multimodal functions, especially for those with very rugged and deceptive topological structures.