Striking a balance between objective optimization and constraint satisfaction is essential for solving constrained multi-objective optimization problems (CMOPs). Nevertheless, most existing evolutionary algorithms face significant challenges on CMOPs with intricate infeasible regions. To tackle these challenges, this paper proposes an adaptive two-population evolutionary algorithm, named ATEA, which dynamically exploits promising information under infeasible solutions to facilitate objective optimization and constraint satisfaction. Specifically, a two-population collaboration mechanism is designed to balance the unconstrained Pareto front search and constrained Pareto front search. Moreover, an adaptive constraint handling strategy is presented to reasonably deploy search resources. Furthermore, a promising infeasibility-based environmental selection and an elitist feasibility-based environmental selection are developed for the two populations to break through complex infeasible barriers and enhance selection pressure, respectively. Comparison experimental results of ATEA with five state-of-the-art algorithms on 33 benchmark test problems and 4 real-word CMOPs demonstrate that ATEA performs competitively with the chosen designs.