Constrained multi-objective optimization problems (cMOPs) are complex because the optimizer should balance not only between exploration and exploitation, but also between feasibility and optimality. This article suggests a parameter-free constraint handling approach called constrained non-dominated sorting (CNS). In CNS, each solution in a population is assigned a constrained non-dominated rank based on its constraint violation degree and Pareto rank. An improved hybrid multi-objective optimization algorithm called cMOEA/H for solving cMOPs is proposed. Additionally, a dynamic resource allocation mechanism is adopted by cMOEA/H to spare more computational efforts for those relatively hard sub-problems. cMOEA/H is first compared with the baseline algorithm using an existing constraint handling mechanism, verifying the advantages of the proposed constraint handling mechanism. Then cMOEA/H is compared with some classic constrained multi-objective optimizers, experimental results indicating that cMOEA/H could be a competitive alternative for solving cMOPs. Finally, the characteristics of cMOEA/H are studied.