When tackling imbalanced constrained multi-objective optimization problems (CMOPs) with simultaneous convergence-hard and diversity-hard constraints, a critical issue is to balance the diversity and convergence of populations. To address this issue, this paper proposes a hybrid algorithm which combines an improved epsilon constraint-handling method (IEpsilon) with a multi-objective to multi-objective (M2M) decomposition approach, namely M2M-IEpsilon. The M2M decomposition mechanism in M2M-IEpislon has the capability to deal with imbalanced objectives. The IEpsilon constraint-handling method can prevent populations falling into large infeasible regions, thus improves the convergence performance of the proposed algorithm. To verify the performance of the proposed M2M-IEpsilon, a series of imbalanced CMOPs with simultaneous convergence-hard and diversity-hard constraints, namely ICD-CMOPs, is designed by using the DAS-CMOPs framework. Six state-of-the-art constrained multi-objective evolutionary algorithms (CMOEAs), including CM2M, CM2M2, NSGA-II-CDP, MOEA/D-CDP, MOEA/D-IEpsilon and PPS-MOEA/D, are employed to compare with M2M-IEpsilon on the ICD-CMOPs. Through the analysis of experimental results, the proposed M2M-IEpsilon is superior to the other six algorithms in solving ICD-CMOPs, which illustrates the superiority of the proposed M2M-IEpsilon in dealing with ICD-CMOPs with simultaneous convergence-hard and diversity-hard constraints.