A hybrid adaptive normalization based constraint handling approach is proposed in the present study. In most constrained optimization problems, constraints may be of different scale. Normalization of constraints is crucial for the efficient performance of a constraint handling algorithm. A growing number of researchers have proposed different strategies using bi-objective methodologies. Classical penalty function approach is another common method among both evolutionary and classical optimization research communities due to its simplicity and ease of implementation. In the present study, we propose a hybrid approach of both bi-objective method and the penalty function approach where constraints are normalized adaptively during the optimization process. The proposed bi-objective evolutionary method estimates the penalty parameter and the starting solution needed for the penalty function approach. We test and compare our algorithm on seven mathematical test problems and two engineering design problems taken from the literature. We compare our obtained results with our previous studies in terms of function evaluations and solution accuracy. The obtained optima are also compared with those of other standard algorithms. In many cases, our proposed methodology perform better than all algorithms considered in this study. Results are promising and motivate further application of the proposed adaptive normalization strategy.