This paper proposes a learning guided parameter setting method for constrained multi-objective optimization. To be more specific, the proposed method can generate penalty factors adaptively, which is inspired by the learning rate setting from deep learning. The suggested penalty function employs an exponential decay model by integrating constraint violation values, objectives values, the current generation counter and the maximum number of generations. Furthermore, the proposed self-adaptive penalty method is embedded in the push and pull search framework (PPS-SA) to deal with constrained multi-objective optimization problems (CMOPs). In PPS-SA, the search process is divided into two different stages — push and pull search stages. In the push stage, a CMOP is optimized without considering any constraints. In the pull stage, the CMOP is optimized with a self-adaptive penalty constraint-handling method. To evaluate the performance regarding convergence and diversity, two commonly used metrics, including IGD and HV, are used to test the proposed PPS-SA and other four state-of-the-art CMOEAs, including PPS-MOEA/D, MOEA/D-IEpsilon, MOEA/D-Epsilon and MOEA/D-CDP. The experimental results indicate that the proposed PPS-SA outperforms the other four algorithms in most of the test cases, which demonstrates the superiority of the proposed PPS-SA.