In this paper, new classes of generalized invex functions called B-(p, r)-pseudo-invex and B-(p, r)-quasi-invex (with respect to η and b) are introduced. These classes of functions are defined by relaxing the definitions of the classes of quasi-invex and pseudo-invex functions earlier defined by Hanson (J. Math. Anal. Appl. 1981, 80:545–550), and also the concept of B-(p, r)-invexity (with respect to η and b) earlier introduced by Antczak (J. Math. Anal. Appl. 2003, 286:187–206). Further, mathematical programming problems involving generalized B-(p, r)-invex functions with respect to η and b are considered. The sufficient optimality conditions and various duality results are established for such optimization problems.