Demands for higher fuel efficiency for off-highway applications motivate manufacturers to replace existing drive transmissions with more complex, high-efficiency transmissions. Increased intelligence and more advanced architectures are, however, more difficult to design and prototype. This leads to longer product development processes and a greater need for early product evaluation. The great variety of existing concepts also requires a methodology to support the choice of architecture. This paper proposes a design methodology for complex hydromechanical transmissions based on optimization. The main objective is to maximize energy efficiency and adapt the design to suit the typical operating behavior of the application. The methodology is also implemented on a multiple mode transmission concept sui for a heavy wheel loader application. It is shown that the design of the gearbox heavily influences the energy consumption and the necessity to use optimization when designing the gearbox.