In this chapter, a time-scale expansion-based scheme is presented for approximately solving the optimal control problem of continuous-time underactuated nonlinear systems subject to input constraints and system dynamics. By time-scale Taylor approximation of the original performance index, the optimal control problem is relaxed into an approximated optimal control problem. Based on the system dynamics, the problem is further reformulated as a quadratic program, which is solved by a projection neural network. Theoretical analysis on the closed-loop system synthesized by the controlled system and the projection neural network is conducted, which reveals that, under certain conditions, the closed-loop system possesses exponential stability and the original performance index converges to zero as time tends to infinity. In addition, two illustrative examples, which are based on a flexible joint manipulator and an underactuacted ship, are provided to validate the theoretical results and demonstrate the efficacy and superiority of the presented control scheme.