Simulation of natural human motion is challenging because the relevant system dynamics is high-dimensional, underactuated—no direct control over global position and orientation—and non-smooth—frequent and intermittent ground contacts. In order to succeed, control policy must look ahead to determine stabilizing actions and it must optimize to generate lifelike motion. In this talk, we will review recently developed control systems that yield high-quality agile movements for three-dimensional human simulations. Creating such controllers requires intensive computer optimization and reveals a need for reusing as many control policies as possible. We will answer this problem partially with an efficient combination that creates a new optimal control policy by reusing a set of optimal controls for related tasks. It remains to be seen if the same approach can also be applied to control systems needed to generate lifelike human motion.