In recent years, machine learning methods have enjoyed great success in a variety of applications. Unfortunately, on-line learning in autonomous agents has not generally been one of them. Reinforcement-learning methods that were developed to address problems of learning agents have been most successful in off-line applications. This talk will briefly review the basic methods of reinforcement learning, point out some of their shortcomings, argue that we are expecting too much from such methods, and speculate about how to build complex, adaptive autonomous agents. These speculations are backed up by recent results demonstrating that a small amount of human-provided input can dramatically speed learning in a real mobile robot.