Deep learning has enabled computers to approach human-level performance on many practical perception and language understanding tasks, ranging from speech recognition to computer vision and machine translation. One of today's grand AI challenges is to bring these new capabilities into the physical world, and teach machines how to behave and make themselves useful in human centered environments. In this talk, I'll argue how robotics may be on the cusp of its very own deep learning revolution, but that for this endeavor to succeed, machine learning practitioners have to break from the relative comfort of the large-scale supervised learning setting that has buoyed the field for the past decade and humbly face some thorny problems that have comparatively been neglected: data scarcity and skill transfer, active and lifelong learning, as well as safety and predictabil- ity. The good news is that tackling these problems is also one of the necessary next steps towards bridging the gap between mere learning and actual intelligence.