Traditional approaches to machine learning and statistical inference are passive, in the sense that all data are collected prior to analysis in a non-adaptive fashion. One can envision, however more active strategies in which information gleaned from previously collected data is used to guide the selection of new data. This talk discusses the emerging theory of such "active learning" methods. I will show that feedback between data analysis and data collection can be crucial for effective learning and inference. The talk will describe two active learning problems. First, I will consider binary-valued prediction (classification) problems, for which the prediction errors of passive learning methods can be exponentially larger than those of active learning. Second, I will discuss the role of active learning in the recovery of sparse vectors in noise. I will show that certain weak, sparse patterns are imperceptible from passive measurements, but can be recovered perfectly using selective sensing.