Extracting information, and more generally extracting knowledge from large datasets is arguably one of the main frontiers of modern science, common to a broad variety of disciplines. Bayesian approaches to machine learning and signal processing provide a conceptual framework in which information bits interact through constraints (due to prior knowledge or to measurements). Statistical physics has helped to develop new approaches and very powerful algorithms in this context, where collective phenomena, like phase transitions and the occurrence of glassy phases, play a major role. This talk will review some of the main developments in this field, illustrated by specific examples like compressed sensing.