With the success of machine learning, we increasingly see learning algorithms make decisions in the real world. Often, however, this is in stark contrast to the classical train-test paradigm, since the learning algorithm affects the very data it must operate on. I will explain how statistical confidence bounds can guide data acquisition in a principled way to make effective decisions in a variety of complex settings. I will discuss several applications, ranging from autonomously designing wetlab experiments in protein structure optimization, to safe automatic parameter tuning on a robotic platform.