Machine learning studies the design of models and training algorithms in order to learn how to solve tasks from data. Whereas historically machine learning has concentrated primarily on static predefined training datasets and respective test scenarios, recent advances also take into account the fact that the world is constantly evolving. In this course, we will go beyond the train-validate-test phase and explore modern approaches to machines that can learn continually. In addition to a comprehensive overview of the breath of factors to consider in such continual learning, the course will outline the basics of techniques that span mitigation of forgetting across multiple tasks, selection of new data in ongoing training, and robustness with respect to unexpected data inputs.