This survey considers the design of methods for learning mathematical models from data. The contemporary toolbox of machine learning consists of a wide set of techniques, essentially reducing to a few formal arguments. The aim of this presentation is then to give insight in some of them, and to argue how they could be implemented successfully. When doing so, we will touch topics as risk-based modeling, probabilistic inference, convex optimization and kernel-based learning amongst others. I will exemplify two application areas, namely (A) identification of dynamic systems, and (B) modeling and prediction of reliability data.