I will show how the "optimism in the face of uncertainty" principle developed in multiarmed bandits can be extended to address large scale decision making problems. Initially motivated by the empirical success of the Monte-Carlo tree search (MCTS) methods popularized in computer-go and further extended to many other optimization problems, I will report elements of theory that characterize the complexity of the underlying search problems and describe efficient algorithms with performance guarantees.