The exploration-exploitation dilemma has been an intriguing and unsolved problem within the framework of reinforcement learning. Optimism in the face of uncertainty and model building play central roles in advanced exploration methods. Here, we integrate several concepts and obtain a fast and simple algorithm. We show that the proposed algorithm finds a near-optimal policy in polynomial time, and give experimental evidence that it is robust and efficient compared to its ascendants.