In this talk I will discuss first-order models and algorithms for sequential decision-making, specifically those approaches that admit exact lifted solutions. The first emphasis of the talk will be on the insights that underlie these models and algorithms along with potential caveats for their practical application. The second emphasis of the talk will be on a variety of extensions of the first-order Markov decision process (MDP) framework such as the factored first-order MDP and the first-order partially observable MDP. The third emphasis of the talk will be on the algorithmic tricks-of-the-trade that allow the practical application of these models; this includes (a) useful data structures, (b) efficient solution techniques for first-order linear programs, (c) new techniques for first-order variable elimination, and (d) practical methods for maintaining compact, consistent first-order representations without theorem proving.