One of the frustrations of machine learning theory is that many of the underlying algorithmic problems are provably intractable (e.g., NP-hard or worse) or presumed to be intractable (e.g., the many open problems in Valiant's model). This talk will suggest that this seeming intractability may arise because many models used in machine learning are more general than they need to be. Careful reformulation as well as willingness to consider new models may allow progress. We will use examples from recent work: Nonnegative matrix factorization, Learning Topic Models, ICA with noise, etc.