Spectral methods are satisfying in that they provide statistically consistent estimators. However, two issues still challenge their widespread adoption: they tend to be inaccurate in smaller data regimes, and they do not directly apply to richer classes of latent-variable models. In this talk, we show some initial progress on both of these issues by using spectral methods to obtain partial information about the parameters rather than using them as a standalone estimator. We also show preliminary results on learning log-linear and loopy graphical models.