Incremental Forward Stagewise Regression (FS") is a statistical algorithm that produces sparse coefficient profiles for linear regression. Using the tools of first-order methods in convex optimization, we analyze the computational complexity of FS" and its flexible variants with adaptive shrinkage parameters. We also show that a simple modication to FS" yields an O(1=k) convergent algorithm for the least squares LASSO t for any regularization parameter and any data-set | thereby quantitatively characterizing the nature of regularization implicitly induced by FS".