I will describe a generic matching pursuit algorithm that can be used in machine learning for regression, subspace methods (kernel PCA and kernel CCA) and classification (given time). I will also describe some generalisa- tion error bounds upper bounding their loss. Some of these bounds will be formed using standard sample compression bounds whilst others will be amalgamations of traditional learning theory techniques such as VC theory and Rademacher complexities. This is joint work with John Shawe-Taylor.