We consider a robust classification problem and show that standard regularized SVM is a special case of our formulation, providing an explicit link between reg- ularization and robustness. At the same time, the physical connection of noise and robustness suggests the potential for a broad new family of robust classification algorithms. Finally, we show that robustness is a fundamental property of classi- fication algorithms, by re-proving consistency of support vector machines using only robustness arguments (instead of VC dimension or stability).