We consider the problem of discriminative factor analysis for data that are in general nonGaussian. A Bayesian model based on the ranks of the data is proposed. We first introduce a new max-margin version of the ranklikelihood. A discriminative factor model is then developed, integrating the max-margin ranklikelihood and (linear) Bayesian support vector machines, which are also built on the max-margin principle. The discriminative factor model is further extended to the nonlinear case through mixtures of local linear classi-fiers, via Dirichlet processes. Fully local conjugacy of the model yields efficient inference with both Markov Chain Monte Carlo and variational Bayes approaches. Extensive experiments on benchmark and real data demonstrate superior performance of the proposed model and its potential for applications in computational biology.