We study a convex formulation of low-rank matrix factorization, in a special case where additional information on the factors is known. Our formulation is typically adapted to source separation scenarii, where additional information on the sources may be provided by an expert. Our formulation promotes low-rank with a nuclear-norm based penalty. As it is non-smooth, generic first-order algorithms suffer from slow convergence rates. We study and compare several algorithms that fully exploit the structure of our problem while keeping memory requirements linear in the size of the problem.