We present a new algorithm with provable guarantees for discrete factor analysis from binary data, enabling the discovery of hidden variables and their causal relationships with observed data. Using our algorithm we can learn large bipartite Bayesian networks where the top layer's variables are unobserved in the training data. Our approach involves a novel quartet test and new techniques for unraveling a Bayesian network's parameters using the method-of-moments. These methodologies have applications throughout computational biology, medicine, and the social sciences.