The recent availability of large scale data sets profiling single nucleotide polymorphisms (SNPs) and gene expression across different human populations, has directed much attention towards discovering patterns of genetic variation and their association with gene regulation. Two aspects of the nature of expression profiles make the identification and interpretation of such associations difficult. Firstly, we expect that a variety of environmental, developmental and other factors influence gene expression which can obscure such associations. Secondly, the regulatory network linking genes makes it difficult to pinpoint causal relationships between SNPS and regulatory elements. We address the first issue by proposing FA-eQTL, a factor-model that explicitly takes non-genetic variability into account, and thereby can significantly improve the power of an expression Quantitative Trait Loci (eQTL) study. We discuss a variational Bayesian implementation of this model, and point out rapid approximations that are applicable in certain situations. Applying our model to simulated and real world data we can demonstrate a significant improvement in performance. On data from the HapMap project, we find more than three times as many significant associations than a standard eQTL method. To address co-expression of genes, we further extended FA-eQTL by jointly reducing the dimensionality of the ex-pression profile and modelling non-genetic factors. We discuss results applying this enhanced QTL-model to biological data, including human as well as datasets from yeast.