We consider the situation in which there is a basic learning task but different sub-tasks define different data generating distributions. Examples include learning to identify spam for various different email users, or parts-of-speech tagging for different text corpora. Our goal is to allow the use of training data coming from one sub-task for prediction under another sub-task distribution.