We present a general framework and learning algorithm for the task of concept labeling: each word in a given sentence has to be tagged with the unique physical entity (e.g. person, object or location) or abstract concept it refers to. Our method allows both world knowledge and linguistic information to be used during learning and prediction. We show experimentally that we can handle natural language and learn to use world knowledge to resolve ambiguities in language, such as word senses or coreference, without the use of hand-crafted rules or features.