We propose a knowledge-driven activity recognition and seg- mentation framework introducing the notion of context connections. Given an RDF dataset of primitive observations, our aim is to identify, link and classify meaningful contexts that signify the presence of complex activities, coupling background knowledge pertinent to generic contextual dependencies among activities. To this end, we use the Situation concept of the DOLCE+DnS Ultralite (DUL) ontology to formally capture the context of high-level activities. Moreover, we use context similarity measures to handle the intrinsic characteristics of pervasive environments in real-world conditions, such as missing information, temporal inaccuracies or activities that can be performed in several ways. We illustrate the performance of the proposed framework through its deployment in a hospital for monitoring activities of Alzheimer's disease patients.