We address an unsupervised object detection and segmentation problem that goes beyond the conventional assumptions of one-to-one object correspondences or modeltest settings between images. Our method can detect and segment identical objects directly from a single image or a handful of images without any supervision. To detect and segment all the object-level correspondences from the given images, a novel multi-layer match-growing method is proposed that starts from initial local feature matches and explores the images by intra-layer expansion and inter-layer merge. It estimates geometric relations between object entities and establishes ‘object correspondence networks’ that connect matching objects. Experiments demonstrate robust performance of our method on challenging datasets.