We formulate a layered model for object detection and multi-class segmentation. Our system uses the output of a bank of object detectors in order to define shape priors for support masks and then estimates appearance, depth ordering and labeling of pixels in the image. We train our system on the PASCAL segmentation challenge dataset and show good test results with state of the art performance in several categories including segmenting humans.