We will describe our ongoing work on learning-based methods for recovering 3D human body pose and motion from single images and from monocular image sequences. The methods work directly with raw image observations and require neither an explicit 3D body model nor a prior labelling of body parts in the image. Instead, they recover the body pose or motion by direct nonlinear regression against shape descriptors extracted automatically from image silhouettes or contours.