Approximately 85% of internet traffic is estimated to be visual data. Conventional object detection algorithms are not yet suitable to harness this unconstrained, massive visual data because they require laborious bounding box annotations for training and large scale inference is infeasibly slow due to model complexity. In this thesis, I present two instantiations of model parsimony for large scale object detection and discovery. For model inference, I present sparselet models which significantly reduce model inference complexity by utilizing a shared representation, reconstruction sparsity, and parallelism to enable real-time multiclass object detection with deformable part models at 5Hz with almost no decrease in task performance. For model learning, I present a framework for training object detectors using only one-bit image level annotations of object presence without any instance level annotations (i.e. bounding boxes). This framework provides approximately 50% relative improvement in localization accuracy (as measured by average precision) over the current state of the art weakly supervised learning methods on standard benchmark datasets.