In this manuscript, we explore three different viewpoints that each provide different quantitative guarantees of performance. First, we present a generalization of the classical theory of integral quadratic constraints. This generalization leads to a tractable computational procedure for finding exponential stability certificates for partially-unknown feedback systems. Second, we present non-asymptotic lower and upper bounds for core problems in the field of system identification. Finally, using the recently developed system-level synthesis framework and tools from high-dimensional statistics, we establish finite-sample performance guarantees for robust output-feedback control of an unknown dynamical system.