A multi-stage compilation framework transforms portions of programs written in a productivity-level language into an efficiency-level language, such as C, with explicit hardware-specific optimizations. It is challenging for compiler programmers to debug errors in the compilation because they must perform complicated end-to-end reasoning, relating the programs across the multiple stages of compilation. To simplify this debugging effort, we present multi-level debugging, a novel combination of error-checking algorithms in a multi-stage compilation environment. Our method particularly aims to model and check sequential and parallel notions of nondeterminism and related bugs introduced by the compilation. Using our method, the programmer can systematically eliminate potential sources of the bug in the compilation process and focus only on the real source. We demonstrated on two real multi-stage compilers the effectiveness of multi-stage debugging in simplifying the diagnosis of manually-injected bugs as well as in an actual bug encountered during compiler development.