Quantum computing has recently transitioned from a theoretical prediction to a nascent technology. Nevertheless, there is still a gap between the computational ability and reliability of current QIP technologies and the requirements of the first useful quantum computing applications. It will require tremendous efforts and investment to solve the engineering and research challenges to close the gap. Because of the uncertainties and difficulties in relying on hardware breakthroughs, it will also be crucial to close the gap using higher-level quantum optimizations and software-hardware co-design, which could maximally utilize noisy devices and potentially provide an accelerated pathway to real-world quantum computing applications. This thesis develops methods to explore the vast design space for highly optimized and reliable quantum computing architectures and software and focuses on the compilation, verification, and error correction aspects of the quantum computing stack. Specifically, the three main chapters in this thesis discuss, respectively, a direct-to-pulse compiler, a protocol to fault-tolerantly prepare approximate GKP (Gottesman-Kitave-Preskill) qubit states, and a verification framework that helps quantum programmers write provably correct compiler components through formal verification. Though touching many aspects of the stack and based on different technologies in physics and computer science, the majority of this thesis can be connected by a main theme --- improving the efficiency of existing quantum computing stacks by breaking the abstraction between layers in the quantum computing stack.