Synthetic materials made of engineered quasiparticles are a powerful platform for studying manybody physics and strongly correlated systems due to their bottom-up approach to Hamiltonian modeling. Photonic quasiparticles called polaritons are particularly appealing since they inherit fast dynamics from light and strong interaction from matter. This thesis describes the experimental demonstration of cavity Rydberg polaritons, which are composite particles arising from the hybridization of an optical cavity with Rydberg EIT, as well as the tools for probing and stabilizing the cavity. ,We first describe the design, construction, and testing of a four-mirror Fabry-Perot cavity, whose small waist size on the order of 10 microns is comparable to the Rydberg blockade radius. By achieving strong coupling between the cavity photon and an atomic ensemble undergoing electromagnetically induced transparency (EIT), we observe the emergence of the dark-state polariton and characterize its single-body properties as well as the single-quantum nonlinearity.,We then describe the implementation of a holographic spatial light modulator for exciting different transverse modes of the cavity, an essential tool for studying polariton-polariton scattering. For compensating optical aberrations, we employ a digital micromirror device (DMD), combining beam shaping with adaptive optics to produce diffraction-limited light. We quantitatively measure the purity of the DMD-produced Hermite-Gauss modes and confirm up to 99.2% efficiency. One application of the technique is to create Laguerre-Gauss modes, which have been used to probe synthetic Landau levels for photons in a twisted, nonplanar cavity.,Finally, we describe the implementation of an FPGA-based FIR filter for stabilizing the cavity. We digitally cancel the acoustical resonances of the feedback-controlled mechanical system, thereby demonstrating an order-of-magnitude enhancement in the feedback bandwidth from 200 Hz to more than 2 kHz. Harnessing the massive processing power of a state-of-the-art FPGA, we present a novel, low-latency digital architecture for loop-shaping, with applications in atomic physics and beyond.