Cold atoms in cavity quantum electrodynamics experiments, and their counterparts using superconducting or solid-state qubits, have enabled impressive and complementary results in quantum-enhanced technology. Combining the advantages of these platforms has clear benefits for quantum networking and metrology, but is complicated by their operation at different ranges of the electromagnetic spectrum. Atomic gases provide a possible solution due to their ground state optical transitions and dense energy level structure with GHz transitions between highly excited Rydberg states. This thesis will describe how a gas of ultracold Rydberg atoms can be simultaneously coupled to single optical and millimeter wave photons using cavity electromagnetically-induced transparency. I will describe the development of a new hybrid quantum apparatus, including high finesse optical and mm-wave resonators at 5 Kelvin, a cryogenic laser-cooled atom source, and techniques for manipulating atoms in a superconducting cavity. This system has immediate applications for upconverting mm-wave photons to optical frequencies for long-distance quantum communication, with predicted efficiency of greater than 90% and bandwidth of ~3 MHz. Furthermore, the very high cooperativity coupling achievable at mm-wave frequencies could enable cavity-mediated interactions between Rydberg atoms and creation of metrologically useful entangled atomic states. I will report recent results from the experiment, including evidence of coupling between Rydberg polaritons and a mm-wave resonator with photon occupation n~1, with a clear path forward to enter the strong coupling regime.