B cells are the central component of the humoral immune system and critical for long term immunity against all forms of pathogens. Antibodies represent one of the earliest lines of defense within the adaptive memory response and are the major effectors of vaccine mediated protection. By applying an oligo-linked bait sorting approach and single cell sequencing to investigate infection by a novel pathogen, severe acute respiratory coronavirus 2, we were able to elucidate that there was considerable heterogeneity in the B cell response. We found that while protective anti-spike protein responses were readily fixed into B cell memory after recovery from infection, memory towards the intracellular and intraviral viral ORF8 and NP proteins increased in relative abundance and maturation over time and found antibodies targeting these antigens to be exclusively non-protective. Additionally, we found that memory to endemic human coronaviruses is activated upon infection with SARS-CoV-2, and that the B cell receptors of these cells show remarkably high amounts of mutation and selective pressure, potentially revealing a rich source to be targeted for future universal coronavirus vaccine efforts. In exploring these data we have developed new computational tools to analyze high throughput, multi-omics data sets and show that it reveals further heterogeneity in B cells responding to SARS-CoV-2 than RNA sequencing alone. Further, our dataset of 13,000 antigen specific B cells with matched transcriptome and receptor sequence and 55,000 B cells total with receptor and transcriptome will provide a foundation for future studies. Ultimately, we have built new tools to investigate B cell responses and shown them capable of generating discoveries of immediate translational impact and of revealing previously unappreciated aspects of basic B cell biology.