Integrating Multi-omic and Imaging Data to Investigate the Targetable Cancer Landscape

Background: Cancer is a widespread disease with commonalities among patients, like cellular proliferation and immune evasion, but also immense diversity of specific genomic alterations, transcriptional signatures, morphological features, and, critically, response to treatment. While new cancer therapies are developed each year, more than 90% fail in clinical trials, partially due to our inability to accurately predict heterogeneous responses to therapies. Methods: This work uses machine learning techniques to learn a biologically meaningful landscape of cancer, distilling high-dimensional assay measurements into a set of interpretable biomarker axes. Results: First, from information-rich gene expression data we identified biological programs of squamous cell carcinomas (SCCs), detected these programs in clinically available tumor histology slide images using a neural network (NN), then, used a conditional generative adversarial network (cGAN) to generate twin-images representing a single synthetic patient with or without that program activated in order to interpret the visual features that were indicative of NN predictions. Next, we used existing histology foundation models to extract vector representations from pan-cancer histology images and then applied supervised dimensionality reduction to identify pathologist-interpretable "biomarker axes", i.e., axes reflecting histology features that correlate with patient status. These biomarker axes were then used to predict biomarker status directly from histology and performed comparably to state-of-the-art deep learning methods. Additionally, we developed a cross-modal autoencoder to jointly embed clinical and histology data from a multinational non-small cell lung cancer (NSCLC) cohort into a shared latent space. This multimodal latent space was then sampled to generate synthetic patients to augment training for machine learning models. Conclusions: These results demonstrate that machine learning methods can find biologically and clinically meaningful, low-dimensional representations of multimodal cancer data that support better identification of biomarkers and interpretation of them in the context of the tumor microenvironment.