US News and World Report Cancer Hospital Rankings: Do They Reflect Measures of Research Productivity?

Context: Prior research has faulted the US News and World Report hospital specialty rankings for excessive reliance on reputation, a subjective measure of a hospital's performance.

Objective: To determine whether and to what extent reputation correlates with objective measures of research productivity among cancer hospitals.

Design: A retrospective observational study.

Setting: Automated search of NIH Reporter, BioEntrez, BioMedline and Clinicaltrials.gov databases.

Participants: The 50 highest ranked cancer hospitals in 2013's US News and World Report Rankings.

Exposure: We ascertained the number of NCI funded grants, and the cumulative funds received by each cancer center. Additionally, we identified the number of phase I, phase II, and phase III studies published and indexed in MEDLINE, and registered at clinicaltrials.gov. All counts were over the preceding 5 years. For published articles, we summed the impact factor of the journals in which they appeared. Trials were attributed to centers on the basis of the affiliation of the lead author or study principal investigator.

Main Outcome: Correlation coefficients from simple and multiple linear regressions for measures of research productivity and a center's reputation.

Results: All measures of research productivity demonstrated robust correlation with reputation (mean r-squared  = 0.65, median r-squared = 0.68, minimum r-squared = .41, maximum r-squared = 0.80). A multivariable model showed that 93% of the variation in reputation is explained by objective measures.

Conclusion: Contrary to prior criticism, the majority of reputation, used in US News and World Rankings, can be explained by objective measures of research productivity among cancer hospitals.