Prosody, the musical facet of speech, is pivotal in human communication, and its structure and meaning remain subjects of ongoing research. In this study, we introduce a data-driven model for English prosody, based on large-scale analysis of spontaneous conversations. As a first step, we identify approximately 200 discernible prosodic patterns—which we view as building blocks of the prosodic vocabulary—and outline their properties and range of meanings. Next, we reveal a Markovian logic, akin to a syntax, for concatenating these elementary building blocks into coherent utterances. We identify distinct compound functions associated with pairs of consecutive patterns and show that the Markovian syntax is more prevalent in spontaneous prosody, as compared to scripted speech. These findings offer invaluable insights into the underlying mechanisms of conversational prosody: They empirically inform and refine existing theoretical concepts. The methodology we present, combining unsupervised analysis of large datasets of spontaneous speech with manual sampling of the results, could guide future research aimed at refining our model and expanding it to other languages.