How do people reason about things they cannot hear, see, or touch? In metaphorical mental representations, people understand abstract conceptual domains, like time and number, using knowledge of other domains, like space. The tendency to spatialize time and number may be a cognitive universal, but the specifics of these metaphorical mappings vary across cultures. In Western cultures, both time and numbers are arranged in people’s minds along an imaginary horizontal line, from left to right, but in other cultures the directions of the mental timeline (MTL) and mental number line (MNL) are reversed. How does culture shape our abstract concepts? Using time and number as a testbed, here I propose and test a general principle, which I call the CORrelations in Experience (CORE) principle, according to which different aspects of experience should selectively affect different metaphorical mappings. Across three training experiments, I show that the MTL is shaped by experiences that provide a correlation between space and time, whereas the MNL is shaped by experiences that provide a correlation between space and number. These findings reveal that the MTL and MNL have distinct experiential bases, supporting the CORE principle and challenging the widespread assumption that both mappings are determined by a common set of cultural experiences. The CORE principle provides an account of how domains like time and number, universal fixtures of the natural world, can be conceptualized in culture-specific ways: People map abstract domains in their minds according to the ways those domains are spatialized in their experience.