The effects of heat exposure on negative affect are thought to be central to the observed relationships between hot summer days and deleterious outcomes, such as violent crime or mental health crises. As these relationships are likely to be magnified by the effects of climate change, a better understanding of how consistent or variable the effects of hot weather on affective states is required. The current work combines data gathered from an ecological momentary assessment (EMA) study on individuals’ thermal perceptions, comfort, and affective states in outdoor environments during their daily lives with high spatiotemporal resolution climate-modeled weather variables. Using these data, associations between objective weather variables (temperature, humidity, etc.), perceived heat (thermal perception and comfort), and affective states are examined. Overall, objective weather data reasonably predicted perception and comfort, but only comfort predicted negative affective states. The variance explained across individuals was generally very low in predicting negative affect or comfort, but within-person variance explained was high. In other words, while there may be a relatively consistent relationship between temperature and psychological experience for any given person, there are significant individual differences across people. Age and gender were examined as moderators of these relationships, and while gender had no impact, participant age showed several significant interactions. Specifically, while older adults tended to experience more thermal discomfort and perceived higher temperatures as hotter, the relationship between discomfort and negative affect was lower in older adults. Taken together, these results emphasize the importance of thermal discomfort specifically in predicting negative affect, as well as the high inter-individual variability in thermal perceptions and comfort for the same ambient temperatures.