It is well established that gestures, the hand movements that accompany speech are an integral part of communication, ubiquitous across cultures, and a unique feature of human behavior (e.g., Goldin-Meadow & Brentari, in press). However, until now, no one has asked how listeners know when hand movements are supposed to be gestures, and not some other class of movement, such as instrumental actions, or meaningless movements produced for the sake of movement (e.g., Schachner & Carey, 2013). In this dissertation, I will explore what features of an event lead observers to see movement as gesture, and how seeing gesture changes across ontogeny. I begin by reviewing the previous literature on the role of gesture in learning and discuss a framework for understanding the functions of gesture on cognitive processes. I will then present three studies exploring how humans, across the lifespan, come to see and categorize hand movements. In Chapter 1, I will explore the features of movement that lead adults to interpret it as representational (as opposed to meaningless movement in the air, or instrumental actions-on-objects). Chapter 2 will investigate the development of movement categorization in childhood (ages 4-9). Chapter 3 will ask how infants first begin to categorize movements, and whether a system for categorizing hand movements is in place in infancy. Finally, I will integrate the findings from these studies into the broader question of how seeing movement as gesture affects learning from gesture.