Women continue to be underrepresented in leadership positions. This underrepresentation is at least partly driven by gender stereotypes that associate men, but not women, with achievement-oriented, agentic traits (e.g., assertive and decisive). These stereotypes are expressed and perpetuated in language, with women being described in less agentic terms than men. The present research suggests that appointing women to the top tiers of management can mitigate these deep-rooted stereotypes that are expressed in language. We use natural language processing techniques to analyze over 43,000 documents containing 1.23 billion words, finding that hiring female chief executive officers and board members is associated with changes in organizations’ use of language, such that the semantic meaning of being a woman becomes more similar to the semantic meaning of agency. In other words, hiring women into leadership positions helps to associate women with characteristics that are critical for leadership success. Importantly, our findings suggest that changing organizational language through increasing female representation might provide a path for women to break out of the double bind: when female leaders are appointed into positions of power, women are more strongly associated with the positive aspects of agency (e.g., independent and confident) in language but not at the cost of a reduced association with communality (e.g., kind and caring). Taken together, our findings suggest that female representation is not merely an end, but also a means to systemically change insidious gender stereotypes and overcome the trade-off between women being perceived as either competent or likeable.