Beyond the Numbers: An Analysis of the Subjectivity and Gender Biases of Course Feedback Comments from Students

Analyzing textual comments beyond numerical ratings in course feedback reports is essential for instructors, administrators, and future students to gain a comprehensive understanding of the course, such as teaching effectiveness, potential evaluation subjectivity, biases, etc. This thesis presents a comprehensive analysis of approximately 2,500 course feedback re- ports containing 113,278 comments from CMSC and SOSC departments at the University of Chicago, from the 2020 Spring quarter to the 2023 Winter quarter. We apply text summarization, natural language processing (NLP), sentiment analysis, and machine learning techniques to organize individual comments, explore key topics, identify potential subjectivity and gender bias in the comments. Preliminary analysis reveals common concerns related to the course structure, content, effectiveness, and clarity in the educational process. The contributions are twofold: (1) the extensive dataset of course feedback comments and instructor profiles we collected, tagged with credible sentiment and subjectivity labels, constitutes a valuable resource for scholars in education evaluations; (2) we develop a text summarization and subjectivity identification pipeline, which enables the extraction of representative nouns and adjectives as well as the calculation of the degree of subjectivity expressed in the comments. These insights can be used to inform education policies and practices, thereby facilitating data-driven decision-making and fair evaluation mechanisms.