Leveraging Facebook for Behavioral Insights: The Influence of Algorithms on Experimental Research

This study investigates the influence of Facebook's algorithms on the outcomes of behavioral research experiments conducted on the platform. By analyzing a series of A/B tests, we highlight the existence of important but opaque underlying algorithms that perform targeting beyond demographic variables and how they affect the experimental results. The research primarily focuses on two types of campaign optimization options—click-optimized and view-optimized—and examines how these approaches influence the demographic composition of the audience and, consequently, the results of ad-effectiveness tests. Using a combination of logistic regression and chi-square tests, we provide empirical evidence that Facebook's algorithms significantly shape the substantial differences in both reach and click-through rates (CTR) between treatment and control groups, even after controlling for available demographics. The findings emphasize that experiments on Facebook do not operate under traditional random assignment, complicating the interpretation of treatment effects. This study contributes to our understanding of Facebook experimental results in social science research and the generalizability of its interpretations.