This dissertation studies the econometrics of the design and analysis of randomized controlled trials (RCTs). Chapter 1, coauthored with Joseph Romano and Azeem Shaikh, studies inference for the average treatment effect (ATE) in RCTs where treatment status is determined according matched-pair designs. We assume that units are paired according to observed baseline covariates instead of some function of the covariates, and Chapter 2 extends these results to settings where units are paired according to (random) functions of the covariates. This type of design is used routinely throughout the sciences, but results about its implications for inference about the average treatment effect are not available. The main requirement underlying our analysis is that pairs are formed so that units within pairs are suitably "close" in terms of the baseline covariates, and we develop novel results to ensure that pairs are formed in a way that satisfies this condition. Under this assumption, we show that, for the problem of testing the null hypothesis that the average treatment effect equals a pre-specified value in such settings, the commonly used two-sample t-test and "matched pairs" t-test are conservative in the sense that these tests have limiting rejection probability under the null hypothesis no greater than and typically strictly less than the nominal level. We show, however, that a simple adjustment to the standard errors of these tests leads to a test that is asymptotically exact in the sense that its limiting rejection probability under the null hypothesis equals the nominal level. We also study the behavior of randomization tests that arise naturally in these types of settings. When implemented appropriately, we show that this approach also leads to a test that is asymptotically exact in the sense described previously, but additionally has finite-sample rejection probability no greater than the nominal level for certain distributions satisfying the null hypothesis. Chapter 2 studies the optimality of matched-pair designs in RCTs. Matched-pair designs are examples of stratified randomization, in which the researcher partitions a set of units into strata based on their observed covariates and assign a fraction of units in each stratum to treatment. Despite the prevalence of stratified randomization in RCTs, implementations differ vastly. We provide an econometric framework in which, among all stratified randomization procedures, the optimal one in terms of the mean-squared error of the difference-in-means estimator is a matched-pair design that orders units according to a scalar function of their covariates and matches adjacent units. Our framework captures a leading motivation for stratifying in the sense that it shows that the proposed matched-pair design additionally minimizes the magnitude of the ex-post bias, i.e., the bias of the estimator conditional on realized treatment status. We then consider empirical counterparts to the optimal stratification using data from pilot experiments and provide two different procedures depending on whether the sample size of the pilot is large or small. We run an experiment on the Amazon Mechanical Turk using one of the proposed procedures, replicating one of the treatment arms in Dellavigna and Pope (2019), and find the standard error decreases by 29%, so that only half of the sample size is required to attain the same standard error. Chapter 3 studies the more fundamental question of why randomization should be used in controlled trials when the objective is to estimate the ATE precisely. In particular, I study the minimax optimality of certain randomization schemes and assignment schemes in estimating "reasonable" parameters including the average treatment effect, when treatment effects are heterogeneous. By a randomization scheme, I mean the distribution over a group of permutations of a given treatment assignment vector. By an assignment scheme, I mean the joint distribution over assignment vectors, linear estimators, and permutations of assignment vectors. I show that for any given assignment vector and any estimator, the complete randomization scheme is minimax optimal for any objective function satisfying quasi-convexity, where the worst-case is over a permutation-invariant class of distributions of the data. Under further conditions on the distribution of the data, I characterize the minimax optimal assignment scheme, where the worst-case is again over a permutation-invariant class of distributions of the data.




Downloads Statistics

Download Full History