Causal Inference for Incomplete and Heterogeneous Multisite and Blocked Experiments

This research project will expand randomization-based causal inference methodology for complex blocked and multisite randomized control trials. Researchers aiming to learn about the causal effects of treatments or interventions often use blocked and multisite trials. With these designs, experimental units are categorized into blocks or sites and then independent experiments occur within each block or site. The designs are especially common in the social and behavioral sciences; currently, however, they have some limitations. This project will fill many notable holes in the randomization-based causal inference literature by developing methodology for analyzing complex blocked and multisite experiments. The results from this project will be of value to both methodologists and practitioners. The significant involvement of graduate students will aid in the training of researchers in randomization-based causal inference. Materials from this project will be used in the development of an undergraduate causal inference course. Freely available statistical software packages also will be developed.This research project will develop novel methodology for randomization-based causal inference for complex blocked and multisite experiments. When there are many treatments, it may not be practical or even feasible to implement all treatments within each block or site. Use of an incomplete block design, in which a random subset of treatments is assigned and implemented within each block, can overcome this challenge while keeping the structure of the blocked design. The project will develop methodology to analyze incomplete block designs from the randomization-based potential outcome framework which requires fewer assumptions than existing model-based approaches. Further, when there are many factorial treatments of interest, it is possible that not all blocks or sites will contain full information on all factors, or that a subset of factors may not be randomized in some blocks or sites. This project will investigate what causal effects can be identified in these situations and will develop inferential techniques to learn about these effects. In addition to average causal effects of interventions, there is often great interest in learning about cross-site heterogeneity. The project will explore in what settings cross-site heterogeneity is identifiable and how to best capture this heterogeneity. The focus on randomization-based inference will make the tools developed especially relevant to social and behavioral scientists who primarily run experiments using human subjects, for which model-based assumptions may not be appropriate for the data collected. Understanding the amount of treatment effect heterogeneity across blocks or sites also will help researchers assess whether programs and policies should be implemented at a large scale.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.