Sampling-based Randomised Designs for Causal Inference under the Potential Outcomes Framework

Zach Branson, Tirthankar Dasgupta

Research output: Contribution to journalArticle

Abstract

We establish the inferential properties of the mean-difference estimator for the average treatment effect in randomised experiments where each unit in a population is randomised to one of two treatments and then units within treatment groups are randomly sampled. The properties of this estimator are well understood in the experimental design scenario where first units are randomly sampled and then treatment is randomly assigned but not for the aforementioned scenario where the sampling and treatment assignment stages are reversed. We find that the inferential properties of the mean-difference estimator under this experimental design scenario are identical to those under the more common sample-first-randomise-second design. This finding will bring some clarifications about sampling-based randomised designs for causal inference, particularly for settings where there is a finite super-population. Finally, we explore to what extent pre-treatment measurements can be used to improve upon the mean-difference estimator for this randomise-first-sample-second design. Unfortunately, we find that pre-treatment measurements are often unhelpful in improving the precision of average treatment effect estimators under this design, unless a large number of pre-treatment measurements that are highly associative with the post-treatment measurements can be obtained. We confirm these results using a simulation study based on a real experiment in nanomaterials.

Original languageEnglish (US)
JournalInternational Statistical Review
DOIs
StateAccepted/In press - Jan 1 2019
Externally publishedYes

Fingerprint

Potential Outcomes
Causal Inference
Estimator
Average Treatment Effect
Experimental design
Scenarios
Unit
Superpopulation
Randomized Experiments
Nanomaterials
Framework
Design
Potential outcomes
Sampling
Causal inference
Assignment
Simulation Study
Pretreatment

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Keywords

  • causal inference
  • experimental design
  • potential outcomes
  • randomised experiments
  • survey sampling

Cite this

@article{89a9c36c0d7a4b54837c10c0d5079e8a,
title = "Sampling-based Randomised Designs for Causal Inference under the Potential Outcomes Framework",
abstract = "We establish the inferential properties of the mean-difference estimator for the average treatment effect in randomised experiments where each unit in a population is randomised to one of two treatments and then units within treatment groups are randomly sampled. The properties of this estimator are well understood in the experimental design scenario where first units are randomly sampled and then treatment is randomly assigned but not for the aforementioned scenario where the sampling and treatment assignment stages are reversed. We find that the inferential properties of the mean-difference estimator under this experimental design scenario are identical to those under the more common sample-first-randomise-second design. This finding will bring some clarifications about sampling-based randomised designs for causal inference, particularly for settings where there is a finite super-population. Finally, we explore to what extent pre-treatment measurements can be used to improve upon the mean-difference estimator for this randomise-first-sample-second design. Unfortunately, we find that pre-treatment measurements are often unhelpful in improving the precision of average treatment effect estimators under this design, unless a large number of pre-treatment measurements that are highly associative with the post-treatment measurements can be obtained. We confirm these results using a simulation study based on a real experiment in nanomaterials.",
keywords = "causal inference, experimental design, potential outcomes, randomised experiments, survey sampling",
author = "Zach Branson and Tirthankar Dasgupta",
year = "2019",
month = "1",
day = "1",
doi = "https://doi.org/10.1111/insr.12339",
language = "English (US)",
journal = "International Statistical Review",
issn = "0306-7734",
publisher = "International Statistical Institute",

}

TY - JOUR

T1 - Sampling-based Randomised Designs for Causal Inference under the Potential Outcomes Framework

AU - Branson, Zach

AU - Dasgupta, Tirthankar

PY - 2019/1/1

Y1 - 2019/1/1

N2 - We establish the inferential properties of the mean-difference estimator for the average treatment effect in randomised experiments where each unit in a population is randomised to one of two treatments and then units within treatment groups are randomly sampled. The properties of this estimator are well understood in the experimental design scenario where first units are randomly sampled and then treatment is randomly assigned but not for the aforementioned scenario where the sampling and treatment assignment stages are reversed. We find that the inferential properties of the mean-difference estimator under this experimental design scenario are identical to those under the more common sample-first-randomise-second design. This finding will bring some clarifications about sampling-based randomised designs for causal inference, particularly for settings where there is a finite super-population. Finally, we explore to what extent pre-treatment measurements can be used to improve upon the mean-difference estimator for this randomise-first-sample-second design. Unfortunately, we find that pre-treatment measurements are often unhelpful in improving the precision of average treatment effect estimators under this design, unless a large number of pre-treatment measurements that are highly associative with the post-treatment measurements can be obtained. We confirm these results using a simulation study based on a real experiment in nanomaterials.

AB - We establish the inferential properties of the mean-difference estimator for the average treatment effect in randomised experiments where each unit in a population is randomised to one of two treatments and then units within treatment groups are randomly sampled. The properties of this estimator are well understood in the experimental design scenario where first units are randomly sampled and then treatment is randomly assigned but not for the aforementioned scenario where the sampling and treatment assignment stages are reversed. We find that the inferential properties of the mean-difference estimator under this experimental design scenario are identical to those under the more common sample-first-randomise-second design. This finding will bring some clarifications about sampling-based randomised designs for causal inference, particularly for settings where there is a finite super-population. Finally, we explore to what extent pre-treatment measurements can be used to improve upon the mean-difference estimator for this randomise-first-sample-second design. Unfortunately, we find that pre-treatment measurements are often unhelpful in improving the precision of average treatment effect estimators under this design, unless a large number of pre-treatment measurements that are highly associative with the post-treatment measurements can be obtained. We confirm these results using a simulation study based on a real experiment in nanomaterials.

KW - causal inference

KW - experimental design

KW - potential outcomes

KW - randomised experiments

KW - survey sampling

UR - http://www.scopus.com/inward/record.url?scp=85071777767&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85071777767&partnerID=8YFLogxK

U2 - https://doi.org/10.1111/insr.12339

DO - https://doi.org/10.1111/insr.12339

M3 - Article

JO - International Statistical Review

JF - International Statistical Review

SN - 0306-7734

ER -