全球智库信息集成服务系统

To credibly measure the effectiveness of public policies, when feasible, pilot programs with randomized control groups should be considered and rolled out first, to facilitate reliable program evaluations—before widespread implementation.

We compare experimental and nonexperimental estimates from a social and informational messaging experiment. Our results show that applying a fixed effects estimator in conjunction with matching to pre-process nonexperimental comparison groups cannot replicate an experimental benchmark, despite parallel pre-intervention trends and good covariate balance. The results are a stark reminder about the role of untestable assumptions—in our case, conditional bias stability—in drawing causal inferences from observational data, and the dangers of relying on single studies to justify program scaling-up or cancelling.