Title：

Why ‘Does this Effect Generalize’ Is a Bad Question and What to Ask Instead

Abstract：

Social scientists often ask whether experimental results generalize across contexts, and the question is usually posed in terms of whether average treatment effects replicate. This paper argues that for many theories, the more relevant question is whether a parsimoniously parameterized model of effects and effect heterogeneity yields stable results across settings. We call this problem parametric generalizability. The paper develops a causal framework that distinguishes unconditional generalizability of marginal effects from conditional generalizability of treatment effects given effect modifiers. It then shows how conditional generalizability can be operationalized in three ways: nonparametric comparison of conditional effects, reweighting to target populations, and parametric projection of effect heterogeneity onto a low-dimensional model. The parametric approach yields a practical set of diagnostics for multi-site experiments: site-by-site tests of the theoretical coefficient pattern, cross-site tests of coefficient homogeneity, and equivalence-style confidence regions describing the range of coefficient variation across contexts. This shifts the inferential target from asking whether effects are identical within agnostically defined regions of the covariate space to asking whether a parsimonious theoretical approximation yields stable results and is thus portable.