Inference from Field and Laboratory Experiments in Economics: Empirical Evidence

Experimental economics publications test and make claims that are based on inferences
from the experimental datasets they are based on. A claim is within-domain if, with respect
to a given domain (profession, age, gender, country of experiment, or experimental asset),
it is representative of the domain that characterizes its underlying experimental dataset.
The field avails access to a range of samples and assets, but do field experiments
publications engage in within-domain inference, in general and relative to laboratory
experiment publications? Our study evaluates the extent to which data from field and
laboratory experiments match (i.e. correspond) to the key claims tested in 520 publications
in 2018 and 2019 at leading general and field journals in Economics. We find that claims
from field experiments, particularly in the realm of policy testing, are more likely to match
the key claims compared to laboratory experiments. However, depending on the domain,
less than 20% or only up to around 65% of field experiments achieve a match. Around four
out of five field experiments fail to match in at least three out of the five domains. We
conclude that the methodological practice of drawing inferences beyond the domain of the
experiments themselves also applies to many field experiments. Further, we find that
publications by top 20 institutions authors or with experiments conducted in majority
White countries are more likely to generalize.

http://www.danielzizzo.com