Exchangeability
Exchangeability
In constructing statistical models, social scientists are nearly universally required to incorporate residual terms to capture those aspects of a phenomenon that the model under analysis cannot explain. Linear models are a leading case, where a dependent variable yi is assumed to be explained by a vector of observables xi as well as an unobservable residual є i via the equation yi = xiβ + єi, i = 1, …, I. Estimation and inference concerning the model parameters comprised by the vector β, in turn, requires a researcher to make assumptions about єi. Examples of assumptions include independence of the residuals across I, and homoskedasticity (constant variance) of the residuals. Such assumptions are employed because they allow one to construct precise inferential statements such as asymptotic distributions under a null hypothesis, but not because they correspond to any substantive social-scientific ideas.
From the social-scientific perspective, one way to think about what it means to specify a model is to place enough structure on the variable of interest, say yi, so that, from the perspective of the modeler, the residuals are indistinguishable. In other words, a modeler should have no reason to believe that the probability description of єi differs from that of єj. If the modeler does believe that these residuals differ, then it would seem that the model needs to be reassessed to see whether these differences call into question the purpose of the model.
To understand the import of this claim, suppose that one is attempting to explain the differences in economic growth between Japan and the United States since World War II and is using a model that does not account for differences in the savings rates between the two countries. For the researcher using the model to find it interpretable as reflecting a causal relation, he presumably must believe that Japan’s much higher average rate of saving versus the United States has no bearing on the residuals associated with the two countries. If he believes that this difference in savings rates induces a difference in the probability descriptions of the two residuals, then he needs to determine how the model may be interpreted in this light.
Exchangeability is a mathematical formalization of the idea that random variables are, from the perspective of their probability description, indistinguishable. A sequence of I random variables єi is exchangeable if μ(є1, …, єI ) = μ(єρ(1), …, єρ(I) ), where ρ() is an operator that permutes the I indices. Note that I may be infinite. The concept of exchangeability originated in the writings of the Italian probabilist Bruno DeFinetti (see DeFinetti 1972 for a wide-ranging statement of his views in English).
Infinite exchangeable sequences have the important property that they may be interpreted as independent and identically distributed (i.i.d.) sequences. This is known as DeFinetti’s theorem. Formally, the theorem says that the probability measure describing any infinite exchangeable sequence can be written as a mixture of i.i.d. probability measures. Each sample path realization will obey one of the probability measures, so each sample path will behave as an i.i.d. sequence. DeFinetti’s theorem thus provides a basis for the i.i.d. assumption: the interchangeability of errors. A proof of DeFinetti’s theorem along with many related results may be found in Olav Kallenberg’s work Probabilistic Symmetries and Invariance Principles (2005). DeFinetti’s theorem does not apply to finite exchangeable sequences.
To be clear, exchangeability is not necessary for a statistical model to be interpretable as a behavioral structure. Heteroskedastic regression errors (regression errors of varying variance) violate exchangeability, but do not affect the interpretability of a regression per se. Rather, exchangeability represents a criterion by which a researcher can evaluate his modeling choices. If a researcher believes that the errors in his model are not exchangeable, then good empirical practice requires that he consider whether the violations invalidate the substantive claims for which the model will be used. This inevitably requires judgment, but judgments are part of any substantive empirical exercise. Draper et al. (1993) place judgments of exchangeability at the heart of empirical analysis and discuss how data may be used to assess these judgments. Brock and Durlauf (2001) place exchangeability judgments at the center of a general critique of empirical research on growth. This emphasis on judgment helps explain why exchangeability notions are more common in Bayesian contexts than in frequentist contexts; exchangeability captures the notion of what is meant by a researcher’s subjective beliefs that a sequence is a random sample from a population (see Lindley and Novick 1981 for elaboration and links to DeFinetti’s original thinking).
Exchangeability has recently been shown to have important uses in econometric theory. Donald Andrews (2005) developed a theory for asymptotic inference in cross-sections that addresses the long-standing problem of cross-sectional residual correlations. Unlike the time-series case, a cross-sectional index does not provide a natural ordering across random terms, so there is no reason to think that dependence across residuals diminishes as the difference between their respective indices grows. Andrews shows that if this dependence is generated by a set of common shocks so that, conditional on the shocks, the errors are independent, one can develop a range of asymptotic results. This independence is motivated in turn by exchangeability arguments. This mode of analysis suggests that exchangeability may play an increasingly prominent role in econometric theory.
SEE ALSO Bayes’ Theorem; Bayesian Econometrics; Probability Theory
BIBLIOGRAPHY
Andrews, Donald. 2005. Cross-Section Regression with Common Shocks. Econometrica 73: 1551–1585.
Brock, William, and Steven Durlauf. 2001. Growth Empirics and Reality. World Bank Economic Review 15 (2): 229–272.
DeFinetti, Bruno. 1972. Probability, Induction, and Statistics. New York: Wiley.
Draper, David, James Hodges, Colin Mallows, and Daryl Pregibon. 1993. Exchangeability and Data Analysis (with Discussion). Journal of the Royal Statistical Society, series A, 156: 9–37.
Kallenberg, Olav. 2005. Probabilistic Symmetries and Invariance Principles. New York: Springer-Verlag.
Lindley, David, and Melvin Novick. 1981. The Role of Exchangeability in Inference. Annals of Statistics 9: 45–58.
William A. Brock
Steven N. Durlauf