VarianceCovariance Matrix
VarianceCovariance Matrix
ADDITIONAL MATHEMATICAL PROPERTIES
The variancecovariance matrix is a convenient expression of statistics in data describing patterns of variability and covariation. The variancecovariance matrix is widely used both as a summary statistic of data and as the basis for key concepts in many multivariate statistical models.
VERBAL DEFINITION
The variancecovariance matrix, often referred to as Cov(), is an average crossproducts matrix of the columns of a data matrix in deviation score form. A deviation score matrix is a rectangular arrangement of data from a study in which the column average taken across rows is zero. The variancecovariance matrix expresses patterns of variability as well as covariation across the columns of the data matrix. In most contexts the (vertical) columns of the data matrix consist of variables under consideration in a study and the (horizontal) rows represent individual records. Variancecovariance matrices may, however, be calculated from any pairwise combination of individuals, measurement occasions, or variables. Even this by no means exhausts the possible covariance matrices that may be considered for a statistical model (see Cattell [1988] for an extensive list of the possibilities involving several dimensions).
MATHEMATICAL DEFINITION
If k variables are assumed in a study and letting X denote a raw score version of the data matrix and µ_{k} a vector of means for the variables under consideration, the covariance matrix is defined as E (X′X ) – µ_{k} ′µ_{k}, where E () denotes the expectation operator. If the columns of X are centered to a mean of 0, the variancecovariance matrix is more conveniently expressed as E (x′x ). Within linear algebra, covariance matrices belong to the class of matrices known as nonnegativedefinite symmetric matrices.
CALCULATION
The sample covariance matrix can be calculated as 1/n *1_{k} *(X′X – µ_{k} ′µ_{k} ) if raw score matrix X is used, µ_{k} denotes a vector of sample means, and where ’ denotes the transpose operator. If data are expressed in columncentered form, x = (X – X ), Cov(x ) is calculated as 1/n *1_{k} *x’x, where 1_{k} denotes a k column vector of 1s and n denotes the number of observations. The sample variancecovariance matrix, although efficient, is a biased estimate of population variability. As a result, the estimated population covariance matrix divides by the reciprocal of n – 1 of n. If the x matrix is further transformed to have a variance of 1 (usually termed Z_{x} ), the resulting sample Cov() matrix is known as a correlation matrix. If the X matrix is retained in raw score form and an additional unit column is added to the data, 1/n *1_{k} *X′X is referred to as an average sum of squares and crossproducts matrix, a data summary convenient for models in which overall elevation as well as patterns of covariation are of interest, as often occurs in longitudinal studies of growth.
PROPERTIES
The Cov() matrix has as many rows and columns as the columns of X and is symmetric (meaning that the value associated with the j th row and k th column in Cov() is equal to the value in the k th row and j th column). Diagonal elements of Cov() represent the variances of the column variables; offdiagonal elements represent covariances or, if based on Z_{x}, correlation coefficients.
ADDITIONAL MATHEMATICAL PROPERTIES
Covariance of a sum: Assuming three matrices x 1, x 2, and y,
Cov(x 1 + x 2, y ) = Cov(x 1, y ) + Cov(x 2, y ).
Covariances involving matrix products: Assuming two conformable matrices A and B,
Cov (AX,BX ) = A Cov(X,X )B’ where ’ denotes the transpose operator.
USES
As mentioned before, covariance matrices, by themselves, are compact summaries of the variability and covariation present in data. More generally, the covariance matrix and vector of means constitute sufficient statistics for models that assume a multivariate normal distribution. As such, the covariance matrix may be used in lieu of the raw data in calculating a number of multivariate statistical models, such as confirmatory and exploratory factor analysis (assuming the diagonal of the matrix is appropriately adjusted by the estimated communality), path analysis, or other general linear models, including the special cases of multiple regression, analysis of variance, and repeated measures analysis of variance or MANOVA. In many statistical models, finding an optimal basis for representing the covariance matrix in a compact fashion is of primary interest. Such reduced or optimal bases are referred to as principal components analysis (PCA), or in image processing as the KarhunenLoève transform, which has time series applications within psychology (Molenaar and Boomsma 1987). Covariance matrices alone are not sufficient statistics for other more sophisticated models, such as those involving weighted least squares, sampling weights, or other categorical or distributional adjustments to reflect the dichotomous, polytomous, or other distributional characteristics of the variables under consideration.
SEE ALSO Classical Statistical Analysis; Covariance; Econometric Decomposition; Inverse Matrix; Least Squares, Ordinary; Matrix Algebra; Ordinary Least Squares Regression; Path Analysis; Regression; Regression Analysis; Statistics
BIBLIOGRAPHY
Cattell, Raymond B. 1988. The Meaning and Strategic Use of Factor Analysis. In Handbook of Multivariate Experimental Psychology, eds. John R. Nesselroade and Raymond B. Cattell, 174–243. 2nd ed. New York: Plenum.
Molenaar, Peter C. M., and Dorrett I. Boomsma. 1987. The Genetic Analysis of Repeated Measures. II: The KarhunenLoève Expansion. Behavior Genetics 17 (3): 229–242.
Phillip K. Wood
Cite this article
Pick a style below, and copy the text for your bibliography.

MLA

Chicago

APA
"VarianceCovariance Matrix." International Encyclopedia of the Social Sciences. . Encyclopedia.com. 24 Sep. 2018 <http://www.encyclopedia.com>.
"VarianceCovariance Matrix." International Encyclopedia of the Social Sciences. . Encyclopedia.com. (September 24, 2018). http://www.encyclopedia.com/socialsciences/appliedandsocialsciencesmagazines/variancecovariancematrix
"VarianceCovariance Matrix." International Encyclopedia of the Social Sciences. . Retrieved September 24, 2018 from Encyclopedia.com: http://www.encyclopedia.com/socialsciences/appliedandsocialsciencesmagazines/variancecovariancematrix
Citation styles
Encyclopedia.com gives you the ability to cite reference entries and articles according to common styles from the Modern Language Association (MLA), The Chicago Manual of Style, and the American Psychological Association (APA).
Within the “Cite this article” tool, pick a style to see how all available information looks when formatted according to that style. Then, copy and paste the text into your bibliography or works cited list.
Because each style has its own formatting nuances that evolve over time and not all information is available for every reference entry or article, Encyclopedia.com cannot guarantee each citation it generates. Therefore, it’s best to use Encyclopedia.com citations as a starting point before checking the style against your school or publication’s requirements and the mostrecent information available at these sites:
Modern Language Association
The Chicago Manual of Style
http://www.chicagomanualofstyle.org/tools_citationguide.html
American Psychological Association
Notes:
 Most online reference entries and articles do not have page numbers. Therefore, that information is unavailable for most Encyclopedia.com content. However, the date of retrieval is often important. Refer to each style’s convention regarding the best way to format page numbers and retrieval dates.
 In addition to the MLA, Chicago, and APA styles, your school, university, publication, or institution may have its own requirements for citations. Therefore, be sure to refer to those guidelines when editing your bibliography or works cited list.