Residuals

views updated

Residuals

To define the notion of residuals, let us introduce a linear model describing the relationship between K + 1 independent variables x_j with j = 0, 1, 2, …, K and a dependent variable y.

y = β ₀ + β ₁x ₁ + β ₂x ₂+ … + β _kx_k +u

with u being the stochastic error representing all factors affecting y not included in the x_j ’s.

Assuming E(uǀx) = E(u) = 0, the population regression function is:

E(yǀx) = β ₀ + β ₁x ₁ + β ₂x ₂ + … + β _kx _k

This is estimated by the sample regression function:

ŷ = β̂ ₀ + β̂ ₁x ₁ + β̂ ₂x ₂ + … + β̂ _Kx_K

where β̂ ₀, β̂ ₁, …, β̂ _K identify a set of estimated parameters calculated using the estimation rule β.

Assuming a sample of N observations, the residual for observation i, with i = 0, 1, 2, …, N, is the difference between the actual value of y i and the fitted value of the estimated regression, that is:

û_i = y_i -ŷ_i = y_i - β̂ ₀ - β̂ ₁x _{i 1} - β ̂₂x _{i 2} - … - β̂ _Kx_iK

If the residual is positive, the estimated regression underpredicts y_i and vice versa if the residual is negative. Each residual û_i can be interpreted as an estimate of the unobservable error u _i and as such can be employed in constructing a number of statistical tests assessing the properties of the estimated model and a set of indicators determining the goodness of fit.

Ideally, a good model is characterized by small residuals, that is, by fitted values ŷ_i that are close to the actual values y_i. The ordinary least squares (OLS) estimator of the parameters β _j is calculated by fitting the regression line that best approximates the data through the minimization of the sum of squared residuals. The resulting first-order conditions for the OLS estimator are and with j = 1, 2, …, K.

As regards the goodness of fit, defining the total sum of squares , the explained sum of squares, the explained sum of squares and the residual sum of the squares the part of variation in y that is explained by the regressors, is given by the coefficient R ₂ = 1 - SSR /SST. R ₂ is between 0 and 1, with the fit of the regression improving as R ² approaches 1. A consistent estimator of the variance of the stochastic error is given by Its square root σ̂ is then an estimator of the standard deviation of the unobservable factors affecting the dependent variable y. It indicates how well the model predicts y given the information set represented by the observables x _j’s. The estimate σ̂ is used to construct the standard error of the OLS estimator, equal to where SST _j is the total sum of squares of x _ij and R ²_j is the of the regression of x_j on all other regressors. The standard error se(β_j ) is then employed to test for statistical significance of each estimated parameter through the t -statistic.

If the errors were homoskedastic, we would have that Var (u ǀx ₁, x ₂, …,x_k ) = E (u ²ǀx ₁, x ₂, …, x_K ) = E (u ²) = σ ². On the contrary, in the presence of heteroskedasticity the expected value of u ² can be assumed to depend on some function of the explanatory variables. Following this logic, Breusch and Pagan suggest a test for heteroskedasticity that consists of an F -test on the regression of the squared residuals on the explanatory variables. The White test for heteroskedasticity simply consists of a similar regression including nonlinearities, that is, the squares and cross-products of the explanatory variables.

white suggests using the residuals û_i to calculate a heteroskedasticity-robust standard error for β_j. This is given by , where r̂ _ij is the i -th residual of the regression of x _j on all other regressors and SSR _j is the residual sum of squares of this regression.

SEE ALSO Hausman Tests; Least Squares, Ordinary; Least Squares, Three-Stage; Least Squares, Two-Stage; Ordinary Least Squares Regression; Regression; Test Statistics; White Noise