Properties of Estimators (Asymptotic and Exact)

BIBLIOGRAPHY

Economic models are useful in making statistical predictions and policy evaluations. These models include statistical mean models, regression models, simultaneous equations models, limited dependent variable models, and panel and spatial models, among others. Based on the sample data, econometric methods are used to estimate these models, and then the testing of relevant economic hypotheses can be conducted. The commonly used estimators are least squares or generalized least squares, maximum likelihood, generalized method of moments, and empirical likelihood. The hypothesis-testing procedures are Wald’s, Rao’s score, and likelihood ratio methods. Since all these methods are based on sample information, the statistical properties of these procedures are of great interest for both small and large samples. The properties include studying the distribution; unbiasedness, which implies the average value of the difference between a sample estimator and the corresponding population parametric is zero; and efficiency, which means a smaller variability. For large samples, the estimators are consistent if they converge to their true parameter values. In the context of hypothesis testing, a test having a correct size and high power implies the probability of rejecting an assumed economic model when the model is true and not true, respectively. All these have led to the development of asymptotic theory econometrics (White 2001) and finite sample econometrics (Ullah 2004).

Asymptotic theory provides the properties of estimators and test statistics for large samples. The statistical methods used to develop such properties are the law of large numbers and central limit theorems (see Chung 2001). There is extensive literature on the asymptotic properties of econometric estimators and test statistics (see White [2001] for linear models; Jennrich [1969] and Gallant [1987] for nonlinear models).

Finite sample econometrics deals with the exact and approximate properties of estimators and test statistics. The exact properties are valid for any size of the sample; however, the derivation of the exact analytical properties is often difficult, especially for the large class of applied economic models that can be nonlinear, nonnormal, or dynamic. Furthermore, even when the analytical exact results are obtained, they are usually expressed in terms of complicated multivariate integrals or infinite series, and they do not provide meaningful interpretations and conclusions (see Ullah 2004). In view of these problems, a vast amount of literature has developed on the approximate analytical properties of estimators and test statistics using the procedures developed by Francis Edgeworth (1896) for distributions and Anirudh Nagar (1959) for moments. The simulation-based approximate properties have also been extensively analyzed, especially using bootstrapping procedures (see Hall 1992; Horowitz 2001). The analytical and simulation-based approximate results can tell us how much we lose by using asymptotic theory results and how far we are from the known exact results.

It is well known that large sample properties may not imply finite sample behavior of econometric estimators and test statistics, and this can give misleading results for small or even moderately large samples. For example, an estimator can be biased for the finite sample but unbiased asymptotically. Also, two estimators may be asymptotically unbiased and have the same variances and asymptotic distributions, but, in the finite sample case, these two estimators may have different biases, variances, and distributions. Thus the statistical inference for the asymptotic case can be quite different compared to the finite sample case. Also, if the asymptotic results are used for small samples, then the conclusions or interpretations of the econometric analysis may be misleading.

R. A. Fisher (1921, 1922) and Harald Cramér (1946) laid the foundations of statistical finite sample theory on the exact distributions and moments. This exact theory on distributions and moments was brought into econometrics by the seminal work of Trygve Haavelmo (1947), T. W. Anderson and Herman Rubin (1949), Leonid Hurwicz (1950), R. L. Basmann (1961), and Peter C. B. Phillips (1983), among others. Another major development took place through Nagar’s work (1959) on obtaining the approximate moments of econometric estimators. This was followed by research by J. D. Sargan (1975) and Phillips (1980), who rigorously developed the theory and applications of the Edgeworth expansions to derive the approximate distribution functions of econometric estimators. Most of the contributions, however, were confined to the analytical derivation of the moments and distributions of the econometric statistics with independent and identically distributed (i.i.d.) normal observations. These also included the finite sample results using the Monte Carlo methodology (Hendry 1984) and advances in bootstrapping (resampling) procedures (Hall 1992). The analytical and bootstrap results for models that are nonlinear with nonnormal and non-i.i.d. observations remain a challenging task for future development. Some development has begun to take place for approximate analytical results (see Ullah 2004; Bao and Ullah 2006). Similarly, there are developments in the bootstrapping procedures for studying the properties of estimators with i.i.d. as well as dependent and nonstationary observations (see Horowitz 2001).

The study of asymptotic and finite sample properties is a fundamental issue of statistical inference, since the quality of data-based inference depends on the properties of estimators and test statistics used in the inference. The developments described provide analytical and simulation-based procedures for the properties of estimators and test statistics. The frontier of this research area has moved forward over the years, but some challenging issues remain. With advances in computer technology, this subject will further develop in both the analytical and the bootstrapping domains.

BIBLIOGRAPHY

Anderson, T. W., and Herman Rubin. 1949. Estimation of the Parameters of a Single Equation in a Complete System of Stochastic Equations. Annals of Mathematical Statistics 20 (1): 46–63.

Bao, Yong, and Aman Ullah. 2006. The Second-order Bias and Mean Squared Error of Nonlinear Estimators in Time Series. Unpublished Manuscript. University of California, Riverside, 2006.

Basmann, R. L. 1961. Note on the Exact Finite Sample Frequency Functions of Generalized Classical Linear Estimators in Two Leading Overidentified Cases. Journal of the American Statistical Association 56: 619–636.

Chung, Kai Lai. 2001. A Course in Probability Theory. 3rd ed. San Diego, CA: Academic Press.

Cramér, Harald. 1946. Mathematical Methods of Statistics. Princeton, NJ: Princeton University Press.

Edgeworth, Francis Y. 1896. The Asymmetrical Probability Curve. Philosophical Magazine 41: 90–99.

Efron, Bradley. 1979. Bootstrap Methods: Another Look at the Jackknife. Annals of Statistics 7: 1–26.

Fisher, R. A. 1921. On the “Probable Error” of a Coefficient of Correlation Deduced from a Small Sample. Metron 1: 1–32.

Fisher, R. A. 1922. The Goodness of Fit of Regression Formulae, and the Distribution of Regression Coefficients. Journal of the Royal Statistical Society 85 (4): 597–612.

Gallant, A. Ronald. 1987. Nonlinear Statistical Models. New York: Wiley.

Haavelmo, Trygve. 1947. Methods of Measuring the Marginal Propensity to Consume. Journal of the American Statistical Association 42 (237): 105–122.

Hall, Peter. 1992. The Bootstrap and Edgeworth Expansion. New York: Springer-Verlag.

Hendry, David F. 1984. Monte Carlo Experimentation in Econometrics. In Handbook of Econometrics, vol. 2, eds. Michael Intriligator and Zvi Griliches, 934–976. Amsterdam: North-Holland.

Horowitz, Joel L. 2001. The Bootstrap in Econometrics. In Handbook of Econometrics, vol. 5, eds. James Heckman and Edward Leamer, 3160–3228. Amsterdam: North-Holland.

Hurwicz, Leonid. 1950. Least Square Bias in Time Series. In Statistical Inference in Dynamic Economic Models, ed. Tjalling C. Koopmans. New York: Wiley.

Jennrich, R. I. 1969. Asymptotic Properties of Nonlinear Least Squares Estimators. Annals of Mathematical Statistics 40: 633–643.

Nagar, Anirudh. L. 1959. The Bias and Moment Matrix of the General k-Class Estimators of the Parameters in Simultaneous Equations. Econometrica 27 (4): 575–595.

Phillips, Peter C. B. 1980. Finite Sample Theory and the Distributions of Alternative Estimators of the Marginal Propensity to Consume. Review of Economic Studies 47 (1): 183–224.

Phillips, Peter C. B. 1983. Exact Small Sample Theory in the Simultaneous Equations Model. In Handbook of Econometrics, vol. 1, eds. Michael Intriligator and Zvi Griliches, 449–516. Amsterdam: North-Holland.

Sargan, J. D. 1975. Gram-Charlier Approximations Applied to t Ratios of k-class Estimators. Econometrica 43 (2): 327–346.

Ullah, Aman. 2004. Finite Sample Econometrics. New York: Oxford University Press.

White, Halbert. 2001. Asymptotic Theory for Econometricians. Rev. ed. San Diego, CA: Academic Press.

Aman Ullah