Alternatives to the Chi-Square Test for Evaluating Rank Histograms from Ensemble Forecasts

From: Weather and Forecasting | Date: October 1, 2005| Author: | Copyright information

ABSTRACT

Rank histograms are a commonly used tool for evaluating an ensemble forecasting system's performance. Because the sample size is finite, the rank histogram is subject to statistical fluctuations, so a goodness-of-fit (GOF) test is employed to determine if the rank histogram is uniform to within some statistical certainty. Most often, the χ^sup 2^ test is used to test whether the rank histogram is indistinguishable from a discrete uniform distribution. However, the χ^sup 2^ test is insensitive to order and so suffers from troubling deficiencies that may render it unsuitable ...

Read all of this article with a FREE trial to HighBeam Research

Related newspaper, magazine, and trade journal articles from HighBeam Research

(Including press releases, facts, information, and biographies)

Interpretation of rank histograms for verifying ensemble forecasts
; ABSTRACT Rank histograms are a too] for evaluating...its mean and spread. Rank histograms are generated by repeatedly...uncritical use of the rank histogram can lead to misinterpretations...Similarly, a U-shaped rank histogram, commonly...also shown that flat rank ...
Evaluating Rank Histograms Using Decompositions of the Chi-Square Test Statistic
; ABSTRACT Rank histograms are often plotted to...forecasting system-an ideal rank histogram is "flat" or uniform...Introduction It is common to use rank histograms to evaluate the performance...statistic Consider a rank histogram for ensemble forecasts...bins or classes in ...
Ensemble Calibration of 500-hPa Geopotential Height and 850-hPa and 2-m Temperatures Using Reforecasts
; ...Forecasts were evaluated using rank histograms and the continuous ranked probability skill score. T2M rank histograms showed a high population of extreme...slightly. The extreme ranks of Z500 rank histograms were slightly underpopulated...
Verification of an Ensemble Prediction System against Observations
; ...distribution, we also looked at the rank histogram (Anderson 1996; TaIagrand et al. 1999...at the spread-skill correlation. The rank histogram measures the statistical consistency...from some weaknesses. First, the RMSE, rank histogram, and spread-skill correlation are evaluated...
Theory and Applications of the Minimum Spanning Tree Rank Histogram
; ABSTRACT A minimum spanning tree (MST) rank histogram (RH) is a multidimensional ensemble reliability verification...forecasts, this degree can be assessed by the shape of a rank histogram (RH), or Talagrand diagram (Anderson 1996; Talagrand...
Evaluating probabilistic forecasts using information theory
; ...relative operating characteristics (Swets 1973; Mason 1982), rank histograms (Anderson 1996; Hamill and Colucci 1996; Talagrand et al. 1997), and the generalization of rank histograms to higher dimensions (Smith 2000). Information theory provides...
A New Verification Method to Ensure Consistent Ensemble Forecasts through Calibrated Precipitation Downscaling Models
; ...precipitation fields forecasted by calibrated downscaling models. The method is based on a generalization of the verification rank histogram and tests the exceedance probability of a fixed precipitation threshold calculated from the observed or ensemble fields...
Implications of Stochastic and Deterministic Filters as Ensemble-Based Data Assimilation Methods in Varying Regimes of Error Growth
; ...ensemble assessment diagnostics: rms analysis error, ensemble rank histograms, and measures of ensemble skewness and kurtosis. Similar...diagnostics, root-mean-square (rms) analysis error, ensemble rank histograms, and measures of ensemble skewness and kurtosis. We then...
Extending the Limits of Ensemble Forecast Verification with the Minimum Spanning Tree
; ...the reliability of the forecasts and 2) estimating the probability distribution of the future state of the system. Current rank histogram ensemble verification techniques can only evaluate scalars drawn from ensembles and associated verification; a new method...
Calibrated Surface Temperature Forecasts from the Canadian Ensemble Prediction System Using Bayesian Model Averaging
; ...forecasts as an independent sample. This process was repeated through the year, and forecast quality was evaluated using rank histograms, the continuous rank probability score, and the continuous rank probability skill score. An examination of the BMA weights...