MANAGEMENT SCIENCE

VoL 34, No. 4. Afiril 1988

fmudinVSA.

A SIMPLE METHOD OF COMPUTING PREDICTION INTERVALS FOR TIME SERIES FORECASTS*

EVERETTE S. GARDNER, JR. College of Business Administration, University of Houston, Houston, Texas 77004
The(Htkl approaches to computing prediction intervals require strong assumptions that do not ^fpesr to hold in practice. This pi^ier presents an empirical approach to prediction into-vals tiiat assumes v o y little. During mo^l-fitting, variances of the errors are computed at different forecast le^times. Using these variances, the Qiebyshev inequality is applied to (tetermine prediction interval. Empirical evitknce is iesented to show that this amHoach gives reasonaUe results. For example, uang the 111 series in the Af-competition, 95% prediction intervals actually contain 95.8% of post-samtrfe <rf)servations. (FORECASTINGTIME SERIES) 1. IntrodHctioii

Almost all point forecasts are wrong. Thus prediction intervals are needed to indicate the likely precision of the forecasts for management planning. Prediction intervals are especially helpful in forecasting for inventory (xtntrol, where safety stocks depend on the prol^bility distribution of lesuitime demand. Another application of prediction intervals is to identify outliers in time series. Traditionally, prediction intervals are computed by making one of two critical assumptions: (1) the correct model has been identifi^ or (2) the generating process for the time series is known. In most cases, these assumptions make it straightforward to derive closed-form expressions for variances of the forecast errors at different leadtimes. For examine. Box and Jenkins (1976) rely on the first assumption to obtain prediction intervals for ARMA models. Advocates of exponential smoothing do not assume that the correct model is used. Instead they assume that the generating proems is known and ^rive variances based on the relationship between this process and the model at hand. See, for example, the expressions for variances of exponential smoothing models in Brown (1963), McKenzie( 1976,1984,1986), and Swt( 1985). To simplify matters, both ARMA modders and e^^mnential smoothers almost al^x^ys mi^e another critical assumption, that the errors are normally distributed. One probtem with this theoretical woik on prediction intervals is that it is impossible to obtain dosed-form expressions for the variances of nonlinear forecasting systems. For exanqde, there is no closed form for the popular Holt-Winters class of exponential smoothii^ sy^ms with linear trend and multiplicative seasonality (McKenzie 1984). For such systems, an empirical method is the only alternative for computing prediction Anotter proUem is that ttecHctical prediction intervals often yidd poor results in practice. In a ieexamination of the 111 time series from the Af-competition (Makridakis et al. 1982), Lusk and Bdhadjali (1986) found that 95% prediction intervals for ARMA modds contained only about 80% of post-samirie observations. Makridakis and Hibon (1986) <ri^ained amilar results for oth- time series models. Om solution to these ixoUems is the ^ i m c h taken by Williams and Goodman (1971), v4io (tevdtoped emprical distributions of post-samide forecast errors. Their procedure is to fit a modd to a samide of a time series, make a set of forecastsfipomone
* AcceptedfayVqayMiAsaan; received January 12,1987.110$ paper has been with the audior 1 month for 1 CBvision. 541 0025-1909^/34O4/O541$01J25
CopyiigM e 1988, H K liHtiaiie<rfMaaKM Sdaoes

542

EVERETTE S. GARDNER, JR.

time origin, compile the post-sample errors, and refit the model with one additional observation. These steps are repeated until the end of the data is reaci^. The Williams and Gocximan procedure is so tedious that no applications sinc^ 1971 appear to have been reported. This paper suggests a simple method of computing predication intervals that avoids the need to make assumptions about the validity of the model, the form of the generating function, or the form of the distribution of forecast errors. Although the method is empirical, it is far less tedious than the Williams and Goodman approach. 2. The Method Given that a forecasting model has been fitted to a time series, the first step is to compute variances of fitted errors at different leadtimes. For example, suppose we have a model selected on the usual basis of one-step-ahead fit. One pass through the data is made to compute the variance of the fitted errors at one-step-ahead. A second pass is made to compute the variance at two-steps-ahead. It is important to understand that the forecasting model is not re-estimated. We simply make two-step-ahead forecasts with the same model and compute the variance of the fittl errors. This process is continued until an individual variance is computed for each desired leadtime. For example, if prediction intervals are desired for leadtimes 1-12, there will be 12 individud variances. The advantage of this method of computing variances is that the validity of the model and the form of the generating funcrtion are irrelevant We simply record the performance of the model. The second step in the procedure is to compute standard errors at each leadtime. The final step is to apply a multiplier to each standard error that yields the desired prediction intervals. The multiplier is based on the Chebyshev inequality (Wilks 1962), which sets a bound on the amount of probability within given limits for any distribution with finite variance. Since the Chebyshev in^uality is well known, only the main result is given here. Let F be a random variable with mean n and variance <P. The inequality is P[\(Y-n)/a\>]^l/'. (1)

No matter what the actual distribution, ^uation (1) states that the probability of an otervation falling beyond e standard enors from the mean is at most 1/e^. Admittedly the Chebyshev inequality generate crude bouncis for many known distributions. For example, if the true distribution of errors is normal, the Qiebyshev bounds are mtich too wide. But in practice the true distribution of ernsrs is never known. Makridakis and Winkler (1985) were unable tofindany distribution that gave a reasonable fit to the errors in a coUortion of 1,001 time seri. Their sti^y suggests that it is dangerotis to assume some arbitrary distribution for the errors. The simj^ expression in (1) is at least a starting point for the development of prediction intervals. 3. EraiHrical Resnlte The time series from the M-ojmpetition (Makridakis et al. 1982) were used to test this method of computing prediction intervals. Forecasts were jntxluc^ using the class of exponential smoothing systems with damped trends ikveioped by Gardner and McKenzie (1985). Other computational details sufficient to r ^ c a t e tiK readts below are availal^ in Gardner (1986). Forecastii^ was done in exactly the same manner as Makridakis. Let the lei^th of a !ries be N. For annual series, the forecasting modd was fittal to the first JV - 6 c^servations. A set of 6 for:aste (for l^dtimes 1 -6) was made at time o r i ^ N~6. For quarterly data, 8 forecasts were made at .^ - 8. For monthly data, 18 forecasts were made at i V - 18.

COMPUTING PREDICTION INTERVALS FOR TIME SERIES FtoURE 1. Percentage of Po^-Sami^ Obs>'ations Inade Prediction Int^vsds: 111 Time Seri. 90% Target rorecast Leadtime 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 AU Chebydiev 94.6% 93.7 94.6 91.9 84.7 86.5 91.2 92.3 92.6 89.7 94.1 89.7 86.8 91.2 89.7 92.6 88.2 89.7 90.8% Normal 82.9% 77.5 73.0 71.2 72.1 66.7 70.3 69.2 76.5 73.5 73.5 72.1 70.6 73.5 66.2 79.4 75.0 67.6 72.9% Chebyshev 97.3% 99.1 96.4 95.5 93.7 91.9 95.6 96.7 98.5 98.5 97.1 95.6 94.1 97.1 94.1 95.6 94.1 94.1 95.8% 95% Target

543

Normal 88.3% 83.8 85.6 80.2 77.5 72.1 73.6 74.7 80.9 75.0 77.9 79.4 73.5 79.4 73.5 83.8 82.4 76.5 79.1%

The Makridakis data include 1,001 time series. Figure I gives results for a sample of 111 series taken from the 1,001. This is the same sample discussed in Makridakis et al. (1982). For each series, 90% and 95% prediction intervals were computed and the number of post-sami^e observations inside the intervals was recorded. The Chebyshev prediction intervals contain approximately the desired percent^es: 90% intervals contain 90.8% of post-sample otervations, while 95% intervals contain 95.8%. Figure 1 also gives results for prediction intervals assuming the normal distribution and using the method of computing variances described above. Normal 90% intervals contain only 72.9% of post-sample observations, while normal 95% intervals contain only 79.1%. The normal percent^es inside prediction intervals are comparable to the results reported by Williams and Goodman (1971), Lusk and Belhadjali (1986), and Makridakis and Hibon (1986). Figure 2 gives results for the complete collection of 1,001 time series. Percentage insi(te the prediction intervals are similar to F%ure 1. Doestileperformance of Chebydiev iHdiction intervals depend on the type of data? To answo- this question, furtter tests were made using subsets of the 111 time series. The r^ults ^lown in Figure 3 are averse percent^es over all forecast horizons. There is littie diflference in Chebyshev performance betw^n seasonal and nonseasonal data. However, there is an important difference in performance when the time series are da^fied by frequency of observation (annual, quarteriy, and monthly). Chebydiev percentage are n ^ r t a ^ t s for quaiteriy and monthly series but are substantially below t^^ets in annual series. The problem is that most of the annual series in tiie Makridakis coBection are too siiort tofitan actequate forecasting model. In the 20 annual seri<^ the a v o ^ ^ number of fitted oteervations is 19.7. Nine of the annual series have 13 fitted observations ami one has mdy 12. Tbe quarterly aiM) monthly series u e mudi lon^r a ^ a v e r ^ 38.4 id 68.4 fitted (rt>servations, t^iectively. l%e saiativky rf C3ieb^lffiv iHie(fictk>n intvate to tl^ number offittedobservations is ^aeawn in iie d^^l by Figaae 4. With ^s thaat 20 fitted t^iservations, tb^ Cteby-

544

EVERETTE S. GARDNER, JR.

FIGURE 2. Percentage of Pcst-Sam|de CNxervations Inside Prediction Intervals: 1,001 Time Series. 90%Taiget Forecast Leadtime 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 All ChebyAev 96.2% 95.1 93.2 91.6 89.2 88.0 91.7 90.1 92.2 93.4 92.1 92.9 91.4 91.9 91.1 90.4 90.6 89.3 91.8% Normal 80.1% 77.3 75.9 72.7 71.3 66.3 71.5 69.6 72.8 73.7 73.9 72.8 71.6 72.5 69.5 71.4 70.7 70.2 72.7% Owbyshev 98.6% 98.0 96.4 96.1 94.7 93.3 96.8 96.0 97.6 97.9 96.8 97.7 96.6 96.6 96.3 95.6 96.1 95.5 96.4% 95% Target
NiMinal

86.2% 84.1 82.0 77.7 76.0 71.8 77.2 75.9 77.0 78.6 78.1 77.2 76.8 77.8 74.6 77.8 78.9 74.4 78.2%

FteiURB 3. Performance by Type of Data. 90% Target Type of Data AUdata Seasonal Nonseasonal Annual Quarterly Monthly Nbr. of Series Ul 60 51 20 23 68 Nbr. of Fmecasts 1,528 990 538 120 184 1,224 Chebyshev 90.8% 91.7 89.0 78.3 88.0 92.0 Normal 72.9% 72.2 73.1 58.3 59.2 75.4 95% Tanpet C3i^>ysiMv 95.8% 96.7 93.1 83.3 95.7 95.9 Ncmnal 79.1% 77.8 79.2 63.3 71.7 80.3

FIGURE^k

Performance by Number of Fitted (Msnvations. 90% Targrt 95%TaiKet Chebysbev 82.7 95.0 95.6 98J 95.8 95.4 Ktomal 57.3% 80.0 81.4 76.0 80.1 79.8

Nbr. of Fitted dK. <20 20-29 30-39 40-49 50-59 >60

NlMT.rf Series 17 6 16 13 16 43

Nlw. of Fweiaists 110 40 204 204 216 754 71.8% 92.5 922 93.6 89.8 91.9

N<wmal 51.8% 75.0 74.5 67.2 74.5 74.9

^ v int^vab are not reliaUe. With 20 or mooefittedobKrvations, O^q^^bev ifltra-^^ are t u ^ y rdiatde. These are at lst two exirianatioDs fcnr the good pesfomiamce t^ tbe O^liyAev inequdity. Fir^ as discKsed in Maiaidlalds and WinUer (1985), the esrom are it

COMPUTING PREDICTION INTERVALS FOR TIME SERIES 95W PREDICTION itfTBRVALS. SERIES SOI.
t60

545

ISO 140 130 120 110 100 90 BO 70 60 SO 40 30 20 10 0 20 40 QUARTER

FIGURE 5 ACTUAL CHEBYSHEV LIMIT

++

^NORMAL LIMIT ,

60

80

normally distributed in these data, regardless of the forecasting method. Second, in this research as well as in the M-competition, post-sample forecast errors were larger than within-sample errors. The Chebyshev inequality yields wider prediction intervals than the normal distribution in order to compensate for larger post-sample errors. One reason that larger errors occur during the post-sample period is that time series firequentiy display changes in pattern or discontinuities. Except for Carbone and Makridalds (1986), tiiis problem has been ignored in the literature. There are many series in the Makridakis data that display change in the direction of trend after the forecasts are made. Figure 5 illustrates this problem for the quarterly time series (number 301) that was analyzed in Carbone and Makridakis (1986). This series is comix>sed of 64 observations, with a weak seasonal pattern and an unstable trend. An exponential smoothii^ model with damped trend and multiplicative s^sonality was fitted to the series, using quarters 1-56. During model-fitting, prediction intervals were developed for quartos 57-64. The Chebyshev intervals accommodate the drastic change in trend whereas the normal intervals do not. All eight post-sample observations are inside the Chebyshev intervals, while only three are inside tiie normal intervals. 4. Omdnsums llie normal distribution is a staiulard assumption for determining prediction intervals in time series forecasting. However, normal prediction intervals are dangerously mideading. Norm^ intervals are too small r^ardless of the forecast leadtime, the type of data, or the number of fittol observations. Thisfindingis consistent with other work a variety of different foretasting methods by Williams and Goodman (1971), and Belh^jali (1986), and Makridakis and Hibon (1986). Since it sap^sas to be imp<^ble to find any adequate distribution for forecast errots, tbe Chebyshev inequality seems an ob\dous alternative for setting bounds on the errors.

546

EVERETTE S. GARDNER, JR.

In the Makridakis data, the Chebyshev yields accurate prediction intervals. Compared to the normal distribution, the primary reason that the Chebyshev works is that it increases prediction intervals to compensate for larger errors during the post-sample period. There is no guarantee that this strategy will perform as well in other data. However, deteriorating accuracy during the post-sample period has been olerved in numerous empirical studies and is by now a generally accepted outcome in time series forecasting. In order to apply the Chebyshev in^uality, variances of the errors by forecast leadtime are necessary. The method of estimating variances in this paper is simple and does not rely on any assumptions about the validity of the model or the form of the generating function for the time series. The method is certainly time-consuming but it is less so than the Williams and Goodman approach.'
' The author is indebted to two anonymous referees and Professor Lynn Lamotte of the University of Houston for helpful suggestions on this research.

References
Box, G. AND G. JENKINS, Time Series Analysis: Forecasting and Control, revised ed., Holden-Day, San Francisco, 1976. BROWN, R., Smoothing, Forecasting, and Prediction ofDiscrete Time Series, Prentice-Hall, Ei^ewood Cliffs, NJ, 1963. CARBONE, R. AND S. MAKRIDAKIS, "Forecasting When I^ttem Changes Occur Beyond the Original E>ata," Management Sci., 32 (March 1986), 257-271. GARDNER, E . S., Autocast Exponential Smoothing System User's Guide, Core Analytic, Bridgewater, NJ, 1986. AND E. MCKENZIE, "Forecasting Trends in Time Series," Management Sci., 31 (October 1985), 1237-1246. LUSK, E AND K. BELHADJAU, "Prediction Intervals for ARIMA Mo&ls for tiie Makrictekis 111 Series," Irttemat. J. Forecasting, fiHthcoming (1986). MAKRIDAKIS, S. ET AL., "The Accur^^ of Extrapcdation (Time Series) Methods: Results of a Foreca^ng Competition," J. Forecasting, 2 (Aiwil-June 1982), 111-153. AND M. HiBON, "Prediction Intervals: an Eminrical Investigation for the Series in the A/-Competition," Intemat. J. Forecasting, forthcoming (1986). AND R. WiNKLER, "The Samf^i^ IDistribution <rf I^>^-samrde Forecast ErrtMS," INSEAD wcnkii^ paper, 1985. MCKBNZIE, E., "A Comparison of Some Standaid Srasonal Forecasting Systems," The Statistician, 25 (January 1976X 3-14. , "Genersd Expcmential Smoothii^ and the Equivalent ARMA Process," / . Fa-ecasting, 3 (July-September 1984), 333-344. , "Error Analysis fw Winters* Additive Seasonal Foreca^ng System," Intemat. J. Fcrecastmg, 2 (Apdl-June 1986), 373-382. SWEET, A., "Computing the Variance of the Forecasti^ Error for the Holt-Winters Seasonal Modds," J. Favcasting, 4 (October-December 1985), 235-243. WDLKS, S. S., Mtahemmical Statistics, Witey, New York, 1962. WiLUAMS, W. AND M. GocXHkf AN, "A ISn^de Method (rf Omstnicting Omfidrace limits for Ecomnoic Forecasts," J. Amer. Statist. Assoc., 66 (Decembo^ 1971), 752-754.