Can Futures Outperform a Random Walk in Commodity Markets?

Andrew Krog* University of Michigan April 17, 2013

Abstract
The idea that commodity futures are efficient forecasters of future spot prices has never been proven definitively. Nonetheless, it is a widely held belief among market participants. This paper explores the theory and tests the forecasting ability of the futures price against a simple random walk model, that has proven difficult to beat in many forecasting applications. Six highly traded commodities (corn, crude oil, gasoline, gold, natural gas, silver) from three commodity classes are included to identify differences in forecasting ability across classes. Recursive forecasts are constructed for 1-, 3-, 6-, 9-, and 12-month horizons using a spread regression to obtain mean square prediction errors, which are compared between the regression and random walk model. The modified Diebold-Mariano statistic is used to test the null hypothesis of equal predictive power. This statistic is then bootstrapped using a nave block bootstrap to find 95% confidence intervals. The results show no statistical difference between the random walk and spread regression, although the MSPE is lower for the random walk for the majority of cases.

Keywords: Recursive Forecasts, Diebold-Mariano Statistic, Block Bootstrap * Department of Economics, University of Michigan, Ann Arbor, MI 48109. Email: akrog@umich.edu

1 Introduction
The question of whether or not commodity futures prices are good forecasters of future spot prices is one that has been frequently addressed yet never answered definitively. Commodity price forecasts are widely used by policymakers as well as businesses, so understanding the reliability of these forecasts is crucial to making optimal policy and investment decisions. Intuitively, a forecasting method should outperform a random walk benchmark in order to be considered for policy of investment decision making. A frequently used method, which is the one addressed in this paper, is the futures price. Futures prices are often looked to as a measure of expected future spot price due to the assumption that all information available to the market is reflected in the current future price. However, futures prices have shown differing levels of reliability depending on commodity class and forecast horizon. This paper will test the forecasting ability of futures prices against a random walk at 1-, 3-, 6-, 9-, and 12-month horizons for three classes of commodities: softs, metals, and energy. The six commodities analyzed are corn, crude oil, gasoline, gold, natural gas, and silver. Forecasting ability will be determined by constructing recursive forecasts regressing the change in spot price on the basis (difference between futures price and spot) and an intercept using least squares. The Diebold-Mariano test statistic will then be used to determine whether the difference in the predicted mean squared errors for random walk and spread regression is significantly greater than zero (one-sided test). Since we are considering nested models with parameter uncertainty we will use the variance of the spread regression predicted mean square errors to determine the standard error. Furthermore, it is necessary to bootstrap this statistic since the distribution is not asymptotically N(0,1), a mistake frequently made in past research using the Diebold-Mariano statistic.

2 Data
The data begins in December 1993 and is collected monthly through December 2012 except for gasoline which only has futures data starting in 2006. Prices are taken in a three day window between the 14th and 16th of every month depending on which are trading days. Furthermore, the month used for each horizon depends on the last trading day on the contract. Specifications vary by commodity, and the last trading day for contracts differs widely. Trading for gold and silver ends on the 3rd last business day of the month; corn ends on the business day prior to the 15th of delivery month; natural gas terminates trading three days prior to the first day of delivery month, gasoline on the last day of month before delivery, and crude oil on the 3rd business day prior to the 25th of the month preceding delivery. Table 1. Commodity Contract Details Commodity Corn Crude Oil Gasoline Gold Natural Gas Silver Currency/Units /Bu $/bbl /gal $/Troy oz. $/MMBtu $/Troy oz. Spot Corn No. 2 Yellow WTI Spot Cushing Gasoline RBOB Gold Bullion London Bullion Market Natural Gas-Henry Hub Silver Fix LBM Cash Futures Exchange CBOT NYMEX NYMEX COMEX NYMEX COMEX

The futures price data is collected from Bloomberg using all available contracts for each commodity. The energy and metals commodities have contracts expiring in each of the twelve months while corn only offers five. The spot prices were collected from Datastream over the same period. The future spot prices are taken for the expiration month of the futures contract and the day is chosen to match the horizon.

3 Methods
3.1 Forecasting Methods The model that will be tested for forecasting efficiency of the futures price is a spread regression used to predict the change in spot price. This model will be tested against a random walk benchmark. The future spot price in the random walk model at the h month horizon is simply the current spot price plus the sum of errors from next period through period h. Since the expected value of the future period errors is zero, the random walk model implies a no change forecast and the current spot price is the expected future spot price. (1) t+h | t = St

For the spread regression, least squares is used to regress the log difference of future and current spot price on the log difference of current futures price at horizon h and current spot price. (2) (3) st+h - st = + *(fth st) + t+h t+h | t = fth - st

The expectation is that =1 and =0 (equation 3) under the null hypothesis of forecast efficiency, but the intercept term is allowed to be nonzero in the regression. A key assumption in this least squares construction of the regression is that the policy makers or investors using the forecasts have a quadratic loss function which may not be the case. An absolute loss function does not penalize large deviations from the estimated price disproportionately whereas the least squares method does. Furthermore, the loss function may not even be symmetric as both the quadratic and absolute loss functions are. One example would be policy makers concerned with gas prices rising above a certain level but not concerned with price drops. However, since it is impossible to know the loss function of all commodity price

forecasters, the quadratic loss function is a reasonable assumption. Symmetry of the loss function implies risk neutrality and the quadratic nature is used mainly for convenience.

3.2 Recursive Pseudo Forecasts The two models are used to generate pseudo out-of-sample forecasts to simulate forecasting with the data in real-time. The initial forecasting period of 24 months is large enough to give reliable estimates of the regression parameters and small enough to give reliable estimates of the PMSE with the remaining sample. For each period from May 1996 (the 25th month of data) until November 2012, the spread regression is re-estimated using the least squares method described in section 3.1 on all past data. The regression at each period is used to generate forecasts of the change in spot price at the corresponding horizon, which be compared with the actual future spot price to construct the PMSE for the model. For the random walk, the PMSE is simply obtained by using the spot price and h-month horizon future spot.

3.3 Diebold-Mariano Pseudo Out-of-Sample Test for Predictive Accuracy In order to test for differences in predictive accuracy between the spread regression and random walk model, the Diebold-Mariano test statistic is employed. The original statistic supplied by Diebold and Mariano is:
1 -

DM =

u0t - u1t
1 -

ar

u0t - u1t

~ N(0,1)

The notation is as follows:

and

are the recursive pseudo out-of-sample

forecast errors from the null model and alternative model respectively, T is the total sample size, and R is the initial recursive window. The spread regression is a nested model of the random walk since the models are equi alent when the parameters are zero. herefore, under the null hypothesis of =0, the
4

ariance of the difference in PMSEs will con erge to zero and the statistic will go to infinity. The solution proposed by McCracken uses the variance of the PMSE of the alternative model, which must be non-zero (Clark & McCracken, 1999). The new statistic is therefore:
1 -

DM =

u0t - u1t ar u1t

~F

However, the distribution of test statistic is no longer asymptotically N(0,1) but rather an unknown distribution F that must be bootstrapped.

3.4 Bootstrapping the distribution of the modified Diebold-Mariano statistic Since the distribution F is unknown, nonparametric bootstrapping methods are used. A nave block bootstrap is chosen due to autocorrelation in the prediction errors that can be seen in the autocorrelation plots in the appendix. Nave block bootstraps allow serial correlation to be retained in the resampling by selecting blocks whose length is increased as sample size increases. Furthermore, we do not need to use a blocks of blocks bootstrap because the statistic of interest is symmetric since it involves only a mean and variance. The bootstrap replication is therefore: {y* t =( * 1,
*

t=1

,,

* r)

where s /l

The length of the original sample is T, the number of blocks used to resample is s, and block length is l. Also, y* denotes the difference in squared prediction error between random walk and t spread regression at time t (
0t

- u1t ). For gasoline, l is chosen to be 5 since the sample size is

very short; for all other commodities, 15 is used. To construct the Diebold-Mariano statistic for each of the 20,000 trials, first a simple mean function is used: y * = t=1 yt* Then this mean is divided by the variance of u1t using Newey-West standard errors:

ar u1t Once the vector of 20,000 simulated statistics is constructed, 95% confidence intervals are found by calculating the 2.5 and 97.5 percentiles. The null hypothesis is that there is no difference in forecasting efficiency between the random walk and spread regression. The alternative is that they do not have the same forecasting efficiency. If the calculated modified Diebold-Mariano test statistic is outside of the 95% confidence interval then the null hypothesis is rejected at the 5% level. 3.5 Possible Issues with Proposed Methods It should be noted that there are multiple ambiguities in the above methods. First of all, the confidence interval is sensitive to the choice of block length, and there is no general rule for choosing it. The goal is to preserve any serial correlation in the data, so as sample size increases, bootstrap block length will also increase. However, there is no obvious reason for choosing specific numbers in this case. Another issue is the choice of lag for HAC standard errors. The autocorrelation plots were observed to help with the decision, but these plots are by no means definitive. This is less of a problem since the same standard errors are included in the calculation of the bootstrapped distribution.

4 Results
The results show that the spread regression using the futures price had a lower MSPE in only five of the thirty total cases for six commodities at five different horizons. The only cases where the random walk has a larger MSPE are corn 3- and 6-month horizons as well as gasoline 3-, 6-, and 12-month horizons. None of these five are significant at the 5% level (Diebold Mariano statistics and 95% confidence intervals are included in Appendix 1). However, even though random walk has the lower MSPE in each of the other 25 cases, none of those are statistically significant at the 5% level. Although there are no statistically significant results, the
6

fact that the MSPE is lower for the random walk in the majority of cases across three categories of highly traded commodities shows that the futures price used in the spread regression is not an ideal measure of the expected spot price. Table 2. Mean Square Prediction Errors Commodity Model Corn RW Spread Crude Oil RW Spread Gasoline Gold RW Spread RW Spread Natural Gas RW Spread Silver RW Spread 1-month 1486.5 1500.3 9.1 29.6 541.8 588.5 1694.7 1703.2 .6476 .6584 2.3 2.4 3-month 3112.6 3030.9 80.1 126.2 1162 1157.8 3763.9 3826.7 1.9426 2.2415 5.7 5.8 6-month 7669.5 7643 257.8 288 2210.7 2126.1 8579.3 9049.9 3.5552 4.1123 12.4 12.6 9-month 11187 11594 348.2 360 2761 2825 13705 14984 4.5541 5.4413 19.6 20.2 12-month 12775 13498 377.8 404.7 2969.1 2964 20405 21840 5.0215 5.6826 25.4 25.7

One explanation for this, as noted in Alquist & Kilian (2010), is that the difference between the futures price and spot price is highly variable and may tend to fluctuate around the current spot price. This is found to also be the case here although to a lesser extent. The variability is not so large that it made the random walk a statistically significant more efficient forecaster, although it does. Another problem with the spread regression is that the bias and MAPE are very close to those of the random walk for most horizons and commodities. Since the spread is higher variance than the spot price, it needs to make up for this by being less biased, but that does not appear to be the case here. One notable exception was corn for which the spread was significantly less biased than the spot price. This led to lower mean square prediction errors for two horizons. Furthermore, forecasts for all commodities were biased toward overestimating the future spot price except for corn and the spread regression for natural gas.
7

5 Conclusion
As the results show, using the futures price in a spread regression does not provide a better forecast for future spot price than a simple random walk model that uses a no-change forecast. This has important implications for policy makers and investors using futures markets to forecast or hedge. The assumption of futures prices as an efficient forecasting measure does not seem to hold across any of the three commodity classes. There was no significant evidence that the current spot price is a preferred method although it did have lower MSPE for nearly all commodities and horizons. Further research might explore alternative uses of the futures price in forecasting, particularly for the commodities in which it was continually overestimating the future spot.

References
Alquist, R., & Kilian, L. (2010). What Do We Learn from the Price of Crude Oil Futures? Journal of Applied Econometrics, 25(4), 539-573. Bloomberg L.P. (2013, April 10). Commodity Futures Historical Contracts. Clark, T. E., & McCracken, M. W. (1999). Tests of equal forecast accuracy and encompassing for nested models. Journal of Econometrics, 105(1), 85-110. Datastream International Commodity Spot Price Database (2013, April 10). Datastream International. Diebold, F. X., & Mariano, R. S. (1995). Comparing Predictive Accuracy. Journal of Business & Economic Statistics, 13(3), 253-263. CME Group (2013, April 10). Corn Futures Contract Specifications. Retrieved from CME Group: http://www.cmegroup.com/trading/agricultural/grain-andoilseed/corn_contract_specifications.html CME Group (2013, April 10). Gold Futures Contract Specifications. Retrieved from CME Group: http://www.cmegroup.com/trading/metals/precious/gold_contract_specifications.html CME Group (2013, April 10). Henry Hub Natural Gas Futures Contract Specifications. Retrieved from CME Group: http://www.cmegroup.com/trading/energy/natural-gas/naturalgas_contract_specifications.html CME Group (2013, April 10). Light Sweet Crude Oil (WTI) Futures Contract Specifications. Retrieved from CME Group: http://www.cmegroup.com/trading/energy/crude-oil/light-sweetcrude_contract_specifications.html CME Group (2013, April 10). RBOB Gasoline Futures Contract Specifications. Retrieved from CME Group: http://www.cmegroup.com/trading/energy/refined-products/rbobgasoline_contract_specifications.html CME Group (2013, April 10). Silver Futures Contract Specifications. Retrieved from CME Group: http://www.cmegroup.com/trading/metals/precious/silver_contract_specifications.html Kilian, L. (1999). Exchange Rates and Monetary Fundamentals: What Do We Learn from Long-Horizon Regressions? Journal of Applied Econometrics, 14(5), 491-510.

Appendix 1 Tables

Bias
Commodity Corn Crude Oil Gasoline Gold Natural Gas Silver Model RW Spread RW Spread RW Spread RW Spread RW Spread RW Spread 1-month -3.37 -3.87 0.43 0.43 3.46 6.13 6.14 6.38 0.01 -0.01 0.12 0.09 3-month 3.51 -6.60 1.26 1.03 13.66 21.36 19.56 19.69 0.01 -0.08 0.40 0.35 6-month 20.25 0.40 2.58 2.32 27.99 34.28 38.52 37.54 0.02 -0.08 0.78 0.67 9-month 23.48 2.07 3.61 3.56 35.83 39.21 57.52 55.05 0.02 -0.11 1.13 0.99 12-month 34.50 10.66 4.69 4.92 43.25 45.56 78.64 81.09 0.04 -0.07 1.57 1.58

Mean Absolute Prediction Error Commodity Model Corn RW Spread Crude Oil RW Spread Gasoline RW Spread Gold RW Spread Natural Gas RW Spread Silver RW Spread 1-month 24.57 24.55 3.68 2.02 19.79 18.45 23.35 23.18 0.54 0.54 0.80 0.77 3-month 38.39 38.67 6.72 5.64 24.63 25.64 38.55 38.01 1.03 0.97 1.28 1.27 6-month 59.11 59.18 10.20 9.48 34.20 37.31 63.17 61.24 1.33 1.26 1.97 1.93 9-month 70.35 72.36 11.95 11.62 44.74 44.35 84.16 80.59 1.55 1.48 2.54 2.47 12-month 79.39 82.56 13.62 13.08 46.43 47.62 100.68 97.16 1.62 1.66 2.87 2.82

10

Diebold-Mariano Statistics and Bootstrapped 95% Confidence Intervals

Corn Horizon 1-month 3-month 6-month 9-month 12-month D-M Statistic 4.20 -12.68 -2.13 22.61 32.13 Bootstrapped 95% Confidence Interval [ 0.67, 11.44 ] [ -55.27, 26.86 ] [ -46.50, 90.80 ] [ -39.28, 127.92 ] [ -41.35, 142.20 ]

Crude Oil Horizon 1-month 3-month 6-month 9-month 12-month D-M Statistic 41.78 33.46 12.87 4.60 8.61 Bootstrapped 95% Confidence Interval [ 12.45, 87.21 ] [ 2.13, 87.70 ] [ .36, 30.59 ] [ -.56, 10.99 ] [ -.96, 21.71 ]

Gasoline Horizon 1-month 3-month 6-month 9-month 12-month D-M Statistic 17.26 -0.69 -9.25 10.65 -0.67 Bootstrapped 95% Confidence Interval [ -4.0, 25.10 ] [ -73.82, 43.75 ] [ -77.80, 11.61 ] [ -42,14, 22.90 ] [ -26.82, 19.50 ]

Natural Gas Horizon 1-month 3-month 6-month 9-month 12-month D-M Statistic 0.18 1.83 2.00 2.69 1.64 Bootstrapped 95% Confidence Interval [ -0.06, 0.38 ] [ -0.52, 5.36] [ -1.33, 6.99 ] [ -0.17, 7.26 ] [ -1.13, 5.67 ]

11

Gold Horizon 1-month 3-month 6-month 9-month 12-month D-M Statistic 3.29 9.15 37.71 57.09 49.34 Bootstrapped 95% Confidence Interval [ -.98, 10.50 ] [ -0.10, 29.77 ] [ 7.71, 85.73 ] [ 20.17, 159.45 ] [ 18.26, 147.51 ]

Silver Horizon 1-month 3-month 6-month 9-month 12-month D-M Statistic 1.34 0.17 0.40 0.77 0.36 Bootstrapped 95% Confidence Interval [ -0.08, 2.11 ] [ -0.19, 0.48 ] [ -0.35, 1.38 ] [ -0.23, 2.27] [ -1.22, 1.48 ]

12

Appendix 2 Figures
Autocorrelation Function Plots
Corn
1-Month Horizon
1 0.5 0 -0.5 1 0.5 0 -0.5

3-Month Horizon

10

15

20

25

30

10

15

20

25

30

6-Month Horizon
1 0.5 0 -0.5 1 0.5 0 -0.5

9-Month Horizon

10

15

20

25

30

10

15

20

25

30

12-Month Horizon
1 0.5 0 -0.5

10

15

20

25

30

Crude Oil
1-Month Horizon
1 0 -1 1 0 -1

3-Month Horizon

10

20

30

10

20

30

6-Month Horizon
1 0 -1 1 0 -1

9-Month Horizon

10

20

30

10

20

30

12-Month Horizon
1 0 -1

10

20

30

13

Gasoline

1-Month Horizon
1 0 -1 1 0 -1

3-Month Horizon

10

20

30

10

20

30

6-Month Horizon
1 0 -1 1 0 -1

9-Month Horizon

10

20

30

10

20

30

12-Month Horizon
1 0 -1

10

20

30

Gold

1-Month Horizon
1 0 -1 1 0 -1

3-Month Horizon

10

20

30

10

20

30

6-Month Horizon
1 0 -1 1 0 -1

9-Month Horizon

10

20

30

10

20

30

12-Month Horizon
1 0.5 0

10

20

30

14

Natural Gas

1-Month Horizon
1 0 -1 1 0 -1

3-Month Horizon

10

20

30

10

20

30

6-Month Horizon
1 0 -1 1 0 -1

9-Month Horizon

10

20

30

10

20

30

12-Month Horizon
1 0 -1

10

20

30

Silver

1-Month Horizon
1 0 -1 1 0 -1

3-Month Horizon

10

20

30

10

20

30

6-Month Horizon
1 0 -1 1 0 -1

9-Month Horizon

10

20

30

10

20

30

12-Month Horizon
1 0 -1

10

20

30

15

Futures Prices
Crude Oil Futures Price for all Horizons
140 1-month 3-month 6-month 9-month 12-month

120

100

Price ($/bbl)

80

60

40

20

1994

1996

1998

2000

2002

2004

2006

2008

2010

2012

Date

Gasoline Futures Price for all Horizons

350 1-month 3-month 6-month 9-month 12-month

300

Price (cents/gal)

250

200

150

2007

2007.5

2008

2008.5

2009

2009.5

2010

2010.5

2011

2011.5

2012

Date

16

Gold Futures Price for all Horizons

1800 1-month 3-month 6-month 9-month 12-month

1600

1400

Price ($/Troy oz.)

1200

1000

800

600

400

1994

1996

1998

2000

2002

2004

2006

2008

2010

2012

Year
Silver Futures Price at all Horizons
40 1-month 3-month 6-month 9-month 12-month

35

30

Price ($/Troy oz.)

25

20

15

10

5 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012

Year

17

Natural Gas Futures Price for all Horizons

14 1-month 3-month 6-month 9-month 12-month

12

10

Price ($/MMBtu)

2 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012

Year

Corn Futures Price for all Horizons

1-month 3-month 6-month 9-month 12-month

700

600

Price (cents/bu)

500

400

300

200 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012

Year

18

Spreads
Corn Spread for All Horizons
0.2 0.2 0.1 0.1 0.1 0 -0.1 0 -0.1 -0.2 -0.05 -0.2 1995 2000 2005 2010 1995 2000 2005 2010 -0.3 1995 2000 2005 2010

0.05

1-month
0.3 0.2 0.1 0 0 -0.1 -0.2 -0.1 -0.2 1995 2000 2005 2010 1995 0.4 0.3 0.2 0.1

3-month

6-month

2000

2005

2010

9-month

12-month

Crude Oil Spread for All Horizons

0.2 0.1 0.1 0.1 0 0 0 -0.1 -0.1 -0.1 -0.2 -0.2

-0.2

1995

2000

2005

2010

1995

2000

2005

2010

1995

2000

2005

2010

1-month

3-month

6-month

0.2 0.1 0 -0.1 -0.2 -0.3 1995 2000 2005 2010

0.2 0.1 0 -0.1 -0.2 -0.3 1995 2000 2005 2010

9-month

12-month

19

Gasoline Spread for All Horizons

0.1 0.2 0.2 0 0.1 0 -0.1 -0.1 -0.2 -0.2 -0.3 2007 0.1 0 -0.1 -0.2 -0.3 2007

2008

2009

2010

2011

2012

2007

2008

2009

2010

2011

2012

2008

2009

2010

2011

2012

1-month
0.2 0.1 0.1 0 0 -0.1 -0.1 -0.2 -0.2 -0.3 2007 2008 2009 2010 2011 2012 2007 2008

3-month

6-month

0.2

2009

2010

2011

2012

9-month

12-month

Gold Spread for All Horizons

0.015 0.01 0.005 0 -0.005 -0.01 1995 2000 2005 2010 1995 2000 2005 2010 0.01 0.02 0 0.01 0 1995 2000 2005 2010 0.03 0.06 0.05 0.02 0.04 0.03

1-month
0.2 0.08 0.06 0.05 0.04 0.02 -0.1 0 -0.15 1995 2000 2005 2010 1995 0 -0.05 0.15 0.1

3-month

6-month

2000

2005

2010

9-month

12-month

20

Natural Gas Spread for All Horizons

0.5 0.3 0.2 0.1 0 0 -0.1 -0.2 -0.5 1995 2000 2005 2010 -0.3 1995 2000 2005 2010 -0.4 1995 2000 2005 2010 0 -0.2 0.4 0.2

1-month
0.5 0.4 0.2

3-month

6-month

-0.2

-0.4 -0.5 1995 2000 2005 2010 1995 2000 2005 2010

9-month

12-month

Silver Spread for All Horizons

0.04 0.02 0 -0.02 -0.04 -0.04 -0.06 -0.06 1995 2000 2005 2010 1995 2000 2005 2010 1995 2000 2005 2010 0.06 0.04 0.02 0 -0.02 0.08 0.06 0.04 0.02 0 -0.02 -0.04

1-month

3-month

6-month

0.1

0.4

0.2 0.05 0 0 -0.2

1995

2000

2005

2010

1995

2000

2005

2010

9-month

12-month

21