Vous êtes sur la page 1sur 25

Lecture Notes of Bus 41202 (Spring 2011)

Ruey S. Tsay

Simple AR models: (Regression with lagged variables.)

Motivating example: The growth rate of U.S. quarterly real
GNP from 1947 to 1991. Recall that the model discussed before is

This is called an AR(3) model because the growth rate rt depends

on the growth rates of the past three quarters. How do we specify
this model from the data? Is it adequate for the data? What are the
implications of the model? These are the questions we shall address
in this lecture.

AR(1) model:

1. Form: rt = 0 + 1rt1 + at, where 0 and 1 are real numbers,

which are referred to as parameters (to be estimated from the
data in an application). For example,

rt = 0.005 + 0.2rt1 + at

2. Stationarity: necessary and sufficient condition |1| < 1. Why?

0
3. Mean: E(rt) = 11

1
U.S. quarterly real GNP growth rate: 1947.II to 1991.I

0.04
0.03
0.02
0.01
gnp

0.00
0.01
0.02

Time

4. Alternative representation: Let E(rt) = be the mean of rt so

that = 0/(1 1). Equivalently, 0 = (1 1). Plugging
in the model, we have

(rt ) = 1(rt1 ) + at. (1)

This model also has two parameters ( and 1). It explicitly uses
the mean of the series. It is less commonly used in the literature,
but is the model representation used in R.
a2
5. Variance: Var(rt) = 121
.

6. Autocorrelations: 1 = 1, 2 = 21, etc. In general, k = k1

and ACF k decays exponentially as k increases,

2
10
8
unrate

6
4

1950 1960 1970 1980 1990 2000 2010

Time

Figure 2: U.S. monthly unemployment rate (total civilian, 16 and older) from January 1948
to February 2011

3
7. Forecast (minimum squared error): Suppose the forecast origin
is n. For simplicity, we shall use the model representation in (1)
and write xt = rt . The model then becomes xt = 1xt1 +at.
Note that forecast of rt is simply the forecast of xt plus .

(a) 1-step ahead forecast at time n:

xn(1) = 1xn
(b) 1-step ahead forecast error:
en(1) = xn+1 xn(1) = an+1
Thus, an+1 is the un-predictable part of xn+1. It is the shock
at time n + 1!
(c) Variance of 1-step ahead forecast error:
Var[en(1)] = Var(an+1) = a2.
(d) 2-step ahead forecast:
xn(2) = 1xn(1) = 21xn.
(e) 2-step ahead forecast error:
en(2) = xn+2 xn(2) = an+2 + 1an+1
(f) Variance of 2-step ahead forecast error:
Var[en(2)] = (1 + 21)a2
which is greater than or equal to Var[en(1)], implying that
uncertainty in forecasts increases as the number of steps in-
creases.
4
(g) Behavior of multi-step ahead forecasts. In general, for the
`-step ahead forecast at n, we have

xn(`) = `1xn,

and the variance of forecast error

2(`1)
Var[en(`)] = (1 + 21 + + 1 )a2.

In particular, as ` ,

This is called the mean-reversion of the AR(1) process. The

variance of forecast error approaches
1 2
Var[en(`)] = a = Var(rt).
1 21
In practice, it means that for the long-term forecasts serial
dependence is not important. The forecast is just the sample
mean and the uncertainty is simply the uncertainty about the
series.

Half-life: A common way to quantify the speed of mean reversion

is the half-life, which is defined as the number of periods needed so

5
that the magnitude of the forecast becomes half of that of the forecast
origin. For an AR(1) model, this mean
1
xn(k) = xn.
2
Thus, k1 xn = 12 xn. Consequently, the half-life of the AR(1) model
ln(0.5)
is k = ln(|1 |) . For example, if 1 = 0.5, the k = 1. If 1 = 0.9, then
k 6.58.
AR(2) model:

1. Form: rt = 0 + 1rt1 + 2rt2 + at, or

(1 1B 2B 2)rt = 0 + at.

3. Characteristic equation: (1 1x 2x2) = 0

0
4. Mean: E(rt) = 11 2
1
5. ACF: 0 = 1, 1 = 12 ,

` = 1`1 + 2`1, ` 2.

6. Stochastic business cycle: if 21 + 42 < 0, then rt shows char-

acteristics of business cycles with average length
2
k= ,
cos1[1/(2 2)]
where the cosine inverse is stated in radian. If we denote the

solutions of the polynomial as a bi, where i = 1, then we
6
have 1 = 2a and 2 = (a2 + b2) so that
2
k= .
cos1(a/ a2 + b2)

In R or S-Plus, one can obtain a2 + b2 using the command
Mod.

Simulation in R: Use the command arima.sim

1. y1=arima.sim(model=list(ar=c(1.3,-.4)),1000)

2. y2=arima.sim(model=list(ar=c(.8,-.7)),1000)

Check the ACF and PACF of the above two simulated series.

Building an AR model

Order specification

1. Partial ACF: (naive, but effective)

Use consecutive fittings
See Text (p. 40) for details
Key feature: PACF cuts off at lag p for an AR(p)
model.
Illustration: See the PACF of the U.S. quarterly growth
rate of GNP.

7
2. Akaike information criterion
2`
AIC(`) = ln(`2) + ,
T
for an AR(`) model, where `2 is the MLE of residual vari-
ance.
Find the AR order with minimum AIC for ` [0, , P ].
3. BIC criterion:
` ln(T )
BIC(`) = ln(`2) + .
T
Needs a constant term? Check the sample mean.

Estimation: least squares method or maximum likelihood method

Model checking:

1. Residual: obs minus the fit, i.e. 1-step ahead forecast errors
at each time point.
2. Residual should be close to white noise if the model is ade-
quate. Use Ljung-Box statistics of residuals, but degrees of
freedom is m g, where g is the number of AR coefficients
used in the model.

Example: Analysis of U.S. GNP growth rate series.

R demonstration:
> setwd("C:/Users/rst/teaching/bs41202/sp2011")
> library(fBasics)
> x=da[,1]
> par(mfcol=c(2,2)) % put 4 plots on a page

8
Series : dgnp
0.3 0.2
Partial ACF
0.1 0.0
0.1

0 5 10 15 20
Lag

> plot(x,type=l) % first plot

> plot(x[1:175],x[2:176]) % 2nd plot
> plot(x[1:174],x[3:176]) % 3rd plot
> acf(x,lag=12) % 4th plot

> pacf(x,lag.max=12) % Compute PACF (not shown in this handout)

> Box.test(x,lag=10,type=Ljung) % Compute Q(10) statistics

Box-Ljung test

data: x
X-squared = 43.2345, df = 10, p-value = 4.515e-06

> m1=ar(x,method=mle) % Automatic AR fitting using AIC criterion.

> m1

Call:
ar(x = x, method = "mle")

Coefficients:
1 2 3 % An AR(3) is specified.
0.3480 0.1793 -0.1423

9
Order selected 3 sigma^2 estimated as 9.427e-05

> names(m1)
 "order" "ar" "var.pred" "x.mean" "aic"
 "n.used" "order.max" "partialacf" "resid" "method"
 "series" "frequency" "call" "asy.var.coef"

> Box.test(m1\$resid,lag=10,type=Ljung) % Model checking

Box-Ljung test

data: m1\$resid
X-squared = 7.0808, df = 10, p-value = 0.7178

> m2=arima(x,order=c(3,0,0)) % Another approach with order given.

> m2
Call:
arima(x = x, order = c(3, 0, 0))

Coefficients:
ar1 ar2 ar3 intercept % Fitted model is
0.3480 0.1793 -0.1423 0.0077 % y(t)=0.348y(t-1)+0.179y(t-2)
s.e. 0.0745 0.0778 0.0745 0.0012 % -0.142y(t-3)+a(t),
% where y(t) = x(t)-0.0077

sigma^2 estimated as 9.427e-05: log likelihood = 565.84, aic = -1121.68

> names(m2)
 "coef" "sigma2" "var.coef" "mask" "loglik" "aic"
 "arma" "residuals" "call" "series" "code" "n.cond"
 "model"

> Box.test(m2\$residuals,lag=10,type=Ljung)

Box-Ljung test

data: m2\$residuals
X-squared = 7.0169, df = 10, p-value = 0.7239

> tsdiag(m2) % obtain 3 plots of model checking (not shown in handout).

>
> p1=c(1,-m2\$coef[1:3]) % Further analysis of the fitted model.

10
> roots=polyroot(p1)
> roots
 1.590253+1.063882e+00i -1.920152-3.530887e-17i 1.590253-1.063882e+00i
> Mod(roots)
 1.913308 1.920152 1.913308

> k=2*pi/acos(1.590253/1.913308)
> k
 10.65638

> predict(m2,8) % Prediction 1-step to 8-step ahead.

\$pred
Time Series:
Start = 177
End = 184
Frequency = 1
 0.001236254 0.004555519 0.007454906 0.007958518
 0.008181442 0.007936845 0.007820046 0.007703826

\$se
Time Series:
Start = 177
End = 184
Frequency = 1
 0.009709322 0.010280510 0.010686305 0.010688994
 0.010689733 0.010694771 0.010695511 0.010696190

Another example: Monthly U.S. unemployment rate from Jan-

uary 1948 to February 2011.
Demonstration: in class, including the R scripts fore, foreplot,
and backtest.
> setwd("C:/Users/rst/teaching/bs41202/sp2011")
> rate=da\$rate
> unrate=ts(rate,frequency=12,start=c(1948,1))
> plot(unrate)
year mon day rate
1 1948 1 1 3.4
.....
6 1948 6 1 3.6
> dim(da)

11
 758 4
> acf(rate)
> acf(diff(rate))
> par(mfcol=c(2,1))
> acf(rate)
> acf(diff(rate))
> m1=ar(diff(rate),method="mle")
> m1\$order
 12
> m1

Call:
ar(x = diff(rate), method = "mle")

Coefficients:
1 2 3 4 5 6 7 8
0.0172 0.2229 0.1523 0.0943 0.1309 -0.0022 -0.0287 0.0114
9 10 11 12
-0.0080 -0.1072 0.0300 -0.1121

Order selected 12 sigma^2 estimated as 0.03865

> t.test(diff(rate))
One Sample t-test

data: diff(rate)
t = 0.9252, df = 756, p-value = 0.3551
alternative hypothesis: true mean is not equal to 0
95 percent confidence interval:
-0.008150105 0.022681148
sample estimates:
mean of x
0.007265522

> m1=arima(rate,order=c(12,1,0))
> m1
Call:
arima(x = rate, order = c(12, 1, 0))

Coefficients:
ar1 ar2 ar3 ar4 ar5 ar6 ar7 ar8 ar9
0.0176 0.2233 0.1526 0.0946 0.1311 -0.0021 -0.0286 0.0115 -0.0079
s.e. 0.0362 0.0364 0.0372 0.0378 0.0380 0.0382 0.0383 0.0380 0.0379
ar10 ar11 ar12
-0.1069 0.0304 -0.1117
s.e. 0.0374 0.0366 0.0366

12
sigma^2 estimated as 0.03867: log likelihood = 156.67, aic = -287.34
> tsdiag(m1,gof=24)
> m2=arima(y,order=c(2,1,1),seasonal=list(order=c(1,0,1),period=12))
Error in NCOL(x) : object y not found
> m2=arima(rate,order=c(2,1,1),seasonal=list(order=c(1,0,1),period=12))
> m2
Call:
arima(x=rate,order=c(2,1,1),seasonal=list(order=c(1,0,1),period=12))

Coefficients:
ar1 ar2 ma1 sar1 sma1
0.5868 0.2385 -0.5826 0.5717 -0.8269
s.e. 0.0628 0.0395 0.0582 0.0708 0.0512

sigma^2 estimated as 0.03721: log likelihood = 169.88, aic = -327.76

> tsdiag(m2,gof=24)
###### use forecast origin at t=750.
> source("fore.R")
> p2=fore(m2,rate,750,8)
Time Series:
Start = 751
End = 758
Frequency = 1
 9.472048 9.429974 9.405080 9.274666 9.290746 9.167263 9.136058 9.091824
Time Series:
Start = 751
End = 758
Frequency = 1
 0.1926610 0.2723187 0.3626496 0.4502437 0.5378878 0.6243315 0.7094266
 0.7928668
#### Forecating plot
> source("foreplot.R")
> foreplot(p2,rate,750,730)
> foreplot(p2,rate,750,700)
#### Backtesting
> source("backtest.R")
> pm1=backtest(m1,rate,730,1)
 "RMSE of out-of-sample forecasts"
 0.2119780
 "Mean absolute error of out-of-sample forecasts"
 0.1651847
> pm2=backtest(m2,rate,730,1)
 "RMSE of out-of-sample forecasts"
 0.1948283
 "Mean absolute error of out-of-sample forecasts"
 0.1445230

13
Moving-average (MA) model
Model with finite memory!
Some daily stock returns have minor serial correlations and can be
modeled as MA or AR models.
MA(1) model

Form: rt = + at at1

Autocovariance:

1. Lag 1: Cov(rt, rt1) = a2

2. Lag `: Cov(rt, rt`) = 0 for ` > 1.

Thus, rt is not related to rt2, rt3, .

ACF: 1 = 1+2
, ` = 0 for ` > 1.
Finite memory! MA(1) models do not remember what happen
two time periods ago.

1. 1-step ahead: rn(1) = an. Why? Because at time n,

an is known, but an+1 is not.
2. 1-step ahead forecast error: en(1) = an+1 with variance a2.

14
3. Multi-step ahead: rn(`) = for ` > 1.
Thus, for an MA(1) model, the multi-step ahead forecasts
are just the mean of the series. Why? Because the model
has memory of 1 time period.
4. Multi-step ahead forecast error:
en(`) = an+` an+`1

5. Variance of multi-step ahead forecast error:

(1 + 2)a2 = variance of rt.

Invertibility:

Concept: rt is a proper linear combination of at and the past

observations {rt1, rt2, }.
Why is it important? It provides a simple way to obtain the
shock at.
For an invertible model, the dependence of rt on rt` con-
verges to zero as ` increases.
Condition: || < 1.
Invertibility of MA models is the dual property of stationarity
for AR models.

MA(2) model
Form: rt = + at 1at1 2at2. or
rt = + (1 1B 2B 2)at.
15
Stationary with E(rt) = .

Forecasts go the the mean after 2 periods.

Building an MA model
Specification: Use sample ACF
Sample ACFs are all small after lag q for an MA(q) series. (See
test of ACF.)

Conditional: Assume at = 0 for t 0

Exact: Treat at with t 0 as parameters, estimate them to
obtain the likelihood function.

Forecast: use the residuals as {at} (which can be obtained from

the data and fitted parameters) to perform forecasts.

Model form in R: R parameterizes the MA(q) model as

rt = + at + 1at1 + + q atq ,
16
instead of the usual minus sign in . Consequently, care needs to be
exercised in writing down a fitted MA parameter in R. For instance,
an estimate 1 = 0.5 of an MA(1) in R indicates the model is
rt = at 0.5at1.
Example:Daily log return of the value-weighted index
R demonstration
> setwd("C:/Users/rst/teaching/bs41202/sp2011")
> library(fBasics)
> dim(da)
 10194 4

> vw=log(1+da[,3])*100 % Compute percentage log returns of the vw index.

> acf(vw,lag.max=10) % ACF plot is not shon in this handout.
> m1=arima(vw,order=c(0,0,1)) % fits an MA(1) model
> m1

Call:
arima(x = vw, order = c(0, 0, 1))

Coefficients:
ma1 intercept
0.1465 0.0396 % The model is vw(t) = 0.0396+a(t)+0.1465a(t-1).
s.e. 0.0099 0.0100

sigma^2 estimated as 0.7785: log likelihood = -13188.48, aic = 26382.96

> tsdiag(m1)
> predict(m1,5)
\$pred
Time Series:
Start = 10195
End = 10199
Frequency = 1
 0.05036298 0.03960887 0.03960887 0.03960887 0.03960887

\$se
Time Series:
Start = 10195
End = 10199
Frequency = 1
 0.8823290 0.8917523 0.8917523 0.8917523 0.8917523

17
Mixed ARMA model: A compact form for flexible models.
Focus on the ARMA(1,1) model for
1. simplicity

2. useful for understanding GARCH models in Ch. 3 for volatility

modeling.
ARMA(1,1) model
Form: (1 1B)rt = 0 + (1 B)at or

rt = 1rt1 + 0 + at 1at1.

RHS.

Invertibility: same as MA(1)

0
Mean: as AR(1), i.e. E(rt) = 11

ACF: Satisfies k = 1k1 for k > 1, but

1 = 1 [1a2/Var(rt)] 6= 1.

Building an ARMA(1,1) model

18
Specification: use EACF or AIC

Forecast: MA(1) affects the 1-step ahead forecast. Others are

similar to those of AR(1) models.

AR representation: (by long division)

rt = 0 + at + 1rt1 + 2rt2 +

MA representation: (by long division)

rt = + at + 1at1 + 2at2 +

For a stationary series, i converges to zero as i . Thus, the

effect of any shock is transitory.
The MA representation is particularly useful in computing variances
of forecast errors.

19
For a `-step ahead forecast, the forecast error is

The variance of forecast error is

Var[en(`)] = (1 + 12 + + `1
2
)a2.

Unit-root Nonstationarity
Random walk

Form pt = pt1 + at

Nonstationary: Why? Because the variance of rt diverges to

infinity as t increases.

pt = ati = iati
X X

i=0 i=0

where i = 1 for all i. Thus, i does not converge to zero. The

effect of any shock is permanent.

20
Nonstationary

Strong memory

differencing

1st difference: rt = pt pt1

If pt is the log price, then the 1st difference is simply the log
return. Typically, 1st difference means the change or incre-
ment of the original series.

Seasonal difference: yt = pt pts, where s is the periodicity,

e.g. s = 4 for quarterly series and s = 12 for monthly series.
If pt denotes quarterly earnings, then yt is the change in earning
from the same quarter one year before.

MA model: mean

Practical implication in financial time series

Example: Monthly log returns of General Electrics (GE) from 1926
to 1999 (74 years)
Sample mean: 1.04%, std() = 0.26
21
Very significant!
is about 12.45% a year
\$1 investment in the beginning of 1926 is worth
annual compounded payment: \$5907

monthly compounded payment: \$9570

Continuously compounded?

Unit-root test
Let pt be the log price of an asset. To test that pt is not predictable
(i.e. has a unit root), two models are commonly employed:

pt = 1pt1 + et
pt = 0 + 1pt1 + et.

The hypothesis of interest is Ho : 1 = 1 vs Ha : 1 < 1.

Dickey-Fuller test is the usual t-ratio of the OLS estimate of 1 being
1. This is the DF unit-root test. The t-ratio, however, has a non-
standard limiting distribution.
Let pt = pt pt1. Then, the augmented DF unit-root test for an
AR(p) model is based on
p1
pt = ct + pt1 + ipti + et.
X

i=1
The t-ratio of the OLS estimate of is the ADF unit-root test statis-
tic. Again, the statistic has a non-standard limiting distribution.
22
Example: Consider the log series of U.S. quaterly real GDP se-
ries from 1947.I to 2009.IV. (data from Federal Reserve Bank of St.
Louis). See q-gdpc96.txt on the course web.
R demonstration
> library(fUnitRoots)
> help(UnitrootTests) % See the tests available
>gdp=log(da[,4])

> adfTest(gdp,lag=4,type=c("c")) #Assume an AR(4) model for the series.

Title:
Augmented Dickey-Fuller Test

Test Results:
PARAMETER:
Lag Order: 4
STATISTIC:
Dickey-Fuller: -1.7433
P VALUE:
0.4076 # cannot reject the null hypothesis of a unit root.

*** A more careful analysis

> x=diff(gdp)
> ord=ar(x) # identify an AR model for the differenced series.
> ord

Call:
ar(x = x)

Coefficients:
1 2 3
0.3429 0.1238 -0.1226

Order selected 3 sigma^2 estimated as 8.522e-05

> # An AR(3) for the differenced data is confirmed.
# Our previous analysis is justified.

1. Dealing with the constant term. If there is any differencing, no

23
constant is used.
The subcommand include.mean=T in the arima command.

2. Fixing some parameters. Use subcommand fixed in arima.

Use unemployment rate series as an example.

R Demonstration: Handling outliers

> r1=residuals(m1)
> idx=c(1:length(rate))[r1==min(r1)] <=== locate the minimum residuals
> idx
 23
> idx=c(1:length(rate))[r1==max(r1)] <== locate the maximum residuals
> idx
 22
> outlier23=rep(0,length(rate)) <== Create indicator variable
> outlier23=1
> m3=arima(rate,order=c(12,1,0),xreg=data.frame(outlier23))
> m3
Call:
arima(x = rate, order = c(12, 1, 0), xreg = data.frame(outlier23))

Coefficients:
ar1 ar2 ar3 ar4 ar5 ar6 ar7 ar8 ar9
0.0716 0.1754 0.1455 0.1152 0.1152 0.0042 -0.0264 0.0143 -0.0132
s.e. 0.0366 0.0374 0.0367 0.0375 0.0377 0.0379 0.0378 0.0376 0.0374
ar10 ar11 ar12 outlier23
-0.1099 0.0474 -0.1343 -0.810
s.e. 0.0369 0.0365 0.0366 0.135

sigma^2 estimated as 0.03696: log likelihood = 173.77, aic = -319.54

> tsdiag(m3,gof=36)
> outlier22=rep(0,length(rate))
> outlier22=1
> m3=arima(rate,order=c(12,1,0),xreg=data.frame(outlier22,outlier23))
> m3

Call:
arima(x = rate, order = c(12, 1, 0), xreg = data.frame(outlier22, outlier23))

Coefficients:
ar1 ar2 ar3 ar4 ar5 ar6 ar7 ar8 ar9
0.1037 0.2033 0.1238 0.1043 0.0763 0.0308 -0.0179 -0.0045 -0.0141

24
s.e. 0.0362 0.0365 0.0372 0.0378 0.0383 0.0382 0.0380 0.0380 0.0377
ar10 ar11 ar12 outlier22 outlier23
-0.0939 0.0631 -0.1351 1.1735 -0.2840
s.e. 0.0374 0.0366 0.0366 0.1388 0.1374

sigma^2 estimated as 0.03379: log likelihood = 207.67, aic = -385.34

> tsdiag(m3,gof=36)
> m2=arima(rate,order=c(2,1,1),seasonal=list(order=c(1,0,1),period=12),
xreg=data.frame(outlier22))
> m2
Call:
arima(x = rate, order = c(2, 1, 1), seasonal = list(order = c(1, 0, 1),
period = 12), xreg = data.frame(outlier22))

Coefficients:
ar1 ar2 ma1 sar1 sma1 outlier22
0.6871 0.1524 -0.5902 0.5457 -0.8152 1.2994
s.e. 0.0700 0.0443 0.0637 0.0724 0.0518 0.1175

sigma^2 estimated as 0.03225: log likelihood = 224, aic = -434.01

> tsdiag(m2,gof=36)
> m2=arima(rate,order=c(2,1,2),seasonal=list(order=c(1,0,1),period=12),
xreg=data.frame(outlier22))
> m2
Call:
arima(x = rate, order = c(2, 1, 2), seasonal = list(order = c(1, 0, 1),
period = 12), xreg = data.frame(outlier22))

Coefficients:
ar1 ar2 ma1 ma2 sar1 sma1 outlier22
1.5913 -0.6824 -1.5017 0.6946 0.5702 -0.8289 1.3103
s.e. 0.1058 0.1033 0.1035 0.0886 0.0700 0.0483 0.1171

sigma^2 estimated as 0.03188: log likelihood = 228.1, aic = -440.2

> tsdiag(m2,gof=36)

25