Time Series - ARIMA Models APS 425 - Advanced Managerial Data

Analysis

APS 425 – Winter 2008

Time Series
Ti S i Analysis:
A l i
ARIMA Models

Instructor: G
G. William Schwert
585-275-2470
schwert@schwert.simon.rochester.edu

Topics

• Typical
i l time
i series
i plot
l

• Pattern recognition in auto and partial

autocorrelations

• Stationarity & invertibility

(c) Prof. G. William Schwert, 2002-2008 1

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

AR(1): Zt = α + φ1 Zt-1 + at
φ1 = .9

1
Note: exponential decay
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
0
A

-0.6
-0.8
-1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Lag k

Auto Partial

AR(1): Zt = α + φ1 Zt-1 + at
φ1 = .5

1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
0
A

-0.6
-0.8
-1

Lag k

Auto Partial

(c) Prof. G. William Schwert, 2002-2008 2

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

AR(1): Zt = α + φ1 Zt-1 + at
φ1 = -.5

1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
0
A

-0.6
-0.8
-1

Lag k

Auto Partial

AR(1): Zt = α + φ1 Zt-1 + at
φ1 = -.9

1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
0
A

-0.6
-0.8
-1 Note: oscillating autocorrelations
Lag k

Auto Partial

(c) Prof. G. William Schwert, 2002-2008 3

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

AR(1): Zt = α + φ1 Zt-1 + at
φ1 = .9
5
4
3
2
1
0
-1
-2
-3
-4
-5
0 10 20 30 40 50 60 70 80 90 100

Note: smooth long swings away from mean

AR(1): Zt = α + φ1 Zt-1 + at
φ1 = -.9
5

-1
1

-2

-3
0 10 20 30 40 50 60 70 80 90 100

Note: jagged, frequent swings around mean

(c) Prof. G. William Schwert, 2002-2008 4

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

AR(2): Zt = α + φ1 Zt-1 + φ2 Zt-2 + at

φ1 = 1.4 φ2 = .45

1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
0
A

-0.6
-0.8
-1

Lag k

Auto Partial

AR(2): Zt = α + φ1 Zt-1 + φ2 Zt-2 + at

φ1 = .4 φ2 = .45

1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
0
A

-0.6
-0.8
-1

Lag k

Auto Partial

(c) Prof. G. William Schwert, 2002-2008 5

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

AR(2): Zt = α + φ1 Zt-1 + φ2 Zt-2 + at

φ1 = .7 φ2 = -.2

1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
0
A

-0.6
-0.8
-1

Lag k

Auto Partial

AR(2): Zt = α + φ1 Zt-1 + φ2 Zt-2 + at

φ1 = -.7 φ2 = .2

1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
0
A

-0.6
-0.8
-1

Lag k

Auto Partial

(c) Prof. G. William Schwert, 2002-2008 6

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

AR(2): Zt = α + φ1 Zt-1 + φ2 Zt-2 + at

φ1 = 1.4 φ2 = .45
6

-2

-4

-6
6

-8

-10
0 10 20 30 40 50 60 70 80 90 100

Note: smooth long swings away from mean

AR(2): Zt = α + φ1 Zt-1 + φ2 Zt-2 + at

φ1 = -.7 φ2 = .2
5
Note: jagged, frequent swings around mean
4

-1

-2
0 10 20 30 40 50 60 70 80 90 100

(c) Prof. G. William Schwert, 2002-2008 7

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

Autoregressive Models:
Summary

1) Autocorrelations decay or oscillate

2) Partial Autocorrelations cut-off after lag p, for

AR(p) model

3) Stationarity is a big issue

• very slow decay in autocorrelations
• should you difference?

MA(1): Zt = α + at - θ1 at-1
θ1 = -.9

1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
0
A

-0.6 Note: autocorrelations cut off,

-0.8 pacf oscillates
-1

Lag k

Auto Partial

(c) Prof. G. William Schwert, 2002-2008 8

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

MA(1): Zt = α + at - θ1 at-1
θ1 = -.5

1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
0
A

-0.6
-0.8
-1

Lag k

Auto Partial

MA(1): Zt = α + at - θ1 at-1
θ1 = .5

1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
0
A

-0.6
-0.8
-1

Lag k

Auto Partial

(c) Prof. G. William Schwert, 2002-2008 9

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

MA(1): Zt = α + at - θ1 at-1
θ1 = .9

1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
0
A

-0.6
-0.8
-1

Lag k

Auto Partial

MA(1): Zt = α + at - θ1 at-1
θ1 = -.9
5

-1
1

-2

-3
0 10 20 30 40 50 60 70 80 90 100

Note: smooth long swings away from mean

(c) Prof. G. William Schwert, 2002-2008 10

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

MA(1): Zt = α + at - θ1 at-1
θ1 = .9
5

-1
1

-2

-3
0 10 20 30 40 50 60 70 80 90 100

Note: jagged, frequent swings around the mean

Moving Average Models: Summary

1) Autocorrelations cut off after lag q for MA(q)

model

2) Partial autocorrelations decay or oscillate

(c) Prof. G. William Schwert, 2002-2008 11

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

Autoregressive Moving Average Models

ARMA(p,q): Zt = α + φ1 Zt-1 + . . . + φp Zt-p +

at − θ1 at-1 − . . . − θq at-q
Combines both AR & MA characteristics:

• equivalent
q to infinite order MA process
p
• if stationary

• equivalent to infinite order AR process

• if “invertible”

ARMA(1,1): Zt = α + φ1 Zt-1 + at - θ1 at-1

φ1 = .9, θ1 = .5
1 Note: exponential decay, starting after lag 1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
-0.6
-0.8
-1

Lag k

Auto Partial

(c) Prof. G. William Schwert, 2002-2008 12

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

ARMA(1,1): Zt = α + φ1 Zt-1 + at - θ1 at-1

φ1 = .5, θ1 = .9
1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
-0.6
-0.8
-1

Lag k

Auto Partial

ARMA(1,1): Zt = α + φ1 Zt-1 + at - θ1 at-1

φ1 = -.5, θ1 = .5
1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
-0.6
-0.8
-1

Lag k

Auto Partial

(c) Prof. G. William Schwert, 2002-2008 13

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

ARMA(1,1): Zt = α + φ1 Zt-1 + at - θ1 at-1

φ1 = -.9, θ1 = .9
1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
-0.6
-0.8
-1

Lag k

Auto Partial

ARMA(1,1): Zt = α + φ1 Zt-1 + at - θ1 at-1

φ1 = .9, θ1 = .5
6

-2

-4

-6

-8
0 10 20 30 40 50 60 70 80 90 100

Note: smooth long swings around the mean

(c) Prof. G. William Schwert, 2002-2008 14

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

ARMA(1,1): Zt = α + φ1 Zt-1 + at - θ1 at-1

φ1 = -.9, θ1 = .9
3

2.5

1.5

0.5

-0.5

-1
0 10 20 30 40 50 60 70 80 90 100

Note: jagged, frequent swings around the mean

Autoregressive Moving Average

Models: Summary

1) Autocorrelations decay or oscillate

2) Partial Autocorrelations decay or oscillate

(c) Prof. G. William Schwert, 2002-2008 15

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

Autoregressive Integrated Moving

Average ARIMA(p,d,q) Models

1) ARMA model in the dth differences of the data

2) First step is to find the level of differencing

necessary

3) Next steps are to find the appropriate ARMA model

for the differenced data

4) Need to avoid “overdifferencing”

ARIMA(0,1,1):
(Zt - Zt-1) = at - .8 at-1 [T=150]
1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
-0.6
-0.8 Note: autocorrelations decay very slowly
-1 (from moderate level); pacf decays at rate .8

Lag k

Auto Partial

(c) Prof. G. William Schwert, 2002-2008 16

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

ARIMA(0,1,1):
(Zt - Zt-1) = at - .5 at-1 [T=150]
1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
-0.6
-0.8 Note: autocorrelations decay very slowly
-1 (from moderate level); pacf decays at rate .5

Lag k

Auto Partial

ARIMA(0,1,1):
(Zt - Zt-1) = at + .5 at-1 [T=150]
1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
-0.6
-0.8 Note: autocorrelations decay very slowly
-1 (from moderate level); pacf decays at rate -.5

Lag k

Auto Partial

(c) Prof. G. William Schwert, 2002-2008 17

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

ARIMA(0,1,1):
(Zt - Zt-1) = at + .8 at-1 [T=150]
1
0.8
0.6
0.4
Autocorrelation

0.2
0
-0.2
-0.4
-0.6
-0.8
Note: autocorrelations decay very slowly
-1
(from moderate level); pacf decays at rate -.8
Lag k

Auto Partial

ARIMA(0,1,1)
(Zt - Zt-1) = at - .8 at-1
2

-1

-2

-3

-4 Note: slowly wandering level of series, with lots

of variation around that level
-5
0 10 20 30 40 50 60 70 80 90 100

(c) Prof. G. William Schwert, 2002-2008 18

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

ARIMA(0,1,1)
(Zt - Zt-1) = at + .8 at-1
20

15

10

-5 Note: slowly wandering level of series, with

smooth variation around that level
-10
0 10 20 30 40 50 60 70 80 90 100

Integrated Moving Average Models:

Summary

1) Autocorrelations decay slowly

• initial level is determined by how close MA parameter
is to one

2) Partial Autocorrelations decay or oscillate

• determined by MA parameter

(c) Prof. G. William Schwert, 2002-2008 19

Time Series - ARIMA Models APS 425 - Advanced Managerial Data
Analysis

Links

Return to APS 425 Home Page:

http://schwert.simon.rochester.edu/A425/A425main.htm

(c) Prof. G. William Schwert, 2002-2008 20