Introduction to Time Series Analysis. Lecture 1.

Peter Bartlett 1. Organizational issues. 2. Objectives of time series analysis. Examples. 3. Overview of the course. 4. Time series models. 5. Time series modelling: Chasing stationarity.

Organizational Issues
Peter Bartlett. bartlett@stat. Ofce hours: Tue 11-12, Thu 10-11 (Evans 399). Joe Neeman. jneeman@stat. Ofce hours: Wed 1:302:30, Fri 2-3 (Evans ???). http://www.stat.berkeley.edu/bartlett/courses/153-fall2010/ Check it for announcements, assignments, slides, ... Text: Time Series Analysis and its Applications. With R Examples, Shumway and Stoffer. 2nd Edition. 2006.

Organizational Issues
Classroom and Computer Lab Section: Friday 911, in 344 Evans. Starting tomorrow, August 27: Sign up for computer accounts. Introduction to R. Assessment: Lab/Homework Assignments (25%): posted on the website. These involve a mix of pen-and-paper and computer exercises. You may use any programming language you choose (R, Splus, Matlab, python). Midterm Exams (30%): scheduled for October 7 and November 9, at the lecture. Project (10%): Analysis of a data set that you choose. Final Exam (35%): scheduled for Friday, December 17.

A Time Series
400

350

300

250

200

150

100

50

1000

2000

3000

4000

5000

6000

7000

A Time Series
400

350

300

250

200

150

100

50

0 1960

1965

1970

1975 year

1980

1985

1990

A Time Series
400

350

300

250

200

150

100

50

0 1960

1965

1970

1975 year

1980

1985

1990

A Time Series
SP500: 19601990 400

350

300

250

200

150

100

50

0 1960

1965

1970

1975 year

1980

1985

1990

A Time Series
SP500: JanJun 1987 340

320

300

280

260

240

220 1987

1987.05

1987.1

1987.15

1987.2

1987.25 year

1987.3

1987.35

1987.4

1987.45

1987.5

A Time Series
SP500 JanJun 1987. Histogram 30

25

20

15

10

0 240

250

260

270 $

280

290

300

310

A Time Series
SP500: JanJun 1987. Permuted. 340

320

300

280

260

240

220

20

40

60

80

100

120

10

Objectives of Time Series Analysis

1. Compact description of data. 2. Interpretation. 3. Forecasting. 4. Control. 5. Hypothesis testing. 6. Simulation.

11

Classical decomposition: An example

Monthly sales for a souvenir shop at a beach resort town in Queensland.
(Makridakis, Wheelwright and Hyndman, 1998)
x 10
4

12

10

10

20

30

40

50

60

70

80

90

12

Transformed data
12

11.5

11

10.5

10

9.5

8.5

7.5

10

20

30

40

50

60

70

80

90

13

Trend
12

11.5

11

10.5

10

9.5

8.5

7.5

10

20

30

40

50

60

70

80

90

14

Residuals
1.5

0.5

0.5

10

20

30

40

50

60

70

80

90

15

Trend and seasonal variation

12

11.5

11

10.5

10

9.5

8.5

7.5

10

20

30

40

50

60

70

80

90

16

Objectives of Time Series Analysis

1. Compact description of data. Example: Classical decomposition: 2. Interpretation. 3. Forecasting. 4. Control. 5. Hypothesis testing. 6. Simulation.

Xt = Tt + St + Yt . Example: Seasonal adjustment. Example: Predict sales.

17

Unemployment data
Monthly number of unemployed people in Australia.
8 x 10
5

(Hipel and McLeod, 1994)

7.5

6.5

5.5

4.5

4 1983

1984

1985

1986

1987

1988

1989

1990

18

Trend
8 x 10
5

7.5

6.5

5.5

4.5

4 1983

1984

1985

1986

1987

1988

1989

1990

19

Trend plus seasonal variation

8 x 10
5

7.5

6.5

5.5

4.5

4 1983

1984

1985

1986

1987

1988

1989

1990

20

Residuals
8 x 10
4

6 1983

1984

1985

1986

1987

1988

1989

1990

21

Predictions based on a (simulated) variable

8 x 10
5

7.5

6.5

5.5

4.5

4 1983

1984

1985

1986

1987

1988

1989

1990

22

Objectives of Time Series Analysis

1. Compact description of data: Xt = Tt + St + f (Yt ) + Wt . 2. Interpretation. 3. Forecasting. 4. Control. Example: Seasonal adjustment. Example: Predict unemployment. Example: Impact of monetary policy on unemployment. Example: Global warming.

5. Hypothesis testing. 6. Simulation.

Example: Estimate probability of catastrophic events.

23

Overview of the Course

1. Time series models 2. Time domain methods 3. Spectral analysis 4. State space models(?)

24

Overview of the Course

1. Time series models (a) Stationarity. (b) Autocorrelation function. (c) Transforming to stationarity. 2. Time domain methods 3. Spectral analysis 4. State space models(?)

25

Overview of the Course

1. Time series models 2. Time domain methods (a) AR/MA/ARMA models. (b) ACF and partial autocorrelation function. (c) Forecasting (d) Parameter estimation (e) ARIMA models/seasonal ARIMA models 3. Spectral analysis 4. State space models(?)

26

Overview of the Course

1. Time series models 2. Time domain methods 3. Spectral analysis (a) Spectral density (b) Periodogram (c) Spectral estimation 4. State space models(?)

27

Overview of the Course

1. Time series models 2. Time domain methods 3. Spectral analysis 4. State space models(?) (a) ARMAX models. (b) Forecasting, Kalman lter. (c) Parameter estimation.

28

Time Series Models

A time series model species the joint distribution of the sequence {Xt } of random variables. For example: P [X1 x1 , . . . , Xt xt ] for all t and x1 , . . . , xt . Notation: X1 , X2 , . . . is a stochastic process. x1 , x2 , . . . is a single realization. Well mostly restrict our attention to second-order properties only: EXt , E(Xt1 , Xt2 ).

29

Time Series Models

Example: White noise: Xt W N (0, 2 ). i.e., {Xt } uncorrelated, EXt = 0, VarXt = 2 . Example: i.i.d. noise: {Xt } independent and identically distributed. P [X1 x1 , . . . , Xt xt ] = P [X1 x1 ] P [Xt xt ]. Not interesting for forecasting: P [Xt xt |X1 , . . . , Xt1 ] = P [Xt xt ].

30

Gaussian white noise

1 P [Xt xt ] = (xt ) = 2
2.5 2

xt

x2 /2

dx.

1.5

0.5

0.5

1.5

2.5

10

15

20

25

30

35

40

45

50

31

Gaussian white noise

2.5

1.5

0.5

0.5

1.5

2.5

10

15

20

25

30

35

40

45

50

32

Time Series Models

Example: Binary i.i.d.
1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 0 5 10 15 20 25 30 35 40 45 50

P [Xt = 1] = P [Xt = 1] = 1/2.

33

Random walk
St =
t i=1

Xi .
8

Differences: St = St St1 = Xt .

10

15

20

25

30

35

40

45

50

34

Random walk
ESt ? VarSt ?
10

10

15

10

15

20

25

30

35

40

45

50

35

Random Walk
Recall S&P500 data.
SP500: JanJun 1987 340

(Notice that its smooth)

320

300

280

260

240

220 1987

1987.05

1987.1

1987.15

1987.2

1987.25 year

1987.3

1987.35

1987.4

1987.45

1987.5

36

Random Walk
Differences:
SP500, JanJun 1987. first differences 10

St = St St1 = Xt .
8

10 1987

1987.05

1987.1

1987.15

1987.2

1987.25 year

1987.3

1987.35

1987.4

1987.45

1987.5

37

Trend and Seasonal Models

Xt = Tt + St + Et = 0 + 1 t +
6

(i cos(i t) + i sin(i t)) + Et

5.5

4.5

3.5

2.5

50

100

150

200

250

38

Trend and Seasonal Models

Xt = Tt + Et = 0 + 1 t + Et
6

5.5

4.5

3.5

2.5

50

100

150

200

250

39

Trend and Seasonal Models

Xt = Tt + St + Et = 0 + 1 t +
6

(i cos(i t) + i sin(i t)) + Et

5.5

4.5

3.5

2.5

50

100

150

200

250

40

Trend and Seasonal Models: Residuals

0.5

0.4

0.3

0.2

0.1

0.1

0.2

0.3

0.4

0.5

50

100

150

200

250

41

Time Series Modelling

1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Transform data so that residuals are stationary. (a) Estimate and subtract Tt , St . (b) Differencing. (c) Nonlinear transformations (log, 3. Fit model to residuals. ).

42

Nonlinear transformations
Recall: Monthly sales.
12 x 10
4

(Makridakis, Wheelwright and Hyndman, 1998)

12

11.5
10

11

10.5
8

10

9.5

9
4

8.5

8
2

7.5

10

20

30

40

50

60

70

80

90

10

20

30

40

50

60

70

80

90

43

Time Series Modelling

1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Transform data so that residuals are stationary. (a) Estimate and subtract Tt , St . (b) Differencing. (c) Nonlinear transformations (log, 3. Fit model to residuals. ).

44

Differencing
Recall: S&P 500 data.
SP500: JanJun 1987 340 10 SP500, JanJun 1987. first differences 8 320 6

4 300 2

280

$ 1987.05 1987.1 1987.15 1987.2 1987.25 year 1987.3 1987.35 1987.4 1987.45 1987.5

2 260 4

6 240 8

220 1987

10 1987

1987.05

1987.1

1987.15

1987.2

1987.25 year

1987.3

1987.35

1987.4

1987.45

1987.5

45

Differencing and Trend

Dene the lag-1 difference operator,
(think rst derivative)

Xt = Xt Xt1 = (1 B )Xt , where B is the backshift operator, BXt = Xt1 . If Xt = 0 + 1 t + Yt , then Xt = 1 + Yt . If Xt =

k i=0

i ti + Yt , then k Xt = k !k + k Yt ,

where k Xt = (k1 Xt ) and 1 Xt = Xt .

46

Differencing and Seasonal Variation

Dene the lag-s difference operator, s Xt = Xt Xts = (1 B s )Xt , where B s is the backshift operator applied s times, B s Xt = B (B s1 Xt ) and B 1 Xt = BXt . If Xt = Tt + St + Yt , and St has period s (that is, St = Sts for all t), then s Xt = Tt Tts + s Yt .

47

Time Series Modelling

1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Transform data so that residuals are stationary. (a) Estimate and subtract Tt , St . (b) Differencing. (c) Nonlinear transformations (log, 3. Fit model to residuals. ).

48

Outline
1. Objectives of time series analysis. Examples. 2. Overview of the course. 3. Time series models. 4. Time series modelling: Chasing stationarity.

49