Statistics for

Financial
Engineering:
Some R
Examples

David Ruppert
Statistics for Financial Engineering: Some R
Introduction

Nonlinear
Examples
Regression
Default probabilities
Data
Transformations:
some theory David Ruppert
Estimating a
dynamic
model Cornell University
Interest rate data
Checking the model:
residual analysis
GARCH models
April 25, 2009
Bayesian
estimation of
expected
returns
 Outline

Statistics for
Financial
Engineering:
Some R
Examples
1 Introduction
David Ruppert
2 Nonlinear Regression
Introduction
Default probabilities
Nonlinear
Regression Data Transformations: some theory
Default probabilities
Data
Transformations:
some theory 3 Estimating a dynamic model
Estimating a
dynamic
Interest rate data
model Checking the model: residual analysis
Interest rate data
Checking the model:
residual analysis
GARCH models
GARCH models

Bayesian
estimation of
4 Bayesian estimation of expected returns
expected
returns
 A little about myself

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert
BA and MA in mathematics
Introduction

Nonlinear
PhD in statistics in 1977
Regression
Default probabilities
Data
taught in the statistics department at North Carolina for
Transformations:
some theory

Estimating a
10 years
dynamic
model
Interest rate data
have been in Operations Research and Information
Checking the model:
residual analysis
GARCH models
(formerly Industrial) Engineering at Cornell since 1987
Bayesian
estimation of
expected
returns
 A little about myself

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert starting teaching Statistics and Finance to undergraduates

Introduction in 2001
Nonlinear
Regression textbook published in 2004
Default probabilities
Data
Transformations: starting teaching Statistics for Financial Engineering to
some theory

Estimating a master’s students in 2008

dynamic
model
Interest rate data working on revised and expanded textbook
Checking the model:
residual analysis
GARCH models
now programming exclusively in R
Bayesian
estimation of
expected
returns
 Undergraduate Textbook

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

Introduction

Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory

Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models

Bayesian
estimation of
expected
returns
 A little about my research

Statistics for
Financial
Engineering:
Some R
have done research in
Examples
asymptotic theory of splines
David Ruppert
semiparametric modeling
Introduction
measurement error in regression
Nonlinear
Regression smoothing (nonparametric regression and density
Default probabilities
Data
Transformations:
estimation)
some theory

Estimating a
transformation and weighting
dynamic
model stochastic approximation
Interest rate data
Checking the model: biostatistics
residual analysis
GARCH models environmental engineering
Bayesian
estimation of modeling of term structure
expected
returns executive compensation and accounting fraud
 Three types of regression

Statistics for
Financial
Engineering: Linear regression
Some R
Examples

David Ruppert
Yi = β0 + β1 Xi,1 + · · · + βp Xi,p + i , i = 1, . . . , n

Introduction Nonlinear regression

Nonlinear
Regression
Default probabilities Yi = m(Xi,1 , . . . , Xi,p ; β1 , . . . , βq ) + i , i = 1, . . . , n
Data
Transformations:
some theory
where m is a known function depending on unknown
Estimating a
dynamic parameters
model
Interest rate data
Checking the model: Nonparametric regression
residual analysis
GARCH models

Bayesian Yi = m(Xi,1 , . . . , Xi,p ) + i , i = 1, . . . , n

estimation of
expected
returns
where m is an unknown “smooth” function
 Usual assumptions on the noise

Statistics for
Financial
Engineering:
Some R
Examples Usually 1 , . . . , n are assumed to be:
David Ruppert
mutually independent (or at least uncorrelated)
Introduction

Nonlinear homoscedastic (constant variance)

Regression
Default probabilities
Data
Transformations:
normally distributed
some theory

Estimating a
dynamic
model
Interest rate data
Much research over the last 50+ years has looked into ways of
Checking the model:
residual analysis
GARCH models
1 checking these assumptions
Bayesian
estimation of
expected
2 statistical methods that require less assumptions
returns
 Transform-both-sides model

Statistics for
Financial Ideal model (no errors):
Engineering:
Some R
Examples
Yi = f (X i , β)
David Ruppert

Introduction Statistical model (first attempt):

Nonlinear
Regression
Default probabilities Yi = f (X i , β) + i
Data
Transformations:
some theory
where 1 , . . . , n are iid Gaussian
Estimating a
dynamic
model TBS model:
Interest rate data
Checking the model:
h{Yi } = h{f (X i , β)} + i
residual analysis
GARCH models
where
Bayesian
estimation of
expected
1 , . . . , n are iid Gaussian
returns
h is an “appropriate” transformation
 Estimation of Default Probabilities

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

Introduction

Nonlinear
Data:
Regression
Default probabilities
Data
ratings: 1=Aaa (best),...,16=B3 (worse)
Transformations:
some theory
default frequency (estimate of default probability)
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models

Bayesian
estimation of
expected
returns
 Some statistical models

Statistics for
Financial
Engineering:
Some R nonlinear model:
Examples

David Ruppert
Pr(default|rating) = exp{β0 + β1 rating}
Introduction

Nonlinear linear/transformation model (in recent textbook):

Regression
Default probabilities
Data
Transformations:
some theory
log{Pr(default|rating)} = β0 + β1 rating
Estimating a
dynamic
model
Interest rate data
Problem: cannot take logs of default frequencies that
Checking the model:
residual analysis
GARCH models
are 0
Bayesian
estimation of (Sub-optimal) solution in textbook: throw out these
expected
returns observations
 A better statistical model

Statistics for
Financial
Engineering:
Some R
Examples Transform-both-sides (TBS) model – see Carroll and
David Ruppert
Ruppert (1984, 1988):
Introduction

Nonlinear
using a power transformation:
Regression
Default probabilities  λ  λ
Data
Transformations:
Pr(default|rating) + κ = exp(β0 + β1 rating) + κ
some theory

Estimating a
dynamic
model λ chosen by residual plots (or maximum likelihood)
Interest rate data
Checking the model: λ = 1/2 works well for this example
residual analysis
GARCH models log transformations are also commonly used
Bayesian
estimation of κ > 0 will shift data away from 0
expected
returns
 The Box-Cox family

Statistics for
Financial
Engineering:
Some R
Examples
the most common transformation family is due to Box and
David Ruppert
Cox (1964):
Introduction

Nonlinear yλ − 1
Regression h(y, λ) = if λ 6= 0
Default probabilities λ
Data
Transformations: = log(y) if λ = 0
some theory

Estimating a
dynamic derivative has simple form:
model
Interest rate data
d
h(y, λ) = y λ−1 for all λ
Checking the model:
residual analysis hy (y, λ) =
GARCH models dy
Bayesian
estimation of
expected
returns
 TBS fit compared to others

Statistics for
Financial
Engineering:
Some R ●
Examples ●
●
●
David Ruppert BOW
0 ●
nonlinear ●
●

tbs
Introduction
o data ●
log(default probability)

● ●
Nonlinear
Regression
−5

Default probabilities
Data
Transformations:
some theory

Estimating a
−10

dynamic
model
Interest rate data
Checking the model:
residual analysis
● ● ● ● ● ●
GARCH models

Bayesian
estimation of 5 10 15
expected
returns rating
 Nonlinear regression residuals

Statistics for
Financial
Engineering:
Some R
Examples
Normal QQ plot
David Ruppert

2
● ●
0.006

Introduction ●

1
Theoretical Quantiles
Nonlinear ●
absolute residual

0.004

●
Regression ●
●
Default probabilities ●

0
●
Data ●
●
Transformations: ●
●
0.002

some theory ●
●

−1
● ●
Estimating a ●
●
dynamic
●● ●
●
model ●
0.000

● ●
● ●

−2
Interest rate data
Checking the model:
0.00 0.02 0.04 0.06 0.08 0.10 0.12 −0.006 −0.002 0.002 0.006
residual analysis
GARCH models fitted values Sample Quantiles

Bayesian
estimation of
expected
returns
 TBS residuals

Statistics for
Financial
Engineering:
Some R
Examples
Normal QQ plot
David Ruppert

2
● ●
0.020

●
●
Introduction ●

1
Theoretical Quantiles
Nonlinear
0.015

●
absolute residual

● ● ●
Regression ●
●
●
●
●
Default probabilities ●

0
●
0.010

●
Data ● ●
●
Transformations: ●
some theory ●
●

−1
0.005

●
Estimating a ●
●
dynamic ●
●
●
model
● ●

−2
Interest rate data
Checking the model:
−0.02 0.02 0.06 0.10 −0.02 −0.01 0.00 0.01 0.02
residual analysis
GARCH models fitted values Sample Quantiles

Bayesian
estimation of
expected
returns
 Estimated default probabilities

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

Introduction

Nonlinear
method Pr{default|Aaa}
c as % of TEXTBOOK est
Regression
Default probabilities
TEXTBOOK 0.005% 100%
Data
Transformations:
nonlinear 0.002% 40%
some theory
TBS 0.0008% 16%
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models

Bayesian
estimation of
expected
returns
 A Similar Problem: Challenger Data

Statistics for
Financial
Engineering:
Some R
Examples

2.0
David Ruppert
*
Introduction
1.5
# o−rings distressed

Nonlinear
Regression
Default probabilities
1.0

Data
Transformations:
some theory

Estimating a
** * *
dynamic
0.5

model
Interest rate data
Checking the model:
residual analysis
0.0

***** ** **
** ** *
GARCH models

Bayesian
estimation of 55 60 65 70 75 80
expected
returns
temp
 Challenger Data: Extrapolation to 31o

Statistics for
Financial
Engineering: Logistic regression
Some R
Examples

David Ruppert 6
all data
5

Introduction
only data with failures
# o−rings distressed

Nonlinear
4

Regression
Default probabilities
Data
3

Transformations:
some theory

Estimating a

*
2

dynamic
model

** * *
Interest rate data
1

Checking the model:

residual analysis

***********
**
GARCH models
0

Bayesian
estimation of 30 40 50 60 70 80 90
expected
returns
temp
 Variance stabilizing transformation: how it works

Statistics for
Financial
Engineering:
Some R
Examples 3.0
David Ruppert

Introduction
2.5

●
Nonlinear Pop 2 ●●●●●
●
●
Regression transformed ● ●
2.0

Default probabilities ●
Data
Transformations:
some theory
1.5
y

Estimating a
dynamic
1.0

model
Interest rate data log(x)
Checking the model: Pop 1 ●
●
residual analysis ●●
0.5

GARCH models
transformed ●
●
●
●
●
●
Bayesian
0.0

estimation of
expected Pop 1 Pop 2
returns

0 5 10
 Strength of Box-Cox family

Statistics for
Financial
Engineering:
Some R
Examples Take a < b
David Ruppert Then  λ−1
hy (b, λ) b
Introduction =
Nonlinear
hy (a, λ) a
Regression
Default probabilities which is increasing in λ and equals 1 when λ = 1
Data
Transformations:
some theory λ = 1 is the dividing point between concave and convex
Estimating a transformations
dynamic
model
Interest rate data
h(y, λ) becomes a stronger concave transformation as λ
Checking the model:
residual analysis
decreases from 1
GARCH models
also, h(y, λ) becomes a stronger convex transformation as
Bayesian
estimation of λ increases from 1
expected
returns
 Strength of Box-Cox family, cont.

Statistics for
Financial
Engineering:
Some R
Examples Example: b/a = 2

David Ruppert

Introduction 2.0

2α−1
1.5

Nonlinear
Derivative ratio

Regression
Default probabilities
Data
Transformations: convex
1.0

some theory

Estimating a
dynamic
model
0.5

Interest rate data

Checking the model: concave
residual analysis
GARCH models
−1.0 −0.5 0.0 0.5 1.0 1.5 2.0
Bayesian
estimation of
α
expected
returns
 Maximum likelihood

Statistics for
Financial
Engineering:
Some R
Examples
h  i2
n
X h(Yi + κ, λ) − h f (X i , β) + κ, λ
David Ruppert L(β, λ, σ) = n log(σ) −
i=1
2σ 2
Introduction
n
X
Nonlinear
Regression
+ (λ − 1) log(Yi )
Default probabilities i=1
Data
| {z }
Transformations:
some theory
from Jacobian

Estimating a
dynamic
model can maximize over σ analytically:
Interest rate data
Pn h i2
b2 = n −1 i=1 h(Yi + κ, λ) − h f (X i , β) + κ, λ
Checking the model:

residual analysis σ
GARCH models

Bayesian
estimation of
they maximize over (β, λ) with optim, for example
expected
returns κ is fixed in advance
 Reference for TBS

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

Introduction

Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory

Estimating a
dynamic
model Transformation and Weighting in Regression by Carroll and
Interest rate data
Checking the model: Ruppert (1988)
residual analysis
GARCH models
Lots of examples
Bayesian
estimation of
expected
returns But none in finance _
¨
 1-Year Treasury Constant Maturity Rate, daily data

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

Introduction 15
Nonlinear
10
rate

Regression
Default probabilities
Data
Transformations:
some theory
5

Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
1970 1980 1990 2000
GARCH models
year
Bayesian
estimation of
expected
returns Source: Board of Governors of the Federal Reserve System
http://research.stlouisfed.org/fred2/
 ∆Rt versus year

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert
1.0
Introduction
0.5
change in rate

Nonlinear
Regression
Default probabilities
0.0

Data
Transformations:
some theory
−0.5

Estimating a
dynamic
model
Interest rate data
−1.0

Checking the model:

residual analysis
GARCH models

Bayesian
estimation of
1970 1980 1990 2000
expected
returns year
 ∆Rt versus Rt−1

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

Introduction 1.0
0.5
Nonlinear
change in rate

Regression
Default probabilities
0.0

Data
Transformations:
some theory
−0.5

Estimating a
dynamic
model
−1.0

Interest rate data

Checking the model:
residual analysis
GARCH models 5 10 15
Bayesian
estimation of lagged rate
expected
returns
 ∆Rt2 versus Rt−1

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

Introduction 1.2
change in rate2

Nonlinear
0.8

Regression
Default probabilities
Data
Transformations:
some theory
0.4

Estimating a
dynamic
model
0.0

Interest rate data

Checking the model:
residual analysis
GARCH models 5 10 15
Bayesian
estimation of lagged rate
expected
returns
 Drift function

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert
Discretized diffusion model:
Introduction

Nonlinear
Regression
∆Rt = µ(Rt−1 ) + σ(Rt−1 )t
Default probabilities
Data
Transformations:
some theory

Estimating a µ(x) is the drift function

dynamic
model σ(x) is the volatility function (as before)
Interest rate data
Checking the model:
residual analysis
GARCH models

Bayesian
estimation of
expected
returns
 Estimating Volatility

Statistics for
Financial
Engineering: Parametric model:
Some R
Examples
β1
David Ruppert Var{(∆Rt )} = β0 Rt−1
Introduction
(Common in practice)
Nonlinear
Regression
Default probabilities
Data
Transformations:
Nonparametric model:
some theory

Estimating a
dynamic
Var{(∆Rt )} = σ 2 (Rt−1 )
model
Interest rate data
Checking the model: where σ(·) is a smooth function
residual analysis
GARCH models
will be modeled as a spline
Bayesian
estimation of
expected In these models: no dependence on t
returns
 Spline Software

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

Introduction
The penalized spline fits shown here were obtained using the
Nonlinear
Regression function spm
Default probabilities
Data
Transformations:
in R’s SemiPar package
some theory

Estimating a
author is Matt Wand
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models

Bayesian
estimation of
expected
returns
 Comparing parametric and nonparametric volatility
fits
Statistics for
Financial
Engineering:
Some R
Examples 1.2
●
David Ruppert ●

0.08
●
● nonpar
1.0

Introduction
● par
Nonlinear ●

0.06
0.8

Regression ●
diff_rate^2

Default probabilities ●
●
0.6

Data ●
●

0.04
Transformations: ● ● ●
●
some theory

●
0.4

●
●
Estimating a ●● ● ●
●●●●
●● ● ● ● ● ●●●●●

0.02
dynamic ● ●●●
● ●●●●
●
model ● ● ●●●
●● ●● ●
●
●
0.2

●● ● ●
● ●● ●●●
●
● ●
● ●●● ●
Interest rate data
● ● ●● ●●
● ● ●● ●
●●
● ●●●●
● ●
● ●●
● ●
●● ● ●●
●●●
●● ●●
●● ● ●● ●
●
● ●●
●
●●
●●●
● ● ●●
●
●
● ●●
●●●
●●
●●
●●
●●● ● ●●
●●●●●● ●●
●●●●
●

0.00
Checking the model: ●● ● ●● ●
●
● ●
●● ● ●●●
● ● ●
●●●● ● ●●
●●
●●
●●●●
●●
●●
●●●
●
● ●●●●●●
●
●●●●
●●
●●
●●
●●●
●● ●
residual analysis ●● ●
●●●
●●
●
●●
●
●
●●
●●
●
●●
●
●●
●●
●●
●
●
●●
●
●
●●
●●
●
●●
●●
●
● ●
●
●●
●●
●●
●
●
●●
●
●
●●●●
●
●●
●●
●●
●●
●
●●●
● ●
●
●●●
0.0

●
●●
●●
●
●
●
●● ●
●●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●●
●●●
●●●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●
●●
●●
●
●
●●
●
●●
●
●●
●
●●
●●
●
●●●
●
GARCH models

Bayesian
estimation of 5 10 15 5 10 15
expected
returns
lag_rate lag_rate
 Comparing parametric and nonparametric volatility
fits: zooming in near 0
Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

0.006
Introduction nonpar
par
Nonlinear
0.002

Regression
Default probabilities
Data
Transformations:
some theory
−0.002

Estimating a
dynamic
model
−0.006

Interest rate data

Checking the model:
residual analysis
GARCH models 0 1 2 3 4 5 6
Bayesian
estimation of lag_rate
expected
returns
 Spline fitting – Estimation of drift function

Statistics for
Financial
Engineering:
Some R
Examples

1.0
● ●

0.00
● ●
● ●
●●
● ●
●●●●● ●● ●●●●●●●

diff_rate
●●
● ●● ● ●● ●
●● ●
●
David Ruppert ● ●
●●
● ●● ●
●
● ●
●●●
●●
●
●●
●
●●●
●● ●
●
●●
●
●
●●
●
●●
●●
●●
●● ●●●●
●
●●
●
●●●
●●
●
●●●
●
●●
●
●●
●
●
●●
●●
●
●
●
●●
●
●●
●● ●
●●
●●
●
●●●
●●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●●
● ●●
●● ●
● ●
●●●●
●●
●
●●
●
●●
●
●●
●●●●
● ●●
●●●
●●●
●●
●●●●
●● ●
●●●●

0.0
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●●●
●
●●
●
●
●
●
●●
●●
●
●
●●
●●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
● ●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●●
●●
●
●
●
●●
●
● ●●●
●●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●●
●●
●●
●●
●●●● ●●●●●
●
●
●
●
●●
●
●●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●●●●●
●
●
●
●
●●●
● ●
●●
●●
● ●
●●
●●
●●
●●
●●
●
●● ● ●●● ● ●
●●●●
● ●●
●●
●●
●●
●●
●●●
●●●●●●●● ●●

−0.06
● ●●●●●●●●●●●
●
● ●●● ●
●
Introduction ● ●● ● ● ●●
−1.0
●
●

Nonlinear
5 10 15 5 10 15
Regression
Default probabilities lag_rate lag_rate
Data
Transformations:
some theory

Estimating a
dynamic

0.005
model
0.00

2nd der
1st der

Interest rate data

Checking the model:
residual analysis

−0.010
−0.03

GARCH models

Bayesian
estimation of 5 10 15 5 10 15
expected
lag_rate lag_rate
returns
 Residuals for diffusion model

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

Introduction
residualt := ∆Rt − µ
b(Rt−1 )
Nonlinear E(residualt ) = 0
Regression
Default probabilities
Data
Transformations:
some theory

Estimating a
dynamic residualt
model std residualt :=
Interest rate data σ
b(Rt−1 )
Checking the model:
residual analysis
GARCH models
E(std residual2t ) = 1
Bayesian
estimation of
expected
returns
 Question

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

Introduction

Nonlinear
Regression
Default probabilities
Are the drift and volatility functions constant in time?
Data
Transformations:
some theory

Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models

Bayesian
estimation of
expected
returns
 Residual plots: ordinary residuals

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

Introduction
0.010

Nonlinear
Regression
residual

Default probabilities
Data
0.000

Transformations:
some theory

Estimating a
dynamic
model
−0.010

Interest rate data

Checking the model:
residual analysis
GARCH models

Bayesian
1970 1980 1990 2000
estimation of
expected year
returns
 Residual plots: standardized residuals

Statistics for
Financial
Engineering:
Some R
Examples
autocorrelation function Normal Q−Q Plot
David Ruppert

4
●

Theoretical Quantiles
Introduction ●
●
●
●●
●
●●
●
●●
●
0.8

●●
●
●
●
●
●●
●
Nonlinear ●
●

2
●
●
●
●
●
●
●
Regression ●
●
●
●●
●
ACF

●
●
Default probabilities
●●
●
●

0
●
●
●
0.4

Data ●
●
●
●
●
●
Transformations: ●●
●
●
●
some theory ●
●

−2
●
●●
●
●
●
●●
●●
●
●
●●
Estimating a ●●
●
●●
●
●
● ●
●
●●
● ●
●
0.0

dynamic ●
●

−4
●
model
Interest rate data
Checking the model:
0 10 20 30 40 −10 −5 0 5
residual analysis
GARCH models
Lag Sample Quantiles
Bayesian
estimation of
expected
returns
 Residual plots: Squared standardized residuals

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

1.5
Introduction
Squared Std residual

Nonlinear
1.0

Regression
Default probabilities
Data
Transformations:
0.5

some theory

Estimating a
dynamic
0.0

model
Interest rate data
Checking the model:
residual analysis
GARCH models

Bayesian
1970 1980 1990 2000
estimation of
expected year
returns
 Residual plots: Squared standardized residuals

Statistics for
Financial
Engineering:
Some R autocorrelation function
Examples

1.0
David Ruppert

Introduction
0.8

Nonlinear
Regression
0.6

Default probabilities
ACF

Data
Transformations:
0.4

some theory

Estimating a
0.2

dynamic
model
Interest rate data
0.0

Checking the model:

residual analysis
GARCH models

Bayesian
0 10 20 30 40
estimation of
expected Lag
returns
 GARCH(p, q) model

Statistics for
Financial
Engineering: The GARCH(p, q) model is
Some R
Examples

David Ruppert at = t σt ,
Introduction
where v
Nonlinear u q p
Regression u X X
Default probabilities σt = tα0 + 2 +
αi at−i 2 .
βi σt−i
Data
Transformations:
some theory
i=1 i=1
Estimating a
dynamic
and
model t is an iid (strong) white noise process
Interest rate data
Checking the model:
residual analysis
GARCH models

Bayesian at is weak white noise

estimation of
expected
returns
uncorrelated but with volatility clustering
 GARCH(1,1) fit using garch in tseries

Statistics for
Financial
Engineering:
Some R
Examples Call:
David Ruppert
garch(x = std_drift_resid^2, order = c(1, 1))

Introduction
Model:
GARCH(1,1)
Nonlinear
Regression
Default probabilities
Coefficient(s):
Data Estimate Std. Error t value Pr(>|t|)
Transformations:
some theory a0 0.27291 0.00148 184 <2e-16 ***
Estimating a
a1 0.44690 0.00252 177 <2e-16 ***
dynamic b1 0.80490 0.00075 1073 <2e-16 ***
model
Interest rate data
Box-Ljung test
Checking the model:
residual analysis
GARCH models
data: Squared.Residuals
Bayesian X-squared = 0.13, df = 1, p-value = 0.7186
estimation of
expected
returns
 GARCH: estimated conditional standard deviations

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

3.5
GARCH conditional std

Introduction

Nonlinear
Regression
2.5

Default probabilities
Data
Transformations:
some theory
1.5

Estimating a
dynamic
model
Interest rate data
0.5

Checking the model:

residual analysis
GARCH models

Bayesian 0 2000 4000 6000 8000 10000

estimation of
expected
year
returns
 GARCH: squared residuals with lowess smooth

Statistics for ● ●
● ●
● ● ● ●
Financial ●
● ●
●
● ●
●
●
●
●
● ●● ● ● ● ●
Engineering: ●
●
●●
●
● ● ●
● ●●
●
●●
●
●● ● ●
●
Some R ●
● ●
●
●
●
●
●
●
●

● ●● ● ● ● ● ●● ●
●● ●
Examples ● ●
●
●
● ●
● ●
● ●
●
● ●
●
● ● ● ●● ● ● ●●
●● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ●
● ● ● ●
David Ruppert ●
● ●● ●
●
● ● ● ●
● ● ● ●
● ●
●
●●
●
●

6
● ●● ● ● ● ● ● ● ●
● ● ● ● ● ● ●
●
● ●● ● ● ● ● ● ● ● ●
●
● ●
●● ● ● ● ● ● ● ●
GARCH squared residuals

● ● ● ● ●
● ●
●● ● ● ● ●● ● ●● ●
●
● ● ● ● ●
● ●
● ● ● ● ● ●
●
Introduction ● ●
●●
● ●● ● ● ● ●
● ● ● ● ●
● ● ●●
●● ● ●●● ●
●●
●
●
5
● ●
● ● ● ● ●● ●● ● ●●● ● ● ●
●
● ● ● ● ● ●● ● ● ●
●● ● ● ● ●
●● ● ●● ●
● ● ● ●
● ●● ●● ●●● ● ● ●
● ● ● ●● ● ● ●● ● ● ●● ● ● ●
●
Nonlinear ● ● ●
● ● ● ●
●●
● ●●
●
●
●
●●
●
●
●
● ●
●
●
●
●
● ●
●
●●
●
●
● ●●●
● ●●
● ● ●
●● ●
● ●
●
● ●
●●● ● ●● ● ● ●● ●
●●● ● ●● ●● ● ●
Regression
4

●● ● ● ●
● ● ● ●●● ● ●● ● ● ●● ●● ● ● ● ●● ●●● ● ●●●● ●
● ●● ● ● ●● ● ●●
● ● ●● ● ●● ●
● ●● ●●● ●
●
● ●● ● ●●
●
●
● ● ● ● ● ● ●
● ● ●● ● ● ● ● ●●
● ● ● ●● ● ●● ●● ●● ● ● ●●
● ●● ● ● ●
Default probabilities ● ●● ● ● ●● ●
● ● ● ●
●
● ● ● ● ●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●● ● ● ● ● ● ● ● ●
● ● ● ●● ● ●● ● ●● ● ●● ● ●●●●● ●● ● ● ● ●● ● ● ● ●●● ● ● ●● ●
● ●● ● ●
● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●
Data ●● ● ●●● ● ●●● ●
● ● ●● ●
● ● ●● ● ● ●●● ●
3

● ● ●●● ● ●● ● ● ● ●● ● ●●●
●●● ●●
●●
● ● ● ● ●● ● ● ●
● ● ● ● ● ●● ● ● ●
●● ●
●●● ●
● ● ● ●● ●
●
Transformations: ● ● ● ●● ● ● ● ●● ● ● ●●●●● ● ●● ● ● ●● ●● ●● ● ●●● ●
● ● ● ● ●● ● ●●● ● ●● ● ● ● ●● ● ● ● ● ●●● ●●
● ●● ● ●●● ●●
●● ● ● ●●● ● ●
●● ● ● ●●●● ● ● ●● ●●● ●
●● ● ● ●● ●
● ●●●● ●●
● ●● ● ●
● ●● ● ●●● ●● ●●● ●● ●● ● ● ●● ● ● ●● ● ● ● ● ● ●●●● ● ● ●
some theory ● ●● ●
●●●
● ● ●
●●●●●
●● ●
● ●●
● ●
● ● ● ●● ●
●●●● ●● ● ● ●
● ● ● ●
●● ●
●●
● ●● ● ●
●● ● ●
●● ●● ●●
●●● ●
●● ● ●
●●● ●● ● ●● ●
●
● ●●● ● ●● ●● ● ●●
● ● ●● ●● ●● ●●● ● ● ● ● ●● ● ●● ●
●
●●
●● ● ● ●●●●●
● ●● ●● ● ●● ● ● ●● ●● ●● ●● ●●● ●● ●● ● ● ●● ●●
● ●●● ● ● ●● ●● ● ●● ●● ●
●● ● ● ●● ●●● ● ● ● ●●
●●●●●●● ●●● ●
● ● ●●● ●
●●● ● ● ● ● ●●●●
● ●● ● ●● ● ●● ●
●
● ● ● ●●● ● ●
● ● ●● ●
●●●
● ● ● ● ●●
● ● ● ●●● ●
● ●●● ●● ● ● ●● ● ●●●● ●● ● ● ●●●
●● ●●● ●● ●● ●
2

● ●●● ● ● ●● ●● ●● ● ● ●●●●●● ●● ● ● ● ● ●●●

● ● ● ● ●●●●
●● ●● ●●
● ●● ● ●●● ● ●●●
● ●●● ● ● ●● ●● ●●
●
Estimating a ●● ●●● ●● ●●
● ● ● ●●● ●● ● ●●●
● ●
●● ● ●
● ●●● ● ●● ● ●●●●●
● ●●
●● ●●● ●● ●●●● ● ● ● ●●●● ●●● ● ●● ● ●● ●●
● ● ● ●● ●●
●
●
● ●● ●● ● ● ●● ●
●●● ●
●● ● ●● ●
● ●●● ●●
● ●●●● ● ●●
● ● ●●●●● ●● ●
●●●● ● ●●● ● ●●●
● ●● ● ● ● ● ●●● ●●● ● ●● ●●●●● ●●●●
●●
●●
● ●● ● ●● ●
●●
●●● ●● ● ● ● ● ● ● ●●●● ● ●●● ● ●● ●● ●●●
● ●
●● ● ●● ●● ●● ● ●● ●●●
● ●●
●● ●
● ●
●
●● ●● ●●●●● ●● ● ● ●●●●●●
●●●● ● ● ● ●●●
● ● ●●
● ● ● ● ●●● ●● ●●
● ● ●● ●
● ● ●● ●● ● ● ●●●
● ●●● ●● ● ●
● ● ●
●●
● ●●
● ● ● ● ●● ●●●● ●
●● ●● ●
●● ●●●
●● ●
●● ●●
● ●
●●● ●
● ● ● ● ●●●●
●
●●
●●●●● ● ●
● ●●●
● ● ●
● ● ● ● ● ● ●
dynamic ●
●●
●●●
●●
●
●● ●
● ●
● ● ●●
●●
●
●●●
●
●●
● ● ●● ●●
●
●● ●
● ●
● ●
●●
● ●
●
●
●
●●●
●● ●
●
●●
●
●
●● ●
●●
●
●
●●
●●●
●
●●
●●
●
● ●● ●
●
●
●
●●●
●
●
●●
● ●
●● ●●●●
●●
●● ●●●
●
●
●●●
●●
●
●●●● ●●●
●
●
●
●
●
●●
●●
● ●●● ●●
●
●●●●●●●●
● ●
●●
●●
●
●●
●●● ●
●●●
●●
●
●●
●
●
●●●
●●
● ●● ●
●●●
●●●●●
●
● ●● ● ●● ●●
●● ●●
●
●● ●
●●
●
●●
●
● ●●●
●●● ●●●●●
●●
●
●●
●
●●●● ●●
●● ● ●
●
● ●●
● ●
●
●
●●●● ●
●
●●● ●
●
●
●
●
●●
●●
●
●● ●●
●●
●
●● ●●
● ●●
●●● ●●
● ●●
●●
●
●●●
●●●●●
●
● ● ●●● ●●●● ● ●
●
●●●●
●
●
●
●
●
●
● ●● ● ●● ●●●●● ●●
●● ●●
●●
●
●
●●●●●●
●
● ●● ●●
●
●
● ●
● ●●●
● ●●
●● ●
●
● ●
●●●●●●●● ●● ● ● ●●
● ●●●●● ● ●● ●●●
●● ●●● ●
●● ●● ●●● ●● ●● ●●● ●● ●●
● ●●● ●
●●●● ●● ●●● ●●●●●● ● ●●●● ● ●● ●●●●
●● ● ●● ●● ● ●●● ● ●●●
● ●●●●● ●●●●●
●●●●● ●
●● ●●
●●●●●●●●
●●● ● ● ●
●
●●
●● ● ● ● ● ● ● ● ● ●● ● ●● ●● ●
●● ● ●● ●
●● ●●●● ● ●●●● ● ● ● ●●● ●●● ●
model ●
●● ●
●●
●●
●●●● ●● ● ●●●
●
●●●●●●●
●● ● ●●
●●●●●
● ● ● ●●
● ●● ●● ●
●● ●● ●●●● ● ●
● ●● ●●● ●● ●●
●●●
● ● ●●● ●●
● ●● ●● ●●●
● ● ●● ●
● ●●●●● ● ●●●
● ● ●
●●●●
●●● ●●●● ●
●●
●●
1

●● ● ● ●● ●● ● ●● ● ●● ●●●● ●● ●● ● ●
● ●● ● ●● ●●●
●
●●
● ●
●●●
● ●● ●●
●●●●
●●●
●● ● ● ●
●●
● ● ●●
●
●●●●●●
●
● ●●●●●
●
● ●●●● ● ●● ●
● ●●
●● ●
●
●
● ●●● ●
●
● ●●
●● ●
●●●
●●
●
●●●●●●●● ●● ●●
● ●
● ●●●
●
●●
●●●●●
●
● ●
●●
● ●●●●
● ●
●●●●
●●●
●●●●
● ●●
●● ● ●
●●
●
● ●●●●
● ● ●●●
●●
● ●●●
●
●●
●● ●
●●
●●●●●●●●●●● ● ● ●
●●
●●●
●●●●●●●
●● ●● ●●●●
● ●●
● ●●
●
●●●●●
●
●●
●●●
●●●●● ●
●● ●●●● ●
●●●
● ●●
● ●●●●●●
● ●● ●●
●
● ●
●●
●●● ●
●● ●
●●
●
●
● ●
●●
●●
● ● ●●●●●
● ●● ● ●●● ●● ● ●●●
● ●
●● ● ●●●●●
●●●
●
●●
●●
●● ●●
●●●● ●
●
●● ●●● ●
●●●
●●●●●
●●
● ●
●●●●●
●
●●
●●
● ●
●● ●●
●●●●
● ●●●
● ●● ●●●●●●●●
●
● ●
●●● ● ●
●●●●● ●●●●● ●●
●●●
●●●● ●●●●●●●● ●
●●●
●
●●
●● ●
●
●●●
●● ●● ●● ●
●●●●●●● ●●●●●●● ● ● ●●●
●
●●●●● ●●
●●● ●●
●
●●●
● ●
● ●●●●
● ●● ●
● ●●●●●
●●
●●●●●● ●●
● ●
●●
● ●●
● ●●●●●
●●●
● ● ●●
●● ●
●●●●●●● ●
●
●●●●
●● ●●● ●●●●● ● ●●●● ●●●●●
●●
● ●
●● ●
●● ●● ●● ●●●
● ●
●●
●●● ●
●●
● ● ●
●●
●●● ●●●●
●●
●●● ●●●
●●●●●●● ●●●● ●●●●● ●● ●●●●●●●● ●●●●
●●
● ●
●
● ●● ●●
●●
● ● ● ●
●●● ●●●●●
●●
●●●●
●
●●●● ●● ●●● ● ●●●● ●
● ●●
●●●●
●●●●●
●●●●●
● ● ●● ●
●
●●
● ●●
●●●●● ●
●
●●● ●●●
●● ●
●
●
●●● ● ● ●
● ●●
●
●●●●●●●
●
●
●●●
● ●
● ●● ●●
●●●
●●●
●
●●● ●
●● ●● ●● ●
●
●●● ●
●●
● ●
● ●●
●
●●
●● ●●
●●●
●
●●●●● ●●
● ●
●
●● ●
●
● ●
● ●● ●
●● ●●● ●● ●
●●
● ●
●●●
●●
●
● ●
●● ●
●
●●●●●●● ●●●●
●
●●
●
●●● ●●
● ● ●
●●
● ●
●
●●●●
●● ●
●●
●●●●●●
●●● ●●
●●● ●
●
●
●●●● ●●●●●●● ●
●●●●●●●●
●●●●●
●
●●●
●●●●
●●
● ●● ●●
●● ●●
● ●●
Interest rate data ●
●●●● ●
●●
●● ●●●
●●● ● ●
●●●●
●●●●
●
●
●● ●●●
●●●
● ●●●●●●●
● ●●
●
●●●
●●
●●●●
● ●●●
●● ●●
●● ●●
●●
●
●●
●● ●●● ●●●
●
● ● ● ●●● ●● ●●● ●
●
●●●
●●● ●●●● ● ●●●●
●●●●
●
● ●
●●●
●● ●
●●
● ●
●●●
● ●
●●
● ●●●● ●●●●
●●
●● ●
●●●
●● ●●
●●● ● ●●
● ●●
●● ●
●●●●
● ●●
●●
●●
● ●
●●
● ● ●● ●
●●●●
●
● ●●●●●●●
●
●
●●●●
●
●●
● ●●
●●● ●
● ● ●
●
●
●●
●●
●
●●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●●● ●●
●
●
●
●●● ●
●●
●
●
●●
●●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
● ●●
●●
●
●●●
●
●
●●●
●
●
●
●●●
●
●
●
●●
●
●●●
● ●
●
●
●
●
●●
●
●●●
●
●
●●
●
●
●●
●●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●●
●● ●
●
●
●
●●
●●
●●●
●●
●●●●
●●
●
●
●●
●●●
●
●
●●●
●●●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
● ●
●
●●●
●
●
●
●●
●
●● ●●
●●
●●●●
●●
●
●
●
●
●
●●
●●
●●● ●●
●●
●●
●
●
●●
●
●
●
●
●
●●● ●
●
●
●
● ●●●
●●
●●
●
●
●●●●●
●● ●
●
●
●●
●●●
●●●●
●●●
●
● ●
●
●
●
●
●●
●
●●
●
●
● ●●
●
●
● ●
●
●
●
●●
● ●
●●●●
●●
●
●●●
● ●●
●
●
●●● ●●
● ●
●●●●
● ● ●
● ●
●● ● ●
●●●●
●●● ●●● ● ●
●●● ●●
●● ● ●
●●
● ●
●●●
●●●● ●
●●●●●● ●●● ● ●
● ●
●●●
●
●● ● ●● ●●● ●
●● ●● ●
●● ● ● ●
●●●
●● ●
●●● ●
● ●● ● ● ● ●
●●● ● ● ● ●●
●●●● ●
●● ● ● ●● ●
●● ●
●● ●●● ●
●● ● ● ●
●●●●●
●● ● ●●●● ●
●
● ●●
● ●
● ●
●●●●● ●●●
● ●
●● ●
●●●
●●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
● ●
●
●●
●●
●
●
●
●
●
●
●
● ●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●●
●●●
●
●
●
●
●
●●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●●●
●●
●●●
●
●●
●
●●
●
●
●●●
●●●●
●
●●●●●
●
●●
●
●
●●●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●● ●
●
●●●
●
●
● ●●●
●●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●●●
●●●
●●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●●
● ●
●
●●
●●
● ●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
● ●
●
●
●●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●● ●
●●●
●●●●
●●●
●●
●●
●●
●● ●
●●●● ●
●●●
●● ●●●
●●●
●
●●●●●● ●
●● ●
● ●●●●● ●●● ●
●●
●●●
●●
●●●
●●●●●
●
●●●●●● ●●
●●●●
●●●●●
●●
●●
●● ●
● ●●
●● ●●
●●
●
●●
● ●
●●
● ●●
●● ●●
●
●●
● ●
●●●●●
●
●●
● ●
●●●●●● ●●●
●
● ●●
●●
●●●●●●
●
●●●● ●●●
●●●●●●
●
●●
●●
●●
● ●●●●●●
●●●
● ●●
● ●●
● ●
●
● ●
●● ● ● ●●●●●●
●
●●
●●● ●●● ●●● ●
●● ●
● ●● ●
Checking the model: ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●●●●● ●●
●●●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●●
●●
●
●
●● ●
●
●●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
0

●
●
●
●
●●
●●
●●
●
●●
●
●●
●
●
●●
●
●
●●
●●
●
●
●●
●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●●
●●
●
●●
●●
●●
●●
●
●●
●
●
●●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●●
●●
●
●●
●
●●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●●
●
●●
●
●
●●
●
●●
●●
●
●●
●
●●
●
●
●●
●●
●●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●●
●●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●●
●●
●●
●
●●●
●
●
●
●●
●
●
●
●
●●
●
●●●
●
●●
●●
●●●
●●
●
●
●●
●
●●
●
●
●
●
●●●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●
●
●●●
●
●●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●●●
●●
●●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●●
●●●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●
●●
●●●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●●
●●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●●
●
●●●
●
●
●
●
●●●●
●●
●
●
●●
●
●
●●
●
●
●●
●●
●●
●●
●
●●
●
●
●●●
●
residual analysis
GARCH models

Bayesian 0 2000 4000 6000 8000 10000

estimation of
expected
year
returns
 GARCH residuals

Statistics for
Financial
Engineering:
Some R GARCH residuals
Examples

David Ruppert

1.0
Introduction
0.8

Nonlinear
Regression
0.6

Default probabilities
ACF

Data
Transformations:
0.4

some theory

Estimating a
dynamic
0.2

model
Interest rate data
0.0

Checking the model:

residual analysis
GARCH models

Bayesian 0 10 20 30 40
estimation of
expected
Lag
returns
 AR(1)/GARCH(1,1)

Statistics for
Financial
Engineering: Call:
Some R garchFit(formula = ~arma(1, 0) + garch(1, 1), data = std_drift_resid)
Examples

David Ruppert Mean and Variance Equation:

data ~ arma(1, 0) + garch(1, 1)
Introduction [data = std_drift_resid]
Nonlinear
Regression Conditional Distribution:
Default probabilities norm
Data
Transformations:
some theory
Std. Errors:
Estimating a based on Hessian
dynamic
model
Interest rate data
Error Analysis:
Checking the model: Estimate Std. Error t value Pr(>|t|)
residual analysis
GARCH models
mu 0.001099 0.007476 0.147 0.883
ar1 0.138691 0.010468 13.248 < 2e-16 ***
Bayesian
estimation of omega 0.008443 0.001163 7.257 3.96e-13 ***
expected alpha1 0.073603 0.005483 13.424 < 2e-16 ***
returns beta1 0.923098 0.005457 169.158 < 2e-16 ***
 AR(1)/GARCH(1,1) residuals

Statistics for
Financial
Engineering:
Some R AR(1)/GARCH(1,1) residuals
Examples

David Ruppert

1.0
Introduction
0.8

Nonlinear
Regression
0.6

Default probabilities
ACF

Data
Transformations:
0.4

some theory

Estimating a
dynamic
0.2

model
Interest rate data
0.0

Checking the model:

residual analysis
GARCH models

Bayesian 0 10 20 30 40
estimation of
expected
Lag
returns
 AR(1)/GARCH(1,1) residuals - QQ plot

Statistics for
Financial
Engineering:
Some R Normal Q−Q Plot
Examples

David Ruppert

4
●
●●●
●
●
●
●●
● ●●
●
●
Introduction ●●
●
●●●
●
●
Theoretical Quantiles

●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
2

Nonlinear ●
●
●●
●
●
●
●
●
●●
●
●
●
●
Regression ●●
●
●
●●
●
●
●
●
●
Default probabilities ●
●
●
●
●●
Data ●
●
●
●
0

●
●
●
Transformations:
●●
●
some theory ●
●
●
●
●
●
●
●●
●
●●
●
Estimating a ●
●●
●
●
●
●
●●
●
●
●
−2

dynamic ●
●
●●
●
●
●
●
●●
●
●●
●
●
●
model ●
●●
●
●●
●●
●
●
●●
●
●
●●
●●
●
●● ●
●●
Interest rate data ●
●● ●
●
Checking the model: ●●
−4

residual analysis ●
GARCH models

Bayesian −10 −5 0 5
estimation of
expected
Sample Quantiles
returns
 AR(1)/GARCH(1,1)

Statistics for
Financial
Engineering: Call:
Some R
Examples garchFit(formula = ~arma(1, 0) + garch(1, 1), data = std_drift_resid,
cond.dist = "std")
David Ruppert

Mean and Variance Equation:

Introduction
data ~ arma(1, 0) + garch(1, 1)
Nonlinear [data = std_drift_resid]
Regression
Default probabilities
Data
Conditional Distribution:
Transformations:
some theory
std
Estimating a
dynamic
Error Analysis:
model Estimate Std. Error t value Pr(>|t|)
Interest rate data mu 0.0001087 0.0059401 0.018 0.98540
Checking the model:
residual analysis ar1 0.0969621 0.0090996 10.656 < 2e-16 ***
GARCH models omega 0.0016722 0.0005955 2.808 0.00498 **
Bayesian alpha1 0.0664390 0.0065895 10.083 < 2e-16 ***
estimation of beta1 0.9413495 0.0052248 180.169 < 2e-16 ***
expected shape 3.9169920 0.1600835 24.468 < 2e-16 ***
returns
 AR(1)/GARCH(1,1) residuals - QQ plot

Statistics for
Financial
Engineering:
Some R QQ−plot using t(3.91)
Examples

David Ruppert
●

10 ●
Introduction ●
●●
Theoretical quantiles

●
●
●
●
●●
Nonlinear ●●
●●
●
●●●
●
●●
5

Regression ●●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●●
●
Default probabilities ●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
Data ●
●
●
●●
●
●
0

●
●
●●
●
●
●
Transformations: ●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
some theory
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
−5

●
●
●●
●
●●
Estimating a ●
●
●
●
●
●
●●
●
dynamic ●●●●
●●●
−10

model ● ●
●
Interest rate data ●
Checking the model:
residual analysis ●
GARCH models

Bayesian −10 −5 0 5
estimation of
expected
Sample quantiles
returns
 Final model for the interest rate dynamics

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert
∆Rt = µ(Rt−1 ) + σ(Rt−1 )at

Introduction

Nonlinear
Regression 1 Model was fit in two steps:
Default probabilities
Data
Transformations:
1 estimate µ() and σ()
some theory
spm in SemiPar
Estimating a
dynamic
model 2 model at as AR(1)/GARCH(1,1)
Interest rate data
Checking the model: garchFit in fGarch
residual analysis
GARCH models

Bayesian
2 Could the two step be combined?
estimation of
expected
returns
3 Would combining them change the results?
 Reference for spline modeling

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

Introduction

Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory

Estimating a
dynamic
model
Interest rate data
Checking the model:
Semiparametric Regression by Ruppert, Wand, and Carroll
residual analysis
GARCH models
(2003)
Bayesian
estimation of
Lots of examples.
expected
returns
But most from biostatistics and epidemiology
 Bayesian statistics

Statistics for
Financial
Engineering:
Some R
Examples
Bayesian analysis allows the use of prior information
David Ruppert
hierarchical priors can:
Introduction

Nonlinear
specify knowledge that a group of parameters are similar to
Regression
Default probabilities
each other
Data
Transformations: estimate their common distribution
some theory

Estimating a
dynamic
WinBUGS can be run from inside R using the R2WinBUGS
model
Interest rate data package
Checking the model:
residual analysis
GARCH models
there is a similar BRugs package that runs OpenBugs
Bayesian
estimation of
BRugs is no longer on CRAN
expected
returns
 Data

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert

Introduction
midcapD.ts in fEcofin package
Nonlinear
Regression
Default probabilities
500 daily returns on:
Data
Transformations:
some theory
20 stocks
Estimating a market
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models

Bayesian
estimation of
expected
returns
 Goal

Statistics for
Financial
Engineering:
Some R
Examples
The goal is to use the first 100 days to estimate the mean
David Ruppert
returns for the next 400 days
Introduction

Nonlinear
Four possible estimators:
Regression
Default probabilities sample means
Data
Transformations:
some theory
Bayes estimation (shrinkage)
Estimating a
dynamic
model mean of means (total shrinkage)
Interest rate data
Checking the model:
residual analysis
CAPM
GARCH models

Bayesian
(expected return) = beta × (expected market return)
estimation of
expected
returns
 Who won?

Statistics for
Financial
Engineering:
Some R
Examples Estimate Sum of squared errors
David Ruppert
sample means 1.9
Introduction
Bayes 0.17
Nonlinear
Regression
Default probabilities
mean of means 0.12
Data
Transformations:
some theory
CAPM 1 0.66
Estimating a
dynamic
CAPM 2 0.45
model
Interest rate data
Checking the model: Squared estimation errors are summed over the 20 stocks
residual analysis
GARCH models

Bayesian CAPM 1: use mean of first 100 market returns

estimation of
expected
returns
CAPM 2: use mean of last 400 market returns
 Why does shrinkage help?

Statistics for
Financial
Engineering:
Some R
Examples

David Ruppert sample means Bayes

Introduction
●
0.6

0.6
Nonlinear
Regression
●
Default probabilities ●
●
Data
mean

mean
Transformations: ● ● ●
0.2

0.2
some theory ●
● ● ●
● ● ●
● ●
Estimating a ●
● ●
●
● ●
●
● ●
● ●
● ●
●
dynamic ● ●
● ●
●
●
● ● ●
model ●
−0.2

−0.2
Interest rate data ●
Checking the model: ●
●
residual analysis ●
GARCH models

Bayesian 1.0 1.4 1.8 1.0 1.4 1.8

estimation of
expected
returns estimate target estimate target
 Likelihood and prior

Statistics for
Financial
Engineering:
Some R
Examples
ri,t = tth return on i stock
David Ruppert Likelihood:
Introduction ri,t = µi + i,t
Nonlinear
Regression i,t ∼ IN (0, σ2 )
Default probabilities
Data
Transformations:
some theory IN = “Independent Normal”
Estimating a
dynamic Hierarchical Prior:
model
Interest rate data
Checking the model:
µi ∼ IN (α, σµ2 )
residual analysis
GARCH models

Bayesian
Diffuse (non-informative) priors on α, σ2 , σµ2
estimation of
expected Auto and cross-sectional correlations are ignored (treated as 0)
returns
 Data-driven shrinkage

Statistics for
Financial
Engineering:
Some R
Examples Hierarchical Prior:
David Ruppert
µi ∼ IN (α, σµ2 )
Introduction

Nonlinear
Regression
the µi are shrunk towards α
Default probabilities
Data
Transformations:
α should be (approximately) the mean of the means
some theory

Estimating a σµ2 /σ2 controls the amount of shrinkage

dynamic
model large σµ2 /σ2 ⇒ less shrinkage
Interest rate data
Checking the model:
residual analysis data-driven shrinkage
GARCH models

Bayesian because σµ2 and σ2 are estimated

estimation of
expected
returns
 WinBUGS output

Statistics for
Financial
Engineering:
Some R > print(means.sim,digits=3)
Examples Inference for Bugs model at "midCap.bug", fit using WinBUGS,
3 chains, each with 5100 iterations (first 100 discarded)
David Ruppert n.sims = 15000 iterations saved
mean sd 2.5% 25% 50% 75% 97.5% Rhat n.eff
mu[1] 1.1e-01 0.169 -0.22 -1.0e-03 1.1e-01 2.1e-01 4.5e-01 1 4000
Introduction mu[2] 1.2e-01 0.170 -0.20 1.5e-02 1.2e-01 2.3e-01 4.7e-01 1 6500
mu[3] 7.7e-02 0.168 -0.27 -2.7e-02 7.9e-02 1.8e-01 4.1e-01 1 3300
Nonlinear mu[4] 4.5e-02 0.176 -0.33 -6.1e-02 5.2e-02 1.6e-01 3.8e-01 1 1300
Regression
Default probabilities mu[18] 8.3e-02 0.170 -0.27 -2.4e-02 8.7e-02 1.9e-01 4.1e-01 1 3000
Data mu[19] 5.1e-02 0.171 -0.32 -5.1e-02 5.7e-02 1.6e-01 3.7e-01 1 1700
Transformations:
some theory mu[20] 4.8e-02 0.175 -0.33 -5.8e-02 5.5e-02 1.6e-01 3.7e-01 1 1800
sigma_mu 1.5e-01 0.065 0.06 9.9e-02 1.3e-01 1.8e-01 3.1e-01 1 520
Estimating a sigma_eps 4.3e+00 0.068 4.18 4.3e+00 4.3e+00 4.4e+00 4.4e+00 1 15000
dynamic alpha 8.8e-02 0.102 -0.11 1.7e-02 8.8e-02 1.6e-01 2.8e-01 1 710
model deviance 1.2e+04 3.989 11510.00 1.2e+04 1.2e+04 1.2e+04 1.2e+04 1 5300
Interest rate data
Checking the model: For each parameter, n.eff is a crude measure of effective sample size,
residual analysis and Rhat is the potential scale reduction factor (at convergence, Rhat=1).
GARCH models
DIC info (using the rule, pD = Dbar-Dhat)
Bayesian pD = 4.1 and DIC = 11521.6
estimation of DIC is an estimate of expected predictive error (lower deviance is better).
expected
returns