Large Sample Sieve Estimation of S-NP Models

Xiaohong Chen; Handbook of Econometrics Chapter 76

Will Matcham
W.O.Matcham@lse.ac.uk

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

1 / 27

Introduction
Sieve Estimation: Examples, Definitions and Sieves
Empirical Examples of S-NP Models
Definition of Sieve Extremum Estimation
Typical Function Spaces and Sieve Spaces
Small Monte Carlo Study
Some Sieve Applications in Econometrics
Large Sample Properties of Sieve Estimation of Unknown Functions
Consistency of Sieve Estimators
Convergence Rates of M-Estimators
Convergence Rates of Series Estimators
Pointwise AN of Series LS Estimators
Large Sample Properties of Sieve Estimation of P Parts in S-NP Models
SP Two-Step Estimators
Sieve Simultaneous M-Estimation
Sieve Simultaneous MD Estimation
Conclusion
Will Matcham

(LSE)

Chapter 76: Chen

October 2015

2 / 27

Abstract
Parametric (P) models often restrictive and sensitive to deviations

from parametric specifications

Semi-nonparametric (S-NP) models are more flexible and robust, but
introduce other complications: potentially non-compact
-dimensional parameter spaces, which lead to ill-posed optimisation
problems
Method of sieves (MoS) provides way to tackle such difficulties
Optimise an empirical criterion over a sequence of approximating
parameter spaces called sieves
Sieves are dense in the original space and less complex; optimisation
will become well posed.
Advantage: MoS very flexible for complex models with or without
heterogeneity and endogeneity.
Will Matcham

(LSE)

Chapter 76: Chen

October 2015

3 / 27

Abstract
Advantage: MoS can incorporate constraints and information from

theory: shape restrictions.

Advantage: MoS can simultaneously estimate parametric and

nonparametric (NP) parts with optimal convergence for both.

Disadvantage: General theory for MoS not complete.
Chapter describes estimation of S-NP models via MoS
Will present general results for large sample properties: consistency,

convergence rates, pointwise normality, some

normality

Will Matcham

(LSE)

Chapter 76: Chen

n asymptotic

October 2015

4 / 27

Introduction

mention S-NP, NP and P, and notation used.

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

5 / 27

Intro to Section 2

MoS consists of two key ingredients

1. Criterion Function: population Q : 7 R (a function); empirical
criterion Qn (a random function)
2. Sieve Parameter Spaces Sequence of approximating spaces
{n }nN
Both can be extremely flexible as we shall see. Almost all criterion

functions in Newey McFadden chapter can be used in MoS

Hence, main new ingredient is choice of sieve parameter space.

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

6 / 27

Empirical Examples of S-NP Models

Impossible to list all existing S-NP models and their empirical

applications. Section presents three, I present the first here.

Example 2.1 (Single spell duration models with unobserved
heterogeneity)
Typical single spell models suggest functional form for structural
duration distribution conditional upon individual heterogeneity. Let
G ( |u, x) be distribution function of duration T conditional upon
unobserved and observed heterogeneity U = u and X = x respectively.
Then modelling U as a random factor with distribution function h(u),
obtain
Z
F ( |x) = G ( |u, x)dh(u)
iid sample {Ti , Xi }ni=1 identifies F .

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

7 / 27

Empirical Examples of S-NP Models

Theoretical models provide parametric functional forms of G up to

finite-dimensional parameter vector.

g (|, u, x) density counterpart to G (|, u, x)
MLE method assumes h known up to finite dimensional
Then MLE gives likelihood
n Z
Y
g (Ti |, u, Xi )dh (u)
i=1

Thus log likelihood scaled is

n

1X
L(, ) =
log
n

Z


g (Ti |, u, Xi )dh (u)

i=1

And

(MLE , MLE )0 = argmaxL(, )

,
Will Matcham

(LSE)

Chapter 76: Chen

October 2015

8 / 27

Empirical Examples of S-NP Models

Heckman and Singer (1984) observe that parametric MLE estimates

of inconsistent if distribution of unobserved heterogeneity h

misspecified.
They suggest S-NP single spell model
Z
F ( |, h, x) = G ( |, u, x)dh(u)
h left unspecified. ( 0 , h) is identified and a sieve MLE method gives

consistent estimator for and h jointly.

Classic example of S-NP model specifying conditional distribution of
observed economic variables semi-nonparametrically, with specific
semi-nonparametric form derived from independence of errors and
regressors.

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

9 / 27

S-NP Conditional Moment Models

Many economic models imply semi-nonparametric conditional

moment restrictions of the form


E [(Zt ; 0 |Xt ] = 0,

0 =

0
h0

1. column vector of residual functions with functional forms known up

to
2. {Zt }nt=1 = {(Yt0 , Xt0 )0 }nt=1 data, where Yt endogenous, Xt exogenous.
3. Worth noting that E [(Zt , )|Xt ] denotes conditional expectation of
(Zt , ) given Xt . True conditional distribution of Yt given Xt left
unspecified.
Parameters of interest 0 = (00 , h00 )0 is split into vector of finite

dimensional unknown parameters 0 and a vector of -dimensional

functions h0 () = (h01 (), . . . , h0q ())0 , which can depend on anything
in the model.
Will Matcham

(LSE)

Chapter 76: Chen

October 2015

10 / 27

S-NP Conditional Moment Models

Hansen (1982) studied conditional moment restriction for stationary

ergodic time series without h0 , i.e. E [(Zt ; 0 |Xt ]

Ai and Chen (2003) and others studied for iid data the general case
E [(Zt ; 0 , h0 |Xt ]
Partition S-NP conditional moment restriction models into two
subclasses:
1. Models without endogeneity : (Zt , ) (Zt , 0 ) doesnt depend on
Yt . In such a case, 0 is the unique maximiser of


0
1
Q() = E (Zt , ) (Xt ) (Zt , )
Where (Xt ) is pd weight matrix
2. Models with endogeneity : negation of above. Then 0 identified as
unique maximiser of


0
1
Q() = E m(Xt , ) (Xt ) m(Xt , )
Where m(Xt , ) = E [(Zt , )|Xt ]

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

11 / 27

S-NP Conditional Moment Models

Although second class includes first class as a special case (trivially),

when contains unknown functions, asymptotic properties for various

nonparametric estimators of are easier to derive in the first case.
First class contains many well studied special cases, such as the

partially linear regression model of Robinson

E [Yi X1i0 0 h0 (X2i )|X1i , X2i ] = 0
The leading, yet difficult example of the second class is the purely

nonparametric instrumental variables regression

E [Y1i h0 (Y2i )|Xi ] = 0.
Even less trivial, the NP IV quantile regression

E [1 (Y1i h0 (Y2i ) ) |Xi ] = 0.

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

12 / 27

General Setup
Let be infinite dimensional parameter space, endowed with pseudo

metric d.1
Typical S-NP econometric model specifies population criterion

Q : 7 R uniquely maximised at 0 . 0 true parameter value.

Choice of Q and existence of are suggested by identification of

econometric model.
True 0 unknown but related to joint probability measure

P0 (z1 , . . . , zn ) from which sample {Zt }nt=1 is available.

n : 7 R is the empirical criterion. For all , Qn is a
Q
measurable function of the data. Qn is a random function.
n converges to Q in some sense as n .
Q
1

Pseudo metric space (X , d), d : X X 7 R with symmetry and triangle inequality

but only d(x, y ) 0, not that x = y d(x, y ) = 0.
Will Matcham

(LSE)

Chapter 76: Chen

October 2015

13 / 27

General Setup
n over . Assuming it exists,
Generally estimate 0 by maximising Q
the maximiser argsupQn () is called the extremum estimate.

When infinite dimensional and not compact with respect to d,

maximising Qn over may not be well defined, or even if it exists,

may be difficult to compute and have undesirable large sample
properties.
Difficulties arise intuitively because problem of optimisation over

infinite dimensional noncompact space is not well posed.

In dimensional metric space (H, d), compact set is d-closed and

totally bounded. Set is totally bounded if > 0, exists finitely many

open balls of radius covering the set.

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

14 / 27

Ill-Posed and Well-Posed Problems

Optimisation problem well posed if {k } in such that

Q(0 ) Q(k ) 0, then d(0 , k 0.

Naturally then problem ill posed if {k } in where

Q(0 ) Q(k ) 0 yet d(0 , k ) 6 0.]

For a given S-NP model, suppose that Q and are such that Q is

uniquely maximised at 0 . Then posedness of the problem

depends on choice of d. Different metrics on dimensional may
not be equivalent. In finite dimensional space, all norms are
equivalent.
In particular, likely that standard norms k ks on dont have

continuity in Q(0 ) Q(). This implies that problem is ill-posed

with s metric. Nevertheless, typically a weaker norm k kw on is
continuous, hence problem well-posed using this norm.

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

15 / 27

Ill-Posed and Well-Posed Problems

No matter whether ill or well-posed, method of sieves provides a

general approach to resolve difficulties with maximising Qn over

dimensional parameter space by maximising Qn over a series of
approximating spaces n called sieves, which are less complex, but
dense in
Sieves are typically compact, nondecreasing and such that ,
n () n such that d (, n ()) 0 as n . Think that n is
a projection mapping from to n .
Approximate sieve extremum estimate n is defined as the
approximate maximiser of Qn over n , i.e.
Qn (n ) sup Qn () Op (n )

n = o(1).
When n = 0, we have exact sieve extremum estimator
n = Qn ()
Will Matcham

(LSE)

n
Chapter 76: Chen

October 2015

16 / 27

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

17 / 27

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

18 / 27

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

19 / 27

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

20 / 27

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

21 / 27

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

22 / 27

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

23 / 27

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

24 / 27

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

25 / 27

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

26 / 27

Will Matcham

(LSE)

Chapter 76: Chen

October 2015

27 / 27