Lecture Notes 7: Dynamic Optimization

Part 1: Calculus of Variations

Peter J. Hammond

Autumn 2013, revised 2014

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 1 of 21

Outline

Introduction
VickreyMirrlees Model
Typical Problem
Economic Application

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 2 of 21

VickreyMirrlees Model
Problem: how much to pay workers of different skills.
Goal: achieve fairness while preserving incentives.
References: William S. Vickrey (1945)
Measuring Marginal Utility by Reactions to Risk
Econometrica 13: 319333.
James A. Mirrlees (1971)
An Exploration in the Theory of Optimal Income Taxation
Review of Economic Studies 38: 175208.
Let n R+ denote a persons skill level, defined to mean
that there is a constant rate of marginal substitution
of n1 /n2 between hours of work supplied by workers
of the two skill levels n1 and n2 .
Thus, a workers productivity is proportional to n, personal skill.
Assume that the distribution of workers skills can be described
by a continuous density function R+ 3 n 7 f (n) R R+

which, like a probability density function, satisfies 0 f (n)dn = 1.
University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 3 of 21
Objective and Constraints

Macro model with a representative consumer/worker

whose preferences for consumption/labour supply pairs (c, `) R2+
are represented by the utility function u(c) v (`),
where u 0 > 0, v 0 > 0, u 00 < 0 v 00 > 0.
The social objectiveR is to maximize
the utility integral 0 [u(c(n)) v (`(n))]f (n)dn
w.r.t. the functions R+ 3 n 7 (c(n), `(n)) R2+ .
The resource balance constraint takes the form C F (L) where
R
I C :=
R0 c(n)f (n)dn is mean consumption;
I L :=
0 n`(n)f (n)dn is mean effective labour supply.
The aggregate production function R+ 3 L 7 F (L) R+
is assumed to satisfy F 0 (L) > 0 and F 00 (L) 0 for all L 0.

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 4 of 21

Pseudo First-Order Conditions
Consider the Lagrangian
Z
L(c(), `()) := [u(c(n)) v (`(n))]f (n)dn
0
Z Z 
c(n)f (n)dn F n`(n)f (n)dn
0 0

as a functional (rather than a mere function)

of the functions R+ 3 n 7 (c(n), `(n)) R2+ .
We derive pseudo first-order conditions by pretending
L L
that the derivatives and both exist, for all n 0.
c(n) `(n)
This gives the pseudo first-order conditions
L
0 = = [u 0 (c(n)) ]f (n)
c(n)
L
0 = = v 0 (`(n))f (n) + F 0 (L)nf (n)
`(n)
University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 5 of 21
Marxist First Best
For any skill level n such that f (n) > 0, these two equations

0 = [u 0 (c(n)) ]f (n) and v 0 (`(n))f (n) + F 0 (L)nf (n)

imply that:
I u 0 (c(n)) = and so c(n) = c ,
where the constant c uniquely solves u 0 (c ) =
(to each according to their need);
d`
I v 0 (`(n)) = F 0 (L)n, implying that v 00 (`(n)) = F 0 > 0,
dn
d`
so > 0 (from each according to their ability)
dn
Exercise
Use concavity arguments to prove
that this is the (essentially unique) solution.
What makes this solution practically infeasible?

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 6 of 21

Sufficiency Theorem: Statement
Theorem
Suppose that there exists > 0
such that c and the function R+ 3 n 7 ` (n)
jointly satisfy the first-order conditions:

u 0 (c ) = andv 0 (` (n)) = F 0 (L )n for all n R+

R
where c = F (L ) and L = 0 n` (n)f (n) dn.
Let R+ 3 n 7 (c(n), `(n)) R2+
be any otherR policy satisfying C = F (L)
R
where C = 0 c(n)f (n) dn and L = 0 n`(n)f (n) dn.
Then
Z Z

[u(c(n)) v (`(n))]f (n)dn < u(c ) v (` (n))f (n)dn
0 0

with strict inequality unless c(n) = c wherever f (n) > 0.

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 7 of 21
Sufficiency Theorem: Proof, I
Because u 00 < 0 and so u is strictly concave, for all n one has

u(c(n)) u(c ) u 0 (c )[c(n) c ] = [c(n) c ]

with strict inequality unless c(n) = c , and so integrating gives

Z
[u(c(n)) u(c )]f (n) dn (C c )
0

with strict inequality unless c(n) = c wherever f (n) > 0.

Similarly, because v 00 0 and so v is convex, for all n one has

v (`(n))v (` (n)) v 0 (` (n))[`(n)` (n)] = F 0 (L )[`(n)` (n)]

and so integrating gives

Z
[v (`(n)) v (` (n))]f (n) dn F 0 (L )(L L )
0

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 8 of 21

Sufficiency Theorem: Proof, II

It follows that
R
0 {[u(c(n)) v (`(n))] [u(c ) v (` (n))]}f (n) dn
[(C c ) F 0 (L )(L L )]

Finally, because F 00 0 and so F is concave, one has

C c = F (L) F (L ) F 0 (L )(L L )

Because > 0, it follows that

Z Z
[u(c(n)) v (`(n))]f (n) dn [u(c ) v (` (n))]f (n) dn
0 0

as required for R+ 3 n 7 (c , ` (n)) R2+ to be optimal.

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 9 of 21

Outline

Introduction
VickreyMirrlees Model
Typical Problem
Economic Application

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 10 of 21

Problem Formulation
The calculus of variations is used to optimize a functional
that maps functions into real numbers.
A typical problem is to choose a function [t0 , t1 ] 3 t 7 x(t) R,
often denoted simply by x,
in order to maximize the integral objective functional
Z t1
J(x) = F (t, x(t), x(t))dt
t0

subject to the fixed end point conditions x(t0 ) = x0 , x(t1 ) = x1 .

A variation involves moving away from the first path x
to the variant path x + u,
where u denotes the differentiable function [t0 , t1 ] 3 t 7 u(t) R,
and  R is a scalar.
To ensure that the end point conditions x(t0 ) + u(t0 ) = x0
and x(t1 ) + u(t1 ) = x1 remain satisfied by x + u,
one imposes the conditions u(t0 ) = u(t1 ) = 0.
University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 11 of 21
Toward a Necessary First-Order Condition
A maximum is a path x satisfying the end point conditions
such that J(x ) J(x) for any alternative path x
that also the satisfies the end point conditions.
A necessary condition for x to maximize J(x) w.r.t. x
is that J(x ) J(x + u) for all small .
Alternatively, the function

R 3  7 fx ,u () := J(x + u)

must satisfy, for all small , the inequality

fx ,u (0) = J(x ) J(x + u) = fx ,u ()

In case the function  7 fx ,u () is differentiable at  = 0,

a necessary first-order condition is therefore fx0 ,u (0) = 0.

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 12 of 21

Evaluating the Derivative

Our definitions of the functions J and fx ,u imply that

Z t1

fx ,u () = J(x + u) = F (t, x (t) + u(t), x (t) + u(t))dt
t0

Differentiating the integrand w.r.t.  implies that

Z t1
fx0 ,u (0) = [Fx0 (t)u(t) + Fx0 (t)u(t)]dt
t0

where for each t [t0 , t1 ], the partial derivatives Fx0 (t) and Fx0 (t)
of F (t, x, x) are evaluated at the triple (t, x (t), x (t)).

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 13 of 21

Integrating by Parts

The product rule for differentiation implies that

 
d 0 d 0
[F (t)u(t)] = F (t) u(t) + Fx0 (t)u(t)
dt x dt x

and so, integrating by parts, one has

Z t1 Z t1  
0 t1 0 d 0
Fx (t)u(t)dt = |t0 Fx (t)u(t) F (t) u(t)dt
t0 t0 dt x

But the end point conditions imply that u(t0 ) = u(t1 ) = 0,

so the first term on the right-hand side vanishes.

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 14 of 21

The Euler Equation
Rt d 0  Rt
Substituting t01 dt Fx (t) u(t)dt for the term t01 Fx0 (t)u(t)dt
Rt
in the equation fx0 ,u (0) = t01 [Fx0 (t)u(t) + Fx0 (t)u(t)]dt,
then recognizing the common factor u(t), we finally obtain
Z t1  
0 0 d 0
fx ,u (0) = Fx (t) Fx (t) u(t)dt
t0 dt

The first-order condition is fx0 ,u (0) = 0

for every differentiable function t 7 u(t)
satisfying the two end point conditions u(t0 ) = u(t1 ) = 0.
This condition holds
iff the integrand is zero for (almost) all t [t0 , t1 ],
d 0
which is true iff the Euler equation dt Fx (t) = Fx0 (t)
holds for (almost) all t [t0 , t1 ].

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 15 of 21

Outline

Introduction
VickreyMirrlees Model
Typical Problem
Economic Application

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 16 of 21

Are We Saving Too Little?

Kenneth Arrow, Gretchen Daily, Partha Dasgupta, Paul Ehrlich,

Lawrence Goulder, Geoffrey Heal, Simon Levin, Karl-Goran Maler,
Stephen Schneider, David Starrett and Brian Walker (2004)
Are We Consuming Too Much?
Journal of Economic Perspectives 18: 147172.
Macroeconomic variation of the SolowSwan growth model.
Given a capital stock K , output Y is given by
the production function Y = f (K ), where f 0 > 0, and f 00 0.
Net investment = gross investment, without depreciation.
So given capital K and consumption C , investment I is given by

I = K = f (K ) C

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 17 of 21

The Ramsey Problem and Beyond
The economys intertemporal objective is taken to be
Z T Z T
e rt U(C (t))dt = e rt U(f (K ) K )dt
0 0

Frank Ramsey (1928) assumed T = (infinite horizon)

and r = 0 (no discounting).
Nicholas Stern (of the Stern Report on Climate Change)
and others take:
I T = ;
I r as the hazard rate in a Poisson process
that determines when extinction occurs;
this implies that e rt is the probability
that the human race has not become extinct
at or before time t.

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 18 of 21

Applying the Calculus of Variations
We apply the calculus
R T rtof variations
to the objective 0 e U(f (K ) K )dt
with the end conditions K (0) = K , which is exogenous,
and K (T ) = 0 at the finite time horizon T .
d 0
Eulers equation takes the form dt FK (t) = FK0 (t)
where F (t, K , K ) = e rt U(f (K ) K ) = e rt U(C ).
So Eulers equation becomes d rt U 0 (C )] = e rt U 0 (C )f 0 (K ).
dt [e
Equivalently, after evaluating the time derivative,

U 00 (C )C e rt + rU 0 (C )e rt = e rt U 0 (C )f 0 (K )

Cancelling the common factor e rt and dividing by U 0 (C ) > 0,

then rearranging, one obtains

U 00 (C )
C = f 0 (K ) r
U 0 (C )

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 19 of 21

Further Interpretation

Define the (negative) elasticity of marginal utility as

d ln U 0 (C ) U 00 (C )C
(C ) := = 0
d ln C U (C )
This is related to the curvature of the utility function,
and to how quickly marginal utility U 0 (C ) decreases as C increases.
Rearranging the equation U 00 (C )C /U 0 (C ) = f 0 (K ) r yet again,
one obtains the equation

C
(C ) = f 0 (K ) r
C
whose left hand side is the proportional rate of consumption growth
multiplied by the elasticity of marginal utility,
or the elasticity of an intertemporal marginal rate of substitution.

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 20 of 21

Final Recommendation

Morton I. Kamien and Nancy L. Schwartz (2012)

Dynamic Optimization, Second Edition:
The Calculus of Variations and Optimal Control
in Economics and Management (Dover Publications)

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 21 of 21