Lagrange

Multipliers

Martin J. Gander

Mechanics
Archimedes
From Lagrangian Mechanics to Optimal Varignon
Bernoulli

Control and PDE Constraints Bernoulli’s Rule

Lever
Double Lever
Varignon example

Lagrange
Martin J. Gander Multiplier
martin.gander@unige.ch Constraints
Multiplier Method
Optimization
Optimal Control
University of Geneva Hamiltonian
Maximum Principle
Pontryagin

Joint work with Felix Kwok and Gerhard Wanner Adjoint

PDE Constraint
Optimization
Lions
MORE workshop, Prague, September 2014 Adjoint

Conclusion
 Lagrange
Archimedes’ Law of the Lever Multipliers

Martin J. Gander
Archimedes (287-212 BC): “Two bodies are in equilibrium
Mechanics
if their weights are inversely proportional to their arm length” Archimedes
Varignon
Bernoulli

Bernoulli’s Rule
Lever
Double Lever
Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control
Hamiltonian
Maximum Principle
Pontryagin

Adjoint
PDE Constraint
Optimization
Lions
Adjoint

Conclusion
 Lagrange
Archimedes’ Law of the Lever Multipliers

Martin J. Gander
Archimedes (287-212 BC): “Two bodies are in equilibrium
Mechanics
if their weights are inversely proportional to their arm length” Archimedes
Varignon
Bernoulli

Bernoulli’s Rule
Lever
Double Lever
Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control

Beautiful proof of Archimedes: Hamiltonian

Maximum Principle
Pontryagin
1) Axiom: equal weights at equal distances are in equilibrium Adjoint
PDE Constraint
Optimization
Lions
Adjoint

Conclusion
2) Redistribute weights to reach symmetric configuration Lagrange
Multipliers

Martin J. Gander

Mechanics
Archimedes
Varignon
Bernoulli

Bernoulli’s Rule
Lever
Double Lever
Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control
Hamiltonian
Maximum Principle
Pontryagin

Adjoint
PDE Constraint
Optimization
Lions
Adjoint

Conclusion
 Lagrange
Pierre Varignon Multipliers

Varignon (1654-1722): Nouvelle Mécanique (frontispiece Martin J. Gander

“dont le projet fut donné en M.DC.LXXXVII”) published Mechanics

Archimedes
posthumously in 1725 Varignon
Bernoulli

Bernoulli’s Rule
Hundreds of results illustrated in 64 plates of figures: Lever
Double Lever
Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control
Hamiltonian
Maximum Principle
Pontryagin

Adjoint
PDE Constraint
Optimization
Lions
Adjoint

Conclusion
 Lagrange
Pierre Varignon Multipliers

Martin J. Gander

Mechanics
Archimedes
Varignon
Bernoulli

Bernoulli’s Rule
Lever
Double Lever
Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control
Hamiltonian
Maximum Principle
Pontryagin

Adjoint
PDE Constraint
Optimization
Lions
Adjoint

Conclusion
 Lagrange
Johann Bernoulli Multipliers

Johann Bernoulli (1667-1748): Invents a general rule Martin J. Gander

(“règle”) and announces it to Mr. le Chev. Renau (Bernard Mechanics

Renau d’Eliçagaray, “le petit Renau”) in a letter, with copy Archimedes

Varignon

to Varignon: Bernoulli

Bernoulli’s Rule
“Votre projet d’une nouvelle mechanique fourmille d’un Lever
Double Lever
grand nombre d’exemples, dont quelques uns à en juger par Varignon example

les figures paroissent assez compliqués; mais je vous deffie de Lagrange

Multiplier
m’en proposer un à votre choix, que je ne resolve sur le Constraints
Multiplier Method
champ et comme en jouant par ma dite règle.” Optimization
Optimal Control
Hamiltonian
“Your project of a new theory of mechanics is Maximum Principle
Pontryagin
swarming of examples, some of which, judging from Adjoint
the figures, appear to be quite complicated; but I PDE Constraint
Optimization

challenge you to propose any one of them to me, Lions

Adjoint

and I solve it on the spot with my so-called rule” Conclusion

Varignon had difficulties admitting that all his work over

decades was declared to be so easy and contested the
general truth of this rule.
 Lagrange
Bernoulli’s Rule in Varignon’s Book Multipliers

Martin J. Gander

Mechanics
Archimedes
Varignon
Bernoulli

Bernoulli’s Rule
Lever
Double Lever
Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control
Hamiltonian
Maximum Principle
Pontryagin
“In every equilibrium of arbitrary forces, no matter how they
Adjoint
are applied, and in which directions they act the ones on the PDE Constraint
Optimization
others, either indirectly or directly, the sum of the positive Lions
Adjoint
energies will be equal to the sum of the negative energies Conclusion
taken positively.”
Varignon gives however the wrong date 1717 for the letter of Bernoulli,
which was later copied by Lagrange.
 Lagrange
Lagrange Explains Bernoulli’s Rule Multipliers

Joseph-Louis Lagrange (1736-1813): “Mécanique Martin J. Gander

analytique” from 1788 Mechanics

Archimedes
Varignon
Bernoulli

Bernoulli’s Rule
Lever
Double Lever
Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control
Assume the lever is in equilibrium: Archimedes implies Hamiltonian
Maximum Principle
Pontryagin
P b
Pa = Qb =⇒ = Adjoint
Q a PDE Constraint
Optimization
Lions
Bernoulli’s idea: apply a “virtual velocity” during an Adjoint

Conclusion
infinitely small time interval to displace the lever. Then
P dq
a ∝ dp and b ∝ dq =⇒ =− =⇒ Pdp + Qdq = 0
Q dp
 Lagrange
So Forget Archimedes ! Multipliers

Applying the virtual velocity principle, Martin J. Gander

Mechanics
Archimedes
Varignon
Bernoulli

Bernoulli’s Rule
Lever
Double Lever
Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
we get Optimization
Optimal Control
Pdp + Qdq = 0. Hamiltonian
Maximum Principle
Pontryagin
Now if dp = dx for an arbitrary dx, then dq = − ba dx, and Adjoint
hence PDE Constraint
Optimization
  Lions
b Adjoint

Pdp + Qdq = 0 =⇒ P −Q dx = 0 ∀dx Conclusion

a
P b
=⇒ = (Archimedes)
Q a
 Lagrange
A More Complicated System Multipliers

Martin J. Gander

Mechanics
Archimedes
Varignon
Bernoulli

Bernoulli’s Rule
Lever
Double Lever
Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
Optimization
Applying the rule of Bernoulli, we get Optimal Control
Hamiltonian
Maximum Principle
Pontryagin
Pdp + Qdq + Rdr = 0 (∗) Adjoint
PDE Constraint

If dp = dx, then dq = − ba dx, and dr = − dc dq = bd

Optimization

ac dx, and Lions

Adjoint
  Conclusion
b bd
(∗) =⇒ P − Q + R dx = 0 ∀dx
a ac
 Lagrange
A System of Varignon Style Multipliers

Martin J. Gander

To use Mechanics
Archimedes
Varignon
Bernoulli
Pdp + Qdq + Rdr = 0 (∗)
Bernoulli’s Rule
Lever

we compute Double Lever

Varignon example

q Lagrange
Multiplier
p = (x −a)2 + (y −b)2 + (z −c)2 Constraints
Multiplier Method
Optimization
Optimal Control
1 Hamiltonian
dp = ·((x−a)dx+(y −b)dy +(z−c)dz) Maximum Principle
p Pontryagin

Adjoint
and similarly q, dq, and r , dr . PDE Constraint
Optimization
Inserted into (∗) we get for equilibrium Lions
Adjoint

Xdx + Ydy + Zdz = 0. Conclusion

y −b y −g y −m
with X = P x−a x−f x−l
p +Q q +R r , Y =P p +Q q +R r
and Z = P z−c z−h z−n
p +Q q +R r .
 Lagrange
Same System with Constraint Multipliers

Martin J. Gander

Mechanics
Motion (dx, dy , dz) Archimedes
Varignon

restricted to a surface L = 0: Bernoulli

Bernoulli’s Rule
∂L ∂L ∂L Lever

dL = dx+ dy + dz = 0 (1) Double Lever

Varignon example
∂x ∂y ∂z
Lagrange
Multiplier
together with the equation Constraints
Multiplier Method
Optimization
Optimal Control
Xdx + Ydy + Zdz = 0 (2) Hamiltonian
Maximum Principle
Pontryagin

Multiply (1) by λ = −Z / ∂L
∂z and add Adjoint

to (2) to obtain PDE Constraint

Optimization
Lions
  Adjoint

X + λ ∂L ∂L
Z + λ ∂L


∂x · dx + Y + λ ∂y · dy = 0 , ∂z = 0 . Conclusion

This means applying the virtual velocity argument, without

constraints, to Xdx + Ydy + Zdz + λdL = 0
“Il n’est pas difficile de prouver par la théorie de l’élimination des équations linéaires...”
 Lagrange
The Multiplier Method Multipliers

Martin J. Gander
Physical interpretation of λ by Lagrange:
Mechanics
λ( ∂L ∂L ∂L
∂x , ∂y , ∂z ) represents the force that holds the particle on Archimedes
Varignon
the surface L = 0 Bernoulli

Bernoulli’s Rule
Lever
Double Lever
“Équation générale” for ALL problems of equilibria: Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control
◮ “Méthode très-simple” in Section IV of the first edition Hamiltonian
Maximum Principle
from 1788. Pontryagin

Adjoint
◮ In the second edition from 1811, Lagrange baptizes the PDE Constraint
Optimization
method Lions
Adjoint

Conclusion
 Lagrange
Lagrange Multiplier in Optimization Multipliers

Lagrange, Théorie de Fonctions Analytiques (1797): “Théorie des Martin J. Gander

fonctions analytiques, contenant les principes du calcul différentiel, Mechanics

Archimedes
dégagés de toute considération d’infiniment petits, d’évanouissans, de Varignon
Bernoulli
limites ou de fluxions, et réduits à l’analyse algébrique des quantités
Bernoulli’s Rule
finies” Lever
Double Lever

Constrained optimization problem: Varignon example

Lagrange
Multiplier
f (x) −→ max, g(x) = 0, Constraints
Multiplier Method
Optimization
with f : Rn → R and g : Rn → Rm , m < n. Optimal Control
Hamiltonian
Maximum Principle
Elimination of the constraints: Pontryagin

Adjoint
x = (y, u), y ∈ Rm , u ∈ Rn−m PDE Constraint
Optimization
Lions
Adjoint
g(x) = 0 and implicit function theorem =⇒ y = y(u)
Conclusion

Unconstrained optimization problem:

f (y(u), u) −→ max .
 Lagrange
Lagrange Multiplier in Optimization Multipliers

Martin J. Gander
Necessary condition for a local maximum of f (y(u), u):
Mechanics
Archimedes
df ∂f ∂y ∂f Varignon
= + =0 Bernoulli
du ∂y ∂u ∂u Bernoulli’s Rule
Lever
Double Lever
∂y
For ∂u , implicitly differentiate constraint g(y(u), u) = 0 Varignon example

Lagrange
Multiplier
∂f ∂y ∂f Constraints
+ = 0, n − m equations Multiplier Method
∂y ∂u ∂u Optimization
Optimal Control
∂g ∂y ∂g Hamiltonian
+ = 0, m(n − m) equations Maximum Principle
∂y ∂u ∂u Pontryagin

Adjoint
g = 0, m equations PDE Constraint
Optimization
Lions
n + m(n − m) equations for the n unknowns in y and u Adjoint

∂y Conclusion
combined, and the m(n − m) unknowns in the Jacobian ∂u

∂y
=⇒ A very big system because of ∂u .
 Lagrange
Gaussian Elimination of Lagrange Multipliers

Martin J. Gander

∂f ∂y ∂f Mechanics
+ = 0, n − m equations Archimedes
∂y ∂u ∂u Varignon
Bernoulli
∂g ∂y ∂g
+ = 0, m(n − m) equations Bernoulli’s Rule
∂y ∂u ∂u Lever
Double Lever
g = 0, m equations Varignon example

 −1 Lagrange
∂g Multiplier
Multiply 2nd equation by λ := − ∂f
∂y ∂y and add to 1st: Constraints
Multiplier Method
Optimization
∂f ∂g Optimal Control
+λ = 0, n − m equations Hamiltonian
∂u ∂u Maximum Principle
Pontryagin
∂f ∂g
+λ = 0, m equations Adjoint
∂y ∂y PDE Constraint
Optimization
Lions
g = 0, m equations Adjoint

Conclusion
for n unknowns in y and u combined, plus m in λ.
Lagrange: get this directly by differentiating
L(u, y, λ) := f (y, u) + λg (y, u)
 Lagrange
Hestenes’ Optimal Control Problem 1950 Multipliers

Martin J. Gander

Mechanics
Archimedes
Varignon
Bernoulli

Bernoulli’s Rule
Lever
Z T Double Lever
Varignon example
f (y, u)dt −→ max, Lagrange
0 Multiplier
ẏ = g(y, u), Constraints
Multiplier Method

y(0) = y0 , Optimization
Optimal Control
Hamiltonian
y(T ) = yT . Maximum Principle
Pontryagin

Introduce the Lagrangian Adjoint

PDE Constraint
Z T Optimization
Z T Lions
L(y, u, λ) := f (y, u)dt + λ (g(y, u) − ẏ) dt, Adjoint

0 0 Conclusion

where all the variables depend on time.

Compute the derivatives with respect to the variables y, u,
and λ using variational calculus (as Euler did in E420)!
 Lagrange
Variational Derivatives Multipliers

Computing a derivative with respect to y of Martin J. Gander

Z T Z T Mechanics
Archimedes
L(y, u, λ) := f (y, u)dt + λ (g(y, u) − ẏ) dt, Varignon
Bernoulli
0 0
Bernoulli’s Rule
we obtain Lever
Double Lever
  Varignon example
d
RT ∂f
RT ∂g
dε L(y + εz, u, λ)|ε=0 = 0 zdt + λ z − ż dt Lagrange

RT ∂y 0 ∂y Multiplier

∂f ∂g T Constraints
= 0 ∂y + λ ∂y + λ̇ z − λ · z|0 = 0 Multiplier Method
Optimization
Optimal Control
Hamiltonian
Since z(0) = z(T ) = 0, but z(t) arbitrary, we get (and with Maximum Principle
Pontryagin
similar derivatives w.r.t. u and λ) Adjoint
PDE Constraint
∂f ∂g Optimization

0 = +λ Lions

∂u ∂u Adjoint

Conclusion
ẏ = g(y, u) y(0) = y0 , y(T ) = yT
∂f ∂g
−λ̇ = +λ
∂y ∂y
 Lagrange
Hamiltonian Structure Multipliers

Defining the Hamiltonian function Martin J. Gander

Mechanics
H(y, u, λ) := f (y, u) + λg(y, u), Archimedes
Varignon
Bernoulli

we see that the first order optimality system is Bernoulli’s Rule

Lever
Double Lever
∂H Varignon example
ẏ = = g(y, u),
∂λ Lagrange
∂H ∂f ∂g Multiplier
λ̇ = − = − −λ , Constraints
∂y ∂y ∂y Multiplier Method
Optimization
Optimal Control

a Hamiltonian system, H is preserved along optimal Hamiltonian

Maximum Principle

trajectories! Pontryagin

Adjoint
PDE Constraint
Optimization
This was already discovered by Carathéodory 1926: Lions
Adjoint

Conclusion
 Lagrange
Necessary Condition from the Hamiltonian Multipliers

Instead of maximizing the Lagrangian Martin J. Gander

Z T Z T Mechanics

L(y, u, λ) = f (y, u)dt + λ (g(y, u) − ẏ) dt Archimedes

Varignon
0 0 Bernoulli

Bernoulli’s Rule
which gives along an optimal trajectory ẏ = g(y, u) Lever
Double Lever
Z T Varignon example

f (y, u)dt −→ max in u(t) as before, Lagrange

Multiplier
0
Constraints

one could also maximize the Hamiltonian Multiplier Method

Optimization
Optimal Control
Hamiltonian
H(y, u, λ) := f (y, u) + λg(y, u), Maximum Principle
Pontryagin

pointwise for each t ∈ [0, T ] (Hestenes’ RAND report 1950) Adjoint

PDE Constraint
Optimization
H(y, u, λ) −→ max with respect to u(t), Lions
Adjoint
∂H Conclusion
ẏ =
∂λ
∂H
λ̇ = − .
∂y
 Lagrange
Discovery of Pontryagin Multipliers

Boltyanski, Gamkrelidze, and Pontryagin (1956): On Martin J. Gander

the theory of optimal processes (in Russian) Mechanics

Archimedes
Varignon
Bernoulli

Bernoulli’s Rule
Lever
Double Lever
Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control
Hamiltonian
Maximum Principle
Pontryagin

Adjoint
PDE Constraint
Optimization
Lions
Adjoint

Conclusion
 Lagrange
Discovery of Pontryagin Multipliers

Boltyanski, Gamkrelidze, and Pontryagin (1956): On Martin J. Gander

the theory of optimal processes (in Russian) Mechanics

◮ Controls are often constraint, e.g. −1 ≤ ui ≤ 1 Archimedes
Varignon
Bernoulli

Bernoulli’s Rule
Lever
Double Lever
Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control
Hamiltonian
Maximum Principle
Pontryagin

Adjoint
PDE Constraint
Optimization
Lions
Adjoint

Conclusion
 Lagrange
Discovery of Pontryagin Multipliers

Boltyanski, Gamkrelidze, and Pontryagin (1956): On Martin J. Gander

the theory of optimal processes (in Russian) Mechanics

◮ Controls are often constraint, e.g. −1 ≤ ui ≤ 1 Archimedes
Varignon
◮ Solution often on the boundary (“bang-bang”) Bernoulli

Bernoulli’s Rule
◮ ÿ = ±M (Feldbaum 1949) Lever
◮ ÿ = ±u, |u| ≤ M (Feldbaum 1953) Double Lever
Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control
Hamiltonian
Maximum Principle
Pontryagin

Adjoint
PDE Constraint
Optimization
Lions
Adjoint

Conclusion
 Lagrange
Discovery of Pontryagin Multipliers

Boltyanski, Gamkrelidze, and Pontryagin (1956): On Martin J. Gander

the theory of optimal processes (in Russian) Mechanics

◮ Controls are often constraint, e.g. −1 ≤ ui ≤ 1 Archimedes
Varignon
◮ Solution often on the boundary (“bang-bang”) Bernoulli

Bernoulli’s Rule
◮ ÿ = ±M (Feldbaum 1949) Lever
◮ ÿ = ±u, |u| ≤ M (Feldbaum 1953) Double Lever
Varignon example
◮ Pontryagin discovers the Hamiltonian formulation Lagrange
without Lagrange multipliers Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control
Hamiltonian
Maximum Principle
(Gamkrelidze (1999) “Discovery of the Maximum Principle”) Pontryagin

Adjoint
PDE Constraint
Optimization
Lions
Adjoint

Conclusion
 Lagrange
Discovery of Pontryagin Multipliers

Boltyanski, Gamkrelidze, and Pontryagin (1956): On Martin J. Gander

the theory of optimal processes (in Russian) Mechanics

◮ Controls are often constraint, e.g. −1 ≤ ui ≤ 1 Archimedes
Varignon
◮ Solution often on the boundary (“bang-bang”) Bernoulli

Bernoulli’s Rule
◮ ÿ = ±M (Feldbaum 1949) Lever
◮ ÿ = ±u, |u| ≤ M (Feldbaum 1953) Double Lever
Varignon example
◮ Pontryagin discovers the Hamiltonian formulation Lagrange
without Lagrange multipliers Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control
Hamiltonian
Maximum Principle
(Gamkrelidze (1999) “Discovery of the Maximum Principle”) Pontryagin

Adjoint
PDE Constraint
Optimization
Lions
Adjoint

Conclusion
 Lagrange
Historical Discovery of the Maxmimum Principle Multipliers

Martin J. Gander
Boltyanski, Gamkrelidze and Pontryagin (1956):
Mechanics
“This fact appears in many cases as a general principle, which we Archimedes
call the maximum principle” Varignon
Bernoulli

Bernoulli’s Rule
Lever
Double Lever
Varignon example

Lagrange
Multiplier
Constraints
Multiplier Method
Optimization
Optimal Control
Hamiltonian
Maximum Principle
Pontryagin

Adjoint
PDE Constraint
Optimization
Lions
Adjoint

Conclusion
Pontryagin (1959): “ In the case that Ω is an open set [. . . ],
the variational problem formulated here turns out to be a special
case of the problem of Lagrange.”
 Lagrange
Early Work with PDE Constraints Multipliers

Martin J. Gander
Egorov (1962,1963): Some problems in the theory of
optimal control, Optimal control in Banach spaces Mechanics
Archimedes
Varignon
Minimum time control problem for the parabolic equation Bernoulli

Bernoulli’s Rule
∂y
∂t + Ay + b(u)y = f +u on Ω × (0, T ) Lever
Double Lever

y = 0 on ∂Ω × (0, T ) Varignon example

Lagrange
Multiplier
with initial condition y (t0 ; u) = y0 and target y (t1 ; u) = yT . Constraints
Multiplier Method
Optimization
Optimal Control
Hamiltonian
Maximum Principle
Pontryagin

Adjoint
PDE Constraint
Optimization
Lions
Adjoint

Conclusion
 Lagrange
Early Work with PDE Constraints Multipliers

Martin J. Gander
Egorov (1962,1963): Some problems in the theory of
optimal control, Optimal control in Banach spaces Mechanics
Archimedes
Varignon
Minimum time control problem for the parabolic equation Bernoulli

Bernoulli’s Rule
∂y
∂t + Ay + b(u)y = f +u on Ω × (0, T ) Lever
Double Lever

y = 0 on ∂Ω × (0, T ) Varignon example

Lagrange
Multiplier
with initial condition y (t0 ; u) = y0 and target y (t1 ; u) = yT . Constraints
Multiplier Method
Optimization
J.-L. Lions: “Le travail de Yu. V. Egorov contient une étude Optimal Control
Hamiltonian
détaillée de ce problème, mais nous n’avons pas pu comprendre Maximum Principle
Pontryagin
tous les points des démonstrations de cet auteur, les résultats Adjoint
étant très probablement tous corrects.” PDE Constraint
Optimization
Lions
Adjoint

Conclusion
 Lagrange
Early Work with PDE Constraints Multipliers

Martin J. Gander
Egorov (1962,1963): Some problems in the theory of
optimal control, Optimal control in Banach spaces Mechanics
Archimedes
Varignon
Minimum time control problem for the parabolic equation Bernoulli

Bernoulli’s Rule
∂y
∂t + Ay + b(u)y = f +u on Ω × (0, T ) Lever
Double Lever

y = 0 on ∂Ω × (0, T ) Varignon example

Lagrange
Multiplier
with initial condition y (t0 ; u) = y0 and target y (t1 ; u) = yT . Constraints
Multiplier Method
Optimization
J.-L. Lions: “Le travail de Yu. V. Egorov contient une étude Optimal Control
Hamiltonian
détaillée de ce problème, mais nous n’avons pas pu comprendre Maximum Principle
Pontryagin
tous les points des démonstrations de cet auteur, les résultats Adjoint
étant très probablement tous corrects.” PDE Constraint
Optimization
Lions
Fattorini (1964): Time-optimal control of solutions of Adjoint

Conclusion
operational differential equations (proof of the “bang-bang”
property, no maximum principle)
Friedman (1967): Optimal control for parabolic equations
 Lagrange
Jacques-Louis Lions Multipliers

Martin J. Gander
R. M. Temam (SIAM News, July 10, 2001):
Mechanics
“A new adventure began for Lions in the early Archimedes
Varignon
1960s, when he met (in spirit) another of his Bernoulli

Bernoulli’s Rule
intellectual mentors, John von Neumann. By then, Lever
using computers built from his early designs, von Double Lever
Varignon example
Neumann was developing numerical methods for Lagrange
Multiplier
the solution of PDEs from fluid mechanics and Constraints

meteorology. At a time when the French Multiplier Method

Optimization

mathematical school was almost exclusively Optimal Control

Hamiltonian

engaged in the development of the Bourbaki Maximum Principle

Pontryagin

program, Lions — virtually alone in France — Adjoint

PDE Constraint
dreamed of an important future for mathematics in Optimization
Lions
these new directions; he threw himself into this Adjoint

new work, while still continuing to produce Conclusion

high-level theoretical work on PDEs.”

 Lagrange
Adjoint Without Lagrange Multipliers Multipliers

Lions (1968): Contrôle optimal de systèmes gouvernés par Martin J. Gander

des équations aux dérivées partielles Mechanics

Archimedes
Varignon
Bernoulli

J(u) = kCy (u) − zd k2H + (Nu, u)U , N self-adjoint, ≥ 0 Bernoulli’s Rule

Lever
Double Lever
Target zd ∈ H, state variable y = y (u) ∈ V , u ∈ Uad , a Varignon example

Lagrange
closed convex subset U, and PDE constraint Multiplier
Constraints

Ay = f + Bu, A : V → V′ Multiplier Method

Optimization
Optimal Control
Hamiltonian
J(v ) − J(u) ≥ 0 for all v ∈ Uad implies after a short Maximum Principle
Pontryagin
calculation Adjoint
PDE Constraint
(Cy (u) − zd , C (y (v ) − y (u)))H + (Nu, v − u)U ≥ 0 Optimization
Lions
Adjoint

which is equivalent to Conclusion

(C ∗ Λ(Cy (u) − zd ), y (v ) − y (u))V + (Nu, v − u)U ≥ 0

Λ : H → H ′ canonical isomorphism from H to its dual H ′

 Lagrange
A Clever Guess Multipliers

Martin J. Gander
Defining p(v ) ∈ V by A∗ p(v ) = C ∗ Λ(Cy (v ) − zd ), we get
Mechanics
Archimedes
(C ∗ Λ(Cy (u) − zd ), y (v ) − y (u))V + (Nu, v − u)U Varignon
Bernoulli

= (A∗ p(u), y (v ) − y (u))V + (Nu, v − u)U Bernoulli’s Rule

Lever
= (p(u), A(y (v ) − y (u))V + (Nu, v − u)U Double Lever
Varignon example

= (p(u), B(v − u))V + (Nu, v − u)U Lagrange

Multiplier

= (Λ−1 ∗
U B p(u) + Nu, v − u)U ≥ 0 Constraints
Multiplier Method
Optimization
Optimal Control
where B ∗ : V → U ′ is the adjoint of B, ΛU : U → U ′ is the Hamiltonian
Maximum Principle
canonical isomorphism from U to U ′ . Pontryagin

Adjoint
Hence the adjoint p(v ) permits elimination of y (u), and PDE Constraint
Optimization
Lions

(Λ−1 ∗ −1 ∗
U B p(u) + Nu, u)U = inf (ΛU B p(u) + Nu, v )U
Adjoint

v ∈Uad Conclusion

Lions (1968): “La formulation peut être considérée comme un

analogue du principe du maximum de Pontryagin”
 Lagrange
Conclusions Multipliers

Martin J. Gander

We have seen four main ideas: Mechanics

Archimedes
Varignon
1. Mechanical systems in equilibrium can easily be Bernoulli

Bernoulli’s Rule
analyzed using “virtual velocities” Lever
Double Lever
Varignon example
2. The Lagrange multiplier is just a multiplier from
Lagrange
Gaussian elimination Multiplier
Constraints
Multiplier Method
3. Using Lagrange multipliers, one can find the adjoint Optimization
Optimal Control
equation in optimal control Hamiltonian
Maximum Principle
Pontryagin

4. One can also find the maximum principle of Pontryagin, Adjoint

PDE Constraint
noticing that the optimality system is Hamiltonian Optimization
Lions
Adjoint

Conclusion
“Constrained Optimization: from Lagrangian Mechanics to
Optimal Control and PDE Constraints”, Gander, Kwok,
Wanner, 2014.