Systems & Control Letters 39 (2000) 173–182

www.elsevier.com/locate/sysconle

Output-feedback stabilization of stochastic nonlinear systems

driven by noise of unknown covariance (
Hua Deng, Miroslav Krstic ∗
Department of AMES, University of California at San Diego, La Jolla, CA 92093-0411, USA

Abstract
We address the class of stochastic output-feedback nonlinear systems driven by noise whose covariance is time varying
and bounded but the bound is not known a priori. This problem is analogous to deterministic disturbance attenuation
problems. We rst design a controller which guarantees that the solutions converge (in probability) to a residual set
proportional to the unknown bound on the covariance. Then, for the case of a vanishing noise vector eld, we design
an adaptive controller which, besides global stability in probability, guarantees regulation of the state of the plant to zero
with probability one. c 2000 Elsevier Science B.V. All rights reserved.

Keywords: Output-feedback stabilization; Noise-to-state stabilization; Adaptive stabilization

1. Introduction
Despite huge popularity of the LQG control problem, the stabilization problem for nonlinear stochastic
systems has been receiving relatively little attention until recently. Eorts toward (global) stabilization of
stochastic nonlinear systems have been initiated in the work of Florchinger [6–8] who, among other things,
extended the concept of control Lyapunov functions and Sontag’s stabilization formula [15] to the stochas-
tic setting. A breakthrough towards arriving at constructive methods for stabilization of broader classes of
stochastic nonlinear systems came with the result of Pan and Basar [13] who derived a backstepping design
for strict-feedback systems motivated by a risk-sensitive cost criterion [1,10,12,14]. In [2,3], for the same
class of systems as in [13], we designed inverse optimal control laws, which, unlike those in [13], can be
designed in an automated manner (via symbolic software). The results of [2,3] were extended in [4] to the
class of output-feedback systems; this is the rst result on global output-feedback stochastic stabilization.
Notable advances on input-to- state (rather than equilibrium) stabilization were made by Tsinias [17,18].
The results in [2– 4] solve only the equilibrium stabilization problem under the assumption that the noise
vector eld is vanishing (which preserves the equilibrium) and the assumption that a bound on the noise
covariance is known. These assumptions allow some interesting nonlinear systems but exclude linear sys-
tems with additive noise! For this reason, in [5] we addressed systems with nonvanishing noise vector eld
and unknown bound on covariance, and derived both “robust” and “adaptive” controllers for the class of
strict-feedback systems. In this paper we extend these results to the class of output feedback systems, and
give a second-order simulation example.
( This work was supported in part by the National Science Foundation under Grant ECS-9624386 and in part by the Air Force Oce
of Scientic Research under Grant F08671-9800319.
∗ Corresponding author. Fax: 619-822-3107.

E-mail addresses: huadeng@mechanics.ucsd.edu (H. Deng), krstic@ucsd.edu (M. Krstic)

0167-6911/00/$ - see front matter c 2000 Elsevier Science B.V. All rights reserved.
PII: S 0 1 6 7 - 6 9 1 1 ( 9 9 ) 0 0 0 8 4 - 5
174 H. Deng, M. Krstic / Systems & Control Letters 39 (2000) 173–182

2. Preliminaries on stochastic stability

In this section, we rst state some basic notation and then brie y review some stochastic stability concepts
and theorems from [11], which will be used in the following sections.

Denition 2.1. A continuous function : R+ → R+ is said to belong to class K if it is strictly increasing

and (0) = 0. It is said to belong to class K∞ if ∈ K and (r) → ∞ as r → ∞.

Denition 2.2. A continuous function  : R+ × R+ → R+ is said to belong to class KL if for each xed
s the mapping (r; s) belongs to class K∞ with respect to r, and for each xed r the mapping (r; s) is
decreasing with respect to s and (r; s) → 0 as s → ∞.

For a matrix X = [x1 ; x2 ; : : : ; x n ], Tr{·} denotes the trace and

|X |F , (Tr{X T X })1=2 = (Tr{XX T })1=2 (2.1)
denotes the Frobenius norm, and obviously,
|X |F = |col(X )| (2.2)
where col(X ) = [x1T ; x2T ; : : : ; xTn ]T . Consider the nonlinear stochastic system
d x = f(x) dt + g(x) dw; (2.3)
n
where x ∈ R is the state, w is an r-dimensional independent Wiener process with incremental covariance
(t)(t)T dt, i.e., E{dw dwT } = (t)(t)T dt, where (t) is a bounded function taking values in the set of
nonnegative denite matrices, f : Rn → Rn and g : Rn → Rn×r are locally Lipschitz, and f(0) = 0.

Denition 2.3. The system (2.3) is noise-to-state stable (NSS) if ∀ ¿ 0, there exists a class KL function
(·; ·) and
 a class K function (·),  such that 
 
P |(x(t)| ¡ (|x0 |; t) + sup (s)(s)T F ¿1 − ; ∀t¿0; ∀x0 ∈ Rn \ {0}: (2.4)
t¿s¿t0

NSS is a stochastic analog of input-to-state stability [16].

Theorem 2.1 (Krstic and Deng [11]). Consider system (2:3) and suppose there exists a C2 function V (x);
class K∞ functions 1 ; 2 and ; and a class K function 3 ; such that
1 (|x|)6V (x)62 (|x|);   (2.5)
 T @V @2
V
|x|¿( F ) ⇒ LV (x; ) = 1 T
f(x) + 2 Tr  g T
g 6 −3 (|x|): (2.6)
@x @x2
Then the system (2:3) is NSS.

Consider the nonlinear stochastic system (2.3) with additional assumptions that g(0) = 0 and (t) ≡ I.

Denition 2.4. The equilibrium x = 0 of the system (2.3) is

• globally stable in probability if ∀ ¿ 0 there exists a class K function (·) such that
P{|x(t)| ¡ (|x0 |)}¿1 − ; ∀t¿0; ∀x0 ∈ Rn \ {0}; (2.7)
• globally asymptotically stable in probability if ∀ ¿ 0 there exists a class KL function (·; ·) such that
P{|x(t)| ¡ (|x0 |; t)}¿1 − ; ∀t¿0; ∀x0 ∈ Rn \ {0}: (2.8)

Theorem 2.2 (Krstic and Deng [11]). Consider system (2:3) and suppose there exists a C2 function V (x)
and class K∞ functions 1 and 2 ; such that
1 (|x|)6V (x)62 (|x|); (2.9)
 
@V @2 V
LV (x) = f(x) + 12 Tr gT 2 g 6 − W (x); (2.10)
@x @x
 H. Deng, M. Krstic / Systems & Control Letters 39 (2000) 173–182 175

where W (x) is continuous and nonnegative. Then the equilibrium x = 0 is globally stable in probability and
n o
P lim W (x) = 0 = 1: (2.11)
t→∞

This theorem is a stochastic analog of LaSalle’s theorem.

3. Output-feedback noise-to-state stabilization

In this section we deal with nonlinear output-feedback systems driven by white noise with bounded but
unknown covariance. This class of systems is given by the following nonlinear stochastic dierential equations:
d xi = xi+1 dt + ’i (y)T dw; i = 1; : : : ; n − 1
d x n = u dt + ’n (y)T dw; (3.1)
y = x1 ;
where ’i (y) are r-vector-valued smooth functions, and w is an independent r-dimensional Wiener process
with incremental covariance E{dw dwT } = (t)(t)T dt.
Since the states x2 ; : : : ; x n are not measured, we rst design an observer which would provide exponentially
convergent estimates of the unmeasured states in the absence of noise. The observer is designed as
x̂˙i = x̂i+1 + ki (y − x̂1 ); i = 1; : : : ; n (3.2)
where x̂n+1 = u. The observer error x̃ = x − x̂ satises
 
−k1
 I 
 
d x̃ =  .  x̃ dt + ’(y)T dw = A0 x̃ dt + ’(y)T dw; (3.3)
 .. 
−kn 0 · · · 0
where A0 is designed to be Hurwitz. Now, the entire system can be expressed as
d x̃ = A0 x̃ dt + ’(y)T dw;
dy = (x̂2 + x̃2 )dt + ’1 (y)T dw;
d x̂2 = [x̂3 + k2 (y − x̂1 )] dt; (3.4)
..
.
d x̂n = [u + kn (y − x̂1 )] dt:
Our output-feedback design will consist in applying a backstepping procedure to the system (y; x̂2 ; : : : ; x̂n ),
while taking care of the feedback connection through the x̃ system. In the backstepping method, the error
variables zi are given by
z1 = y; (3.5)

zi = x̂i − i−1 (x̂i−1 ; y); i = 2; : : : ; n; (3.6)

where x̂i = [x̂1 ; : : : ; x̂i ]T . According to Itô’s dierentiation rule, we have
d z1 = (x̂2 + x̃2 ) dt + ’1 (y)T dw (3.7)
" i−1
X @i−1 @i−1
d zi = x̂i+1 + ki x̃1 − (x̂l+1 + kl x̃1 ) − (x̂2 + x̃2 )
@x̂l @y
 l=2
 
1 @2 i−1 T T @i−1
− ’1 (y)  ’1 (y) dt − ’1 (y)T dw; i = 2; : : : ; n: (3.8)
2 @y2 @y
As usual in the stochastic case [11], we employ a quartic Lyapunov function
n
1 1X 4 b T
V (z; x̃) = y4 + zi + (x̃ P x̃)2 ; (3.9)
4 4 2
i=2
176 H. Deng, M. Krstic / Systems & Control Letters 39 (2000) 173–182

where b is a positive constant and P satises AT0 P + PA0 = −I . The rst two terms in (3.9) constitute a
Lyapunov function for the (y; x̂2 ; : : : ; x̂n )-system, while the third term in (3.9) is a Lyapunov function for the
x̃-system.
Now we start the process of selecting the functions i (x̂i ; y) to make LV in the form
LV 6 − (x̃; y; x̂2 ; : : : ; x̂n ) + (||) (3.10)
where  is positive denite, radially unbounded, and is a class K∞ function. Since ’(y) is a smooth
function, according to mean value theorem, we can write it as
’(y) = ’(0) + y (y) (3.11)
where (y) is a smooth function. Along the solutions of (3.3), (3.7) and (3.8), we have
X n  i−1
X @i−1
LV = y3 (x̂2 + x̃2 ) + 32 y2 ’1 (y)T T ’1 (y) + zi3 x̂i+1 + ki x̃1 − (x̂l+1 + kl x̃1 )
@x̂l
  i=2  l=2
@i−1 1 @2 i−1 T T
− (x̂2 + x̃2 ) − ’ 1 (y)  ’ 1 (y)
@y 2 @y2
n  
3 X 2 @i−1
2

+ zi ’1 (y)T T ’1 (y) − bx̃ T P x̃|x̃|2 + b Tr ’(y)T (2P x̃x̃T P +x̃ T P x̃P)’(y)T
2 @y
i=2   √
T 2
√ 2 1 T 1 1 4 3bn n|P|2 22
6 −bx̃ P x̃|x̃| + 3bn n|P| |’(0) ’(0)| + 2 + 2 |x̃| + |’(0)T ’(0)k|4
222 3 24 2
√ √  2 
3bn n|P|2 32 8 bn n|P|2 42 24 √ 2 33 T 4 542 24=5 24=5
+ || + || + bn n|P| |’(0) (y)| y + | (y)| y y3
2 4 2 4
3
+y3 (1 + z2 + x̃2 ) + y2 ’1 (y)T T ’1 (y)
2
X n  i−1
X @i−1 @i−1
+ zi3 i + zi+1 + ki x̃1 − (x̂l+1 + kl x̃1 ) − (x̂2 + x̃2 )
@x̂l @y
i=2 l=2
   n  2
1 @2 i−1 T T 3 X 2 @i−1
− ’ 1 (y)  ’ 1 (y) + z i ’1 (y)T T ’1 (y)
2 @y2 2 @y
" 
i=2
 n
#
√ 2 1 T 1 1 1X 1 1
6 − b − 3bn n|P| |’(0) ’(0)| + 2 + 2 − − 4 |x̃|4
222 3 24 4
i=2 i
 4 41

3
+y3 1 + |’1 (0)|4 y + 32 y3 (’1 (0)T 1 (y))2 + 38 54=3 y7=3 | 1 (y)|8=3 + 34 4=3 3 4=3
1 y + 4 1 y
4
 2 
√ 33 52
+bn n|P|2 |’(0)T (y)|4 y + 4 | (y)|24=5 y9=5
" 2 4
X n i−1
X  2
3 @i−1 @i−1 1 @2 i−1
+ zi i + ki x̃1 − (x̂l+1 + kl x̃1 ) − x̂2 + ’1 (y) ’1 (y) zi3
T
@x̂l @y 4 @y2
i=2 l=2
 4=3  4 #
4=3 1 4=3 @ i−1 3 @ i−1
+ 34 i zi + 4 zi + 34 i zi + 2 zi (’1 (y)T ’1 (y))2
4i−1 @y 4i @y
√ n
!
3bn n|P|2 22 T 3X 2 5
+ |’(0) ’(0)| + i + 2 ||4
2 4
 √  i=2
√
6 3bn n|P|2 32 bn n|P|2 42 24
+ 4+ ||8 + || (3.12)
5 2 4
where  ¿ 0 is the smallest eigenvalue of P. The inequalities come from substituting x̂i = zi + i−1 , and
Young’s inequalities in Appendix A and (A.2), (A.3), (A.4), (A.5) in [4]. At this point, we can see that all
the terms can be cancelled by u and i . If we choose 1 ; 2 ; 3 ; 4 and i large enough to satisfy
  n
√ 1 1 1 1X 1 1
b − 3bn n|P|2 |’(0)T
’(0)| + + − − 4 = p ¿ 0; (3.13)
222 32 242 4
i=2 i
 4 41
 H. Deng, M. Krstic / Systems & Control Letters 39 (2000) 173–182 177

and i and u as
1 = −c1 y − 34 |’1 (0)|4 y − 32 y3 (’1 (0)T 1 (y))2 − 38 54=3 y7=3 | 1 (y)|8=3 − 34 4=3 3 4=3
1 y − 4 1 y
 2 
√ 33 52
−bn n|P|2 |’(0)T (y)|4 y + 4 | (y)|24=5 y9=5 ; (3.14)
2 4
i−1
X @i−1  2
@i−1 1 @2 i−1 T
i = −ci zi − ki x̃1 + (x̂l+1 + kl x̃1 ) + x̂2 − ’1 (y) ’ 1 (y) zi3
@x̂l @y 4 @y2
l=2
 4=3  4
3 4=3 1 3 4=3 @i−1 3 @i−1
− 4 i zi − 4 zi − 4 i zi − 2 (’1 (y)T ’1 (y))2 zi ; (3.15)
4i−1 @y 4i @y
u = n ; (3.16)
where ci ¿ 0 and n = 0, then the innitesimal generator of the closed-loop system (3.3), (3.7), (3.8) and
(3.16) satises
n √ n
!
X 3bn n|P|2 22 3X 2 5
4 4 T
LV 6 − ci zi − p|x̃| + |’(0) ’(0)| + i + 2 ||4
2 4
i=1
 √  √ i=2
3bn n|P|2 32 6 8 bn n|P|2 42 24
+ + 4 || + || : (3.17)
2 5 4
With (3.17), according to Theorem 2.1, we have the following stability result.

Theorem 3.1. The closed-loop stochastic system (3:1); (3:2); (3:16) is NSS.

4. Output-feedback adaptive stabilization

In this section, we deal with output-feedback systems (3.1) with an additional assumption that ’i (0) = 0.
Since ’i (0) = 0, by the mean value theorem, ’(y) can be expressed as
’(y) = y (y) (4.1)
where (y) is a smooth function. As we will see in the sequel, to achieve adaptive stabilization in the
presence of unknown , it is not necessary to estimate the entire matrix . Instead, we will estimate only one
unknown parameter  = kT k2∞ using an estimate . ˆ Employing the same observer (3.2), the entire system
is
d x̃ = A0 x̃ dt + ’(y)T dw
dy = (x̂2 + x̃2 ) dt + ’1 (y)T dw
d x̂2 = [x̂3 + k2 (y − x̂1 )] dt (4.2)
..
.
ˆ + kn (y − x̂1 )] dt
d x̂n = [n (x̂; y; )
˙ ˆ
ˆ =  (x̂; y; );
n

where n and n are functions to be designed. In the backstepping method, the error variables zi are given by
z1 = y; (4.3)

ˆ
zi = x̂i − i−1 (x̂i−1 ; y; ); i = 2; : : : ; n: (4.4)
According to Itô’s dierentiation rule, we have
d z1 = (x̂2 + x̃2 ) dt + ’1 (y)T dw (4.5)
" i−1
X @i−1 @i−1
d zi = x̂i+1 + ki x̃1 − (x̂2 + x̃2 )
(x̂l+1 + kl x̃1 ) −
@x̂l @y
  l=2 
1 @2 i−1 T T @i−1 ˙ˆ @i−1
− ’1 (y)  ’1 (y) −  dt − ’1 (y)T dw i = 2; : : : ; n: (4.6)
2 @y 2
@ˆ @y
178 H. Deng, M. Krstic / Systems & Control Letters 39 (2000) 173–182

As usual in the stochastic case, we employ a Lyapunov function of a quartic form

Xn
ˆ = 1 y4 + 1
V (z; x̃; )
b 1 2
zi4 + (x̃T P x̃)2 + ˜ ; (4.7)
4 4 2 2
i=2

where b is a positive constant, P satises AT0 P + PA0 = −I , and ˜ = kT k2∞ − .

ˆ Now we start the process
 ˆ
of selecting the functions i (x̂i ; y; ) to make LV in the form
ˆ
LV 6 − (x̃; y; x̂2 ; : : : ; x̂n ; ) (4.8)
where  is a positive semidenite function. Along the solutions of (3.3), (4.5) and (4.6), we have
Xn 
3 3 2 T T 3
LV = y (1 + z2 + x̃2 ) + 2 y ’1 (y)  ’1 (y) + zi i + zi+1 + ki x̃1
i=2
i−1
X   
@i−1 @i−1 1 @2 i−1 T T @i−1 ˙ˆ
− (x̂l+1 + kl x̃1 ) − (x̂2 + x̃2 ) − ’1 (y)  ’1 (y) − 
l=2
@x̂l @y 2 @y2 @ˆ
n  2
3 X 2 @i−1
+ zi ’1 (y)T T ’1 (y) − bx̃T P x̃|x̃|2
2 @y
i=2
˜˙ˆ
+b Tr{’(y)T (2P x̃x̃T P + x̃T P x̃P)’(y)T } −
" #
√ n 
3bn n 2 1 X 1 1 32
6 − b − 2
|P| − 4
− 4
|x̃| 4
+ y 3
1 + 3 ( 1 (y)T 1 (y))2 y + 34 4=3 1 y
22 4 i 41 4
√ i=2
3bn n|P|2 22
+ 34 14=3 y + | (y)|4 ykT k2∞
2 
3 T 2 32 T 2 332 T 2
+ 2 yk k∞ + ( 1 (y) 1 (y)) y + (n − 1)( 1 (y) 1 (y)) y
43 " 4 4
X n i−1
X  2 2
@i−1 @i−1 1 @ i−1
+ zi3 i + ki x̃1 − (x̂l+1 + kl x̃1 ) − x̂2 + 2 zi3 2
kT k2∞
@x̂l @y 4 3 @y
i=2 l=2
 4=3  4 
1 @i−1 3 @i−1 @i−1 ˙ˆ ˜˙ˆ
+ 34 4=3
i z i + z i + 3 4=3

4 i z i + z i k T 2
k ∞ −  −
" 44i−1 @y
# 432 @y @ˆ
√ Xn  2
3bn n 2 1 1 1 (3n + 1)3
= − b − |P| − − 4 |x̃|4 + y3 1 + ( 1 (y)T 1 (y))2 y + 34 4=3 1 y
222 4
i=2
 4
i 4 1 4
√  n " i−1
3 4=3 3bn n|P|2 22 4 ˆ 3 ˆ X 3 X @i−1 @i−1
+ 4 1 y + | (y)| y + 2 y + zi i + ki x̃1 − (x̂l+1 + kl x̃1 ) − x̂2
2 43 @x̂l @y
 2
i=2
 4=3
l=2
 4 #
1 3 @2 i−1 1 @ 3 @ @ ˙
ˆ + 4 i zi + zi ˆ − ˆ
3 4=3 3 4=3 i−1 i−1 i−1
+ 2 zi zi + 4 i zi + 2
43 @y2 44i−1 @y 43 @y @ˆ
" √  2  4 #
˙ˆ 3bn n|P|2 22
n n
˜ 4 4 3 4 1 X 6 @2 i−1 3 X 4 @i−1
− − | (y)| y − 4 y − 2 zi − 2 zi (4.9)
2 43 @y2 43 @y
i=2 i=2
where  ¿ 0 is the smallest eigenvalue of P, x n+1 =u, zn+1 =0, n =u. The inequalities come from substituting
x̂i = zi + i−1 , and Young’s inequalities in Appendix B and (A.2), (A.3), (A.4), (A.5) in [4]. Let
√
3bn n|P|2 22
1 = | (y)|4 y4 + 34 y4 ; (4.10)
2
3
i = i−1 + zi !i ; i = 2; : : : ; n; (4.11)
ˆ˙ =  ;
n (4.12)
where
 2  4
1 3 @2 i−1 3 @i−1
!i = z + zi : (4.13)
432 i @y2 432 @y
 H. Deng, M. Krstic / Systems & Control Letters 39 (2000) 173–182 179

Then " #
√ n 
3bn n 2 1 X 1 1 (3n + 1)32
LV 6 − b − 2
|P| − 4
− 4
|x̃| 4
+ y 3
1 + ( 1 (y)T 1 (y))2 y + 34 4=31 y
22 4  i 4 1 4
i=2 "
√  X n i−1
X
3 4=3 3bn n|P|2 22 4 ˆ 3 ˆ 3 @i−1
+ 4 1 y + | (y)| y + 2 y + zi i + ki x̃1 − (x̂l+1 + kl x̃1 )
2 43 @x̂l
i=2 l=2
@i−1 1
− x̂2 + 34 4=3
i zi + zi
@y 44i−1
 4=3 i i−1
#
@ @ X X @ j−1
zi − !i ˆ −
i−1 i−1
+ 34 4=3
i zj3 !j − zj3 !i : (4.14)
@y @ˆ j=2 j=2 @ˆ
If we choose 1 ; 2 and i large enough to satisfy
√ n
3bn n 2 1 X 1 1
b − 2
|P| − 4
− 4 = p ¿ 0; (4.15)
22 4 i 41
i=2
and i and u as
√
(3n + 1)32 3bn n|P|2 22 3 ˆ
1 = −c1 y − T 2 3 4=3
( 1 (y) 1 (y)) y − 4 1 y − 4 1 y −3 4=3
| (y)|4 yˆ − 2 y; (4.16)
4 2 43
i−1
X @i−1 @i−1
i = −ci zi − ki x̃1 + (x̂l+1 + kl x̃1 ) + x̂2 − 34 4=3
i zi
@x̂l @y
l=2
 4=3 X i i−1
X
1 @i−1 ˆ + @i−1 @j−1
− 4 zi − 34 4=3 i z i + ! i  z 3
j ! j + zj3 !i ; (4.17)
4i−1 @y @ˆ j=2 j=2 @ˆ
u = n ; (4.18)
where ci ¿ 0 and n = 0, then the innitesimal generator of the closed-loop system (3.3), (4.5), (4.6) and
(4.18) satises
n
X
LV 6 − ci zi4 − p|x̃|4 : (4.19)
i=1
Since z = 0 and x̃ = 0 implies x = 0, by Theorem 2.2, we have the following result.

Theorem 4.1. The equilibrium x = 0; ˆ = kT k2∞ of the closed-loop system (3:1); (3:2); (4:12) and (4:18)
is globally stable in probability and
n o
P lim x(t) = 0 and lim x̂(t) = 0 = 1: (4.20)
t→∞ t→∞

Remark 4.1. Since LV is nonpositive, EV (t) is nonincreasing. Since V is also bounded from below by zero,
˜ 2 } has a limit.
EV (t) has a limit. Since z(t) and x̃(t) converge to zero with probability one, thus E{(t)

5. Example

We give a second-order example to illustrate the two methods given in above sections. Consider the system
d x1 = x2 dt + 12 x12 dw;
d x2 = u dt; (5.1)
y = x1 :
For this system, the estimator is
x̂˙1 = x̂2 + k1 (y − x̂1 ); x̂˙2 = u + k2 (y − x̂1 ): (5.2)
According to the NSS design, the virtual control 1 and control u are
√
34=3 5 2b|P|2 42 33=5
1 = −c1 y − 5 y5 − 34 4=3
1 y − 3 4=3

4 1 y − y ; (5.3)
32 64
180 H. Deng, M. Krstic / Systems & Control Letters 39 (2000) 173–182

Fig. 1. The states and control eort of the NSS design.

 2
@1 1 @2 1 1
u = −c2 z2 − k2 x̃1 + x̂2 − y8 z23 − 34 4=3
2 z2 − z2
@y 64 @y 2 441
 4=3  4
@1 3 @1
− 34 4=3
2 z 2 − y 8 z2 : (5.4)
@y 6422 @y
We choose k1 = 3, k2 = 4:5, c1 = 1 = 4 = 5 = 2 = 0:01, c2 = b = 0:1, 1 = 2 = 0:4, 2 = 50, and set the
initial condition at x1 (0) = 1:3, x2 (0) = 0, x̂1 (0) = 0, x̂2 (0) = 1 (0), the states and control of the system are
shown in Fig. 1 for  = 1.
According to the adaptive design, the virtual √ control 1 and control u are:
732 5 3 4=3 4=3 3 2b|P|2 22 5 ˆ 3 ˆ
1 = −c1 y − y − 4 (1 + 1 )y − y  − 2 y; (5.5)
64 16 43
 4=3
@1 1 @1 @1 3
u = −c2 z2 − k2 x̃1 + x̂2 − 34 4=3
2 z2 − z2 − 34 4=3
2 z2 + !2 ˆ + z2 !2 ; (5.6)
@y 441 @y @ˆ
and the adaptive law is
˙ˆ = 2 = (1 + z 3 !2 );
2 (5.7)
where √
3 2b|P|2 22 8 3 4
1 = y + 4y ; (5.8)
16
 2 2  4
1 @ 1 3 @1
!2 = 2 z23 + z 2 : (5.9)
43 @y2 432 @y
We choose the same estimator as in NSS design, set parameters as c1 = 1 = 2 = 3 = b = 0:01, c2 = 2 = 0:1,
1 = 2 = 0:4, = 10−5 , and the initial condition at x1 (0) = 1:3, x2 (0) = 0, x̂1 (0) = 0, x̂2 (0) = 1 (0), the states
and control of the system are shown in Fig. 2 for  = 1.
From Figs. 1 and 2, we can see that (5.4) achieves noise to state stability (i.e., regulation to within a residual
set proportional, in appropriate sense, to  = 1) and (5.6) achieves stability of x = x̂ = ˆ = 0 and regulation of
x and x̂ (in probability). A closer inspection of Fig. 1 (left) indicates the possibility that controller (5.4) may
be achieving not only NSS but also regulation to zero, however, we do not have hard simulation evidence
that x1 continues approaching zero over a longer time window, thus we only claim NSS.

Appendix A
In this and the following appendix, we use Young’s inequality [9, Theorem 156]:
p 1
xy6 |x|p + q |y|q ; (A.1)
p q
 H. Deng, M. Krstic / Systems & Control Letters 39 (2000) 173–182 181

Fig. 2. The states, estimate and control eort of the adaptive design.

where  ¿ 0, the constants p ¿ 1 and q ¿ 1 satisfy (p − 1)(q − 1) = 1, and (x; y) ∈ R2 . Applying these
inequalities leads to
n  2 n  4 n
3 X 2 @i−1 T T 3 X 1 @i−1 4 T 2 3X 2
zi ’1 (y)  ’1 (y)6 zi (’ 1 (y) ’ 1 (y)) + i |T |2 ; (A.2)
2
i=2
@y 4 2i @y
i=2
4
i=2

b Tr{’(y)T (2P x̃x̃T P + x̃T P x̃P)’(y)T }

√
63bn n|’(y)T ’(y)| |P|2 |x̃|2 ||2
√ √ √
=3bn n|P|2|’(0)T ’(0)||x̃|2 ||2 +6bn n|P|2 |y’(0)T (y)||x̃|2 ||2 + 3bn n|P|2 y2 | (y)T (y)||x̃|2 ||2
√ √
3bn n|P|2 3bn n|P|2 22 √
6 2
|’(0)T ’(0)kx̃|4 + |’(0)T ’(0)k|4 + 3bn n|P|2 32 |’(0)T (y)|2 y2 ||4
22 2
√ √ √
3bn n|P|2 4 3bn n|P|2 4 3bn n|P|2 42 4
+ | x̃| + |x̃| + y | (y)T (y)|2 ||4
32 242 2
  √ √
√ 1 1 1 3bn n|P|2 22 3bn n|P|2 32 8
63bn n|P|2 |’(0)T
’(0)| + + |x̃| 4
+ |’(0)T
’(0)| ||4
+ ||
222 32 242 2 2
√ √ √
3bn n|P|2 32 bn n|P|2 42 24 5bn n|P|2 42
+ |’(0)T (y)|4 y4 + || + | (y)|24=5 y24=5 ; (A.3)
2 4 4
182 H. Deng, M. Krstic / Systems & Control Letters 39 (2000) 173–182

3 2 T T
2 y ’1 (y)  ’1 (y)
6 2 y |’1 (y)T ’1 (y)k|2
3 2

= 32 y2 (’1 (0)T ’1 (0) + 2y’1 (0)T 1 (y) + y2 1 (y)

T
1 (y))||
2

6
6 34 |’1 (0)|4 y4 + 32 y6 (’1 (0)T + 38 54=3 y16=3 | 1 (y)|8=3 + 94 ||4 + 4 ||8
1 (y))
2
(A.4)
 2 2  5
1 @2 i−1 3 T T 1 6 @ i−1 T 1 4
− z ’1 (y)  ’1 (y)6 z ’ 1 (y) ’1 (y) + || : (A.5)
2 @y2 i 4 i @y2 4

Appendix B
Similar to Appendix A, in the following inequalities, 2 is a constant to be chosen.
n  2
3 X 2 @i−1
zi ’1 (y)T T ’1 (y)
2 @y
i=2
n  4 n
3 X @i−1 32 X 4
6 2 zi4 kT k2∞ + 3 y ( 1 (y)T 1 (y))2 ; (B.1)
43 @y 4
i=2 i=2
T T T
√ x̃2 x̃ P +2x̃ P2x̃P)’(y)
b Tr{’(y)(2P }
2 T
63bn √ny | (y)| |P| |x̃| k k∞ √
3bn n22 |P|2 4 3bn n|P|2 4
6 y | (y)|4 kT k2∞ + |x̃| ; (B.2)
2 222
3 2 T T
2 y ’1 (y)  ’1 (y)
2
3 3 32 3 4
6 3 (’1 (y)T ’1 (y))2 + 2 y4 kT k2∞ = 3 y4 ( 1 (y)T 1 (y))
2
+ y kT k2∞ ; (B.3)
4 43 4 432
1 @2 i−1 3 T T
− z ’1 (y)  ’1 (y)
2 @y2 i
 2 2
1 6 @ i−1 T 2 32 4
6 2 zi k k ∞ + y ( 1 (y)T 1 (y))2 : (B.4)
43 @y2 4

References
[1] T. Basar, P. Bernhard, H ∞ -Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach, 2nd Edition,
Boston, MA, Birkhauser, 1995.
[2] H. Deng, M. Krstic, Stochastic nonlinear stabilization — Part I: A backstepping design, Systems Control Lett. 32 (1997) 143–150.
[3] H. Deng, M. Krstic, Stochastic nonlinear stabilization — Part II: Inverse optimality, Systems Control Lett. 32 (1997) 151–159.
[4] H. Deng, M. Krstic, Output-feedback stochastic nonlinear stabilization, IEEE Trans. Automat. Control 44 (1999) 328–333.
[5] H. Deng, M. Krstic, Stabilization of stochastic nonlinear systems driven by noise of unknown covariance, Proceedings of ACC,
Philadelphia, 1998.
[6] P. Florchinger, A universal formula for the stabilization of control stochastic dierential equations, Stochastic Anal. Appl. 11 (1993)
155–162.
[7] P. Florchinger, Lyapunov-like techniques for stochastic stability, SIAM J. Control Optim. 33 (1995) 1151–1169.
[8] P. Florchinger, Global stabilization of cascade stochastic systems, Proceedings of the 34th Conference on Decision and Control,
New Orleans, LA, 1995, pp. 2185 –2186.
[9] G. Hardy, J.E. Littlewood, G. Polya, Inequalities, 2nd Edition, Cambridge University Press, Cambridge, 1989.
[10] M.R. James, J. Baras, R.J. Elliott, Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems,
IEEE Trans. Automat. Control 39 (1994) 780–792.
[11] M. Krstic, H. Deng, Stabilization of Nonlinear Uncertain Systems, Springer, Berlin, 1998.
[12] H. Nagai, Bellman equations of risk-sensitive control, SIAM J. Control Optim. 34 (1996) 74–101.
[13] Z. Pan, T. Basar, Backstepping controller design for nonlinear stochastic systems under a risk-sensitive cost criterion, SIAM J.
Control Optim. 37(3) (1999) 957–995.
[14] T. Runolfsson, The equivalence between innite horizon control of stochastic systems with exponential-of-integral performance index
and stochastic dierential games, IEEE Trans. Automat. Control 39 (1994) 1551–1563.
[15] E.D. Sontag, A ‘universal’ construction of Artstein’s theorem on nonlinear stabilization, Systems Control Lett. 13 (1989) 117–123.
[16] E.D. Sontag, Smooth stabilization implies coprime factorization, IEEE Trans. Automat. Control 34 (1989) 435–443.
[17] J. Tsinias, The concept of ‘exponential ISS’ for stochastic systems and applications to feedback stabilization, preprint, 1997.
[18] J. Tsinias, Stochastic input-to-state stability and applications to global feedback stabilization, (special issue on Breakthrough in the
Control of Nonlinear Systems) Int. J. Control, 1998 71(5) (1998) 907–930.