Propagation of Uncertain Inputs Through Networks of Nonlinear
Components
Tamás Kalmár-Nagy, Mihai Huzmezan
United Technologies Research Center, 411 Silver Lane, East Hartford, CT 06108
email: imece@kalmarnagy.com, huzmezm@utrc.utc.com
Abstract
Physics based models are often converted to
monolithic systems of uncertain nonlinear differen-
tial/algebraic equations. Graph decomposition meth-
ods can be used to decompose such system into sub-
systems evolving on different time scales. This time
scale separation can be exploited to increase com-
putational efficiency when propagating input uncer-
tainty in a subsystem-by-subsystem manner. In this
paper the propagation of uncertain inputs through
series, parallel and feedback interconnections of dy-
namical systems with simple asymptotic behavior is
studied by employing discrete density mapping (anal-
ogous to the input-output Perron-Frobenius opera-
tor). A process control example is used to illustrate
the method.
1 Introduction
Engineering performance and productivity can be im-
proved through systematic analysis and integrated
and design that takes uncertainty into account [7].
A key objective for such an analysis tool is to study
uncertainty propagation through networks of nonlin-
ear components. This view is also shared by [20]
or [1], where large-scale, interconnected systems are
designed using model-based techniques that employ
explicit descriptions of uncertainty. Since all system
models have some level of uncertainty [3], designers
often use large safety margins which result in more
complex and expensive systems [14]. Finding the con-
tribution of model or initial condition uncertainty to
1
the system output would thus enable designing more
robust and effective sytems.
Uncertainty propagation has traditionally been ad-
dressed by Monte Carlo like methods. This classic
approach (see [5]) employs a large number of sim-
ulations with a random selection of variables from
their prescribed distribution. To address slow con-
vergence rate and clustering issues associated with
Monte Carlo methods, new methods such as: poly-
nomial chaos [13] (i.e. stochastic finite elements), sto-
chastic surface response methods [9] and probabilistic
collocation methods [17] were used with significant
success. Complementing these approaches, the prop-
agation of uncertainty in the distribution of initial
conditions for a dynamical systems can be studied
using the corresponding Liouville’s equation as in [4].
Unfortunately, Monte Carlo methods do not scale
well with system size. To overcome this problem a
large system can be broken down into loosely cou-
pled pieces (such that the coupling is strong inside
the pieces). Uncertainty propagation techniques can
then be used on the smaller subsystems together with
iterations accounting for the coupling between these
(akin to the Waveform Relaxation method). Large
systems can for example be decomposed into subsys-
tems evolving on different time scales using graph
decomposition methods ([16]). In [10] graph theory
is used in the context of autocatalytic networks/sets
to classify the uncertainty of the network and predict
its influence over short and medium time-scales. It
is essential to perform related computations in cor-
responding time scales [19], when dealing with irre-
ducible (i.e. strongly connected) graphs (e.g. dynam-
ical subsystems).
This time scale separation can be exploited to in-
crease computational efficiency when propagating in-
put uncertainty in a subsystem-by-subsystem man-
ner. This approach has been also advocated in [2],
where arbitrary interconnections of multivariable sys-
tems (represented either in a continuous or discrete
form) with nonlinear or linear dynamics (nonlinear
time varying, distributed linear time invariant or
lumped linear time invariant) are decomposed into
aggregate, strongly connected subsystems. Another
interesting alternative could be decomposition based
on state space behavior. Subsets of the state space
of a dynamical system where typical trajectories stay
longer before entering different regions are called al-
most invariant sets [6]. Such decomposition could be
used in uncertainty analysis based on regions of dif-
ferent dynamics.
Finally, to obtain the global results due to uncer-
tain parameters or initial conditions, the weak cou-
pling between subsystems should be taken into ac-
count by using an appropriate iteration scheme. The
Waveform Relaxation method has been successfully
employed in [11] to address large scale systems, such
as integrated circuits. Alternatively, a Recursive Pro-
jection Method ([15]) type approach could be used to
accelerate iteration convergence.
2 Signal Flow Decomposition:
Chains, Parallel and Feed-
back Connections
The simplest topology that can arise in the intercon-
nection of systems is a chain, as shown in Figure 1.In
Figure 1: Density propagation through chains
this case the output of a block serves as input to the
following block. If the input-output maps f1 , f2 , ...
are known for all blocks, then the output of the chain
can simply be calculated as the composition of these
maps f1 ◦ f2 ◦... acting on the input of the first block.
To consider uncertain inputs, it is necessary to extend
the notion of single input-output mapping to proba-
bility densities. The resulting formalism, analogous
to the Perron-Frobenius operator is discussed below.
The computational advantage of propagating in-
put densities in a block-by-block manner, which
corresponds to the composition of maps, becomes
clear when the dynamics of different blocks include
completely different timescales. Complications arise

In this paper we focus on the propagation of un-

certain inputs through interconnections of dynamical
systems with simple asymptotic behavior (i.e. ones
that can be replaced with static blocks). In Sec-
tion 2 the conventional signal flow decomposition is
described. A simple discrete density mapping, anal-
ogous to the input-output Perron-Frobenius opera- Figure 2: Parallel summation
tor is employed in Section 3. The rule for summing
correlated signals due to typical parallel or feedback when considering a parallel connection, see Figure 2.
connections is also given here. Convergence issues for In this case, the resulting probability densities (out-
feedback topologies are addressed in Section 3.2. To put uncertainties) have to be summed. For den-
illustrate the method a simple example is offered in sities of independent random variables, this opera-
Section 4. Conclusions are then drawn in Section 5. tion would be a simple convolution. However, when

2
these densities are dependent, correlation informa- corresponds to the action of a block upon un , the
tion should be used to sum them correctly (discussed system input. This action results by applying on
in the next Section). un the mapping corresponding to the asymptotic dy-
namics of the system f and has as result un+1 , the
output. The uncertainty of un will be represented
by its probability density G. Therefore the map has
a corresponding Perron-Frobenius operator U acting
on G:
Z
Gn+1 = U Gn = δ (u − f (x)) Gn (x) dx = (2)
¡ ¢
X Gn fα−1 (u)
= ¯ ¡ ¢¯
¯f 0 fα−1 (u) ¯
α

where the summation should be taken over all inverse

branches fα−1 (u).

Figure 3: Feedback loop with loop operator G

Finally, the feedback connection, shown in Fig-

ure 3, can be thought of as infinite series of paral-
lel connections for which the convergence aspects are
discussed in Subsection 3.2.

3 The Density Mapping

Method
The paper proposes a simple algorithm that can be
used to propagate input uncertainty through non-
linear components. While this algorithm can be
thought of as a simple implementation of the Perron-
Frobenius operator [12], it has been extended to ac-
count for the mapping of the distribution support
(akin to cell-to-cell mapping [8]). This extension was
necessary to enable operations with correlated densi-
ties (e.g. parallel or feedback connections).
To understand the proposed technique first we re-
view the Perron-Frobenius operator of a scalar map
in the context of the proposed input-output uncer-
tainty mapping. The one-dimensional map:
un+1 = f (un ) (1)
Figure 4: Density mapping
In the following a straightforward numerical im-
plementation of this mapping is described. Taking
a cell-based approximation of the input probability
density (histogram) {(τi , τi+1 ) , Gi } with cell-width w
a collection of rectangles (bins) is produced. Consec-
utively the mapping f is applied to the corner points
of all rectangles. Their resulting heights are derived
from the density conservation condition. This proce-
dure yields a new set
Gi w
{(f (τi ) , f (τi+1 )) , hi } hi =
|f (τi ) − f (τi+1 )|
(3)

3
If the map f is many-to-one, this collection will con-
tain overlapping rectangles.
The overall output density can be produced by ’re-
binning’, a procedure which involves finding the total
area over a bin on the support [min f (τi ) , max f (τi )]
of the resulting distribution. This corresponds to
summing over the inverse branches of the map. Re-
binning is also useful when propagating densities in
a system with chain topology. On the other hand,
re-binning destroys information about the mapping Figure 5: Example with nonlinear maps
of the original support into the support of the out-
put. This information is however crucial when deal- 0.04

ing with densities mapped through parallel or feed-

back connections. 0.035

Initial Distribution
0.03
3.1 Summing Correlated Probability
Densities 0.025

Consider the parallel connection shown in Fig- 0.02

ure 2. At the summing junction we have two lists

{(f1 (xi ) , f1 (xi+1 )) , hi }, {(f2 (xi ) , f2 (xi+1 )) , hi } 0.015

produced from the same input density G. Note that

0.01
the lists contain the same hi ’s, because of the density Final Distribution

conservation. Consequently, to produce the parallel 0.005

structure output density, the sum of these lists are
then taken as 0
0 50 100 150 200 250 300

{(f1 (xi ) , f1 (xi+1 )) , hi } ⊕ {(f2 (xi ) , f2 (xi+1 )) , hi } =

(4) Figure 6: Convergence of the Density Mapping iter-
= {(f1 (xi ) + f2 (xi ) , f1 (xi+1 ) + f2 (xi+1 )) , hi } ations

This represents the list produced by the simple appli-

cation of the operator f1 + f2 to the original density. chosen metric). A proof for the convergence of the
iterative scheme proposed is offered in [18].

3.2 Feedback Loop: Convergence Is-

sues 4 A Simple Example
As mentioned earlier, feedback connection represents This example for uncertainty propagation can be
a natural extension of the parallel connection to- thought as the abstraction of a typical industrial
gether with the summation of correlated density func- closed loop control system. This process control
tions. Propagating densities instead of scalar signals structure (see Figure 5) involves inner control loops,
in the loop is facilitated by keeping track of the as- feedforwards and outer control loops.The intercon-
sociated density lists. Computational convergence is nected structure is captured through series, parallel
achieved when the output distributions for two con- and feedback interconnections of nonlinear compo-
secutive iterations become close (in an appropriately nents.

4
 The evolution of an initial uncertain input distrib-
ution u (solid thick line) under the action of the sys-
tem is shown in Figure 6. The successive iterations
convergencing to the known closed loop solution (ex-
plicitly known in this example), are also shown in
Figure 6 (dotted lines). Note that to produce this
plot the density lists were ’re-binned’ (see Section 3),
with the same bin size as for the original histogram.
5 Conclusions and Further
Work
The propagation of uncertain inputs through series,
parallel and feedback interconnections of dynamical
systems with simple asymptotic behavior has been
studied by employing the discrete density mapping
method. Understanding the role of density lists in
dealing with correlated signal density functions pro-
vides a framework for an extension of current work to
use advanced uncertainty propagation (such as Poly-
nomial Chaos) methods for large-scale systems. The
basic idea of this approach is to decompose the dis-
tribution of an uncertain input into small pieces of
uniform distributions and propagate these distribu-
tions by Polynomial Chaos methods. This way the
correlation information is preserved for later opera-
tion on dependent densities. This extension may also
allow for the propagation of uncertain inputs through
interconnections of uncertain dynamical systems with
uncertain initial conditions.
The density of products of correlated signals can
be calculated by using a logarithmic conversion and
summation, thus extensions of the current method to
multivariable systems is also possible.
Acknowledgements
[9] Isukapalli, S.: 1999, ‘Uncertainty Analysis of
Funding from the DARPA/DSO: 2003 Contract on
Analytical Systems Engineering Advanced Mathe-
matics (F49620-03-C-0035) is appreciated. We would
like to acknowledge the fruitful discussions with An-
drzej Banaszuk, Michael Dellnitz, Mihai Dorobantu, [10] Jain, S. and S. Krishna: 2002, Graph The-
Clas Jacobson, Bob LaBarre, Igor Mezic, Mark My-
ers, Thordur Runolfsson, and Subbarao Varigonda.
References
[1] Axelsson, J.: 2002, ‘Model Based Systems Engi-
neering Using a Continuous-Time Extension of
the Unified Modeling Language (UML)’. Sys-
tems Engineering 5(3), 165—179.
[2] Callier, F. M., W. S. Chan, and C. A. Desoer:
1978, ‘Input-Output Stability of Interconnected
Systems Using Decompositions: An Improved
Formulation’. IEEE Transactions on Automatic
Control 23(2), 150—163.
[3] Carlson, J. M. and J. C. Doyle: 2000, ‘Highly
Optimized Tolerance: Robustness and Design
in Complex Systems’. Physics Review Letters
84(11), 2529—2532.
[4] Ehrendorfer, M.: 2003, ‘The Liouville Equation
in Atmospheric Predictability’. In: Proceedings
ECMWF Seminar on Predictability of Weather
and Climate. pp. 47—81.
[5] Friedel, I. and A. Keller: 2000, ‘Fast generation
of randomized low discrepancy point sets’. In: H.
Niederreiter, K. Fang, and F. Hickernell (eds.):
Monte Carlo and Quasi-Monte Carlo Methods
2000. pp. 257—273.
[6] Froyland, G. and M. Dellnitz: 2003, ‘Detecting
and Locating Near-Optimal Almost-Invariant
Sets and Cycles’. Society for Industrial and Ap-
plied Mathematics 24(6), 1839—1863.
[7] Helton, J.: 1994, ‘Treatment of Uncertainty in
Performance Assessments for Complex Systems’.
Risk Analysis 14(4), 483—511.
[8] Hsu, C.: 1987, Cell-to-Cell Mapping. New York:
Springer-Verlag.
Transport-Transformation Models’. Ph.D. the-
sis, Rutgers, The State University of New Jer-
sey.
ory and the Evolution of Autocatalytic Networks,
Chapt. 16. Wiley - VCH.
5
[11] Lelarasmee, E., A. E. Ruehli, and A. L.
Sangiovanni-Vincentelli: 1982, ‘The Waveform
Relaxation Method for Time-Domain Analysis
of Large Scale Integrated Circuits’. IEEE Trans.
on CAD of IC and Syst. 1, 131—145.
[12] McCauley, J. L.: 1993, Chaos, Dynamics, and
Fractals, Vol. 2 of Nonlinear Science. Cam-
bridge University Press.
[13] R., G. and S. P.: 1991, Stochastic Finite El-
ements: A Spectral Approach. New York:
Springer.
[14] Sage, A. P.: 1998, ‘Systems Engineering: Pur-
pose, Function, and Structure’. The Journal Of
The Iinternational Council On Systems Engi-
neering 1(1), 1.
[15] Shroff, G. and H. B. Keller: 1993, ‘Stabilization
of Unstable Procedures: The Recursive Projec-
tion Method’. SIAM J. Numer. Anal. 30(4),
1099—1120.
[16] Strogatz, S.: 2001, ‘Exploring Complex Net-
works’. Nature 410, 268—276.
[17] Tatang, M. A.: 1995, ‘Direct Incorporation of
Uncertainty in Chemical and Environmental En-
gineering Systems’. Ph.D. thesis, Massachusetts
Institute of Technology.
[18] Varigonda, S.: 2004, ‘Iterative Method for Un-
certainty Propagation in Feedback Systems’. In:
Control and Decision Conference. Bahamas.
[19] Vidyasagar, M.: 1981, Input-Output Analysis of
Large-Scale Interconnected Systems: Decompo-
sition, Well-Posedness & Stability. Springer-
Verlag.
[20] Wojtkiewicz, S., M. Eldred, R. Field, and et al.,
‘Uncertainty Quantification In Large Computa-
tional Engineering Models’.