Aircraft Design Thesis

A Probabilistic Multi-Criteria Decision Making
Technique for Conceptual and Preliminary

Aerospace Systems Design
A Thesis
presented to
The Academic Faculty
by
Oliver Bandte
In Partial Fulfillment
of the Requirements for the Degree
Doctor of Philosophy in Aerospace Engineering
Georgia Institute of Technology
September 2000
Copyright 2000 by Oliver Bandte
Oliver Bandte ii
A Probabilistic Multi-Criteria Decision Making Technique for
Conceptual and Preliminary Aerospace Systems Design
Approved:
_________________________________
Professor Dimitri Mavris, Advisor
_________________________________
Professor Daniel Schrage, Advisor
_________________________________
Professor Leonid Bunimovich
Date Approved ____________________
_________________________________
Professor James Craig
_________________________________
Professor Christian Houdr
Oliver Bandte iii
ACKNOWLEDGEMENTS
The hardest part of a dissertation is finding the proper words of appreciation for
all the people that where involved in its creation process. There are my advisors,
Dr. Schrage, without whom I would not be at Georgia Tech in the first place, and
Dr. Mavris, who convinced me of staying on for the Master and Ph.D. after my German
thesis. They truly made my work at the Aerospace Systems Design Laboratory an
unforgettable experience I will benefit from for life. There is the thesis committee with
Dr. Bunimovich and Dr. Houdr, whose patience and strong will to meet me half way on
the bridge between engineering and mathematics can not be commended enough, as well
as Dr. Craig who had the vision to bring probabilistic techniques to fields of engineering
other than aerospace. Of course, great contributions were made by almost all students of
ASDL, with a special thanks to everybody that was answering my nagging computer
related questions again and again and again.
A very special thanks has to go to my parents, who have always stood by my side
and supported me one hundred percent. Particularly, the last three years have been the
hardest they had to suffer through, without me being able to give back as much as they
deserved. Included in this appreciation is of course my brother without whom I simply
would not have been able to finish my thesis after my dads stroke.
Last but not least, my dearest love and thanks to Joanna (and her family for the
recent months). Without your presence and love, my U.S. experience would have been
less colorful. You challenged and provided the balance in my life that carried me through
times of much uncertainty. I am more than looking forward to spending the rest of my
life with you.
Thank you all!
OB
August 2000
Atlanta, GA.
Oliver Bandte iv
TABLE OF CONTENTS
ACKNOWLEDGEMENTS _______________________________________________iii
TABLE OF CONTENTS _________________________________________________iv
LIST OF FIGURES_____________________________________________________ vii
LIST OF TABLES _____________________________________________________ xii
NOMENCLATURE ____________________________________________________xiii
SUMMARY __________________________________________________________xvi
CHAPTER I - INTRODUCTION___________________________________________ 1
Systems Engineering_______________________________________________ 1
Systems Design ___________________________________________________ 4
Uncertainty in Systems Design ______________________________________ 13
Research Quest __________________________________________________ 20
CHAPTER II - DETERMINISTIC MULTI-CRITERIA DECISION MAKING
TECHNIQUES _____________________________________________________ 24
Product Selection_________________________________________________ 26
Optimization ____________________________________________________ 34
CHAPTER III - PROBABILISTIC DESIGN METHODS_______________________ 49
Metamodel/Monte-Carlo Simulation _________________________________ 52
Fast Probability Integration_________________________________________ 55
Oliver Bandte v
Illustrative Example ______________________________________________ 60
CHAPTER IV - MULTI-VARIATE PROBABILITY THEORY _________________ 66
CHAPTER V - A NEW PROBABILISTIC MULTI-CRITERIA DECISION MAKING
TECHNIQUE ______________________________________________________ 71
Introduction _____________________________________________________ 71
Algorithms______________________________________________________ 74
Five Schemes for Implementation____________________________________ 85
Joint Probabilistic Decision Making Technique _________________________ 98
Optimization ___________________________________________________ 103
Product Selection________________________________________________ 112
Requirement Trade-Off ___________________________________________ 118
CHAPTER VI - IMPLEMENTATION INTO SYSTEMS DESIGN______________ 122
Feasibility Problem ______________________________________________ 122
Optimization of a Supersonic Commercial Transport____________________ 131
Product Selection________________________________________________ 140
Requirements Trade-Off Analysis and Discussion ______________________ 150
Example Equation System with Ten Criteria __________________________ 158
CHAPTER VII - CONCLUSIONS________________________________________ 166
Joint Probabilistic Decision Making Technique in Design________________ 166
Algorithms for Determining the Joint Probability Distribution ____________ 167
JPDM for Optimization___________________________________________ 169
JPDM for Product Selection _______________________________________ 171
Oliver Bandte vi
Research Questions and Answers ___________________________________ 173
Recommendations _______________________________________________ 174
APPENDIX A - DISTRIBUTIONS _______________________________________ 177
APPENDIX B - CORRELATION FUNCTIONS ____________________________ 179
APPENDIX C - NUMBER OF SAMPLES FOR THE MONTE-CARLO
SIMULATION ____________________________________________________ 184
APPENDIX D - JOINT PROBABILISTIC DECISION MAKING TECHNIQUE
- THE COMPUTER PROGRAM -_____________________________________ 187
List of Files ____________________________________________________ 189
BIBLIOGRAPHY_____________________________________________________ 209
VITA _______________________________________________________________ 220
Oliver Bandte vii
LIST OF FIGURES
Figure 1.1: Design As a Decision Making Process 21
Figure 2.1: Decision Tree for MADM Technique Selection 28
Figure 2.2: Euclidean Distance to the Ideal and Negative-Ideal Solutions 33
Figure 2.3: Decision Tree for MODM Technique Selection 35
Figure 2.4: Parametric Method of Two Criteria with a Convex Set 44
Figure 2.5: Parametric Method of Two Criteria with a Nonconvex Set 44
Figure 2.6: Surface Plot for f
1
and f
2
45
Figure 3.1: Probabilistic Design Method #1 50
Figure 3.4: Face Centered Central Composite Design 54
Figure 3.5: Objective Function Contours 56
Figure 3.6: Joint Probability Distribution 56
Figure 3.7: Most Probable Point Location 57
Figure 3.8: Visualization of MPP 57
Figure 3.9: AMV Method 59
Figure 3.10: Change in Optima for f
1
and f
2
with Changing y
1
and y
2
61
Figure 3.11: CDF Comparison of Method #1 and Method #3 for F
1
63
Figure 3.12: CDF Comparison of Method #1,#2, and #3 for F
2
63
Figure 3.13: Response Surface for Equation 3.5 64
Figure 3.14: Response Surface for Equation 3.6 64
Figure 3.15: Response Surface for Equation 3.5 (10,000 Samples) 65
Figure 3.16: Response Surface for Equation 3.5 (1,000 Samples) 65
Oliver Bandte viii
Figure 4.1: Joint and Marginal PDF of Continuous Criteria X and Y 68
Figure 5.1: Filling the Gap in the Design Process 73
Figure 5.2: Applicability of MADM, MODM, and JPDM 74
Figure 5.3: Gamma-Distribution for X 84
Figure 5.4: Weibull-Distribution for Y 84
Figure 5.5: Example Joint Probability Distribution in 3D 84
Figure 5.6: Contour Plot of Example Joint Probability Distribution 84
Figure 5.7: Five Schemes for the Evaluation of the Joint Probability Distribution 85
Figure 5.8: Example Distribution Generated by Scheme #I (3D) 87
Figure 5.9: Example Distribution Generated by Scheme #I (2D) 87
Figure 5.10: Distribution Regression for F
1
in Scheme #II 88
2
in Scheme #II 88
Figure 5.12: Example Distribution Generated by Scheme #II (3D) 89
Figure 5.13: Example Distribution Generated by Scheme #II (2D) 89
Figure 5.14: Example Distribution Generated by Scheme #III (3D) 91
Figure 5.15: Example Distribution Generated by Scheme #III (2D) 91
1
in Scheme #IV 92
2
in Scheme #IV 92
Figure 5.18: Example Distribution Generated by Scheme #IV (3D) 93
Figure 5.19: Example Distribution Generated by Scheme #IV (2D) 93
2
in Scheme #V 95
Figure 5.21: Example Distribution Generated by Scheme #V (3D) 96
Figure 5.22: Example Distribution Generated by Scheme #V (2D) 96
Figure 5.23: Comparison of Joint Distributions from Schemes #I and #II 97
Figure 5.24: Comparison of Joint Distributions from Schemes #III and #IV 97
Figure 5.25: Comparison of Joint Distributions from Schemes #II, #IV and #V 98
Oliver Bandte ix
Figure 5.26: Display of the Joint Probability Distribution for Two Criteria 99
Figure 5.27: Joint Probabilistic Decision Making Technique 100
Figure 5.28: Shifting the Joint Probability Distribution During Optimization 104
Figure 5.29: Joint Probabilistic Decision Making Technique for Optimization 105
Figure 5.30: POS Surface Plot Over Design Space 109
Figure 5.31: Location of Optima of MaxiMin, OEC, and Goal Attainment Method for
100 Samples 110
Figure 5.32: Location of Optima for Different Probabilistic Multi-Objective Optimization
Methods 110
Figure 5.33: Comparing Joint Probability Distributions for Product Selection 114
Figure 5.34: Joint Probabilistic Decision Making Technique for Product Selection 115
Figure 5.35: Comparison of Alternatives Based on POS with or without Preferencing 118
Figure 5.36: Requirement Trade-Off to Gain POS 120
Figure 5.37: Trade-Off to Tighten Requirement 120
Figure 5.38: Joint Probabilistic Decision Making Technique for Requirement Trade-Offs 121
Figure 6.1: Feasible Space in a Design Space 123
Figure 6.2: Five Steps to Aircraft Design 124
Figure 6.3: Five Steps to Aircraft Design with JPDM 125
Figure 6.4: Supersonic Transport Concept 126
Figure 6.5: Illustration of the Kink Location 128
Figure 6.6: Determination of Feasibility with JPDM 129
Figure 6.7: Joint EDF for Take-Off Field Length and Approach Speed (2D) 130
Figure 6.8: Joint EDF for Take-Off Field Length and Approach Speed (3D) 130
Figure 6.9: Joint Normal Distribution for Take-Off Field Length and Approach Speed (2D) 130
Figure 6.10: Joint Normal Distribution for Take-Off Field Length and Approach Speed (3D) 130
Figure 6.11: Notional Supersonic Transport 131
Oliver Bandte x
Figure 6.12: JPDM as Optimization Process 136
Figure 6.13: Optimization Iteration History of POS and Vapp 137
Figure 6.14: Optimization Iteration History of OEC, POS, and Vapp 138
Figure 6.15: B-747 140
Figure 6.16: B-777 140
Figure 6.17: A340 141
Figure 6.18: Supersonic Transport 141
Figure 6.19: JPDM for Product Selection 142
Figure 6.20: Joint Probability Plot for ROI and TOC 144
Figure 6.21: Joint Probability Plot for REV and TOC 144
Figure 6.22: Magnified Joint Probability Plot for ROI and TOC 145
Figure 6.23: Magnified Joint Probability Plot for REV and TOC 145
Figure 6.24: Joint Probability Plot for REV and TOC with New Area of Interest 149
Figure 6.25: Comparison of POS as a Function of Revenue and Cost Requirements 151
Figure 6.26: Joint Probability Plot for REV and TOC (A340 is best) 152
Figure 6.27: Joint Probability Plot for REV and TOC (B-777 is best) 152
Figure 6.28: Joint Probability Plot for REV and TOC (B-747 is best) 153
Figure 6.29: Joint Probability Plot for REV and TOC (SST is best) 153
Figure 6.30: Sensitivity Plot for Changes in Mean for the B-747 and B-777 157
Figure 6.31: Sensitivity Plot for Changes in Standard Deviation for the B-747 and B-777 157
Figure 6.32: Surfaces of Ten Equations 159
Figure 6.33: Maxima of Ten Equations 159
Figure 6.34: POS Response Surface 162
Figure 6.35: OEC Response Surface 162
Figure 6.36: Optimal Solutions 163
Figure B.1: Surface Plot of Equation 5.18 for Two Variables 179
Oliver Bandte xi
Figure B.2: Surface Plot of Equation B.1 180
Figure B.3: Surface Plot of Equation B.2 180
Figure B.4: Joint Probability Plot for Two Beta-Distributions ( = 0) 181
Figure B.5: Joint Probability Plot for Two Beta-Distributions with Correlation
Function B.1 ( = 0.09) 181
Function B.1 ( = 0.9) 181
Figure B.7: Joint Probability Plot for Beta-Distributions with Correlation
Function B.3 ( = 4.9) 182
Function B.4 ( = 4.9) 182
Function B.5 ( = 4.9) 183
Function B.6 ( = 4.9) 183
Figure C.1: Number of Samples as a Function of Probability Level and Percent Error 186
Figure D.1: Flow Chart of the Joint Probabilistic Decision Making Technique 187
Oliver Bandte xii
LIST OF TABLES
Table 2.1: Comparison of Three Multi-Criteria Optimization Techniques ________________ 46
Table 2.2: Normalized Comparison of Three Multi-Criteria Optimization Techniques ______ 48
Table 5.1: Comparison of Joint and Univariate Probabilities__________________________ 112
Table 6.1: Description of the Baseline ___________________________________________ 126
Table 6.2: Design Variable Description and Range _________________________________ 128
Table 6.3: Range of Allowable Design Variable Values for Optimization _______________ 132
Table 6.4: Economic Parameter Distributions _____________________________________ 134
Table 6.5: Four Alternatives for Product Selection Problem __________________________ 141
Table 6.6: Noise Variable Distributions __________________________________________ 143
Table 6.7: Summary of POS for each Alternative __________________________________ 146
Table 6.8: Comparison of POS for Different ROI
min
Values __________________________ 150
Table 6.9: Comparison of POS Values for Different Criterion Preferences_______________ 154
Table 6.10: Old and New Noise Variable Distributions ______________________________ 155
Table 6.11: Old and New POS Values and Sensitivities for B-747 and B-777 ____________ 157
Table 6.12: Comparison of Joint and Univariate Probabilities for Different Objective
Functions ____________________________________________________________ 165
Table 7.1: Advantages and Disadvantages of EDF and JPM__________________________ 169
Oliver Bandte xiii
NOMENCLATURE
: Mean; expected value of a random variable
Correlation Coefficient; normalized covariance of two random
variables
: Standard Deviation; measure of dispersion or variability of random
variable values, determined by their deviation from the mean

2
: Variance of a random variable
AMV: Advanced Mean Value method; FPI technique (see page 55)
CDF: Cumulative Distribution Function
DOE: Design of Experiments; set of evaluation points that a metamodel
is based on
EDF: Empirical Distribution Function; discrete probability distribution
based on random samples
FPI: Fast Probability Integration; computer program; collection of
techniques for efficient evaluation of probability distributions of
system characteristics
JPDM: Joint Probabilistic Decision Making technique; product of this
thesis
JPM: Joint Probability Model; joint probability distribution (product of
this thesis)
Oliver Bandte xiv
MADM: Multi-Attribute Decision Making = product selection
MCDM: Multi-Criteria Decision Making; comprised of MADM and
MODM; decision making on the basis of multiple, often
conflicting, criteria
MCS: Monte-Carlo simulation: technique for generating random numbers
MODM: Multi-Objective Decision Making = optimization
OEC: Overall Evaluation Criterion; utility function technique for product
selection and optimization [OEC =
N
i
i i
f w
1
) (x ]
PDF: Probability Density Function
POS: Probability of Success; probability of meeting customer
requirements and desirements; objective function for probabilistic
multi-criteria optimization and product selection
RSE: Response Surface Equation
RSM: Response Surface Methodology; linear regression technique for
creating metamodels (RSE)
Attribute: Criterion that describes in part the state of a system
Covariance: Measure of the degree of (linear) interrelationship between values
of two random variables
Correlation: Mathematical correspondence of values of two random variables
Criterion: Standard of judgment to test acceptability
Desirement: Customer supplied want that impacts product design
Oliver Bandte xv
Engineering Design: Process that creates a product, which satisfies a need that was not
(sufficiently) satisfied by already existing products
Joint Distribution: Probability distribution of two or more random variables
Metamodel: Model that approximates a more complex model
Model: Mathematical representation of a system
Model Fidelity: Prediction accuracy of a model
Objective: Customer supplied criterion that forms (in part) the bases for the
decision making process
Optimization: Process of finding the best solution among an infinitely large
(open) set of alternatives
Probabilistic Design: Design process that accounts for uncertainties probabilistically
Product Selection: Process of finding the best solution among a fixed (closed) set of
alternatives
Random Variable: Variable with uncertain value; likelihood of values is described by
a probability distribution
Readiness: Level to which a particular technology or concept is implemented
into the system
Requirement: Customer supplied need that must be fulfilled
System: Group of components, attributes, and relationships needed to
accomplish an objective
Trade-Off: Process of giving up one thing for another
Uncertainty: Inability to predict the future with certainty; three classes:
environmental/operational, fidelity, readiness
Oliver Bandte xvi
SUMMARY
It has always been the intention of systems engineering to invent or produce the
best product possible. Many design techniques have been introduced over the course of
decades that try to fulfill this intention. Unfortunately, no technique has succeeded in
combining multi-criteria decision making with probabilistic design. The design
technique developed in this thesis, the Joint Probabilistic Decision Making (JPDM)
technique, successfully overcomes this deficiency by generating a multivariate
probability distribution that serves in conjunction with a criterion value range of interest
as a universally applicable objective function for multi-criteria optimization and product
selection. This new objective function constitutes a meaningful metric, called Probability
of Success (POS), that allows the customer or designer to make a decision based on the
chance of satisfying the customers goals. In order to incorporate a joint probabilistic
formulation into the systems design process, two algorithms are created that allow for an
easy implementation into a numerical design framework: the (multivariate) Empirical
Distribution Function and the Joint Probability Model. The Empirical Distribution
Function estimates the probability that an event occurred by counting how many times it
occurred in a given sample. The Joint Probability Model on the other hand is an
analytical parametric model for the multivariate joint probability. It is comprised of the
product of the univariate criterion distributions, generated by the traditional probabilistic
Oliver Bandte xvii
design process, multiplied with a correlation function that is based on available
correlation information between pairs of random variables.
JPDM is an excellent tool for multi-objective optimization and product selection,
because of its ability to transform disparate objectives into a single figure of merit, the
likelihood of successfully meeting all goals or POS. The advantage of JPDM over other
multi-criteria decision making techniques is that POS constitutes a single optimizable
function or metric that enables a comparison of all alternative solutions on an equal basis.
Hence, POS allows for the use of any standard single-objective optimization technique
available and simplifies a complex multi-criteria selection problem into a simple ordering
problem, where the solution with the highest POS is best. By distinguishing between
controllable and uncontrollable variables in the design process, JPDM can account for the
uncertain values of the uncontrollable variables that are inherent to the design problem,
while facilitating an easy adjustment of the controllable ones to achieve the highest
possible POS. Finally, JPDMs superiority over current multi-criteria decision making
techniques is demonstrated with an optimization of a supersonic transport concept and ten
contrived equations as well as a product selection example, determining an airlines best
choice among Boeings B-747, B-777, Airbus A340, and a Supersonic Transport. The
optimization examples demonstrate JPDMs ability to produce a better solution with a
higher POS than an Overall Evaluation Criterion or Goal Programming approach.
Similarly, the product selection example demonstrates JPDMs ability to produce a better
solution with a higher POS and different ranking than the Overall Evaluation Criterion or
Technique for Order Preferences by Similarity to the Ideal Solution (TOPSIS) approach.
Oliver Bandte Chapter I 1
CHAPTER I
INTRODUCTION
The history of creating tools and objects that people need is as long as the history
of humanity itself. Man has always striven to better his position. For many situations,
this meant the invention of new tools or devices that helped to overcome obstacles people
were facing. In each case, however, there was an initial need identified that led to the
invention and production of the new device. If those needs are of material nature, it is
fair to say that engineers today typically answer those needs by designing products or
designing processes that produce products. Beakley and Leach point out that, in general,
engineers do the things required to serve the needs of the people and their culture. Their
job is to take knowledge and make practical use of it and in doing so provide for mans
material needs and well being.[Beakley, Leach, 1972] Thus, engineering design can be
described as a process that creates a product, which satisfies a need that was not
(sufficiently) satisfied by already existing products.
Systems Engineering
However, the satisfaction of a particular need often produced new ones. Recent
decades have seen material needs that are so complex that traditional engineering design
methods are no longer suited to yield products that satisfy these needs. These products
are not simple devices that perform a single function, but rather complex systems that
fulfill a number of tasks. The term system stems from the Greek word systema, which
means organized whole. With the engineering process in mind, a system is, by
definition, a group of activities or components that can be bounded where the bounding
rule is: all relevant interdependencies and interactions must be enclosed.[Starr, 1963]
Blanchard and Fabrycky refine this definition by stating that the total system, at
whatever level in the hierarchy, consists of all components, attributes, and relationships
needed to accomplish an objective. Each system has an objective, providing a purpose
for which all system components, attributes, and relationships have been organized.
Constraints placed on the system limit its operation and define the boundary within which
it is intended to operate. Similarly, the system places boundaries and constraints on its
subsystems.[Blanchard, Fabrycky, 1998]
That the objects of concern in aerospace engineering are systems is quite obvious.
An aircraft (system), for example, is comprised of such components as the wing,
fuselage, engines, empennage, landing gear, and so on. Each of these components can be
divided up into smaller units or components. Thus, they are often referred to as
subsystems, showing all characteristics of a system but at a lower level of hierarchy in the
aircraft system. Therefore, general systems engineering methods are extremely
applicable to aerospace systems. Specifically, the following three system elements can
be defined [Blanchard, Fabrycky, 1998]:
1. Components are the operating parts of a system consisting of input, process, and
output. Each system component may assume a variety of values to describe a
system state as set by some control action and one or more restrictions.
2. Attributes are the properties or discernible manifestations of the components of a
system. These attributes characterize the system.
3. Relationships are the links between components and attributes.
If the system is then to satisfy a material need, as argued by Beakley and Leach,
the process that brings the system about can be called systems engineering.[Beakley,
Leach, 1972] Websters defines systems engineering as a branch of engineering using
especially information theory, computer science, and facts from system-analysis studies
to design integrated operational systems for specific complexes.[Webster, 1996]
Blanchard adds to this definition the creative aspect of engineering by defining, systems
engineering as the orderly process of bringing a system into being. A system constitutes
a complex combination of resources (in the form of human beings, materials, equipment,
software, facilities, data, etc.) integrated in such a manner as to fulfill a designated
need.[Blanchard, 1998] Both definitions recognize the engineered system as an
assemblage or combination of elements or parts forming a complex or unitary whole,
[while its] interrelated components [work] together toward some common objective or
purpose.[Blanchard, Fabrycky, 1998] However, systems engineering per se is not
considered as an engineering discipline in the same context as civil engineering,
mechanical engineering, reliability engineering, or any other design specialty area.
Actually, system engineering involves the efforts pertaining to the overall design and
development process employed in the evolution of a system from the point when a need
is first identified, through production and/or construction and the ultimate installation of
that system for consumer use. The objective is to meet the requirements of the consumer
in an effective and efficient manner.[Blanchard, 1998]
Systems Design
Blanchards remark points out that a part of systems engineering involves the
design of the system. According to Websters dictionary, to design is to plan and carry
out in a skillful way or to form [something] in ones mind.[Webster, 1996] Dieter,
however, argues that this definition does not emphasize enough the fact, that to design
something means to create something that has never been.[Dieter, 1991] However,
when a design has a particular aim, it is a purposeful activity directed toward the goal of
fulfilling human needs, particularly those which can be met by the technological factors
of our culture.[Asimow, 1962] If the satisfaction of this need is the ultimate aim of
design, a process has to be established that guarantees the fulfillment of this goal. This
process is an essential element of design theory, which extends intuition and experience
in (systems) engineering design by providing a framework for evaluating and extending
design concepts. A design theory would make it possible to answer such questions as: Is
this a good design?; Why is this design better than others?; How many design
parameters (DPs) do I need to satisfy the functional requirements (FRs)?; Shall I
abandon the idea or modify the concept?[Dieter, 1991]
Decision Making in Systems Design
These questions, on the other hand, constitute the decision making problem in
systems engineering. Websters dictionary defines decision making as the act of making
up ones mind, judging, or reaching a conclusion about something.[Webster, 1996] The
connection between decision making and design may not be apparent from this
definition, but when considering the engineers aim of product creation, it can be realized
that design, or resource allocation, involves the process of decision making. As
Hazelrigg reasons, a decision is an irrevocable allocation of resources. The selection of
design parameters for an engineering system such as a computer or an automobile
constitutes an allocation of resources. [Therefore,] design is a decision making process,
and the selections of design parameters represent decisions.[Hazelrigg 1996]
The close relation between design and decision making can also be seen from
Dixons definition: a decision making problem exists, then, when and only when there is
an objective to be reached, alternative methods of proceeding, and a variety of factors
that are relevant to the evaluation of the alternatives or their probability of
success.[Dixon, 1966] The objective in design is the satisfaction of a need, while
alternative methods are courses of action that can be taken during the design process.
Factors relevant for the evaluation of alternatives are the design variables, controlled by
the engineer, that comprise the system and performance parameters by which the system
is evaluated. From these three groups, three elements can be extracted that concern us
in critical decision making, as it appears in the process, [which] are the alternatives, the
benefits, and the difficulties of implementation.[Asimow, 1962] While the difficulties
of implementation have to be seen in a broader context, comprising economic penalties
as well as technical and environmental constraints.
Defining the objective in design somewhat more narrowly, one could say that
the objectives are customer supplied guidelines that form the bases for the decision
making process. This notion links the customers need directly to the decision making
process. The importance of this link is also pointed out by Starr: the decision problem is
to choose that strategy, which best satisfies the decision-makers objectives. If the
designer does not know what objectives apply, then only by fortuitous selection can he
succeed.[Starr, 1963]
This statement clearly shifts the emphasis of decision making towards the
definition and selection of the objectives, which can only be achieved through a needs
analysis. The needs analysis consists of listing the user needs for the design in brief,
succinct phrases. Each user need should be identified with the basic need it represents.
In addition, there will be technological (performance) needs, time needs, and cost needs.
The needs analysis is not complete until we estimate what resources the user will
exchange for satisfying his needs.[Dieter, 1991] However, a needs analysis alone is not
enough to supply the engineer with the information required to proceed with the design
process. First, the problem should be defined more precisely to enable the systems
engineer to make educated decisions about the product with respect to the problem that
needs solving. Asimow and Dieter point out the essentials of the problem definition:
before an attempt is made to find possible solutions for the means of satisfying the need,
the design problem should be identified and formulated. The information we have
available [for this] comes from the results of the preceding step, [establishing the need,]
particularly the specifications of desired outputs, and from relevant technical knowledge
about environments, resources and general engineering principle.[Asimow, 1962] And
further, problem definition is based on identifying the true needs of the user and
formulating them in a set of goals for the problem solution. The problem statement
expresses as specifically as possible what is intended to be accomplished to achieve the
goals. Design specifications are a major component of the problem statement.[Dieter,
1991]
Consequently, the problem definition yields a set of objectives on which the
engineer can base his design decisions. These parameters are called criteria, described
by Ostrofsky as the means by which the performance of a system is evaluated. As such,
then, the criteria must emerge from the needs analysis and the problem formulation of the
[systems engineering process].[Ostrofsky, 1977] These criteria play the essential role in
the decision making process, deeming an alternative solution successful, when their
customer desired levels are met. Asimow states clearly that as soon as we raise the
problem of finding which of the feasible solutions is best, we are forced to state with
precision the rules by which we are to judge the vague quality of excellence. We must
name and define the attributes, which are to be considered, we must specify how they are
to be measured, and we must establish relative importance. We will refer to such a
composite statement as the design criterion. If the optimization is to be performed
mathematically, these considerations must be set forth in an equation which we shall call
the criterion function.[Asimow, 1962]
In some traditional approaches to design, these criteria have been termed
design-to parameters, as Blanchard and Fabrycky point out: evolving from the
operational requirements and [the customer need] is the development of qualitative and
quantitative design-to criteria, [such as system size and weight, range, speed,
performance, capacity, operational availability, reliability, maintainability, supportability,
and cost]. Of particular interest are the quantitative factors or metrics associated with the
system being developed. These metrics, or technical performance measures (TPMs), lead
to the identification of design-dependent parameters (DDPs) and the desired
characteristics that should be incorporated into the design, and must be established
initially as part of the requirements definition process during conceptual
design.[Blanchard, Fabrycky, 1998] It is important to understand that a design-to-
criterion approach to design potentially yields a design that satisfies the specified
criterion exclusively. All other criteria that the system was not specifically designed for
may not be satisfied at all, yielding a lower customer satisfaction and, potentially, a
failure to meet the specified need. Blanchard and Fabrycky even go so far as to claim
that a single criterion system design decision is the exception rather than the rule.
Multiple criteria considerations arise when both economic and noneconomic elements are
present in the evaluation. In these situations, decision evaluation is facilitated by the use
of a decision evaluation display exhibiting both cost and effectiveness
measures.[Blanchard, Fabrycky, 1998]
If the decision problem entails several criteria, experience has shown that not all
of them are of equal importance to the customer. Which criterion is more important than
another is often not for the designer to decide, but must be part of the needs analysis
described earlier. This fact is pointed out again by Blanchard and Fabrycky: some of
these factors may be considered to be more important than others by the customer, which
will, in turn, influence the design process in placing different levels of emphasis on the
selection of design criteria. The result is the identification and prioritization of technical
performance measures (TPMs) for the system overall.[Blanchard, Fabrycky, 1998] The
importance of preferences among criteria in the decision process should not be
underestimated, since the design outcome depends heavily on them and different
preferences can produce drastically different designs.
Mathematical Models in Systems Design
In order to get an accurate prediction of the systems operational behavior or state,
it is vital that the criteria for the decision making process are quantifiable metrics or
TPMs, as termed by Blanchard and Fabrycky. Dixon emphasizes this point when he
argues that an engineering design decision question is one, which can be answered in
terms of parameters that can be calculated or measured. In other words, a question must
be asked that can be answered quantitatively. This is not always easy. It involves
translation of a real physical situation into a paper-and-pencil question. For example, it is
not enough to ask of a system, Will it work? Such a question is neither operational nor
specific. Instead, questions must be asked such as: What temperature is too hot? What
does quickly mean in seconds? An engineering analyst must begin by defining
quantitatively answerable questions.[Dixon, 1966]
Accommodating the need for quantifiable criteria, analytical methods must be
employed in the design process that yield these criteria. Mathematical models typically
lend themselves best to a systems engineering design approach, since they are frequently
used in engineering, fast in producing the sought for answers, and yield the calculable,
hence quantifiable, outcome desired. Further, Asimow emphasizes the availability and
ease of manipulation of information about the product through the use of mathematical
models. Specifically, he points out that, as useful as verbal descriptions and graphic
illustration are, symbolic descriptions are uniquely useful, for they can be manipulated
with the facility of mathematical logic in pursuing the implications, which are dormant in
the concept. The symbolic description becomes a device, which enables the designer
to use information about the concept in order to anticipate analytically the behavior of the
prototype. In this sense, the symbolic description becomes a mathematical archetype of
the physical object which is yet to be materialized.[Asimow, 1962]
The ease of obtaining answers in the design evaluation processes is particularly
important for systems engineering. The complexity of the product and the related design
processes demands a design method that allows the designer to make informed decisions
at any level in the design process. Blanchard and Fabrycky specifically emphasize the
benefit of mathematical models to systems engineering: the use of mathematical models
offers significant benefits. There are many interrelated elements that must be
integrated as a system and not treated on an individual basis. The mathematical model
makes it possible to deal with the problem as an entity and allows consideration of all
major variables of the problem on a simultaneous basis.[Blanchard, Fabrycky, 1998]
Last but not least, Dieter denotes that design solutions need to be communicated and that
mathematical models are especially helpful here: engineers use models for thinking,
communications, prediction, control, and training. Since many engineering problems
deal with complex situations, a model often is an aid to visualizing and thinking about the
problem. Models are vital for communicating, whether via the printed page, the
computer screen, or oral presentation. Generally, we do not really understand a problem
thoroughly until we have predictive ability concerning it. Engineers must make decisions
concerning alternatives. The ability to simulate the operation of a system with a
mathematical model is a great advantage in providing sound information, usually at lower
cost and in less time than if experimentation had been required.[Dieter, 1991]
Phases in Systems Design
On the other hand, in the past, the early front-end analysis as applied to many
new systems has been minimal.[Blanchard, Fabrycky, 1998] It is at the front-end of
the design process, however, where the requirements are established from the need
analysis. The so called conceptual design phase encompasses the activities related to
the identification of customer need and the several steps involved in the definition of
system design requirements.[Blanchard, Fabrycky, 1998] This design phase, where the
basic design concepts are developed, encompasses many critical decisions that drive the
system production and operational characteristics. The decisions are critical, since their
impact is large on the system and changes to the design concept at later stages are
extremely expensive, as Blanchard, Fabrycky, and Ertas point out: it is at this early stage
in the life cycle (i.e., the conceptual design phase) that major decisions are made relative
to adapting a specific design approach, and it is at this stage that the results of such
decisions can have a great impact on the ultimate characteristics and life-cycle cost of a
system.[Blanchard, Fabrycky, 1998] Once the decision to proceed has been made and
funding approval has been granted, it becomes increasingly costly and difficult to make
changes. The increasing cost of making changes as programs develop is, of course,
due to the sunk cost manpower and materials as well as to the difficulty in reassigning or
terminating an increasingly large number of employees.[Ertas, 1993] Thus, these
decisions should be as educated as possible to make the best decisions possible. In other
words, it is during the conceptual design phase where mathematical modeling has the
most potential to make a difference in the decision making process.
Employing mathematical models in the conceptual design phase will inevitably
move efforts traditionally associated with preliminary design into the conceptual design
phase. The purpose of preliminary design is to establish which of the proffered
alternatives is the best design concept. Each of the alternative solutions is subjected to
order of magnitude analyses until the evidence suggests either that the particular solution
is inferior to some of the others, or that it is superior to all of the others. The surviving
solution is tentatively accepted for closer examination. Synthesis studies are initiated for
establishing, to a first approximation, the fineness of the range within which the major
design parameters of the system must be controlled. Further studies investigate the
tolerances in the characteristics of major components and critical materials, which will be
required to insure mutual compatibility and proper fit into the system. Other studies
examine the extent to which perturbations of environmental or internal forces will affect
the stability of the system.[Asimow, 1962] For aerospace systems specifically, this
means that during preliminary design the specialists in areas such as structures, landing
gear, and control systems will design and analyze their portion of the aircraft [system].
Testing is initiated in areas such as aerodynamics, propulsion, structures, and stability
and control. A mockup may be constructed at this point.[Raymer, 1992] With these
descriptions of preliminary and the earlier established intentions of conceptual design, the
use of mathematical models in conceptual as well as preliminary design merges both
phases into one, yielding the ability of the designer to make better, more educated
decisions in satisfying the customer need.
Uncertainty in Systems Design
Despite the use of mathematical models, certain decisions the designer makes will
have to be based on assumptions rather than certain knowledge, due to incomplete
information about the operational environment, availability of new technology, or simply
the uncertainty about the prediction accuracy of the models themselves. The needs
analysis can only describe what kind of qualities the system must or must not have. It
does not provide, in general, the characteristics of the environment in which the system
will operate. This is often left to the designer to determine and assess. However, there
is usually little assurance that predicted futures will coincide with actual futures. The
physical and economic elements on which a course of action depends may vary from
their estimated values because of chance causes. [On the other hand,] this lack of
certainty about the future makes decision making one of the most challenging tasks faced
by individuals, industry, and government.[Blanchard, Fabrycky, 1998]
Asimow claims more generally, that the work of design demands a constant
peering ahead through the curtains of time; for a project started in the present will not be
completed until some time in the future, and the actual product will not be used until an
even more remote time. Two main questions should be asked: one concerns the socio-
economic environment that will exist when the product comes into actual use; the other
refers to the race against technical obsolescence.[Asimow, 1962] The second question
refers to the problem of uncertainty about the availability of anticipated technological
concepts. Obviously, current technology is readily available for implementation in the
system. However, as Asimow states, it may also be obsolete when the system is actually
fielded. Even worse, certain needs of the customer may not even be met by a system that
is solely built with current technologies. Thus, new technological solutions have to be
found, applied to the components, and incorporated into the system. But these
technological solutions may only be at a conceptual stage in their development, i.e.,
several questions remain concerning their readiness for implementation when needed and
their actual performance level once implemented. On the other hand, technology
readiness is a function of money and time allotted for completion of the technology
development. Asimow points out that virtually any solution, even a very difficult one,
provided it is physically permissible (e.g., not a perpetual motion machine), can be
carried through to a physically realizable design if a sufficiently large amount of money
is appropriated and an indefinite amount of design time allowed. If the budget of time
and money is limited, then whether or not the design is realizable becomes uncertain.
The greater the limitation, the more uncertainty it raises.[Asimow, 1962]
The third source of uncertainty, rooted in the prediction accuracy of the models
used in the design process, is often called model fidelity. Dixon summarized this
frequently overlooked form of uncertainty best when he wrote that engineering models
are not exact replicas of real physical situations. Most nonengineers feel or believe that
engineering is an exact and precise activity. By comparison with the activities of, say,
psychologists, this is quite true. In an absolute sense, however, it is not true at all. The
real physical situations with which engineers work are very complex and are never
analyzed exactly or completely. No one even tries to analyze them exactly or completely.
Assumptions, idealizations, and approximations are expected.[Dixon, 1966]
In general, uncertainty can be separated into two system specific categories. Part
of it resides solely with the subsystems or components of the system, while part of it
stems from the characteristics of the interactions among several of the
subsystems.[Asimow, 1962] While operational uncertainty resides at the system level,
readiness uncertainty is surely attributable to the component technologies. Fidelity
uncertainty then resides with the actual components as well as their interactions.
It is thus the designers responsibility to examine how the system will behave in
the future by virtue of its own inherent characteristics. The decision for a particular
design or class of designs will then have to be based on the level of satisfaction of a
certain set of desired outputs in a particular range of environments.[Asimow, 1962] This
design solution is typically called robust design solution, since its performance
parameters are invariant or almost invariant, i.e. robust, with respect to changes in the
production or operational environment.
Probability Theory in Systems Design
Simply acknowledging the existence of uncertainty in the decision making
process of systems engineering, however, does not indicate how to account for it in the
design process. Decision making under risk occurs when the decision-maker does not
suppress acknowledged ignorance about the future but makes it explicit through the
assignment of probabilities. Such probabilities may be based on experimental evidence,
expert opinion, subjective judgment, or a combination of these.[Blanchard, Fabrycky,
1998] Asimow provides an intuitive reason for the use of probability estimates by
writing: before we actually choose a particular action, what do we believe is our
probability of successfully carrying it out? It is precisely at this point in our chain of
reasoning that we make a subjective estimate of our chances. We can elect to leave this
estimate implicit and submerged beneath our impressions, or we can try to draw it out in
an explicit statement as a quantitative expression of our level of confidence.[Asimow,
1962]
While the introduction of probability estimates captures the uncertainty in
decision making, it is not until these estimates are translated into usable information to
the decision-maker that their full benefit can be felt. Traditionally, systems analysis
provides this information. Cleland and King define systems analysis as a process that
involves [first] systematic examination and comparison of those alternative actions
which are related to the accomplishment of desired objectives, [second] comparison of
alternatives on the basis of the costs and the benefits associated with each alternative,
[and third] explicit consideration of risk.[Cleland, King, 1983] While this definition is
global enough to include a wide variety of decision making problems and tools to answer
them, Silver and Silver believe that systems analysis consists predominantly of operations
research (OR).[Silver, Silver, 1976] The focus on OR within systems analysis allows to
prescribe a body of tools and methods to it that highlights its aim of capturing
uncertainties in decision making/design through mathematical techniques. Silver and
Silver emphasize such techniques as sampling, allocation of resources, including linear
programming and queuing theory, quality control, forecasting, including linear
regression, and simulations with mathematical modeling. Particularly the last group
enables a conversion of the decision making problem into mathematical terms that allow
for examining the effect that a change in one or more elements in a system will have on
the rest of the system without actually making the changes [in the real system].[Silver,
Silver, 1976] Unfortunately, the systems engineering community has been slow in
adopting and applying these tools. It has been the structural and reliability engineers
rather that took to it more easily.[SAE, 1998] Perhaps the inherent uncertainty in
predicting a failure of a structural component paired with the severe consequences of that
failure introduced a need for new, non-traditional techniques of evaluating a successful
design. It is not before long, until this need will be felt in systems engineering as well.
Finally, the use of probability estimates for values of uncontrollable parameters,
technology performance and readiness estimates, as well as model fidelity estimates is
stipulated by the use of simulation and mathematical models in the design process.
Available OR methods need the prediction capabilities of these models. On the other
hand, probabilistic OR methods allow for a full exploration of all possible alternatives,
hence making maximum use of the analytical prediction capability of the models. For
example, a Monte-Carlo simulation will randomly select solutions from the design space
specified, generating numerous alternatives and estimating the chance of satisfying the
customer need, an impossible task without the use of mathematical models. But the
probabilistic analysis also yields information about the range of possible outcomes for the
specified inputs, while a simple deterministic analysis offers nothing but single point
estimates, without knowing their likelihood of occurrence. If the generation of
alternatives for a Monte-Carlo simulation takes too much time due to the complexity of
the analysis or system, fast probability estimation techniques can be employed to yield
the same estimates based on a smaller number of design alternatives. Of course, this
comes at the price of accuracy of the probability estimates. For very simple
mathematical models, analytical probability estimates are possible that do not require any
execution of the models at all. Their probability estimate is exact, but the model itself,
particularly in systems engineering, is probably not very accurate.
Despite the outlined advantages, probabilistic design methodologies have been
faced with some outspoken skepticism, if not criticism, in the engineering community. A
good summary of perceived and actual limitations of modern probabilistic design
techniques has been collected and published in the SAE Standard AIR5086.[SEA, 1998]
Based on a survey among structural and reliability engineers, the G-11 Probabilistic
Methods Committee found two groups of limitations: perceived ones and actual ones.
Among the perceived limitations the committee found a suspected radical departure from
existing practices and lack of compatibility with existing tools, difficulty in use and result
interpretation, too much effort, time, and data necessary, and finally a lack of verification.
The perceived radical departure and lack of compatibility of the probabilistic
techniques with the existing tools is mainly due to misinformation. As a matter of fact,
most probabilistic techniques encompass and build on existing tools, rather than replacing
them. The SAE Standard explicitly states that probabilistic methods are, in fact, usually
close descendants of deterministic methods, born from a need to quantify the effects of
input variability.[SAE, 1998] Chapter III of this thesis provides an overview of these
probabilistic design techniques that are specifically build around existing engineering
tools. On the other hand, the perceived difficulty in use and result interpretation is
simply based on a lack of education in probability theory among engineers. For this
reason, the SAE Standard makes a strong call for an adaptation and inclusion of
probability theory into todays college curricula.[SAE, 1998]
The perceived limitation of probabilistic design techniques due to the extensive
time and effort needed for execution is only real for certain techniques, and can in most
cases be alleviated through parallel processing or metamodels,[SAE, 1998] which is
discussed in more detail in Chapter III. Finally, the perceived limitation of a lack of
verification of the new techniques is an accusation that holds only true when looked upon
with deterministic eyes. But probabilistic techniques do not try to predict a single
number, but rather indicate a likelihood for a range of system characteristic values. The
aim is different and comparison therefore flawed. By the same token, the deterministic
tools used in systems engineering almost never predict the correct number achieved by
the product, but this discrepancy is accepted and attributed to the fact the prediction was
done with a model and not the actual system. The aim of probabilistic techniques is to
capture this prediction error and make it explicit rather than have it appear as the big
surprise during operation.[SAE, 1998]
Among the actual limitations of probabilistic techniques, the SAE Standard
further found a lack of guidelines, the aforementioned computational effort, and difficulty
in validation. The lack of guidelines particularly is due to the fact that probabilistic
techniques have only been emerging recently and have not completely made their way
into the design and certification process for engineering systems. This limitation is
therefore just temporal and will hence diminish with time, since probabilistic techniques
will very soon take on a larger role in design. Similarly, the concern about the
computational efforts needed by many probabilistic techniques is a major field of
research and may therefore be considered a temporal limitation. It is important that
probabilistic techniques are not being dismissed in their entirety because of this
limitation, since it is only relevant for designs involving system analyses that require a lot
of computational resources and time. Finally, the most serious limitation of probabilistic
techniques is their lack of validation. In most cases, it is unreasonable to produce and
operate a statistically significant number of systems just in order to verify their design
process. The SAE Standard recommends instead verifying methods through continuos
collection of data at the component rather than the system level. A further possibility is
the use of Bayesian probability rather than pure statistics, with the benefit of being able to
update probabilistic assumptions as new data becomes available.[SAE, 1998]
Research Quest
In summary then, what is the problem that needs solving in systems engineering
design and is addressed in this thesis? The process of systems engineering, from the
needs analysis to the decision making process, has been outlined in this chapter. But how
is the inherent uncertainty in the design process accounted for in the decision making
procedure? This question is depicted symbolically in Figure 1.1.
Customer
Environment
Need for Product
Requirements/Desirements
Decision Making
Criteria
Systems Engineering
Probabilistic Techniques
Uncertainty
PRODUCT ?
Figure 1.1: Design As a Decision Making Process
Starting at the top, the customer supplies the need for a product. Entering the
systems engineering process, this need is translated through a needs analysis into a set of
requirements and desirements
1
that have the potential of being relaxed and traded-off
against each other. Requirements and desirements supply a set of multiple criteria based
upon which a particular product (concept) decision is made. At the opposite end of the
systems engineering process, the environment introduces uncertainty to the process. This
uncertainty is caused by technology developments and the assumptions made about the
products operation and can be modeled via probabilistic and systems analysis
techniques. The question mark in the middle of the figure represents the missing link
between the multi-criteria decision making process and probabilistic techniques, which
ultimately determines the product the customer will use in its operational environment.

1
Desirements - Customer supplied wants that impact the product design.
Unfortunately, no existing technique has succeeded in combining multi-criteria decision
making with probabilistic design. This void is intended to be filled by the Joint
Probabilistic Decision Making technique researched and developed in this thesis.
To facilitate this research and to structure the development of the technique, the
following research questions are posed, summarizing the issues raised in this
introduction, and will be addressed throughout the thesis in varying detail:
Does uncertainty in the systems engineering design obstruct the decision process?
Can the use of probabilistic design be beneficial in the decision making process?
Is there a numerical value representing customer satisfaction?
Does a technique already exist that can help the decision-maker find a best solution
based on multiple criteria arising from a probabilistic design technique?
Is it possible to create such a technique and what should it look like?
Can this technique be used for optimization?
Can this technique be used for product selection?
The technique proposed in this thesis has its theoretical foundation in two areas:
Multi-Criteria Decision Making and Probabilistic Design. Multi-Criteria Decision
Making techniques have been used for more than 30 years in decision making problems
that involve multiple, conflicting criteria.[Hwang, 1979] A summary of the most
important techniques is provided in Chapter II. Probabilistic Design, on the other hand,
is a fairly new discipline, with only a small body of established literature. Its most
common techniques are outlined in Chapter III. The mathematical background of the
proposed Joint Probabilistic Decision Making technique rests on the joint probability
theory, which is summarized in Chapter IV. Chapter V finally introduces the technique
itself, while Chapter VI demonstrates the techniques application in aerospace systems
design, comparing it to current methods of probabilistic design. Chapter VII summarizes
the technique and its achievements by revisiting the questions asked in this introduction
and highlighting the major findings of this thesis.
Oliver Bandte Chapter II 24
CHAPTER II
DETERMINISTIC MULTI-CRITERIA DECISION MAKING
TECHNIQUES
Decision making is characterized by its involvement with information, value
assessments, and optimization. Thus, whereas inventiveness seeks many possible answers
and analysis seeks one actual answer, decision making seeks to choose the one best
answer.[Dixon, 1966] But the one best answer can be difficult to obtain, particularly
when the decision is based on several objectives. Ching-Lai Hwang has been on the
forefront of the development of new techniques and the enhancement of existing ones
that aid the decision-maker. His two references [Hwang, 1981] and [Hwang, 1979] list a
multitude of techniques, grouped in two classes; the Multi-Attribute Decision Making
(MADM) and Multi-Objective Decision Making (MODM) techniques. This chapter
highlights the most important techniques in both references.
According to Hwang, multiple attribute decision [making] problems involve the
selection of the best alternative from a pool of preselected alternatives described in
terms of their attributes.[Hwang, 1979] Attributes are generally defined as
characteristics that describe in part the state of a product or system, while objectives are
attributes with a goal and a direction to do better as perceived by the decision-
maker.[Hwang, 1979] Specifically, goals are things desired by the decision-maker
expressed in terms of a specific state in space and time. Thus, while objectives give the
desired direction, goals give a desired (or target) level to achieve.[Hwang, 1979] In
many cases, however, the terms objective and goal are used interchangeably. With
this definition for objectives in mind, multi-objective decision making problems involve
the design of alternatives which optimize or best satisfy the objectives of the decision-
maker.[Hwang, 1979] In other words, multi-attribute decision making problems are
product selection problems, multi-objective decision making problems are optimization
problems. Together all techniques for solving both problems can be classified as Multi-
Criteria Decision Making (MCDM) techniques. While criteria typically describe the
standards of judgment or rules to test acceptability, here they simply indicate attributes
and/or objectives. In general, a MCDM problem is described by
} ) ( , ), ( ), ( { max
2 1
x x x
x
N
f f f K (2.1)
subject to a set of constraints
g
r
(x) r = 1, , L, (2.2)
where f
i
, i = 1, , N, are the criteria and x is the K-dimensional vector of design
variables the criteria and constraints depend on. Further, a feasible set X = {x|g(x) 0}
is the design variables x which satisfy the constraint vector g(x) 0. But for each point
in X there is an associated vector f(x), so that X can be mapped into a set
S = {f(x)|x X} in the criteria space. Further, x
*
is called an optimal solution iff
x
*
X and f(x
*
) f(x) for all x X. x
*
is called a nondominated solution iff no x X
exists such that f
i
(x) f
i
(x
*
) for all i and f
j
(x) > f
j
(x
*
) for at least one j i, i, j = 1, 2, , N.
In other words, a nondominated solution, also called a Pareto-optimal solution, is
achieved when no criteria can be improved without simultaneous detriment to at least one
other criterion.[Sen, Yang, 1998]
Product Selection
The three elements that concern us in critical decision making, as it appears in
the process, are the alternatives, benefits, and difficulties of implementation.[Asimow,
1962] The techniques in this section are concerned with exactly these three elements of
decision making. This class of Multi-Attribute Decision Making (MADM) techniques
determines the best of a finite and often small set of alternative solutions, based on the
attribute levels achieved and their indicated preferences. The final selection of the best
alternative is made with the help of inter- and intra-attribute comparisons, which may
involve implicit or explicit information.[Hwang, 1981]
Common to all MADM techniques is the concept of a decision matrix, or goal
achievement matrix D. D is an M-by-N-matrix with elements x
ji
that indicate the value of
the attribute X
i
with respect to the alternative A
j
.
]
]
]
]
]
]
]
]
]
]
MN Mi M M
jN ji j j
N i
N i
N i
M
j
x x x x
x x x x
x x x x
x x x x
X X X X
A
A
A
A
D
L L
M O M M M
L L
M M O M M
L L
L L
L L
M
M
2 1
2 1
2 2 22 21
1 1 12 11
2 1
2
1
(2.3)
Another concept typical for MADM techniques is the ideal solution A
*
. It is a
hypothetical solution to the decision problem, combining the best achievements of all
criteria to one solution, i.e. A
*
= (f
1
*
, f
2
*
, , f
i
*
, , f
N
*
). Note, that in MADM problems
the ideal solution is subjective, driven by the existing solutions. In MODM problems, on
the other hand, the objective ideal is the best solution that any alternative could possibly
obtain. Further, for MADM problems, A
*
is infeasible most of the time, meaning it is not
part of the set of alternatives. If it was, there would be no conflict among the criteria and
A
*
would be the solution to the decision problem.
In general, MADM techniques can be separated into noncompensatory and
compensatory techniques by their treatment of attribute information. The
noncompensatory techniques do not allow for a trade-off between criteria, i.e. one
unfavorable criterion value cannot be offset by reducing a favorable value of another
criterion. Hence, comparisons are made on a criterion by criterion basis. The
compensatory techniques, on the other hand, permit trade-offs between criteria, assigning
a number to each multidimensional representation of an alternative. Based on the method
of calculating this number, these techniques can further be classified as scoring method,
compromising method, and concordance method.[Hwang, 1981] The scoring method
selects the alternative with the highest score, reducing the decision problem to obtaining
the appropriate multi attribute utility function. The compromising method identifies the
alternative closest to the ideal solution. The concordance method arranges a set of
preference rankings which best satisfies a given concordance measure. A selection
process to arrive at the right MADM technique for a given product selection problem is
outlined in the flowchart of Figure 2.1, as suggested by [Sen, Yang, 1998]. The
highlighted techniques are explained in more detail.
Which decision rule
is appreciated ?
Ordinal ranking
of all attributes
Preferentially independent
attribute set and
linear utility function
Relative closeness
to ideal and
negative ideal points
Concordance and
discordance
dominance indices
Simple Additive
Weighting Method
Lexicographic
Method
TOPSIS
Method
ELECTRE
Method
Decision table
Ranking of alternatives
by attributes
Linear Assignment
Method
Pairwise comparisons
of all alternatives
Linear Assignment
Method
What type of input
data is available ?
of all alternatives
and attributes
AHP
Method
of all alternatives
LIMAP
Method
What type of input
data is available ?
Given Generated
Minimal attribute
value acceptable for
each current attribute
Conjunctive
Method
Greatest value of
an attribute for
an alternative
Disjunctive
Method
What cut-off values
are favorable ?
Is preference
information required ?
Overall utility
function
UTA
Method
Local utility
function
ILUTA
Method
Implicit utility
function
EDMCM
Method
Is weight given or
will it be generated ?
What types of utility
functions are appreciated ?
Non-dominance
Dominance
Method
Maximin
Maximin
Method
Maximax
Maximax
Method
What decision rules
are appreciated ?
No Yes
Utility functions Relative weight
Standard level
on each attribute
How is preference
information represented ?
Given Generated
Figure 2.1: Decision Tree for MADM Technique Selection (based on [Sen, Yang, 1998])
Techniques for No Preference Information Given
MaxiMin Technique
MaxiMin is a widely used technique for MADM problems, particularly in
economics and game theory, and belongs to the class of noncompensatory techniques.
Any decision making situation where the analogy a chain is only as strong as its weakest
link applies can be solved with this technique. It identifies the weakest criterion for
each alternative and selects the alternative A
+
that has the most acceptable level in its
weakest criterion. In other words, it is selecting the maximum (across alternatives) of the
minimum (across criteria) values, hence MaxiMin. Explicitly:
))} ( min ( max | {
ji
i j
j
x A A
+
, j = 1, 2, , M, i = 1, 2, , N. (2.4)
Principally, this technique reduces the multi-criteria problem to a single criterion
decision, since the single weakest criterion represents the whole alternative, ignoring all
other criteria. Thus, it only utilizes a small amount of the available information during
the decision making process. This obvious shortcoming is illustrated by the fact that an
alternative clearly superior in all other criteria will be thrown out, if its weakest criterion
performs worse than the weakest criterion of an alternative achieving average values for
all other criteria. However, the MaxiMin technique can be applied whenever the
decision-maker has a particularly pessimistic outlook and the criteria are truly of equal
importance and a failure in any criterion would prevent the alternative (system) from
performing in the desired manner.[Hwang, 1981]
Techniques for Standard Level of Attributes Given
Additive Utility Function Method (UTA)
The Additive Utility Function Method (UTA) attempts to rank alternatives based
on utility function values. First, a subset of alternatives is selected and ranked
subjectively. This subset of alternatives is further used to determine a utility function for
all alternatives based on a monotonic piecewise linear utility function for the attributes
and their subjective preferences. Finally, all alternatives are ranked by their utility.[Sen,
Yang, 1998] While the UTA method provides an alternative means to represent
preferences among alternatives, it has some disadvantages that seem to be problematic.
First, the method assumes an additive utility function for the attributes, which in turn
requires the attributes to be independent.[Sen, Yang, 1998] Second the method assumes
the attribute to be piecewise continuos, which is difficult to conclude from a finite set of
attribute values. Third, the utility function for all alternatives is based only on a subset
and their preference information is subjective. Last but not least, UTA assumes the
utility function to be (piecewise) linear and monotonically increasing or decreasing and
thereby limiting the range of problems the utility function can model.
Techniques for Standard Level of Attributes Given
Conjunctive Technique
The Conjunctive Technique is one of the oldest MADM tools and belongs to the
class of noncompensatory techniques. Minimum attribute levels are provided which each
criterion has to satisfy. Alternatives that do not satisfy all criteria levels are rejected.
This process eliminates a certain number of alternatives, depending on how aggressive
these goals are set. In order to avoid dismissing all alternatives on the first trial, an
iterative approach is typically selected that eliminates one alternative per iteration. Even
though no relative importance information is needed for this technique, alternatives are
not being credited for especially good attribute values. The concept of a minimum level
that needs to be satisfied for each attribute has a very intuitive appeal, however, and has
therefore been used in other MCDM techniques.[Hwang, 1981]
Techniques for Cardinal Preference of Attributes Generated
Analytic Hierarchy Process Method
The Analytic Hierarchy Process (AHP) method was originally introduced by
Saaty and is intended to solve such product selection problems that have a hierarchical
structure of attributes.[Saaty, 1980] Attributes in one level are compared in terms of
relative importance with respect to an element in the immediate higher level, treating the
pairwise comparison with the eigenvector method as outlined in [Sen, Yang, 1998]. This
process is executed from the top down starting with the overall goal as the single top
element of the hierarchy and closing with the alternatives at the very bottom, ranking the
attributes/alternatives at each level with respect to the overall goal.
While AHP method is well known, it has several disadvantages as outlined in
[Sen, Yang, 1998]. First, it requires attributes to be independent with respect to their
preferences, which is rarely the case in product selection cases. Second, all attributes and
alternatives are compared with each other (at a given level), which may cause a logical
conflict of the kind: A > B and B > C but C > A. The likelihood of such conflicts
occurring in the hierarchy tree increases dramatically with the number of alternatives and
attributes. Last but not least, AHP has the potential of introducing a rank reversal of
alternatives, depending on the number of alternatives assessed, which is particularly
troublesome for normative decision making environments.[Sen, Yang, 1998]
Techniques for Ordinal Preference of Attributes Given
Lexicographic Technique
In some decision problems a single criterion is predominant. If one alternative
yields a high value for this criterion, it is preferred over the other alternatives, which
yield lower values. However, if some alternatives have equal values for the most
important criterion, the next most important criterion might determine the best
alternative. If alternatives are tied again, the next most important criterion might yield an
answer, and so on. Thus, ordinal ranking of the criteria can provide a down selection
process of alternatives. A special form is the lexicographic semiorder technique, which
considers the next important criterion not only when the criterion values are equal but
also when their differences are negligible, keeping more alternatives in the decision
making process. Because of its need for only limited information, the lexicographic
technique has received serious consideration in the past.[Hwang, 1981]
Techniques for Cardinal Preference of Attributes Given
Technique for Order Preference by Similarity to Ideal Solution (TOPSIS)
TOPSIS is one of the compromising methods among the compensatory
techniques, utilizing preference information provided in the form of weights w
i
for each
criterion. It is based upon the concept that the best alternative should have the shortest
distance to the ideal solution and be farthest away from the negative-ideal solution. The
ideal solution A
*
is composed of the best normalized criterion values
M
j
ji i
x x r
i 1
2 * *
obtained by all existing solutions, while the negative-ideal A
-
is composed of the worst
normalized criterion values
M
j
ji i
x x r
i 1
2
obtained. One approach uses the weighted
minimum Euclidean distance S
j*
to determine the closest solution to the ideal. However,
depending on the location of the alternative A
j
in the solution set S, the shortest Euclidean
distance to the ideal may not be the longest distance from the negative-ideal. Consult
Figure 2.2 for an example where A
1
has a shorter distance to the ideal and the negative-
ideal solution than A
2
. TOPSIS considers both distances simultaneously by computing
the relative closeness C
j
, j = 1, 2, , M, to the ideal solution:
j j
j
j
S S
S
C
*
with ( )

N
i
i i ji i j
r w r w S
1
2
*
*
and ( )

N
i
i i ji i j
r w r w S
1
2
. (2.5)
A
2
A
j
A
1
A
*
A
-
Attribute X
1
(increasing preference)
A
t
t
r
i
b
u
t
e

X
2

(
i
n
c
r
e
a
s
i
n
g

p
r
e
f
e
r
e
n
c
e
)
S
Figure 2.2: Euclidean Distance to the Ideal and Negative-Ideal Solutions [Hwang, 1981]
The solution with the highest relative closeness is the best solution among the
alternatives. The advantages of this technique are clearly its simplicity and the
indisputable ranking order obtained. However, the dependency on cardinal preference
information, such as weights, yields solutions highly dependent on their values. TOPSIS
further requires the criteria to have a monotonically increasing or decreasing utility to the
decision-maker, a condition that is violated in all situations where a particular attribute
value is supposed to be achieved, e.g. in tolerance design.[Hwang, 1981]
Optimization
Generally, optimization is defined as the process of maximizing a desired and/or
minimizing a detrimental quantity or outcome.[Dieter, 1991] On the other hand,
optimization is just a subset of decision making as Hazelrigg points out: Decision
making is the taking of choices from sets of options in order to obtain the most desired
outcomes. This is precisely the process of optimization.[Hazelrigg, 1996] As a decision
making problem, optimization has to be based on criteria that are to be maximized or
minimized. In systems design these criteria are typically the attributes of the system,
indicating its performance, cost, or reliability. Hence, optimization in systems design
requires the maximization or minimization of a multitude of criteria concurrently, which
in many cases results in a conflict. As a matter of fact, it is not intuitive what optimal
solution means in a multi-criteria optimization problem. To solve this problem, several
multi-criteria decision making techniques for optimization have been introduced (see
[Hwang, 1979]), some of which are listed here with a brief explanation. A selection
process to arrive at the right MODM technique for a given optimization problem is
outlined in the flowchart of Figure 2.3, as suggested by [Sen, Yang, 1998]. The
highlighted techniques are explained in more detail.
Generate extreme
efficient solutions
for MOLP problems
MOLP
Method
Represent efficient
solution set as some
parametric functions
Envelope
Method
Use weights as
parameters to generate
efficient solutions
Parametric Method
or Weighting Method
Vary minimum allowable
levels for objectives
to get solution set
Epsilon-constraint
Method
Generate approximate
set of efficient solutions
Efficient Solution
Generation Method
Which decision rule
is favorable ?
A posteriori
articulation
A priori
articulation
Progressive
articulation
How is preference
information elicited ?
Optimization of implicit
concave function based on
set of positive multipliers
Zionts-Wallenius
Method
Optimization of local
additive utility function
based on direct trade-off
REISTM
Method
utility function based on
marginal rates of substitution
Geoffrions
Method
Generation of preferred
solution based on trade-off
rates and surrogate function
Interactive Surrogate
Worth Trade-Off Method
additive separable
utility function
Interactive
Goal Programming
Which decision rule
is favorable ?
Optimization of
utility function
Utility Function
Method
Closeness to
ideal point
Ideal Point
Method
Satisfaction of
goal values
Goal
Programming
Which decision rule
is favorable ?
Explicit trade-off
Directive trade-offs among
objective functions
around ideal point
ISTM
Method
Closeness to ideal point
with satisfaction of one
objective at each interaction
STEM
Method
Generation and reduction
of compromise sets by
displacing ideal points
Displaced Ideal
Point Method
Optimization of surrogate
objective function based on
goals and DMs aspiration
SEMOPS
Method
Which decision rule or interactive
procedure is favorable ?
Implicit trade-off
What type of preference
information is appreciated ?
Ordinal
Lexicographic
Method
Goal
Programming
Cardinal
What type of preference
information is appreciated ?
Figure 2.3: Decision Tree for MODM Technique Selection (based on [Sen, Yang, 1998])
Techniques for No Articulation of Preference Information Given
Techniques in this category do not require any information about preferences
among the defined criteria. While this may seem as an advantage at first, since no
additional information is required from the customer or designer, it is probably more of a
disadvantage, since the decision will be made in isolation, not representing the true
preferences of the customer or company.
Technique of Global Criterion
The vector x
*
that minimizes some global criterion is called the optimal vector.
For all f
i
to be maximized the global criterion can be described by:
,
`
.
|
N
i
p
i
i i
f
f f
1
) (
) ( ) (
min
x
x x
x
. (2.6)
The optimal solution will differ significantly with a change in criteria chosen.
The difficulty with this approach is finding the value for p that will maximize the
satisfaction of the decision-maker. When p = 1 and f
i
(x) are linear functions, the decision
problem becomes a linear programming problem that can be solved with the Simplex
method for example. When p = 2 and f
i
(x) are linear functions, the problem becomes a
quadratic programming problem, solved by a gradient based technique for example.
When the f
i
(x) are nonlinear functions, the decision problem becomes a nonlinear
programming problem which can be solved by the Complex method.[Hwang, 1979]
MaxiMin Technique
The MaxiMin technique
2
is the simplest form of multi-criteria optimization. The
basic principle for this technique is equivalent to the product selection technique
MaxiMin: find the smallest criterion value and change the independent variables such
that this particular criterion value is maximized. Repeat previous step until no
improvement is found. Or expressed mathematically:
))) ( , ), ( ), ( ), ( ( min ( max
2 1
x x x x
x
N i
i
f f f f K K . (2.7)
Principally, this technique reduces the multi-criteria to a single criterion decision
problem, since the single weakest criterion represents the whole alternative, ignoring all
other criteria. Thus, it only utilizes a small amount of the available information during
the optimization process. This obvious shortcoming is illustrated by the fact that an
alternative clearly superior in all other criteria will be thrown out, if its weakest criterion
performs worse than the weakest criterion of an alternative achieving average values for
all criteria. Furthermore, whenever two or more (normalized) criterion values are equal,
the technique suffers from a discontinuous derivative.[Vanderplaats, 1999]
Techniques for A Priori Articulation of Cardinal Preference Information Given
Techniques in this category utilize preference information provided before the
decision making as part of the mathematical modeling of the decision process. They also
require some judgement about specific criterion preference levels or specific trade-offs.

2
Also called MinMax technique in minimization problems.[Vanderplaats, 1999]
Utility Function Technique
For the utility function method, the optimization problem is converted to
max{U(f
1
, f
2
, , f
N
) = U(f)}, where the U(f) is the utility function. The technique is
based on the idea that the decision-maker must have some kind of utility associated with
the criteria, accounting for the preferences. However, for complex problems such as
systems design, determining U(f) is very difficult. The major advantage of this technique
lies in the fact that the utility function, if assessed correctly, will ensure finding the most
desirable solution. There are several ways the utility function can be formed. A common
approach assumes that the utility function is a mere summation of criteria, reducing the
decision problem to:
( )
N
i
i i
f u U
1
) ( max x
x
. (2.8)
A special form of Equation 2.8, which has been widely used in (aerospace)
systems design and other MODM problems, uses weights w
i
to indicate the importance of
each objective. It is formulated as
( )

N
i
i i
f w U
1
max x
x
(2.9)
and is also known as the Overall Evaluation Criterion (OEC). The advantage of this
technique clearly lies in its simplicity. The disadvantages, however, are: (i) there are
very few cases where the utility functions is a linear combination of criteria and (ii) the
weights, w
i
, depend upon the achievement level of f
i
(x) itself and the relative achievement
of f
i
(x) compared to the achievement levels of the other criteria.[Hwang, 1979]
Techniques for A Priori Articulation of Ordinal and Cardinal Preference Information
Given
Goal Programming
Goal programming finds the optimal solution by minimizing the deviation from
goals, specified for each criterion. The decision problem is formulated for p 1 by:
( )
'
'
]
]
]
+
p
N
i
p
i i
d d
1
1
min
x
, with ( ) ,
i i i i
b d d f +
+
x and , 0 0 ,
+ +
i i i i
d d d d (2.10)
where b
i
, i = 1, 2, , N, are the goals to be obtained by the criteria, and d
i
-
and d
i
+
are the
respective under- and overachievements of the ith goal. A very common form of goal
programming requires an ordinal ranking of the criteria also. In this case the formulation
is:
( ) ( ) ( ) {
+ + +
d d d d d d , ..., , , , , min
2 2 1 1 N N
h P h P h P (2.11)
where h
i
(d
-
, d
+
), are linear functions of the deviational variables and are called
achievement functions. The P
i
s are preemptive weights, i.e. P
i
>>> P
i+1
.
The algorithm that solves Equation 2.11 minimizes h
1
(d
-
, d
+
) first, thus
determining h
1
*. Next, h
2
(d
-
, d
+
) is minimized, but subject to h
1
(d
-
, d
+
) h
1
*. In other
words, a lower ranking achievement function can not be satisfied to the detriment of a
higher ranking achievement function. This process is repeated until the last achievement
function is minimized. If the criteria are linear functions of x, a modified Simplex
algorithm for linear programming can be used to solve moderate size problems. For
large, complex problems, like systems design, a basic Simplex algorithm has to be used
within iterations. If the criteria are nonlinear functions, any single objective nonlinear
optimization technique can be used iteratively to solve the problem.[Hwang, 1979]
Requiring only a ranking but no numerical weights is one of the advantages of the goal
programming technique. However, the ordinal ranking implies a trade-off assumption
that is very strong and potentially restrictive. The fact that higher ranking criteria may
not be detrimented while minimizing lower ranking ones limits the possible solutions and
makes them extremely dependent on the ranking order. This can be particularly bad for
decision making problems with no clear preference of one criterion over another.
A variation of Goal Programming is the Goal Attainment Method, introduced by
Gembicki,[Gembicki, 1974] and relies on a goal and weight for each criterion. The
method is of relevance, since it is one of the two multi-criteria optimization methods
available in MATLAB
.[Branch, Grace, 1996] The optimization problem is formulated

as:
min
x
(2.12)
subject to:
i i i
b w f ) (x , w
i
> 0, i = 1, 2, , N,
and the constraints 0 ) ( x
r
g , r = 1, 2, , L.
Depending on the reference, the weights should either be normalized such that
they add to one,[Hwang, 1979] or should be set to zero for the criteria that must be
realized and set equal to the goal when no preferences are provided for the different
criteria.[Fleming, 1986] The Goal Attainment Method shares the disadvantages of all
Goal Programming methods, namely the high dependency of the solution on the weights
and goals provided by the decision-maker, but has fewer variables than the formulation in
Equation 2.10, and is hence computationally less intensive.
Techniques for Progressive Articulation of Implicit Trade-Off Information Given
This class of interactive techniques utilizes preference information progressively
defined during the exploration of the criterion space. They typically involve iteration
processes that require a preference or trade-off input from the decision-maker or his
consent with the current achievement level. This information is assumed to be
unavailable a priori due to the complexity of the problem, but rather at a local level for
a particular solution. Some techniques allow the reversal of the preference order; few
guarantee a final solution within a limited number of iterations. Among their advantages
is the lack of a priori information needed; only local preference information is needed.
The iterative nature of this technique also constitutes a learning process for the decision-
maker, and the assumptions made are less restrictive compared to the previous
techniques. Among their disadvantages are the dependency of the solution on the
accuracy of the local preference information provided, and the fact that many methods
can not guarantee a preferred solution within a finite number of iterations. These
methods also require more effort from the decision-maker than the previous
techniques.[Hwang, 1979]
The STEM Technique
The STEP-method (STEM) is a solution technique for multiple objective linear
programming (MOLP) problems. An MOLP problem can be formulated as:
'
'

K
j
j Nj
K
j
K
j
j j j j
x c x c x c
1 1 1
2 1
,..., , max (2.13)
subject to:
i
K
j
j ij
b x a
1
, i = 1, 2, , N,
0
j
x , j = 1, 2, , K.
The STEM technique is a three step process that allows the decision-maker to
learn to recognize good solutions as well as the relative importance of the objectives, by
alternating phases of computing with phases of decision making. The first step consists
of the construction of a pay-off table, which lists in rows the values z
ij
that are taken on
by the criterion f
i
when criterion f
j
reaches its feasible maximum solution. Step 2 consists
of the calculation phase, determining the feasible solution to Equation 2.13 which is
nearest, in a Minimax sense, to the ideal solution f
j
*
:
min
x
(2.14)
subject to: ( ) {
i i i
f f x
*
, i = 1, 2, , N, and
i
K
j
j ij
b x a
1
plus constraints from the previous cycle,
0
j
x j = 1, 2, , K and 0 .
i
gives the relative importance of the distances to the optimum. Note, it is only
locally effective and does not have such overriding powers as the weights in the utility
technique. If f
i
in the ith column of the pay-off table varies little from the optimum value
f
i
*
when varying x,
i
is assigned a small value since Criterion i is not sensitive to a
variation in weighting values. The third step then compares the criteria vector f

m
of the
mth iteration, corresponding to solution x
m
, with the ideal solution f
*
. In order to be able
to improve on the unsatisfactory criteria, the satisfactory ones need to be relaxed enough
to enable an improvement for the unsatisfactory criteria in the next iteration. Thus, the
Criterion i is allowed to be relaxed by f
i
and the constraints f
i
(x) f
i
(x
m
) - f
i
and
f
j
(x) f
j
(x
m
), i j and j = 1, 2, , N, are added to the linear programming problem.
Finally the weight
i
is set to zero and the calculation for cycle m+1 begins.[Hwang,
1979]
Techniques for A Posteriori Articulation of Preference Information Given
These techniques determine a series of nondominated solutions, of which the most
desirable is selected based on some previously unindicated or nonquantifiable criteria.
This means that the trade-off information is used after the nondominated solutions have
been found. Hence, these techniques do not require assumptions or information
regarding a utility function, which is an advantage. However, in order to be accurate, a
large number of solutions is required, which makes it almost impossible to determine the
most desirable. Consequently, these techniques have rarely been used by themselves, but
rather in conjunction with other interactive techniques.[Hwang, 1979]
Parametric Method (Weighting Method)
Suppose that the relative importance of N objectives is known and represented by
a set of constant weights w
i
, i = 1, 2, , N, with the sum of all w
i
equal to one. The
feasible, nondominated preferred solution can be obtained by solving:
( )
'
'
N
i
i i
f w
1
max x
x
. (2.15)
Although, the assumption of linearity and additivity is difficult to satisfy, the
technique can be used to generate nondominated solutions by varying values for w.
Hence, the weighting coefficients are only parameters for finding the nondominated
solution points and do not reflect the relative importance of the criteria. A geometric
interpretation of the Parametric Method finds the points where the hyperplane
L = {f(x)|w
T
f(x) = c}, c is a constant, is tangential to the set of nondominated solutions.
The preferred solution is the one furthest away from the origin. This has been illustrated
for two criteria with a convex and nonconvex set in Figures 2.4 and 2.5 respectively. L is
the line with the slope w
1
/w
2
. In Figure 2.4, the thick line indicates the set of
nondominated solutions, of which point A is the preferred solution. For the nonconvex
set, some solutions can not be found, since the technique will try to find only extremal
solutions. Thus the Parametric method will find Points B and C, but not A in Figure 2.5.
f
1
f
2
A
Slope = -w
1
/w
2
S
L
Figure 2.4: Parametric Method of Two
Criteria with a Convex Set
[Hwang,1979]
f
1
f
2
B
C
A
Slope = -w
1
/w
2
S
L
Figure 2.5: Parametric Method of Two
Criteria with a Nonconvex Set
[Hwang, 1979]
Illustrative Example
In order to illustrate some of the previously listed optimization techniques, a
simple multi-criteria example is employed to compare their results. Consider the
following equations:
2 1
2
2 2
2
1 1 1
) ( 5 . 0 ) ( 5 . 0 5 y y y x y x f + (2.16)
2
2 2 1
2
1 2
cos cos
2
y x x y y f + + + (2.17)
subject to: 2 2
1
x and 2 2
2
x . (2.18)
The optimal point (maximum) for f
1
is clearly located at (y
1
, y
2
), while the optimum for f
2
is (0, 0). Therefore, assuming y
1
= 0.5 and y
2
= 1.5, the maximum values for f
1
and f
2
are
f
1max
= 5.75 and f
2max
= 5. On the other hand, the value for f
1
at the optimum for f
2
is
f
1
(0,0) = 4.5, and the value for f
2
at the optimum for f
1
is f
2
(0.5,1.5) = 3.7953, both being
quite different from their respective maximum value. To display these differences, both
functions were plotted in Figure 2.6 over the entire range for x
1
and x
2
at y
1
= 0.5 and
y
2
= 1.5.
Figure 2.6: Surface Plot for f
1
and f
2
f
2
f
1
x
2
x
1
So the question arises, which is the best point when both functions need to be
maximized. To answer this question, three techniques are selected as a representation for
all the multi-criteria optimization techniques: MaxiMin, Utility Function, and Goal
Attainment Method. The Utility Function Method in itself does not include an
optimization algorithm, just the composed overall evaluation criterion (OEC), and is
therefore used in conjunction with the univariate sequential programming technique and a
line search.[Reklaitis, 1983], [Vanderplaats, 1999] In order to be able to compare the
results, no preferences are identified for either function. The Utility Function Method is
assigned equal weights, while the Goal Attainment Method is assigned weights of the
same value as the goals; the goals are set high, since both functions are to be maximized.
MaxiMin: ))) , ( ), , ( ( min ( max
2 1 2 2 1 1
x x f x x f
f x
. (2.19)
Utility Function: ) , ( 5 . 0 ) , ( 5 . 0
2 1 2 2 1 1
x x f x x f OEC + (2.20)
Goal Attainment:
min
x
(2.21)
subject to: 10 10 ) , (
2 1 1
+ x x f and 10 10 ) , (
2 1 2
+ x x f
All techniques are easily implemented in MATLAB
and well documented in the

Users Guide to the Optimization Toolbox.[Branch, Grace, 1996] All results are listed
and compared in Table 2.1.
Table 2.1: Comparison of Three Multi-Criteria Optimization Techniques
MaxiMin OEC Goal Attain
x
1
*
0.0520 0.1543 0.0513
x
2
*
0.3152 0.7898 0.3154
f
1
(x
1
*
, x
2
*
) 4.9477 5.4381 4.9477
f
2
(x
1
*
, x
2
*
) 4.9477 4.6773 4.9477
The differences in the results clearly show how difficult it is to solve the multi-
criteria decision making problem. While MaxiMin and the Goal Attainment Method both
yield roughly the same results, it is not clear whether they are better than the result from
the OEC. Both methods drive the function values to be equal thereby drifting far from
the optimum for f
1
.
3
The OEC on the other hand seems to yield a point that balances both
function values better. Unfortunately, the OEC is not an ideal technique either. It will
drive the multi-criteria solution of an optimization problem with vastly different function
values towards the optimum of the function with the largest function values. As a brief
example consider:
4 . 0 000 , 10 ) , ( ) , ( ) , (
2 1 2 1 2 2 2 1 1 1 2 1
+ + w w x x f w x x f w x x OEC . (2.22)
Perturbations of the independent design variables x
1
and x
2
wont change the
function value 0.4 much with respect to the function value 10,000, and the solution
will therefore be driven by the 10,000 more than the 0.4.
It seems that the solution can be improved in all three cases by normalizing the
function values. Unfortunately, this procedure will make the solution dependent on the
normalization value, but can at least alleviate the aforementioned problems as
demonstrated in Table 2.2. Here, all function values were normalized by their respective
maximum values:
MaxiMin: ))
5
) , (
,
75 . 5
) , (
( min ( max
2 1 2 2 1 1
x x f x x f
f x
. (2.23)

3
The difference f
1max
- f
1
*
= 0.8023 is much larger than the difference f
2max
f
2
*
= 0.0523.
Utility Function:
5
) , (
5 . 0
75 . 5
) , (
5 . 0
2 1 2 2 1 1
x x f x x f
OEC + (2.24)
Goal Attainment:
min
x
(2.25)
subject to: 10 10
75 . 5
) , (
2 1 1
+
x x f
and 10 10
5
) , (
2 1 2
+
x x f
Table 2.2: Normalized Comparison of Three Multi-Criteria Optimization Techniques
MaxiMin OEC Goal Attain
x
1
0.1402 0.1396 0.1422
x
2
0.7579 0.7317 0.7579
f
1
*
(x
1
, x
2
) 5.4099 5.3899 5.4099
f
2
*
(x
1
, x
2
) 4.7043 4.7222 4.7043
Note that the Minimax and Goal Attainment method, again, yield roughly the
same results. This time, however, much closer to the result from the OEC, with much
more balanced function values. Finally, it is up to the decision-maker to decide which
point is ideal.
The following question arises as an afterthought: what happens when the values
for y
1
and y
2
are not known deterministically, but instead are uncertain. How do these
techniques handle uncertain information? Unfortunately, they dont. All current multi-
criteria decision making techniques require known, i.e. 100% certain information. An
approach to systems design with uncertain information is the topic of the following
chapter.
Oliver Bandte Chapter III 49
CHAPTER III
PROBABILISTIC DESIGN METHODS
One of the major obstacles in applying probabilistic methods to design is the
accommodation of the large variety of computer codes used in modern systems
engineering. It is impractical to modify all of them just to accommodate a probabilistic
problem formulation. Hence, a more generic methodology is proposed in which a
wrapper is used to link the analysis codes, drive the programs, and collect the output
parameter values. Using this formulation, probability functions are assigned to those
input variables whose actual values are considered to be uncertain, indicating the
likelihood of occurrence for all values within a given set. These variables are also called
noise variables. Since the inputs to the analysis codes are probability distributions, the
output parameters have to be distributions as well. In most cases the output distribution
of interest is a cumulative distribution function (CDF) that describes the likelihood of
achieving values less (or more) than specified values of interest. To generate these
distributions, many probabilistic analyses, e.g. the Monte-Carlo simulation, require a
large number of samples generated by the analysis.[Kleijnen, 1974] While the use of
computer models allows for an easy perturbation of input values, an increase in
complexity of the modeled system increases the complexity of the code and hence the
run-time of the computer. Fox lists three methods that incorporate such complex
computer programs in a probabilistic systems design approach.[Fox, 1994]
Method #1, displayed in Figure 3.1, directly links a computationally intensive
(large number of repetitions) thus inefficient probabilistic method, like the Monte-Carlo
simulation, to the traditional systems design codes used in deterministic design
approaches. Although computer speed has significantly increased in recent years, this
method can only be used with simple, i.e. fast analysis tools and/or parallel processing
(distributing system analysis over several processors). The extreme complexity of some
current design codes yields computation times that may prohibit a large number of
program evaluations within the allotted time frame for the design process. Thus,
Method #1 may not be a feasible option for a probabilistic design procedure if parallel
processing cannot be applied.
Inefficient Probabilistic
Method to Obtain CDF
Fast
Analysis Tool
Exact
Input
Variables
Objective
CDF
Figure 3.1: Probabilistic Design Method #1 [Fox, 1994]
Method #2, displayed in Figure 3.2, proposes the use of a metamodel, which
approximates the exact design codes. The advantage of creating such a metamodel is a
significantly reduced execution time, allowing a Monte-Carlo simulation to run on the
metamodel rather than on the actual computer code. Several different metamodels have
been proposed and applied. Some of the more common regression models are based on
experimental designs [Kleijnen, 1987], artificial neural networks [Cheng, Titterington,
1994], or Fuzzy Graph based metamodeling.[Huber, Berthold, Szczerbicka, 1996]
Inefficient Probabilistic
Metamodel
Exact
Input
Variables
Objective
CDF
Complex
Analysis Tool
Approximation
Method #3, displayed in Figure 3.3, takes a different approach, approximating the
probability distribution function rather than the design code. This approximation is based
on the notion that in order to obtain the cumulative distribution function (CDF) not all
probability levels need to be identified. The method selects several percentile levels and
calculates the corresponding objective value. Note that this calculation is based on the
exact computer code, not on an approximating metamodel. These objective values and
their probabilities can than be used to fit the typical S-shape of a CDF. Method #3 has
found the widest application in the area of structural reliability,[Khalessi, Lin, 1993],
[Wu, Burnside, Dominguez, 1987] while more current publications suggest a use for
systems design also.[Mavris, Bandte, 1998], [Mavris, Kirby, 1998], [Mavris, Kirby,
Qiu,1998]
Efficient Probabilistic
Complex
Analysis Tool
Exact
Input
Variables
Objective
CDF
Approximation
Metamodel/Monte-Carlo Simulation
The metamodel/Monte-Carlo simulation combination (Method #2) has found the
widest application in aerospace systems design. In particular, the use of statistical
regression models based on Taylor series expansions in combination with experimental
designs is very popular. Examples can be found in [Chen, et al., 1995], [DeLaurentis,
Mavris, Schrage, 1996], [DeLaurentis, Calise, Schrage, Mavris, 1996], [Fox, 1994],
[Giunta, et al., 1996], [Kaufman, et al., 1996], [Mavris, Bandte, Schrage, 1995], [Mavris,
Bandte, 1996], [Mavris, Bandte, Schrage, 1996], [Unal, Stanley, Joyner, 1994]. The two
main reasons for its popularity are its simple application to numerous computer
simulation problems, like aircraft synthesis, and the large number of statistical analysis
tools commercially available, such as JMP
, MINITAB
, SPSS, etc. Nonetheless, there

are two major problems in metamodeling of complex computer codes with a high number
of inputs. First, the number of input variables handled by this approach is limited.
However, this problem can often be solved through a screening process [Box, Draper,
1987] that identifies the major contributors to the variation in model output.
The second problem with Method #2 lies in the mathematical background for
such regression methods as Response Surface Methodology (RSM) and Design Of
Experiments (DOE), which are based on random rather than deterministic variables (see
[Box, Draper, 1987], [Box, Hunter, Hunter, 1978], [Kleijnen, 1974], [Kleijnen, 1987]).
Fundamental statistical knowledge is critical for obtaining reasonable approximations of
the computer model. Many of the statistical results offered by the commercial packages
are based on random error estimation and do not completely reflect the accuracy of the
metamodel, since random error does not exist in deterministic computer simulation. A
discussion on accuracy and behavior of statistical regression metamodels in computer
simulations can be found in [Kleijnen, 1987], [Sacks, Schiller, Welch, 1989], [Sacks,
Welch, Mitchell, Wynn, 1989], and [Welch et al., 1992]. In general, the best validation
of the accuracy of the metamodel is an extensive test at randomly distributed points over
the design space to compare predicted values with the exact computer simulation values.
Unfortunately, this test increases the computational effort put into the generation and
validation of the metamodel.
Despite the aforementioned problems, the Response Surface Methodology (RSM)
can, if applied correctly, provide some valuable insight into the systems design code
behavior. Hence, it has been used as a metamodel generator to facilitate probabilistic
aerospace systems design methods.[DeLaurentis, 1998], [Fox, 1994], [Mavris, Bandte,
1996], [Tai, 1998] RSM is based on a statistical approach to build and rapidly assess
empirical metamodels.[Box, Draper, 1987], [Box, Hunter, Hunter, 1978] By designing
and analyzing experiments or simulations, the methodology seeks to relate and identify
the relative contributions of various input variables to the system attributes. However,
modern aerospace systems are extremely complex, and most attributes of interest are a
function of many hundreds of design variables. The first step in constructing a Response
Surface Equation (RSE) as a metamodel is to conduct a screening test to identify the
variables, which have the greatest contribution to the system attribute variation. The
screening test is comprised of a two level fractional factorial Design of Experiments that
accounts for main effects of variables only (i.e. no interactions).[Box, Hunter, Hunter,
1978] It allows the rapid investigation of many variables to gain a first understanding of
the problem.
After identifying the variables that form the RSE, a Design of Experiments has to
be selected. A typical, efficient design is a face-centered central composite design,
displayed in Figure 3.4, a three level composite design formed by combining a two-level
full or fractional factorial with a star design.[Box, Draper, 1987]
Figure 3.4: Face Centered Central Composite Design
Typically, a second order model in k-variables is assumed to exist as an
approximation of the analysis code. This second order polynomial for a response R can
be written as:

+ + +
K
i
i
j
j i ij
K
i
i ii
K
i
i i
x x b x b x b b R
2
1
1 1
2
1
0
(3.1)
where: b
i
are regression coefficients for linear terms
b
ii
are coefficients for pure quadratic terms
b
ij
are coefficients for cross-product terms
x
i
, x
j
are the design variables of interest
Refer to [Box, Draper, 1987] and [Box, Hunter, Hunter, 1978] for a detailed
description of response surface generation. Both references provide several ways of
measuring and estimating the prediction accuracy of the RSE, but due to the lack of
randomness in the data (each experiment run with the computer is repeatable) their
validity is questionable.[Kleijnen, 1987], [Sacks, Schiller, Welch, 1989], [Sacks, Welch,
Mitchell, Wynn, 1989], [Welch et al., 1992] The best validation of an RSE is
undoubtedly a comparison of RSE and analysis code values at random points throughout
the design space.
The developed RSEs, representing the analysis codes in the design process, can
subsequently be used for a Monte-Carlo simulation to incorporate the effect of uncertain
variables in the design process. A Monte-Carlo simulation is effectively a random
number generator that selects values for each random variable with a frequency
proportional to the shape of the corresponding probability distribution. Usually,
dependent on the desired accuracy, 5,000 to 10,000 trials are needed for a good
representation of the response probability distribution.
4
Without the aid of the RSE, this
task would be computationally excessive and in many cases impractical, considering that
a Monte-Carlo simulation would have to execute the design simulation code each time
(Method #1, Figure 3.1).
Fast Probability Integration
As an alternative to the sampling of a metamodel (Method #2), a fast probability
integration (FPI) technique can be used as an approach to Method #3. FPI techniques are
probability analysis techniques based on the Most Probable Point (MPP) analysis,

4
Refer to Appendix B for a detailed description of the number of necessary samples.
frequently used in structural reliability analysis. The MPP analysis utilizes a response
function Z(X) that depends on several random variables X
i
(see Figure 3.5 for a 2-D
example). Each point in the design space spanned by the random variables has a specific
probability of occurrence according to their joint probability distribution function (see
Figure 3.6). However, each point in the design space also corresponds to one specific
response value Z(X). Hence, each response value has the same probability of occurrence
as the corresponding point in the design space.
x
2
Z(x) = z
1
x
1
Z(x) = z
2
Z(x) = z
3
Z(x) = z
4
Figure 3.5: Objective Function
Contours
Figure 3.6: Joint Probability Distribution
[Ang, Tang, 1984]
In systems analysis and other disciplines involving random variables, it is often
desired to find the probability of achieving response values below a critical value of
interest z
0
. This critical value can be used to form a limit-state function (LSF):
g(X) = Z(X) z
0
(3.2)
where values of g(X) > 0 are undesirable. The MPP analysis calculates the
cumulative probability of all points that yield g(X) 0 for the given z
0
(see Figure 3.7).
Since the LSF cuts off a section of the joint probability distribution (see Figure 3.8) a
point with maximal probability of occurrence can be identified on that LSF. This point is
called the Most Probable Point. It is found most conveniently in a transformed u-space
(see Figure 3.8), in which all random variables are normally distributed. Once the MPP
and the cumulative probability are identified, the process can be repeated for several z
0
values, mapping the corresponding probability to z
0
. This cumulative probability
distribution for Z(X) can then be differentiated to obtain the probability density function
of the response.
MPP
g(x) = 0
x
1
x
2
g(x) < 0
g(x) > 0
Figure 3.7: Most Probable Point
Location
g(u)
u
1
MPP
u
2
Figure 3.8: Visualization of MPP
[Southwest Research Institute, 1995]
The FPI techniques utilize several methods to find the MPP and the probability of
a given LSF value z
0
for the response function. Some of these methods are very efficient
and eliminate the need for an expensive Monte-Carlo simulation. The advantage of FPI
techniques is the direct linkage of the probabilistic analysis to the analysis code,
eliminating the need for a metamodel and its limit in the number of variables. However,
all Fast Probability Integration methods approximate the LSF locally at the Most
Probable Point.
Advanced Mean Value Method
The Advanced Mean Value (AMV) method is one of the FPI techniques that
combines a simple Mean Value method with the MPP analysis and determines the CDF
for the response function Z(X). The Mean Value (MV) method is based on a simple
Taylor series expansion of the response function Z(X) (Equation 3.3), assuming Z(X) to
be smooth and the expansion to exist at the mean:
( ) ) ( ) ( ) ( ) ( ) ( ) (
1 1
X X X X X H Z H X a a H X
X
Z
Z Z
MV
N
i
i i o i i
N
i i
+ + + +
,
`
.
|
+

. (3.3)
All derivatives are evaluated at the mean values for the design variables. Z
MV
(X)
represents the sum of the first order terms and H(X) represents higher order terms. For N
random variables, the a
i
s can be estimated with N+1 function evaluations and a
numerical differentiation method. Based on this linear approximation the CDF for
Z
MV
(X) can be obtained directly, since the distributions for the random variables X
i
are
fully defined and Z
MV
(X) is explicit. However, for nonlinear Z-functions the MV solution
for the CDF is not sufficiently accurate. One possibility for increasing accuracy is to
increase the order of the Taylor series expansion, which becomes difficult and inefficient
for implicit response functions and a large number of random variables.
A more efficient approach to increasing the accuracy is proposed by the AMV
method: Z
AMV
= Z
MV
+ H(Z
MV
) (3.4)
H(Z
MV
) is defined as the difference between Z and Z
MV
at the Most Probable Point Locus
(MPPL) of Z
MV
, where the MPPL combines the MPPs for several values of z
0
.[Wu,
Burnside, Dominguez, 1987] In other words, H(Z
MV
) in Equation 3.4 approximates H(X)
in Equation 3.3. Z
AMV
would be exact if the MPPL was known and exact, i.e.
MPPL(Z
MV
(X)) = MPPL(Z(X)). Since the MPPL is not known, the AMV method
approximates the locus based on Z
MV
, which is a good approximation for smooth
response functions.[Southwest Research Institute, 1995] Again, to avoid confusion with
Method #2, the AMV method does not approximate the response function to obtain the
CDF but rather the MPP (MV method). This approximation, however, is corrected by the
move in the AMV method, as depicted in Figure 3.9. The steps for a CDF generation
with the AMV method are also displayed in Figure 3.9.
Analysis Code:
Response Function Z(X)
Z
MV
= a
0
+ a
1
X
1
+ a
2
X
2
+
CDF, MPP(X
*
)
Analysis Code:
Response Function Z(X)
Z
AMV
= Z(X
*
)
1
0
z
Z
AMV
Z
MV
H
Figure 3.9: AMV Method [Southwest Research Institute, 1995]
One of the dominant advantages of the AMV method is the small number of
function calls necessary for each output parameter distribution. N+1 analysis code
executions are sufficient for the linear approximation of the response function Z
MV
and
ten additional program evaluations are needed to obtain the updated Z
AMV
for ten selected
levels of z
0
.[Southwest Research Institute, 1995] This translates into significant time
savings over the RSE/MCS method which usually requires several hundred function
evaluations for the generation of the RSE.[Box, Draper, 1987] Additionally, the AMV is
principally not limited to a small number of variables. The current limit of 100 variables
for the computer based implementations of the FPI technique is only due to vector
dimensioning in the source code and not the technique itself. Nonetheless, there is an
additional gain associated with the extended effort in the RSE generation. It can serve as
a valuable tool in gaining insight to the behavior of the underlying model. The AMV
method, on the other hand, will only return a probability distribution without providing
any further information about the analysis code. A more detailed comparison of
Method #2 and Method #3 can be found in [Mavris, Bandte, 1998].
To illustrate the three different probabilistic design methods, the example problem
from Chapter II is employed here again. For this example, the values for y
1
and y
2
are
assumed to be uncertain, and probability distributions are assign to them. It is easily seen
from Equations 2.16 and 2.17 that with changing y
1
and y
2
values the optimal point for f
1
changes while the optimal point for f
2
stays the same (0,0). This phenomenon is
exemplified in Figure 3.10 with 100 different values for y
1
and y
2
.
-2 -1 0 1 2
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Figure 3.10: Change in Optima for f
1
and f
2
with Changing y
1
and y
2
In order to not overly complicate the problem, Normal-Distributions are assigned
to both random variables, Y
1
~ N(1,1) and Y
2
~ N(1,1). The optimization problem then
tries to seek the solution that maximizes the probability of achieving a certain goal, lets
say f
1
> 5 and f
2
> 5. Since f
1
and f
2
depend on the random variables Y
1
and Y
2
, they
become random variables themselves and the probabilistic optimization problem can be
formulated as:
) 5 ) ( 5 . 0 ) ( 5 . 0 5 ( max ) 5 ( max
2 1
2
2 2
2
1 1
,
1
,
2 1 2 1
+ Y Y Y x Y x P F P
x x x x
(3.5)
) 5 cos cos ( max ) 5 ( max
2
2 2 1
2
1
,
2
,
2
2 1 2 1
+ + + Y x x Y Y P F P
x x x x
(3.6)
subject to: 2 2
1
x , 2 2
2
x , Y
1
~ N(1,1), and Y
2
~ N(1,1).
To demonstrate the differences between the three models outlined in this chapter,
cumulative distribution functions (CDF) generated by each method are compared with
each other. Note that a metamodel approximation of f
1
in form of a second order
polynomial simply reproduces the original function, since it is a second order polynomial
x
1
x
2
Optimum for f
2
Optima for f
1
already. Consequently, the CDFs for f
1
generated by Method #1 and Method #2 are
identical. Unfortunately, the tool used to generate the comparison, CRAX,[Robinson,
1999] does not compute CDFs based on probabilities of random variable values being
larger than a specified value. Hence, for the comparison, the problem is transformed to
) 5 ( ) 5 (
1 1
F P F P , (3.7)
which is used in the subsequent plots.
First, as an example test case for x
1
= x
2
= 1, the CDFs for F
1
generated by the
probabilistic design Methods #1 and #3 are compared with each other. As displayed in
the graph of Figure 3.11, the two CDFs are quite different. Generally, the CDF generated
by the sampling method is assumed to be the truth model,[Fox, 1994],[Fox, Reh, 2000],
[Mavris, Bandte, 1998] which infers that the CDF generated by the AMV method is false.
This misprediction is related to difficulties the AMV method
5
has with such non-
monotonic functions as f
1
.[Fox, Reh, 2000], [Wu, Millwater, Cruse, 1990]
Second, the CDFs for F
2
are compared via the graph in Figure 3.12, again for
x
1
= x
2
= 1. In this case, however, all three methods produce a distinctly different CDF,
since f
2
is not a second order polynomial like f
1
. As a matter of fact, f
2
is so complex with
its y
2
2
cosx
1
term that a regular central composite design, in Figure 3.4, was not
sufficient to produce a reasonable approximation of f
2
. Instead, a 3-level full factorial
design was employed to generate the second order polynomial:
2
2
2
2
2
1 1 2
0559 . 1 354 . 0 4782 . 2 6087 . 7 y x x y f
RSE
+ . (3.8)

5
The AMV and the AMV
+
method both yielded the same CDF.
11.00 9.60 8.20 6.80 5.40 4.00 2.60 1.20 0.20 1.60 3.00
0.000
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
1.000
Figure 3.11: CDF Comparison of Method #1 and Method #3 for F
1
20.0 17.0 14.0 11.0 8.0 5.0 2.0 1.0 4.0 7.0 10.0
0.000
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
1.000
Figure 3.12: CDF Comparison of Method #1,#2, and #3 for F
2
Nonetheless, the RSEs failure to accurately represent f
2
causes the CDF
generated with probabilistic design Method #2 to be significantly different from the truth
model of Method #1. Obviously significant improvement could be achieved if the RSE
was altered to include higher order effects accounting for y
2
2
cosx
1
term. But the
purpose of this illustrative example is not an accurate prediction of some CDF, but rather
the illustration of the three probabilistic design methods and the identification of some
potential pitfalls. On the same token, the CDF generated by the AMV method, while
Method #3
(AMV) Method #1
(Monte-Carlo)
-f
1
P
r
o
b
a
b
i
l
i
t
y
Method #3
(AMV)
Method #1
(Monte-Carlo)
-f
2
P
r
o
b
a
b
i
l
i
t
y
Method #2
(RSE/MCS)
closer to the truth model this time, shows an uncharacteristic dip at very high probability
levels. This dip defies one of the conditions of a CDF, monotonically increasing function
values, and is due to AMVs aforementioned problems with non-monotonic limit state
functions.
Since the functions f
1
and f
2
are explicitly available and simple to evaluate,
Method #1 will be used from here on to solve the optimization problem in Equations 3.5
and 3.6. In essence, Equation 3.5 and Equation 3.6 are two new objective functions that
are dependent on the design variables x
1
and x
2
. Consequently, the objective functions
span a surface over the design space, where each point in the design space corresponds to
a probabilistic analysis, e.g. a Monte-Carlo simulation, which determines the function
value. The surfaces for Equations 3.5 and 3.6 are displayed in Figures 3.13 and 3.14
respectively.
Figure 3.13: Response Surface for
Equation 3.5
Equation 3.6
Equation 3.5 exhibits two maxima, the smaller one barely visible, located around
x
1
= 0 and x
2
= 0. Equation 3.6 exhibits only one local maximum, which is to be
expected, since the location of the maximum for f
2
is not affected by changes in y
1
and y
2
P
r
o
b
a
b
i
l
i
t
y
values. A simplex search method is employed to find the optima for Equations 3.5 and
3.6. The best solution found is x
1
= 0.8230 and x
2
= 0.8759 for Equation 3.5 and
x
1
= 0.0155 and x
2
= 0.0049 for Equation 3.6 with function values of
5501 . 0 ) 5 ) 8759 . 0 , 8230 . 0 ( (
1
F P and 2932 . 0 ) 5 ) 0049 . 0 , 0155 . 0 ( (
2
F P , which is
equal to the true maximum at x
1
= 0 and x
2
= 0.
Note that the surfaces in Figures 3.13 and 3.14 are very smooth, which can be
attributed to the fact that each point on the surface is based on 10,000 Monte-Carlo
simulation samples. Reducing the number of samples increases the ruggedness of the
surface. Compare Figures 3.15 and 3.16, which display the same surface for
Equation 3.5 for 10,000 and 1,000 samples respectively. The rougher surface in
Figure 3.16 makes it more difficult for simple optimization routines to find the true
optimum, since they are likely to get stuck in one of the smaller local ones. On the other
hand, a very high number of samples guaranties a smooth surface, allowing for the use of
a simple and quick optimization routine.
Equation 3.5 (10,000 Samples)
Equation 3.5 (1,000 Samples)
Oliver Bandte Chapter IV 66
CHAPTER IV
MULTI-VARIATE PROBABILITY THEORY
Chapter I established that engineering system designs are evaluated based on a
multitude of criteria and the design process typically relies on uncertain assumptions. To
accommodate both aspects of design concurrently, an extension of the commonly used
univariate probability theory is needed. It is insufficient to look at each criterion and its
distribution independently, since all attribute values are generated by the same design
process representing a common system and are thus interdependent. The assumption of
independent criteria is therefore typically unfounded. The aforementioned necessary
extension is consequently a probability theory for jointly distributed random variables.
Definition: Let X
1
, X
2
, .,X
N
be a set of random variables defined on a (discrete)
probability space . The probability that the events X
1
= x
1
, X
2
= x
2
, .,and X
N
= x
N
happen concurrently, is denoted by f(x
1
, x
2
,., x
N
) = P(X
1
= x
1
, X
2
= x
2
, ., X
N
= x
N
)
6
for
the set of desired solutions A . If the function f(x
1
, x
2
,., x
N
) is discrete, it is called
the joint probability mass function of X
1
, X
2
, .,X
N
and has the following properties
[Hogg, Tanis, 1993]:
1 ) ,...., , ( 0
2 1

N
x x x f 1 ) ,...., , (
) ,..., , (
2 1
2 1

N
x x x
N
x x x f (4.1)

6
P(X
1
= x
1
, X
2
= x
2
, ., X
n
= x
N
) = P[(X
1
= x
1
) (X
2
= x
2
) . (X
N
= x
N
)].
[ ]

A x x x f A X X X P
N
x x x A
N N
, ) ,.., , ( ) ,.., , (
) ,..., , (
2 1 2 1
2 1
(4.2)
If f(x
1
, x
2
,., x
N
) is continuous, it is called a joint probability density function of
X
1
, X
2
, ., X
N
and has the following properties [Hogg, Tanis, 1993]:
) ,...., , ( 0
2 1 N
x x x f 1 .... ) ,...., , (
2 1 2 1

N N
dx dx dx x x x f (4.3)
[ ]

A dx dx dx x x x f A X X X P
A
N N N
, .. ) ,.., , ( ) ,.., , (
2 1 2 1 2 1
(4.4)
If the lower bound of A, the set of desired solutions, is equal to the infimum
7
of
for all X
i
, i.e. if A = (inf
i
(), a
i
] , for all i = 1, 2,..., N, a function F(a
1
,a
2
,.,a
N
) can
be defined, such that:
[ ]

) ,..., , (
2 1 2 1 2 1
2 1
) ,.., , ( ) ,.., , ( ) ,.., , (
N
x x x A
N N N
x x x f A X X X P a a a F
(f is discrete) (4.5)
[ ]

A
N N N N
dx dx dx x x x f A X X X P a a a F .. ) ,.., , ( ) ,.., , ( ) ,.., , (
2 1 2 1 2 1 2 1
(f is continuous). (4.6)
F is called the joint cumulative probability distribution function.[Ross, 1993] For
=
N
and a continuous function f:
8
[ ]

1
.. ) ,.., , ( )] ,.., , ( ), ,.., , (( ) ,.., , ( ) ,.., , (
1 2 1 2 1 2 1 2 1
a
N N
a
N N N
dx dx x x x f a a a X X X P a a a F
N
. (4.7)
The univariate probability function f
Xi
for each criterion X
i
, obtained from the
traditional probabilistic design process, can also be generated with the joint probability

7
Greatest lower bound.
8

n
denotes the set of all real valued N-touples.
The common notation: F(a
1
, a
2
,.., a
N
) = P(X
1
a
1
, X
2
a
2
, ., X
N
a
N
) will be used subsequently also.
function f. f
Xi
is called marginal probability mass or density function (PDF) of X
i
and
defined by:

) ,..., (
2
2
1
) ,...., (
N
x x R
N X
x x f f
(f is discrete) (4.8)

R
N N X
dx dx x x f f .... ) ,...., (
2 2
1
(f is continuous). (4.9)
Figure 4.1: Joint and Marginal PDF of Continuous Criteria X and Y [Ang, Tang, 1984]
To further illustrate the concept of joint probability, an example for two
continuous criteria, X and Y, is displayed in Figure 4.1. The joint probability function,
f
X,Y
(x,y), creates the surface of a probability hump in the x-y-f-space, characterized by
rings of constant probability density. The distribution curves over the x- and y-axis are
the aforementioned marginal probability functions f
X
(x) and f
Y
(y), respectively. Also
displayed in Figure 4.1 are two cuts through the probability hump, marking the
probability distributions f
X,Y
(x = a, y) and f
X,Y
(x, y = b) and their respective areas
underneath f
X
(a) and f
Y
(b).
The last necessary concept to mention here for the development of a joint
probabilistic formulation is the concept of dependence of criteria. Two random variables
X and Y are said to be independent, if
) ( ) ( ) , (
,
y f x f y x f
Y X Y X
; (4.10)
otherwise X and Y are said to be dependent. This dependence is a mathematical notion
and should not be confused with causal dependence. A simple example for
mathematical dependence without causal dependence is the number of times a person
takes an umbrella to work and the number of times he wears long pants in a given month.
The two numbers increase similarly with the number of rainy days in that month, i.e. they
are (mathematically) dependent. They are, however, not causally dependent, since
wearing pants does not depend on taking an umbrella or vice versa, but rather on the rain
the person has to face on the way to work.
From here on, mathematical dependence will be referred to as correlation.
Correlation is measured by the covariance of two criteria, X and Y, defined by [Ang,
Tang, 1984]:
Cov(X,Y) = E[XY] E[X]E[Y]. (4.11)
It is more convenient, however, to use a covariance normalized by the standard
deviations,
X
and
Y
, for both criteria, called correlation coefficient [Ang, Tang, 1984]:
Y X
Y X Cov

) , (
. (4.12)
The correlation coefficient is defined over the interval (-1,1), indicating strongly
positively correlated criteria at values close to 1 and strongly negatively correlated
criteria at values close to 1. The criteria are independent, if = 0. In aerospace systems
design can be quite difficult to calculate by Equation 4.12. It is much more effective to
view the correlation coefficient differently for calculation purposes. Jointly collected
data from a probabilistic or any other analysis can be thought of as vectors of numbers.
The correlation coefficient measures the orthogonality, i.e. independence, of both vectors,
a and b. is simply the cosine of the angle between the two criterion vectors:
b a
b a
cos
. (4.13)
It does not reflect any causal relationship, it merely indicates their alignment. For
= 1, vectors are parallel and point in same direction, for = -1, vectors are parallel and
point in opposite direction. For = 0, vectors are orthogonal and the criteria are
independent. The correlation coefficient plays a significant role in the formulation of
joint probability distribution models as described in the next section.
Oliver Bandte Chapter V 71
CHAPTER V
A NEW PROBABILISTIC MULTI-CRITERIA DECISION
MAKING TECHNIQUE
Introduction
A key problem in complex systems design is measuring the goodness of a
design, i.e. finding a criterion through which a particular design is determined best.
Traditional choices in aerospace systems design, such as gross take-off weight,
acquisition cost, and payload, individually fail to fully capture the life cycle
characteristics of the system. Thus, a common approach has been to combine all criteria
together into one equation termed the overall evaluation criterion, OEC. This equation is
often very simple in its mathematical structure due to lack of any better model for the
decision process. Recognizing this lack of proper decision process modeling, a different
approach is derived in this thesis, using the system attributes concurrently as decision
criteria for the evaluation of designs. This evaluation is not based on a summation of
criteria, like an OEC, but rather the probability of satisfying all criteria at the same time,
a notion similar to a Pareto-optimality.
9
The main difference with respect to Pareto-
optimality lies in the optimizable objective function, called Probability of Success (in

9
State of economic affairs where no one can be made better off without simultaneously making another
worse off.[Nas, 1996]
satisfying all criteria).
This multi-criteria approach to decision making lends itself more suitably to
aircraft design than a single-criterion approach, since customers typically like to see all
decision criteria satisfied. For example, a probabilistic multi-criteria approach can yield
the design solution, which maximizes the probability of low cost, high capacity, speed,
and dependability, while a single objective design will only yield an optimum in one of
these criteria, neglecting all others.
The proposed method also accommodates the modeling and use of uncertain
information intrinsic to system descriptions in conceptual and preliminary design. For
example, the designer may have a flight path or mission scenario for the aircraft, but is
unclear about the operating conditions. The modeling option used in this method treats
the incomplete information probabilistically, accounting for the fact that certain values
may be more prevalent, while the actual value during operation is unknown. By
assigning probability estimates to the values within the range of interest, the method
guarantees that all values are kept as possible solutions. Hence, these values and their
corresponding likelihood constitute a new type of assumption in the systems design
process, which yields the aircrafts attributes, and thus the decision criteria, as random
variables.
If multiple, interdependent criteria are needed for decision making, a joint-
probabilistic formulation is needed to accurately estimate the probability, since the
marginal, or univariate, distribution for each criterion does not indicate the likelihood of
any other criterion value. In fact, most aircraft attributes are interdependent, since they
are evaluated by the same design process or analysis. For example, the probability of
cost being below a particular value depends on the value of payload, speed, and system
reliability. The proposed joint-probabilistic approach to multi-criteria aircraft design,
called the Joint Probabilistic Decision Making (JPDM) technique, will facilitate precisely
this estimate, closing the gap between traditional decision making tools and uncertainty
estimation techniques, as indicated in Figure 5.1.
Customer
Environment
Need for Product
Requirements/Desirements
Decision Making
Criteria
Systems Engineering
Probabilistic Techniques
Uncertainty
PRODUCT JPDM
Figure 5.1: Filling the Gap in the Design Process
The JPDM technique can be categorized as a Multi Attribute Decision Making
(MADM) as well as Multi Objective Decision Making (MODM) tool. As pictured in
Figure 5.2, MADM tools are characterized by a small number of alternatives to be
evaluated based on a large number of criteria, reflecting a selection problem. MODM
tools, on the other hand, are characterized by a small number of criteria used to determine
the best solution from amongst a large pool of alternatives. This formulation reassembles
a design (optimization) problem, optimizing the design variables based on a (small)
number of objective functions. The JPDM technique, however, is not limited to a small
number of alternatives or decision criteria.
Alternatives
Alt 1 Alt 2 Alt 3 Alt 4 Alt 5 .. Alt N
Crit 1 Value Value Value Value Value Value
.
.
C
r
i
t
e
r
i
a
Crit M Value Value Value Value Value Value
JPDM
MODM
MADM
Figure 5.2: Applicability of MADM, MODM, and JPDM
Algorithms
To integrate the joint probabilistic formulation of Chapter IV into the design
process, algorithms need to be created that allow for an easy implementation into a
numerical framework. The necessary transition from the mathematical formulation in
Chapter IV to a probabilistic model that yields the information relevant for multi-variate
decision making is described in this section. For that the following rules are established
based on which algorithms are deemed suitable for implementation into systems design:
Algorithms may not be limited in number of random variables, i.e. criteria.
Algorithms need to be flexible with respect to the criterion distributions.
Algorithms have to satisfy the conditions of a joint probability distribution.
Algorithms cannot require numerical integration.
N
M
The last requirement stems from the fact that a numerical integration would
require too many function evaluations, particularly in problems with many criteria. For
example, the numerical determination of a joint distribution with ten variables, using 100
points for integration in each dimension, requires as many as 10
20
function evaluations, a
task too daunting even for modern desktop computers. Most algorithms found in the
literature, however, violate at least one of these requirements, giving rise to the following
two.
Empirical Distribution Function:
The first algorithm that can be used for a joint probabilistic formulation is the
nonparametric Empirical Distribution Function (EDF), named after the empirically
collected data samples upon which the probability is estimated. Its univariate probability
mass function for a random variable X is defined for M samples as:
) ( ) (
1
1
x a I x f
M
j
j M X

where
'

otherwise 0
for 1
) (
x a
x a I
j
j
. (5.1)
a
j
are the criterion sample values derived from a sampling method such as the Monte-
Carlo simulation, while x is the criterion value of interest. The EDFs cumulative
probability function is consequently defined as:
) ( ) (
1
1
x a I x F
M
j
j M X

where
'

otherwise 0
for 1
) (
x a
x a I
j
j
. (5.2)
Recognizing the joint probabilistic notation from Chapter IV, the univariate EDF
can easily be extended to more random variables. The joint probability mass function
(Equation 4.2) can thus be formulated as:
( )

M
j
N jN j j M N
x x x a a a I x x x f
1
2 1 2 1
1
2 1
) ,.., , ( ) ,.., , ( ) ,.., , ( (5.3)
where ( )
'

otherwise 0
) ,.., , ( ) ,.., , ( for 1
) ,.., , ( ) ,.., , (
2 1 2 1
2 1 2 1
N jN j j
N jN j j
x x x a a a
x x x a a a I .
Similarly, the joint cumulative probability distribution function (Equation 4.5) can be
formulated as:
) ,.., , ( ) ,.., , (
2 2 1
1
1
1
2 1 N jN j
M
j
j M N
x a x a x a I x x x F

(5.4)
where ( )
'

otherwise 0
) ,.., , ( for 1
,.., ,
2 2 1 1
2 2 1 1
N jN j j
N jN j j
x a x a x a
x a x a x a I .
The joint EDF satisfies all previously stated requirements, since it is not limited in
the number of criteria, depends on joint samples for the criteria only, and is not limited by
any a priori assumptions about criterion distributions. It satisfies the conditions of a joint
probability distribution identified in Chapter IV, since its actual distribution function is
derived from the univariate distribution. It does not require any (numerical) integration
or rely on any particular sampling method and can be used as long as sample data is
available. The need for this data, however, is its very limitation, since it can only be used
in a design process with available simulation/modeling or test data. Given enough
sample data, however, the joint EDF yields the most accurate joint distribution
prediction, since it does not rely on any approximation to generate criterion statistics. Its
greatest advantage is that it does not require a correlation coefficient, which can be
difficult to estimate reliably in a design process. For very large numbers of sample data,
the joint EDF can yield the exact solution for the joint distribution.
Joint Probability Model:
A joint probability model is an explicit formulation of a parametric joint
probability density (or cumulative) distribution function that can be used as an algorithm
to compute the joint probability. Its key characteristic is that it is not based on sample
data, like the Empirical Distribution Function, but rather the statistics or characteristics of
parametric univariate criterion distributions. This characteristic is important, since it
allows for the continued use of the probabilistic information generated by the traditional
probabilistic design process with its univariate criterion distribution output.
The few explicit formulations of a joint probability density function, published in
the literature, are based on the joint Normal-Distribution, the two main references being
[Garvey, 1999] and [Tong, 1990]. Garveys book also includes the bivariate Normal,
Normal-Lognormal, and Lognormal joint distributions, published first in [Garvey, Taub,
1992]. For illustration purposes, the bivariate Normal-Distribution is reproduced here:
,
`
.
|
,
`
.
|
,
`
.
|
+

,
`
.
|
2 2
2
2
2
2 2
1
exp
1 2
1
) , (
Y
Y
Y
Y
X
X
X
X
Y X
XY
y y x x
y x f

. (5.5)
Note that the only information needed for this model consists of the means
X
and
Y
, the standard deviations
X
and
Y
, and the correlation coefficient for the criteria X
and Y. The model variables, x and y, are defined over the interval of all possible criterion
values. The advantage of this model is the limited information needed, which makes it
very flexible for use and application. For example, if only expert knowledge and no
simulation/modeling is available in the early stages of design or technology development,
educated guesses for the means, standard deviations, and correlation coefficients can be
used to execute the joint probability model. Unfortunately, it is only valid for two
variables at a time.
While Tongs book does not include Garveys Lognormal and Normal-Lognormal
formulations, it does extend the bivariate joint Normal formulation to N dimensions
[Tong, 1990]:
Definition: An N-dimensional random vector
10
X with mean vector and
covariance matrix is said to have a nonsingular multivariate Normal-Distribution,
> 0, if (i) is positive definite, and (ii) the density function of X is of the form
2 / ) , ; (
2 / 1
2 /
) 2 (
1
) , ; (
x
x
N
Q
N
e f

and
N
x (5.6)
where ) ( ) ( ) , ; (
1
x x x

N
Q and
,
`
.
|
2
2 2 1 1
2 2
2
2 2 1 12
1 1 2 1 12
2
1
N N N N N
N n
N n

L
M O M M
L
L
. (5.7)
X is said to have a singular multivariate Normal-Distribution (X ~ N
N
(, ), || = 0), if (i)
is positive semidefinite, and (ii) for some r < N there exists an N x r real matrix C such
that X and CZ
r
+ are identically distributed, where Z
r
~ N
r
(0, I
r
).
While the bivariate density functions are valid for the entire range of the
correlation coefficient , the multivariate Normal-Distribution is not defined for certain
negative . Specifically, condition (i), being positive (semi-)definite, is violated for
certain , which can easily been seen from the following proof.

10
Vector of random variables.
Proof: Without loss of generality, assume all standard deviations
i
are equal to
one, and all correlation coefficients are equal and have a value -1 < < 1. In this case,
becomes a very simple symmetric matrix of the form:
,
`
.
|
1
1
1
L
M O M M
L
L

. (5.8)
In order for to be positive (semi-)definite, all of its eigenvalues need to be
positive (or zero).[Strang, 1993] The eigenvalues can be determined from [Strang,
1993]:
,
`
.
|
,
`
.
|
,
`
.
|

0
0
0
1
1
1
0
2
1
) (
M M
L
M O M M
L
L
N
x
x
x

x I
(5.9)
which yields a system of linear equations with N-1 equations of the form
solution) zero - non only (the 1
0 ) 1 ( ) 1 ( ) 1 (
2 1

+ + +
N
x x x L
(5.10)
and one equation of the form
( ) ( ) ( )
solution). zero - non only (the ) 1 ( 1
0 ) 1 ( 1 ) 1 ( 1 ) 1 ( 1
2 1

+
+ + + + + +
N
x N x N x N
N
L
(5.11)
While the N-1 eigenvalue solutions in Equation 5.10 are all positive for
-1 < < 1, the solution for in Equation 5.11 is negative for < 1/(1-N). Hence, for
correlation coefficient values < 1/(1-N), is not positive (semi-)definite and the joint
normal density function is not defined.
A similar proof can be made for other standard deviation and correlation
coefficient values. Some values for
i
and
ij
may produce a positive (semi-)definite ,
while others do not. The conclusion to be made is that the algorithm in Equation 5.6 is
not valid for all
ij
(-1,1), and is hence unsuitable for a generally applicable joint
probabilistic decision making technique. Finally, the integral of the characteristic
function for a univariate Normal-Distribution does not exist and consequently neither
does the one for Equations 5.5 and 5.6. As a result, a numerical integration is needed to
determine the cumulative probability for both algorithms. While the bivariate algorithms
listed in [Garvey, 1999] are simple enough to be integrated numerically, Equation 5.6
accommodates many variables and its numerical integration over a large number of
criteria becomes too extensive of a task. Consequently, algorithms based on a joint
Normal-Distribution are considered unsuitable for systems design and other algorithms
need to be identified.
New Algorithm:
To remedy the shortcomings of the existing algorithms and to allow the use of
non-gaussian joint probability distributions, a new algorithm is proposed for this thesis
work. Any function that satisfies the conditions in Equations 4.3 and 4.4 can be
classified as a continuous joint probability density function. It can further be assumed
that a suitable joint density function entails the product of the univariate probability
distributions for all random variables. This is particularly true for independent random
variables (Equation 4.10), and is also reflected in the models in [Garvey, 1999] and
[Tong, 1990]. Without loss of generality, it can further be assumed that a correlation
function, accounting for the correlation between random variables, can be multiplied with
this product of univariate density functions to generate the new joint probability density
function. Explicitly:
( )
N X X X N
x x x g f f f
C
x x x f
N
,...., ,
1
) ,...., , (
2 1 2 1
2 1
L (5.12)
with ( )

N N X X X
dx dx dx x x x g f f f C
N
L L L
2 1 2 1
,...., ,
2 1
. (5.13)
f satisfies the conditions in Equations 4.3 and 4.4 as long as the correlation
function g satisfies these conditions. Specifically, g has to satisfy the following
requirements in order for f to be a joint probability density function:
g 0, since 0 ) ,...., , (
2 1

N
x x x f and 0 , , ,
2 1
N
X X X
f f f K . (5.14)
g(x
1
, x
2
, , x
N
) = 1, when X
1
, , X
N
are independent, since
N
X X X
f f f f L
2 1
.(5.15)
The second condition in Equation 4.3 is always satisfied by f, independent of g, since
( ) 1 ,...., ,
1
.... ) ,...., , (
2 1 2 1 2 1 2 1
2 1

N N X X X N N
dx dx dx x x x g f f f
C
dx dx dx x x x f
N
L L L
with ( )

N N X X X
dx dx dx x x x g f f f C
N
L L L
2 1 2 1
,...., ,
2 1
.
Using Equation 5.12 as an algorithm for the determination of the joint probability
reduces the problem of finding a suitable joint probability density function to finding a
suitable correlation function. In general, all correlation functions, satisfying the
conditions in Equations 5.14 and 5.15, are possible functions, while some are easier to
implement in a computer algorithm than others.
The fundamental problem for the determination of the joint cumulative
probability is the problem of dimensionality that prohibits a numerical integration in the
case of four or more criteria (at current processor speeds). If a hundred integration points
are desired in each dimension for sufficient accuracy, four criteria would require
(10
2
)
4
= 10
8
= 100 million function evaluations, a prohibitively expensive task in systems
design. An alternative approach is to integrate the joint density function analytically,
since the univariate density functions and the correlation function are explicitly known.
This approach however puts an additional restriction on the correlation function and the
univariate PDFs:
i N i i X
dx x x x x g x f
i
) , , , , ( ) (
2 1
K K , i = 1, 2, , N, (5.16)
needs to be defined for all criteria i. This requirement can be derived from simple
integration rules for independent functions:
( )

1 2 2 1 2 1 2 1
,...., ,
1
.... ) ,...., , (
2 1
dx dx dx x x x g f f f
C
dx dx dx x x x f
N N X X X N N
N
L L .(5.17)
Hence, each integration step in Equation 5.17 requires ( )
i i X
dx x g f
i
to exist and
be finite. This, in turn, becomes a requirement for the correlation function, once the
univariate distribution function has been chosen for X
i
. A simple example for g is the
linear function
,
`
.
|
1
1 1
2 1
2
) , , , (
N
i
N
i j j
j j
i
i i
ij N
range
median x
range
median x N
x x x g K . (5.18)
The transformation for x
i
, using range
i
and median
i
, guaranties that g > 0. The
variable range
i
denotes the length of half the range in which the distribution of X
i
has
sufficiently large probability/frequency values, i.e. range
i
= (Max
i
Min
i
)/2. The term
sufficiently large is at the users discretion, making range
i
an input to the algorithm.
By applying Equation 5.18, Equation 5.16 can be simplified using the product rule:
( )

max
min
max
min
) (
] [
i
i
i
i
i i i
x
x
i X
i
i
x
x X i i X
dt F
t
t g
g F dx x g f . (5.19)
If
i
i
t
t g
) (
is not a function of t
i
, i.e. g is linear with respect to t
i
, the integral
( )
i i X
dx x g f
i
exists and is finite for all univariate probability distributions that can be
integrated twice. A more comprehensive list of suitable correlation functions for this new
algorithm is provided in Appendix B.
The following bivariate example illustrates the new algorithm and the use of the
correlation function introduced in Equation 5.18. Assume random variables X and Y are
distributed according to a Gamma-Distribution, X ~ G(4,3,5), Figure 5.3, and a Weibull-
Distribution, Y ~ W(3,2,7), Figure 5.4, respectively.
11
Further assume that both variables
are correlated with a correlation coefficient = -0.6. Consequently, the proposed joint
probability function is formulated as:
4 4 4 3 4 4 4 2 1
4 4 4 3 4 4 4 2 1
4 4 3 4 4 2 1
) , (
) (
) 7 ( 3 1 2
) (
3
5
4
1 4
1
8
20
25
6 . 0 1 ) 7 ( 2 3
) 4 ( 3
) 5 (
) , (
2
y x g
y f
y
x f
x
y x
e y e
x
y x f
Y
X
,
`
.
|

. (5.20)

11
See Appendix A for a definition and description of the distributions used in this thesis.
0 10 20 30 40 50
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
x
Figure 5.3: Gamma-Distribution for X
7 7.5 8 8.5 9
0
0.5
1
1.5
y
Figure 5.4: Weibull-Distribution for Y
The graphs in Figures 5.5 and 5.6 display the joint probability distribution f(x,y) in
three and two dimensions, respectively. In order to be able to display the distribution
better, the graph in Figure 5.5 is rotated 180 around the center axis. The graph in
Figure 5.6 displays concentric lines of constant frequency indicating the location of the
area of highest frequency around the common center. The lines are obtained by slicing
the three-dimensional distribution at several levels of constant frequency and projecting
the cut onto the two-dimensional plane spanned by the two random variables.
Figure 5.5: Example Joint Probability
Distribution in 3D
10 20 30 40
7
7.5
8
8.5
9
x
y
Figure 5.6: Contour Plot of Example
Joint Probability Distribution
Increasing Frequency
P
r
o
b
a
b
i
l
i
t
y

D
e
n
s
i
t
y
P
r
o
b
a
b
i
l
i
t
y

D
e
n
s
i
t
y
Five Schemes for Implementation
In order to be able to use Equations 5.3 and 5.12 for probabilistic systems design,
schemes for practical implementation need to be established. Five such schemes are
introduced in this thesis, all of which are extensions of the three probabilistic design
methods outlined in Chapter III. The schemes are summarized in Figure 5.7, indicating
that all assume some modeling and simulation, either in form of computer programs or
explicit equations.
Problem Definition Modeling and
Criteria Simulation
Problem Definition Modeling and
Criteria Simulation
Monte-Carlo
Simulation
Empirical
Distribution
Function
Joint
Probability
Model
Joint
Probability
Model
Joint
Probability
Model
Metamodel
Fast Probability
Integration
CDF
Regression
CDF
Regression
Correlation
Function
Monte-Carlo
Simulation
Correlation
Function
Scheme #I Scheme #II Scheme #V
Least
Effort
Most
Accurate
CDF
Regression
Correlation
Function
Empirical
Distribution
Function
Scheme #III Scheme #IV
Figure 5.7: Five Schemes for the Evaluation of the Joint Probability Distribution
In order to illustrate the five different schemes, the example problem utilized in
Chapters II and III is employed to generate a joint probability distribution with each
scheme, followed by a comparison of all distributions. As noted in Chapter III, the
values for y
1
and y
2
are assumed to be uncertain and normal distributions are assigned to
them, such that Y
1
~ N(1,1) and Y
2
~ N(1,1). Also, the deterministic variables are held
constant for each distribution, such that x
1
= x
2
= 1.
Scheme #I Monte-Carlo Simulation with Empirical Distribution Function
The first scheme is an extension of the probabilistic design Method #1, identified
in Chapter III. It uses the samples from the system analysis tool generated through a
Monte-Carlo simulation as data for the Empirical Distribution Function (Equation 5.3).
This method is the most accurate, but requires the most system analysis effort, which can
be computationally expensive.
To create the distribution based on the example problem from Chapters II and III,
a Monte-Carlo simulation is employed to generate 10,000 sample values from the noise
variable distributions for Y
1
and Y
2
as inputs to Equations 2.16 and 2.17. The resulting
10,000 samples for F
1
and F
2
are then used in the Empirical Distribution Function
(Equation 5.3):
( )

M
i
M
f F f F I f f f
1
2 2 1 1
1
2 1
, ) , ( , (5.21)
where ( )
'
+ +

otherwise 0
2 2 2 2
for 1
,
2
2 2
2
2
1
1 1
1
1
2 2 1 1

f F f f F f
f F f F I .
1
and
2
denote the ranges around f
1
and f
2
that F
1
and F
2
must fall within,
respectively. Equation 5.21 can be used to generate the plots of the joint probability
distribution displayed in Figures 5.8 and 5.9. Note the sharp frontier in the two-
dimensional representation of the joint probability distribution in Figure 5.9. It is due to
the local (and global) optima of f
1
and f
2
within the distribution range of Y
1
and Y
2
. The
scattered lines in this plot are contours of equal probability density, produced by the
sample points that are found throughout the f
1
-f
2
-plane in this scattered fashion.
Figure 5.8: Example Distribution
Generated by Scheme #I (3D)
-10 -5 0 5 10
0
5
10
15
20
25
30
f1
f
2
Generated by Scheme #I (2D)
Scheme #II Monte-Carlo Simulation with Joint Probability Model
Scheme #II is also an extension of the probabilistic design Method #1 from
Chapter III, regressing the data generated by the same Monte-Carlo simulation for each
criterion individually in order to obtain its univariate probability distribution function. In
addition, a correlation function is determined, e.g. via correlation coefficients for each
pair of criteria. The joint probability model in Equation 5.12 can then be used with the
univariate distributions and the correlation function to create the joint probability
distribution. Due to the regression of the probability distributions, this scheme produces
a joint probability distribution less accurate than the one in Scheme #I, but it yields
graphs that are more amenable (no scatter plots) and therefore to be preferred for plotting.
-10
-5
5
0
10
To create the joint probability distribution with Scheme #II, the 10,000 samples
from Scheme #I are regressed to establish univariate probability distributions for F
1
and
F
2
. The regression is illustrated in Figure 5.10 for F
1
and Figure 5.11 for F
2
, yielding a
Beta-Distribution with = 17, = 7, Min = -10, Max = 11.46 for F
1
and a Gamma-
Distribution with parameters a = 4, b = 2, Loc = -4.31 for F
2
. The correlation coefficient
for F
1
and F
2
was found to be = 0.08.
.000
.016
.032
.048
.064
1
in Scheme #II
.000
.026
.052
.077
.103
2
in Scheme #II
Once the regression has determined the correlation coefficient and the univariate
probability distributions for F
1
and F
2
, the joint probability distribution can be
determined based on the Joint Probability Model in Equation 5.12 with a correlation
function based on Equation 5.18:
F
r
e
q
u
e
n
c
y
Beta-
Distribution
F
1
11.5 6.5 1.5 -3.5 -8.5 f
1
F
r
e
q
u
e
n
c
y
Gamma-
Distribution
F
2
35 25 15 5 -5 f
2
4 4 4 4 3 4 4 4 4 2 1 4 4 4 3 4 4 4 2 1 4 4 4 4 4 4 4 3 4 4 4 4 4 4 4 2 1
) , (
2 1
) (
4
2
31 . 4
1 4
2
) (
1 7
1
1 17
1
2 1
2 1 2
2
2
1
1
20
12
73 . 10
73 . 0
08 . 0 1
) 4 ( 2
) 31 . 4 (
46 . 21
46 . 11
46 . 21
10
) 7 ( ) 17 (
) 7 17 (
) , (
f f g f f
f
f f
f f e f f f
f f f
F F
,
`
.
|

+

,
`
.
|
,
`
.
| +

+

. (5.22)
The joint probability distribution is visualized in Figures 5.12 and 5.13 for three
and two dimensions respectively. Note that the contours of constant density in
Figure 5.13 are much smoother than in Figure 5.9.
Generated by Scheme #II (3D)
-5 0 5 10
-5
0
5
10
15
f1
f
2
Generated by Scheme #II (2D)
Scheme #III Metamodel/Monte-Carlo Simulation with Empirical Distribution Function
Scheme #III is an extension of the probabilistic design Method #2, using samples
for the Empirical Distribution Function (Equation 5.3), obtained from a metamodel and
generated by a Monte-Carlo simulation. This method may require significantly less
simulation analysis to generate the metamodel, but its accuracy in predicting the joint
probability distribution depends heavily on the prediction accuracy of the metamodel
itself. To create the distribution based on the example problem from Chapters II and III,
Equations 2.16 and 2.17 are approximated with a response surface equation (RSE). As
pointed out in the example for Chapter III, while f
1
is a quadratic polynomial already and
can be used directly as an RSE. f
2
on the other hand can be approximated, with Equation
3.8 as the corresponding response surface equation. Subsequently, a Monte-Carlo
simulation is employed to generate 10,000 sample values from the noise variable
distributions for Y
1
and Y
2
as inputs to Equations 2.16 and 3.8. The resulting 10,000
samples for F
1
and F
2
RSE
are then used in the Empirical Distribution Function
(Equation 5.3):
( )

M
i
RSE
M
f F f F I f f f
1
2 1 1
1
2 1
2
, ) , ( , (5.23)
where ( )
'
+ +

otherwise 0
2 2 2 2
for 1
,
2
2
2
2
1
1 1
1
1
2 1 1
2
2

f F f f F f
f F f F I
RSE
RSE
.
1
and
2
denote the ranges around f
1
and f
2
that F
1
and F
2
RSE
must fall within,
respectively. Equation 5.23 can be used to generate the plots of the joint probability
distribution displayed in Figures 5.14 and 5.15. Note again the sharp frontier in the two-
dimensional representation of the joint probability distribution in Figure 5.15, which is
due to the local (and global) optima of f
1
and f
2
RSE
within the distribution range of Y
1
and
Y
2
. The scattered lines in this plot are contours of equal probability density, produced by
the sample points that are found throughout the f
1
-f
2
-plane in this scattered fashion.
Generated by Scheme #III (3D)
-5 0 5 10
5
10
15
20
f1
f
2
Generated by Scheme #III (2D)
Scheme #IV Metamodel/Monte-Carlo Simulation with Joint Probability Model
Scheme #IV is also an extension of the probabilistic design Method #2, identified
in Chapter III. In this case, the data generated by the Monte-Carlo simulation is based on
a metamodel as system analysis code approximation and regressed for each criterion
individually to obtain its univariate probability distribution. The information for the
correlation function, e.g. correlation coefficients for each pair of criteria, of the Joint
Probability Model (Equation 5.12) can be obtained from either the system analysis data
used for the generation of the metamodel, or the Monte-Carlo simulation data based on
the metamodel. When the metamodel generation calls only for few system analysis
simulations, the Monte-Carlo simulation data provides the greater statistical significance
for the correlation coefficients and is to be preferred, despite the fact that the data is
based on a system analysis code approximation. When the metamodel generation
requires a significant amount of system analysis already, the data might as well be used
-5
5
0
10
for the correlation coefficient estimation, since it is more accurate than the Monte-Carlo
simulation data. This scheme is less accurate than Scheme #III, but produces the nicer
graphs, similar to Scheme #II, and is to be preferred for plotting. To create the joint
probability distribution with Scheme #IV, the 10,000 samples from Scheme #III are
regressed to establish univariate probability distributions for F
1
and F
2
. Note that these
samples are based on the metamodels for f
1
and f
2
, Equations 2.16 and 3.8. The
regression is illustrated in Figure 5.16 for F
1
and Figure 5.17 for F
2
, yielding a Beta-
Distribution with = 17, = 7, Min = -10, Max = 11.46 for F
1
and a Gamma-
Distribution with parameters a = 4, b = 1, Loc = 0.78 for F
2
. The correlation coefficient
for F
1
and F
2
was found to be = 0.02.
.000
.016
.032
.048
.064
1
in Scheme #IV
.000
.015
.030
.045
.059
2
in Scheme #IV
F
r
e
q
u
e
n
c
y
Beta-
Distribution
F
1
11.5 6.5 1.5 -3.5 -8.5 f
1
F
r
e
q
u
e
n
c
y
Gamma-
Distribution
F
2
20 15 10 5 0 f
2
Once the regression has determined the correlation coefficient and the univariate
probability distributions for F
1
and F
2
, the joint probability distribution can be
determined based on the Joint Probability Model in Equation 5.12 with a correlation
function based on Equation 5.18:
4 4 4 4 4 3 4 4 4 4 4 2 1 4 4 4 3 4 4 4 2 1 4 4 4 4 4 4 4 3 4 4 4 4 4 4 4 2 1
) , (
2 1
) (
4
1
78 . 0
1 4
2
) (
1 7
1
1 17
1
2 1
2 1 2
2
2
1
1
10
10
73 . 10
73 . 0
02 . 0 1
) 4 ( 1
) 78 . 0 (
46 . 21
46 . 11
46 . 21
10
) 7 ( ) 17 (
) 7 17 (
) , (
f f g f f
f
f f
f f e f f f
f f f
F F
,
`
.
|

,
`
.
|
,
`
.
| +

+

. (5.24)
The joint probability distribution is visualized in Figures 5.18 and 5.19 for three
and two dimensions respectively. Note that the contours of constant density in
Figure 5.19 are much smoother than in Figure 5.15.
Generated by Scheme #IV (3D)
-5 0 5 10
-5
0
5
10
15
f1
f
2
Generated by Scheme #IV (2D)
Scheme #V Fast Probability Integration with Joint Probability Model
The fifth scheme is the only one identified here that extends the probabilistic
design Method #3. It uses a fast probability integration technique, like the Advanced
Mean Value (AMV) method, or one of its derivatives, to obtain a univariate cumulative
distribution function for each criterion, which subsequently can be used in the Joint
Probability Model (Equation 5.12). A disadvantage of this particular scheme, however, is
its lack of correlation information on the criteria to create the correlation function for the
algorithm. This means that the correlation function must be obtained through some other
means. Also, in the case of the AMV method, the scheme only yields one univariate
CDF per probabilistic analysis. Therefore, despite the high efficiency of the AMV
method, Scheme #V can become computationally intensive for large numbers of criteria.
Last but not least, the transformation of the CDF into a probability function that can be
used in Equation 5.12 is not a trivial task, since the information on the CDF is generated
in form of function values only. The accuracy of Scheme #V, using the AMV method, is
similar to the accuracy of Scheme #IV with a response surface equation, since the AMV
method has a prediction accuracy similar to a response surface equation
12
and both
methods rely on a probability distribution regression.
In order to generate the joint probability distribution for the example problem in
Chapters II and III, the univariate distributions generated by the AMV method in the
example for Chapter III can be employed directly. By visual comparison of the
cumulative probability plot (Figure 3.11), the distribution for F
1
is determined the to be a
Beta-Distribution with = 10, = 5, Min = -10, Max = 10. Based on the cumulative
distribution plot in Figure 3.12, the univariate distribution for F
2
, estimated with the
AMV method, is very similar to the distribution based on the Monte-Carlo simulation.

12
See Chapter III for discussion.
For this reason, F
2
in Scheme #V is assumed to have the same distribution as in
Scheme #II, a Gamma-Distribution with parameters a = 4, b = 2, and Loc = -4.31. The
correlation coefficient cannot be determined with Scheme #V, but is borrowed for this
example from Scheme #II: = 0.08.
.000
.026
.052
.077
.103
2
in Scheme #V
With the correlation coefficient and the univariate probability distributions for F
1
and F
2
, the joint probability distribution can be determined based on the Joint Probability
Model in Equation 5.12 with a correlation function based on Equation 5.18:
4 4 4 4 3 4 4 4 4 2 1 4 4 4 3 4 4 4 2 1 4 4 4 4 4 4 3 4 4 4 4 4 4 2 1
) , (
2 1
) (
4
2
31 . 4
1 4
2
) (
1 5
1
1 10
1
2 1
2 1 2
2
2
1
1
20
12
20
0
08 . 0 1
) 4 ( 2
) 31 . 4 (
20
10
20
10
) 5 ( ) 10 (
) 5 10 (
) , (
f f g f f
f
f f
f f e f f f
f f f
F F
,
`
.
|

+

,
`
.
|
,
`
.
| +

+

. (5.25)
The joint probability distribution is visualized in Figures 5.21 and 5.22 for three and two
dimensions respectively.
F
r
e
q
u
e
n
c
y
Gamma
Distribution
F
2
35 25 15 5 -5 f
2
Generated by Scheme #V (3D)
-5 0 5 10
-5
0
5
10
15
f1
f
2
Generated by Scheme #V (2D)
Comparison of Schemes
In order to compare the relative performance of the five schemes, three plots are
employed to provide a visual contrast of the different joint probability distributions. The
first two plots in Figures 5.23 and 5.24 compare the joint distribution from Scheme #I
with the one from Scheme #II and the one from Scheme #III with the one from
Scheme #IV respectively. Essentially this comparison tests the ability of the regression
in Scheme #II and Scheme #IV together with the correlation function to accurately
represent the data from the Monte-Carlo simulation.
Both plots clearly indicate that Schemes #II and #IV are not capturing the steep
ridge of the distributions from Schemes #I and #III. Both distributions yield a certain
likelihood for high values for f
1
and low values for f
2
, which the data does not indicate. It
appears that particularly the correlation function in Equation 5.18 is not adequate for the
presented data, since the correlation function used is symmetric while the data is
asymmetric. Unfortunately, the right correlation function can only be determined after
the Monte-Carlo simulation data has been generated so that Equation 5.18 with its
correlation coefficients is still the best first guess when more detailed information is
lacking. Also, the distributions from Schemes #II and #IV are both yielding the right
location of the distribution so that both can be used in visual comparisons of joint
probability distributions for different products. However, for the estimation of an
accurate joint probability value Schemes #I and #III are to be preferred over Schemes #II
and #IV.
-5 0 5 10
-5
0
5
10
15
20
f1
f
2
Figure 5.23: Comparison of Joint
Distributions from Schemes #I and #II
-5 0 5 10
0
5
10
15
20
f1
f
2
Figure 5.24: Comparison of Joint
Distributions from Schemes #III and #IV
The plot in Figure 5.25 finally compares the three schemes employing the Joint
Probability Model in Equation 5.12. Here the differences are entirely due to the different
univariate distributions, regressed from system analysis data (Scheme #II), regressed
from response surface equation data (Scheme #IV), or generated through the AMV
method (Scheme #V). The distribution from Scheme #II can be seen as the most
accurate, despite the aforementioned criticism, with the distribution from Scheme #V as a
close second best. Scheme #IV seems to miss the target distribution the most, due to the
prediction inaccuracy of the response surface equation. Note that these results cannot be
generalized, since Scheme #IV may yield a distribution quite accurate, if the metamodel
prediction is accurate. This is particularly true for aerospace systems design, where
response surface equations have demonstrated their adequate approximation capability of
the economic objectives specifically.[Mavris, Bandte, 1998], [Mavris, Bandte,
DeLaurentis, 1999], [Mavris, Bandte, Schrage, 1995]
-5 0 5 10
-5
0
5
10
15
f1
f
2
Figure 5.25: Comparison of Joint Distributions from Schemes #II, #IV and #V
Joint Probabilistic Decision Making Technique
Once the joint probability distribution has been obtained through one of the five
schemes identified in the previous section, the joint Probability of Success (POS), can be
determined. The joint Probability of Success is the envelope objective function or overall
evaluation criterion for the Joint Probabilistic Decision Making (JPDM) technique,
measuring the probability of satisfying all criteria. The key concept in this technique is
an area of interest, limited by the values z
jmin
and z
jmax
for each criterion j, that contains
Scheme #II
Scheme #IV
Scheme #V
all values the decision-maker deems satisfactory. The joint probability distribution is
then superposed with this area of interest, yielding the Probability of Success as the
volume underneath the distribution, over the area of success. This concept is visualized
in Figure 5.26 and mathematically represented by:

M
j
j j j
z z z I
M
POS
1
max min
) (
1
, (5.26)
using the Empirical Distribution Function (Equation 5.3), while M are the number of
samples, and
max
min
) (
z
z
t t d f POS
, (5.27)
using the Joint Probability Model (Equation 5.12).
0.0 00
0 .001
0.0 02
0.00 3
0.0 04
0 .00 5
0.0 06
P
r
o
b
a
b
i
l
i
t
y

D
e
n
s
i
t
y
z
1max
Ellipses of Constant Density
z
1min
z
2min
z
2max
z
1
z
2
z
1max
z
1min
z
2min
z
2max
z
1
z
2
Area of
Interest
Figure 5.26: Display of the Joint Probability Distribution for Two Criteria
An information flowchart of the Joint Probabilistic Decision Making technique is
displayed in Figure 5.27, arriving at the joint Probability of Success through a process
that contains eight elements: identification of criteria, assignment of probability
distributions to noise variables, fixation of control variables, identification of the system
analysis tool, evaluation of the joint probability distribution, establishment of criterion
values, determination of preferences among criteria, and calculation of the joint
Probability of Success.
Criteria (Z)
Objectives/Requirements
1
Analysis Tool
Z = f(x,Y)
Criterion
Values
(z
min
, z
max
)
6
Weights (w)
7
4
Alternatives
Baseline
Uncontrollable/
Noise Variables
Y
Non-Deterministic
Control
Variables
x = (0.5, 4, 0.07, )
Deterministic
Joint Probability
Distribution
(Five Schemes)
POS
2
3
5
8
Figure 5.27: Joint Probabilistic Decision Making Technique
Element 1 Identification of Criteria
First, the criteria for the decision making process need to be determined. They are
typically comprised of customer requirements or desirements as well as objectives that
need to be satisfied from the designers perspective. These criteria are usually
established in the early conceptual design phase of the product (see Chapter I). The Joint
Probabilistic Decision Making technique treats this set of criteria as a random vector,
represented by Z.
Element 2 Assignment of Probability Distributions to Noise Variables
All variables that are not under the control of the designer, i.e. their values are not
known with certainty, need to be assigned probability distributions that represent the
likelihood of taking on certain values. This element allows the subsequent simulation to
evaluate a range of values rather than a single deterministic number.
Element 3 Fixation of Control Variables
For each probabilistic analysis, all variables that are part of the analysis and under
the control of the designer
13
need to be held constant. For a product selection, i.e.
evaluation of alternatives, the vector x, containing the control variable values, is a
representation of each alternative under consideration. For optimization, the vector x is
the set of design variables that is manipulated by the optimization routine.
Element 4 Identification of System Analysis Tools
Chapter I underlined the importance of mathematical analysis in systems design,
for example in the form of a computer program. For this reason, the probabilistic design
methods in Chapter III and the five implementation schemes from the previous section all
center around system analysis on a computer. Since the system analysis tool is of such
importance, the following principles for the selection of its tools should be kept in mind:
the analysis tool needs to yield numerical information on the criteria, and
the numerical information should be dependent on the noise and control variables.

13
A variable is under the control of a designer if its value is considered known (with certainty).
Element 5 Evaluation of the Joint Probability Distribution
This element is comprised of one of the five schemes for calculating the joint
probability distribution, outlined in the previous section and Figure 5.7. It is the key
element of the Joint Probabilistic Decision Making technique and is the major scientific
contribution of this thesis work.
Element 6 Establishment of Criterion Values
As outlined in the beginning of this section, the Probability of Success is
determined by two components: the joint probability distribution and the area of interest.
The values defining the area of interest are a minimum and a maximum
14
for each
criterion and are listed in the vectors z
min
and z
max
. Values in between these limits are
considered to be satisfying the decision-makers objective. Typically, these values are
supplied through Requests for Proposals or other means during the initial design phase.
Additional requirements may be imposed through design and performance goals,
government regulations, or customer desirements.
Element 7 Determination of Preferences Among Criteria
The seventh element of the JPDM technique determines preferences supplied by
the customer or designer for each criterion. Preferences indicate which criteria are more
important than others to the decision-maker. They are usually represented by a set, or
vector w, of (preference) weights which are normalized to sum to 1, signifying the

14
infimum and supremum in the case of minus and plus infinity
relative importance of each criterion.[Hwang, 1981] If no criterion is associated with a
prevalent preference over other criteria, all weights w
i
, i = 1, 2, , N, are assigned an
equal value of 1/N, with N being the number of criteria.
Element 8 Calculation of the Probability of Success
The eighth element finally combines the criterion values, the weights, and the
joint probability distribution function to calculate the joint Probability of Success (POS).
The Probability of Success denotes the chance for a design solution to produce criterion
values within the area of interest and constitutes the objective function or evaluation
criterion for the Joint Probabilistic Decision Making technique. POS is always attempted
to be maximized, while values larger than one are impossible.
While this section described the essence of the Joint Probabilistic Decision
Making technique itself, the following sections outline its use and application,
particularly for optimization, product selection, and requirement trade-off.
Optimization
One important use of the Joint Probabilistic Decision Making technique is for
optimization. As outlined in Chapter II, current multi-criteria optimization techniques
suffer from an inability to account for uncertain parameters. As remarked in Chapter III,
current probabilistic design methods only account for one criterion at a time without
supplying information about the likelihood of meeting other criteria. The use of the joint
Probability of Success (POS) as an objective function for optimization, accounts for both,
uncertainty in the analysis parameter values and a multitude of criteria. Principally, the
multi-criteria optimization problem is transformed into a univariate maximization
problem, trying to achieve the highest possible Probability of Success by perturbing the
design variables. As such, the baseline values for the design variables may situate the
joint probability function as indicated in Figure 5.28, yielding a small Probability of
Success. By manipulating the design variables, however, the joint probability
distribution may be shifted into the area of interest, yielding a higher Probability of
Success. Naturally, POS cannot take on values higher than one. The Joint Probabilistic
Decision Making technique in itself does not provide an optimization scheme, but POS
can be used as an objective function for any univariate optimization scheme. Examples
of such schemes can be found in [Reklaitis, Ravindran, Ragsdell, 1983] and
[Vanderplaats, 1999]. To illustrate the use of the Joint Probabilistic Decision Making
technique for optimization, it is extended in Figure 5.29 to include the optimization loop
and an exit criterion.
Criterion 1
C
r
i
t
e
r
i
o
n

2
O
p
t
i
m
i
z
e
Baseline
Optimal
Figure 5.28: Shifting the Joint Probability Distribution During Optimization
Area of
Interest
Weights (w)
Analysis Tool
Z = f(x,Y)
7
4
Baseline
Uncontrollable/
Noise Variables
Y
Non-Deterministic
Control
Variables
x = (0.5, 4, 0.07, )
Deterministic
Joint Probability
Distribution
(Five Schemes)
POS
2
3
5
Update
Deterministic
Variables in
Optimization
Scheme
Satisfactory
POS ?
Solution with
Highest POS
Y
N
Criteria (Z)
1
9
Optimization Loop
8
10
Criterion
Values
(z
min
, z
max
)
6
Figure 5.29: Joint Probabilistic Decision Making Technique for Optimization
First the criteria are identified based on which the decision is made, i.e. based on
which the design is evaluated. Second, probability distributions are assigned to the noise
variables, i.e. the variables that introduce the uncertainty into the optimization process.
Third, a baseline is established that corresponds to a set of control variable values. Next,
the joint probability distribution is evaluated through the use of one of the
aforementioned five schemes and the system analysis tool. Finally, using the weights and
criterion values that identify the area of interest, the Probability of Success is calculated
for the baseline.
The new Element #9 is a checkpoint that determines whether the calculated POS
is large enough. If so, the current setting of design variables represents the best solution.
If the baseline, or starting point, already yields a probability of one, which is unusual for
a baseline, the criterion values should be revisited and more realistic values should be
established. If POS is not deemed satisfactory, the deterministic control variables need to
be updated based on the optimization scheme employed (Element #10).
Using the updated values for the deterministic variables, the probabilistic analysis
is repeated to yield the new POS, thereby closing the optimization loop. Note that the
noise variable distributions, criterion values, and weights, do not change during the entire
optimization process, only the deterministic variables change from one iteration to the
next. Once POS cannot be improved upon, the optimization loop is terminated and the
last design variable setting constitutes the solution. To illustrate the use of the Joint
Probabilistic Decision Making technique for optimization further and compare it to
existing techniques, the example from Chapters II and III is employed here again.
To demonstrate the use of the Joint Probabilistic Decision Making (JPDM)
technique for optimization, the example previously used to generate joint probability
distributions for the five different schemes is employed here again. Furthermore, to
demonstrate its capability, optimization with JPDM is compared to the methods used in
the example of Chapter II: MaxiMin, Overall Evaluation Criterion, and Goal Attainment
method.
To illustrate the process outlined in Figure 5.29, the example problem is first
mapped to all elements of the process. The obvious starting point is the identification of
the criteria: f
1
and f
2
. As before, the values for y
1
and y
2
are assumed to be uncertain and
normal distributions are assigned to them, such that Y
1
~ N(1,1) and Y
2
~ N(1,1). The
values for the deterministic variables are selected for each optimization loop based on a
simplex or sequential programming/linear search algorithm, with a baseline or initial
guess of x
1
= x
2
= 1. The system analysis tool is comprised of two example equations:
2 1
2
2 2
2
1 1 1
) ( 5 . 0 ) ( 5 . 0 5 Y Y Y x Y x F + (5.28)
2
2 2 1
2
1 2
cos cos
2
Y x x Y Y F + + + (5.29)
subject to: 2 2
1
x and 2 2
2
x .
To evaluate the joint probability distribution, Scheme #I suggests itself, since it is
the most accurate scheme and the system analysis is simple enough to generate an
extensive amount of data directly from it. Also, no plotting of joint probability
distributions is necessary at this point. The criterion values to be satisfied have been
established in earlier uses of the example problem to be F
1
5 and F
2
5. The important
difference between this example and that in Chapter III is that POS represents the
probability of both, F
1
and F
2
, being larger than 5 concurrently. Therefore, the
optimization problem can be formulated as:
) 5 cos cos , 5 ) ( 5 . 0 ) ( 5 . 0 5 ( max
) 5 , 5 ( max max
2
2 2 1
2
1 2 1
2
2 2
2
1 1
,
2 1
, ,
2
2 1
2 1 2 1
+ + + +

Y x x Y Y Y Y Y x Y x P
F F P POS
x x
x x x x
(5.30)
subject to: 2 2
1
x , 2 2
2
x , Y
1
~ N(1,1), and Y
2
~ N(1,1).
Using this formulation and the process described above, POS is calculated for the
baseline to be 0.1254. This Probability of Success can certainly be improved upon and
the deterministic variables are adjusted by the optimization schemes, thereby entering a
new iteration in the optimization loop. Once no improvement in POS can be found
through adjustment of the deterministic variables, the solution to the optimization
problem is established with the last setting of the deterministic variables.
To illustrate the use of Equation 5.30 as an objective function for optimization
and in order to get a better feel for the nature of the objective function in this example, a
surface plot is generated over the design space and displayed in Figure 5.30. Note that,
similar to the surface plot in Figure 3.16, POS has a steep drop off and a somewhat
rugged surface. The small local maximum for small values of x
1
and x
2
is barely visible.
However, as mentioned in Chapter III before, a line search method may have a chance of
getting stuck in it. For this reason, the simplex search technique was employed in the
following comparison of the different approaches to probabilistic optimization.
Figure 5.30: POS Surface Plot Over Design Space
In order to truly appreciate the advantages of POS as an objective function, the
following possible approaches to multi-objective probabilistic design/optimization are
compared to each other with the help of the example problem discussed above.
Considering the fact that F
1
and F
2
are explicitly dependent on Y
1
and Y
2
, samples can be
generated based on the distributions for Y
1
and Y
2
and the optimal point for each sample
(y
1
, y
2
) can be determined using the MaxiMin, Overall Evaluation Criterion, or Goal
Attainment method. Once the optimal solution is found for each sample point, the
statistical mean of these solution points can be established and returned as the solution to
the multi-objective optimization problem. A visual representation of this approach can
be found in Figure 5.31 for 100 sample points.
Note that these methods cannot be considered accurate in yielding the optimal
solution, since the sample mean is based on the different solutions for the deterministic
design variables and not the function values for f
1
and f
2
. Hence, they do not guarantee a
high value for the functions themselves or the probability of achieving a certain function
value level. In any case, as in Chapter II, the OEC seems to yield the more balanced
solution. In contrast, the solutions presented in Figure 5.32 are based on achievement
levels for the objectives f
1
and f
2
as elaborated next.
-2 -1 0 1 2
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Figure 5.31: Location of Optima of
MaxiMin, OEC, and Goal Attainment
Method for 100 Samples
-2 -1 0 1 2
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Figure 5.32: Location of Optima for
Different Probabilistic Multi-Objective
Optimization Methods
Alternative approaches to obtaining a solution to the multi-objective probabilistic
optimization problem are the MaxiMin, OEC, or Goal Attainment method with objective
functions presented in Equations 3.5 and 3.6. Since both equations seek to maximize the
probability of achieving function values for F
1
and F
2
larger than 5, the actual objectives
(probabilities) are deterministic and all three methods can be employed in this multi-
objective optimization problem. No normalization is necessary, since the achievable
values for the objectives lie between 0 and 1. The goals for the Goal Attainment method
are naturally equal to 1, since it is the highest probability value attainable.
x
2
x
1
x
2
x
1
MaxiMin
Solutions
OEC
Solutions
Goal Attain
Solutions
Mean for
OEC
Mean for MaxiMin
And Goal Attain
JPDM/
Simplex Search
JPDM/Line Search
P(F
1
5)
OEC
P(F
2
5)
Goal Attain
MaxiMin
The solutions presented in Figure 5.32 indicate the differences in methods very
well. The solutions to the univariate optimization problems of maximizing the
probability of achieving F
1
function values larger than 5 and maximizing the probability
of achieving F
2
function values larger than 5 are added to the graph for reference
purposes. The two solutions generated by the Joint Probabilistic Decision Making
technique correspond to the two optimization techniques employed in conjunction with
POS, a sequential programming/line search and a simplex search technique.[Branch,
Grace, 1996]
The true test of superiority for all methods is listed in Table 5.1. All solutions
presented in Figure 5.32 produce a joint probability of achieving function values larger
than 5 as well as corresponding univariate probabilities. Clearly JPDM in conjunction
with the simplex search method yields the solution with the highest joint probability. The
JPDM/line search solution has a lower joint probability, but yields higher univariate
probabilities. This can be attributed to the fact that this solution produces a joint
probability distribution for F
1
and F
2
that allocates more probability density for higher
values of F
1
, as long as values for F
2
stay small, and vice versa. The same phenomenon
can be observed for the solution produced by the Overall Evaluation Criterion method.
The solution produced by the single objective optimization problem of maximizing the
probability of achieving F
1
function values larger than 5 marks the ideal solution for this
objective, which naturally cannot be achieved by any of the multi-criteria optimization
methods, due to the conflicting criteria. Nonetheless, its solution still yields a Probability
of Success value close to the OEC solution. This result cannot be generalized, which is
indicated by the single objective optimization problem of maximizing the probability of
achieving F
2
function values larger than 5 which yields a joint probability of zero.
Finally, the low value in joint and univariate probability from the solutions found with
the MaxiMin and Goal Attainment method is due to the fact that both methods try to
achieve the same values for the univariate probabilities P(F
1
5) and P(F
2
5). It must
be concluded that these methods clearly are not suitable for solving probabilistic multi-
criteria optimization problems.
Table 5.1: Comparison of Joint and Univariate Probabilities
Solution Point
(x
1
, x
2
)
Joint
P(F
1
5, F
2
5)
Univariate
P(F
1
5)
Univariate
P(F
2
5)
JPDM/Simplex Search (0, 1.5) 0.1794 0.4325 0.2449
JPDM/SQP/Line Search (0.0089, 1.0397) 0.1777 0.4658 0.2674
OEC (0.4608, 0.6606) 0.1565 0.5421 0.2678
MaxiMin (-0.178, 0.4234) 0.1101 0.2886 0.2882
Goal Attainment (-0.178, 0.4234) 0.1101 0.2886 0.2882
Single Obj. P(F
1
5) (0.7569, 0.7867) 0.1393 0.5552 0.2369
Single Obj. P(F
2
5)
(0.026, -0.0832)
true: (0,0)
0 0.0045 0.2949
Product Selection
While optimization is concerned with finding the best possible solution from a
very large pool of potential alternatives, product selection describes the process of
finding the best from a smaller pool of already realized alternatives. On the other hand,
product selection is often based on more criteria than optimization, as indicated in
Figure 5.2. A good example for the difference of product selection and optimization is
the difference between designing and buying a car. While the designer is looking for the
one best solution out of an almost infinite number of combinations of design parameter
values, the customer is looking for his best solution in a much smaller pool of alternatives
from different car manufacturers and models. For a more detailed description of the
differences between optimization and product selection see [Hwang, Masud, 1979],
[Hwang, Yoon, 1981], and [Sen, Yang, 1998].
As indicated in Chapter II, the main problem in multi-criteria product selection is
that of conflicting criteria, i.e. certain attribute values are approved upon with another
alternative only to the detriment of values for another attribute. The use of the Joint
Probabilistic Decision Making technique for product selection alleviates this problem by
transforming the multi-criteria problem into a single criterion problem: maximizing the
Probability of Success (POS). Each alternative in the product selection problem is
associated with a distinct joint probability distribution that yields a particular POS value
over the area of interest. The depiction in Figure 5.33 illustrates this concept by mapping
three joint probability distributions for three different alternatives over the area of
interest, which is spanned by two criteria. It is clear from the distributions overlap with
the area of interest, that Alternative #2 yields the highest POS, while Alternative #3
yields less and Alternative #1 the least amount of POS. So even though the criteria are
conflicting with each other, each alternative yields a unique value for POS that indicates
the usefulness to the decision-maker. It is this ability to rank the alternatives based on
their Probability of Success that is the true advantage of the Joint Probabilistic Decision
Making technique in product selection.
Criterion 1
C
r
i
t
e
r
i
o
n

2
Alternative 1
Alternative 3
Alternative 2
Area of
Interest
Figure 5.33: Comparing Joint Probability Distributions for Product Selection
To illustrate the use of the Joint Probabilistic Decision Making technique for
product selection, it is extended in Figure 5.34 to include the product selection process,
and an exit criterion. First the criteria are identified, based on which the best product is
selected. Second, probability distributions are assigned to the noise variables, i.e. the
variables that introduce the uncertainty into the product selection process. Third, control
variable values are established for each alternative. That is, each control variable can be
thought of as a vector, where each element in this vector corresponds to a different
(control variable) value for each alternative. Next, the joint probability distribution is
evaluated for each alternative through the use of one of the five schemes and the system
analysis tool. Finally, using the weights and criterion values that identify the area of
interest, the Probability of Success is calculated for each alternative.
Criteria (Z)
1
Criterion
Values
(z
min
, z
max
)
Analysis Tool
Z = f(x,Y)
6
Weights (w)
7
4
Alternatives
Uncontrollable/
Noise Variables
Y
Non-Deterministic
Control
Variables
x = (0.5, 4, 0.07, )
Deterministic
Joint Probability
Distribution
(Five Schemes)
POS
2
3
5
Satisfactory
POS ?
Change
z-Values?
Change
Weights?
Solution with
Highest POS
Y Y
Y N
N N
Product Selection Process
8
9 10 11
Figure 5.34: Joint Probabilistic Decision Making Technique for Product Selection
If at least one of the alternatives achieves a sufficient level of Probability of
Success in Element #9, the alternative with the highest value for POS is deemed the best
solution. If none of the alternatives reach levels of POS that seem reasonable, e.g. above
50%, the criterion values may be adjusted in Element #10 to yield higher values for POS.
On the same token, if several alternatives have a Probability of Success of one, the
criterion values should be adjusted to reduce the value of POS in order to be able to
distinguish the alternatives and determine which one is truly best. Note that by adjusting
the criterion values, i.e. changing the area of interest, the probabilistic analysis does not
need to be rerun, since the joint probability distributions for the different alternatives stay
constant. This sets the product selection process apart from the optimization loop, where
the joint probability distribution needs to be reevaluated for each optimization iteration,
while the criterion values stay constant. If changing the criterion values is not an option
or is not desired, and if the original set of weights does not reflect the criteria preference
structure of the decision-maker any longer, preference weights can be adjusted in
Element #11.
Change of Preferences
If the decision-maker decides to look at a different preference structure among the
criteria, new weight values need to be assigned to all criteria. These new weights yield a
change in target values, i.e. changing the area of interest. The modified area in return
yields a different POS value for each alternative and possibly a different best solution.
This process can be repeated until the decision-maker has a sufficient understanding of
the dependence of the best design alternatives on the distribution of weights.
Weight Adjusted Target Values
The technique suggested in this thesis accounts for preferences among criteria by
weight adjusted target values t that limit the area of interest:
( )
min min
z w t N , (5.31)
and
( ) N
w
z
t
max
max
(5.32)
with N being the number of criteria. This formulation essentially narrows the target
range of interest for the criteria with high preference weights and widens its range of
interest for the ones with low weights. The formulation for the Probability of Success in
Equations 5.26 and 5.27 consequently changes to:

M
j
j j j
t z t I
M
POS
1
max min
) (
1
(5.33)
for the Empirical Distribution Function (Equation 5.3), with M being the number of
samples, and
max
min
) (
t
t
s s d f POS (5.34)
for the Joint Probability Model (Equation 5.12). It is essential to note that the POS in this
case does not represent the numerical probability of achieving values within the area of
interest spanned by the criteria, but rather yields a measure of goodness for the design
alternatives, accounting for preferences among decision criteria. Hence, products fielded
based on this formulation promise to yield a higher customer satisfaction than products
that were fielded based on the aforementioned Probability of Success without criteria
preferencing.
15
This concept is also visualized in Figure 5.35, where one design solution
(Alternative 1) appears to be superior when no preferencing is applied, while the other
solution (Alternative 2) appears to be the better one when preferencing is applied.

15
Preferencing = Assigning weighting values to criteria in a decision making environment.
z
2
Area of
Interest
Alternative 1
Alternative 2
Without Preferencing
z
1
z
1min
z
1max
z
2min
z
2max
z
2
Area of
Interest
Alternative 1
Alternative 2
With Preferencing
z
1
t
1min
t
1max
t
2min
t
2max
Figure 5.35: Comparison of Alternatives Based on POS with or without Preferencing
Requirement Trade-Off
While analysis and drafting are important activities in design, no design process is
complete without a needs analysis that includes a gathering and investigation of customer
requirements. For complex products or markets, the designer is often faced with the
dilemma of multiple conflicting requirements. These conflicts have always been
explored, if not solved, through the use of requirement trade-off techniques.[Dieter,
1991] Siddall, for example, introduced an approach of Interaction Curves,[Siddall, 1982]
which represent the locus of all optimal designs in a Pareto sense, where each point on
the line is a trade-off point.[Siddall, 1982] Other names for this curve, such as Pareto-
Front or Pareto-Line, are also common in the literature.[Hwang, Masud, 1979], [Hwang,
Yoon, 1981], [Zeleny, 1982], [Osyczka, 1984], [Steuer, 1986], [Stadler, Dauer, 1992]
Whether through a formal approach or a mind experiment, designers have always used
the information from Interaction Curves, or the like, to answer such questions as: What
happens to Requirement B, if I was able to relax Requirement A a little? How much do
I have to relax Requirement A in order to meet my Requirement B? and so on. It is
these types of questions that are addressed through the use of the Joint Probabilistic
Decision Making technique for requirement trade-offs.
In principal, the technique computes the new Probability of Success for each
combination of requirement values, thereby identifying the impact of criterion target
value changes on the alternatives. To visualize this process, a plot with lines of constant
POS is created, similar to the ones in Figure 5.36 and 5.37. Each point (z
1
, z
2
) in this plot
represents the joint cumulative probability of achieving values smaller (or larger) than z
1
for the one, and smaller (or larger) than z
2
for the other requirement. If the area of
interest is not open ended, i.e. z
imin
-, or z
imax
, both limits have to be traded
separately. In general two requirement trade-off scenarios are possible. First, one or
more requirements can be relaxed in order to gain some Probability of Success. This
scenario is exemplified in Figure 5.36, where the requirement Z
2
> t
2min
is relaxed to
Z
2
> t
2
*
min
, thereby increasing the joint probability from 0.22 to 0.5. In the second
scenario, POS is kept constant, while one requirement is relaxed to allow for more
stringent values of another requirement. In other words, values for one requirement are
being traded in for values of another. This scenario is demonstrated in Figure 5.37,
relaxing the requirement Z
2
> t
2min
to Z
2
> t
2
*
min
in order to allow the requirement
Z
1
> t
1min
to be tightened to Z
1
> t
1
*
min
, while keeping the Probability of Success constant.
Increased
Probability

max 2
t

max 1
t
t
2 min
t
2
*
min
Lines of Constant
Cumulative Probability
z
1
z
2
0.01
0.1
0.2
0.8
0.9
t
1 min
0.99
r
e
l
a
x
0.5
0.22
Figure 5.36: Requirement Trade-Off to
Gain POS
Lines of Constant
Cumulative Probability
z
1
z
2
0.01
0.1
0.2
0.8
0.9
t
1 min
0.99
t
2 min
t
2
*
min
t
1
*
min
r
e
l
a
x
tighten

max 2
t

max 1
t
Figure 5.37: Trade-Off to Tighten
Requirement
To illustrate the use of the Joint Probabilistic Decision Making technique for
product selection, it is extended in Figure 5.38 to include the requirement trade-off
process, and an exit criterion. First the criteria are identified, which provide the
requirements to be traded. Second, probability distributions are assigned to the noise
variables, i.e. the variables that introduce the uncertainty to the systems design process.
Third, control variable values are established for each alternative, similar to the product
selection process. Next, the joint probability distribution is evaluated for each alternative
through the use of one of the five schemes and the system analysis tool. Finally, using
the weights and criterion values that identify the area of interest, the Probability of
Success is calculated for each alternative. If the requirements need to change in
Element #9, the criterion values are adjusted in Element #10 and the new Probability of
Success is computed. This process is repeated until the decision-maker has gained
sufficient insight into how the solution depends on changes in the requirements or until a
sufficient level of POS is reached for the best solution. Note that the probabilistic
analysis does not need to be rerun for the requirement trade-off process, since only the
area of interest changes while the joint probability distributions for the different
alternatives stay constant.
Criteria (Z)
1
Criterion
Values
(z
min
, z
max
)
Analysis Tool
Z = f(x,Y)
6
Weights (w)
7
4
Alternatives
Uncontrollable/
Noise Variables
Y
Non-Deterministic
Control
Variables
x = (0.5, 4, 0.07, )
Deterministic
Joint Probability
Distribution
(Five Schemes)
POS
2
3
5
Change in
Requirements?
Solution with
Highest POS
Y
N
Requirement
Trade-Off Process
Change
z-Values
8
9
10
Figure 5.38: Joint Probabilistic Decision Making Technique for Requirement Trade-Offs
To highlight the significance of the Joint Probabilistic Decision Making technique
to aerospace systems design, the following chapter employs several example applications
for optimization, product selection, and requirement trade-offs.
Oliver Bandte Chapter VI 122
CHAPTER VI
IMPLEMENTATION INTO SYSTEMS DESIGN
Feasibility Problem
The first application example demonstrates the use of the Joint Probabilistic
Decision Making technique for the system reliability or determination of feasible
space problem. This problem is not a decision making problem in the strictest sense,
however, it facilitates the design process, particularly the technology selection
process.[Mavris, Kirby, 1998],[Mavris, Kirby, Qiu, 1998] The aim is to determine as
early as possible in the design phases whether a particular design concept has a chance of
having a feasible design solution.
16
A feasible design is defined as a design that satisfies
all imposed constraints.[Reklaitis, Ravindran, Ragsdell, 1983] Typical constraints in
aircraft design are limitations on approach speed, landing and take-off field length, and
aircraft noise based on FAA regulations. The sketch in Figure 6.1 illustrates this notion
for a simple example with two design variables and two constraints. The whole rectangle
denotes the design space, i.e. all possible design variable setting combinations. The dark
lines mark the constraints as functions of the design variables for a particular constraint
value that needs to be satisfied. The white area in the middle denotes the feasible space.

16
The main difference between a design solution and design concept lies in the level of determination of
the design variables. It is a solution, if all variable settings are known. It is a concept, if only few settings
are determined, but the products major functionalities and components are known.[Dieter, 1991]
Constraint 2
Constraint 1
Design Variable 1
D
e
s
i
g
n

V
a
r
i
a
b
l
e

2
Figure 6.1: Feasible Space in a Design Space
The computation of a deterministically formulated feasibility requires an
exorbitant amount of work, which can be reduced through the use of probability theory.
Such a formulation assumes uniform distributions for the design variables, i.e. assigning
each possible variable value the same probability of occurrence. The feasibility problem
is thereby reduced to determining the probability of satisfying all constraints
concurrently. This probability is equivalent to the volume spanned over the feasible area
by the joint uniform distribution and is therefore a measure of feasibility. With the
previously described probabilistic process, and recognizing that the constraints are simply
random variables, since they are a function of the design variables, a joint probabilistic
approach can be used to determine system feasibility.
This technique is very useful in conceptual and preliminary design, where concept
feasibility needs to be evaluated quickly in order to determine whether a particular
concept should enter the next design phase. For example, if little feasibility is found, new
technologies may be introduced to the design concept in hope of increasing the feasible
space. Through repeated execution of this method, different technologies can be applied
to the system, while a growth or shrinkage of feasible space manifests their benefit. This
process has been formulated in [Mavris, DeLaurentis, 1998] as a five step approach to
aircraft design, suggesting the use of the Fast Probability Integration tool (FPI)
[Southwest Research Institute, 1999] for determining the feasibility. From the flow chart
in Figure 6.2 one can see that the determination of feasibility is the key step in this
process, evaluating whether an expensive investigation of new technologies is necessary.
Y N
N
P(feas)
<
small
Problem Definition
Identify objectives, constraints,
design variables (and associated
side constraints), analyses,
uncertainty models, and metrics
1
x
1
x
2
x
3
Determine System Feasibility
D
e
s
i
g
n

S
p
a
c
e

M
o
d
e
l
Constraint Fault Tree
C
1
C
2
C
3
C
4
AND
P(feas)
2
FPI(AIS) or Monte Carlo
Relax
Constraints?
Y
Examine Feasible Space
x
1
x
2
x
3
3
Constraint
Cumulative Distribution
Functions (CDFs)
C
1
P
C
2
P
C
3
P
D
e
s
i
g
n

S
p
a
c
e

M
o
d
e
l
FPI(AMV)
or
Monte Carlo
Relax Active
Constraints
?
Y
Technology Identification/Evaluation/Selection (TIES)
C
i
P
Old Tech.
New Tech.
Obtain New CDFs
Identify Technology Alternatives
Collect Technology Attributes
Form Metamodels for Attribute Metrics
through Modeling & Simulation
Incorporate Tech. Confidence Shape Fcns.
Probabilistic Analysis to obtain CDFs for the
Alternatives
4
5
Decision Making
MADM Techniques
Robust Design Simulation
Incorporate Uncertainty Models
Technology Selection
Resource Allocation
Robust Design Solution
Figure 6.2: Five Steps to Aircraft Design [Mavris, DeLaurentis, 1998]
The use of the Joint Probabilistic Decision Making technique for determining
feasibility replaces the AND in Step #2 of the flow chart in Figure 6.2, calculating the
Probability of Success rather than P(feas). As an additional benefit from the use of the
JPDM technique, Step #3 is automatically executed when calculating POS, i.e. it does not
need to reapply the Uniform-Distributions to find the Constraint Cumulative
Distributions, as required by FPI. A reformulated flow chart that includes the Joint
Probabilistic Decision Making technique can be found in Figure 6.3. The other
significant advantage of JPDM over FPI is the fact that FPIs system feasibility analysis
does not yield values smaller than 0.5, which can only be determined after all of the
computational effort has been expended.
Problem Definition
Problem Definition
1
Determine System Feasibility
x
1
x
2
x
3
D
e
s
i
g
n

S
p
a
c
e

M
o
d
e
l
System Analysis
C
1
= f(x
1
, x
2
, x
3
)
C
2
= f(x
1
, x
2
, x
3
)
C
3
= f(x
1
, x
2
, x
3
)
JPDM
P
C1
= P(C
1
< Target
1
)
P
C2
= P(C
2
< Target
2
)
P
C3
= P(C
3
< Target
3
)
P(feas) = P(C
1
< Target
1
C
2
< Target
2
C
3
< Target
3
)
2
Decision Making
JPDM
Decision Making
JPDM
5
3
P(feas)
<
small
P(feas)
<
small
Relax Active
Constraints ?
Relax Active
Constraints ?
Y Y N
Technology Selection
Resource Allocation
Robust Design Solution
N
Technology Identification/Evaluation/Selection (TIES)
Identify Technology Alternatives
Collect Technology Attributes
Form Metamodels for Attribute Metrics
through Modeling & Simulation
Incorporate Tech. Confidence Shape Fcns.
Prob. Analysis to Obtain CDFs for Alternatives
4
Obtain New CDFs
Old Tech.
New Tech.
C
i
P
Figure 6.3: Five Steps to Aircraft Design with JPDM
A particularly good example for the determination of feasibility is the supersonic
transport aircraft, depicted in Figure 6.4. This next generation aircraft has very little
chance of satisfying all imposed constraints with todays technology, due to its stringent
performance requirements. In other words, this concept at todays technology levels has
a very small feasible space, if any at all. The baseline aircraft has an area-ruled fuselage
(maximum diameter of 12 ft.), a double delta planform, and four nacelles below the wing,
housing mixed flow turbofan power plants. The values for some of the important design
parameters are given in Table 6.1. Due to the severe impact of the sonic boom on the
ground, it was decided that the aircraft is not allowed to fly supersonically over largely
populated areas. Since many cities are not located adjacent to a large body of water, a
split supersonic/subsonic mission is assumed to be required. Unfortunately, the
requirement of subsonic cruise penalizes the design, since the aerodynamic shape can not
be optimized for pure supersonic flight, yielding a compromised design with higher drag
during supersonic cruise. As a consequence, the aircraft needs to carry more fuel, which
in turn increases the gross weight. The length of the subsonic cruise segment is assumed
to be 15% of the design range, which equals 750 nm.
Figure 6.4: Supersonic Transport Concept
Table 6.1: Description of the Baseline
Parameter Baseline
Range 5000 nm
Payload 300 Passengers
Fuselage length 310 ft.
Span 77.5 ft.
Inboard Sweep 74 deg.
Outboard Sweep 45 deg.
Mach Number 2.4
Supersonic Cruise Altitude ~63,000 ft.
As demonstrated by the subsonic flight requirement over land, designing a
supersonic transport vehicle is a multidisciplinary and difficult task. Choosing a wing
planform shape, for example, is driven by the need for efficient performance at both sub-
and supersonic cruise conditions, a conflicting design objective in itself.[DeLaurentis,
Mavris, Schrage, 1996], [Sakata, Davis, 1977] Furthermore, the trades involved in
planform selection are complicated by different discipline considerations for
aerodynamics, structures, propulsion, etc., and the presence of design and performance
constraints at the system level which are directly related to the wing. The limit on
approach speed, for example, is the main driver for the wing loading. On the other hand,
fuel volume requirements impact the wing size and shape. Both become sizing criteria
that, treated as constraints, tend to increase the wing in size. On the other hand, increased
wing area yields higher induced and skin friction drag, thus increasing fuel consumption.
These examples are just a snap shot of design trades that drive the design process and
give rise to performance constraints, which are complemented by such Federal Aviation
Regulations as the takeoff and landing field length limitations (less than 10,500 ft).
Since the feasibility study is just one application for the joint probability
formulation introduced in this thesis, only a simple example for two constraints, approach
speed and take-off field length, is executed here. Specifically, noise constraints are not
considered, since they cannot be satisfied with the simulated, current technology. The
design variables used in this feasibility study are listed in Table 6.2, representing some of
the key drivers in aircraft design.[Mavris, Bandte, Schrage, 1996] They have been
selected from an elaborate screening process that identified the most significant
variables.[DeLaurentis, Mavris, Schrage, 1996] An illustration of the kink location in a
notional wing planform can be found in Figure 6.5. Finally, as indicated in Figure 6.3,
uniform distributions are assigned to all variables based on their ranges in Table 6.2.
Table 6.2: Design Variable Description and Range
Variable Name Range
Thrust to Weight Ratio TWR 0.28 - 0.32
Wing Area WingArea 8.5 - 9.5 * 10
3
ft
2
Longitudinal Kink Location x1 154% - 162% Semispan
Spanwise Kink Location y1 50% - 58% Semispan
Turbine Inlet Temperature TIT 3 - 3.25 * 10
3
F
Fan Pressure Ratio FPR 3.5 - 4.5
x1
y1
Figure 6.5: Illustration
of the Kink Location
To determine the feasibility, the eight step Joint Probabilistic Decision Making
technique is executed as outlined in Figure 6.6. The first step identifies the criteria: the
two constraints take-off field length (TOFL) and approach speed (Vapp). In Step #2,
uniform distributions are assigned to the variables over their respective ranges, identified
in Table 6.2. In the third step then, values are assigned to the remaining (control)
variables that describe the concept. The system analysis tool is identified in Step #4 to be
the aircraft synthesis/sizing code, Flight Optimization System (FLOPS),[McCullers,
1994] evaluating TOFL and Vapp based on the noise and control variables. To determine
the joint probability distribution, Scheme #I was chosen in Step #5, generating 10,000
Monte-Carlo simulation samples with the system analysis tool for the Empirical
Distribution Function, Equation 5.4, of TOFL and Vapp. The samples are collected in
pairs and are thus distributed jointly. In Step #6, minimum and maximum values for the
criteria are established that limit the area of interest: TOFL
min
= 0 ft (physical limit),
TOFL
max
= 10,500 ft (FAA Regulation), Vapp
min
= 0 kts (physical limit), and
Vapp
max
= 154 kts (FAA Regulation). Finally, Step #8 combines the area of interest with
the joint probability distribution and determines the Probability of Success, i.e.
feasibility. Note that Step #7, assigning weights to the criteria, has been omitted here,
since it does not seem sensible to have different preferences for the different constraints.
Explicitly, the feasibility is calculated to be:
. 0107 . 0 ) 154 , 500 , 10 (
) 154 , 500 , 10 ( ) 154 , 500 , 10 (
000 , 10
1
000 , 10
1

kts a ft a I
kts ft F kts VAPP ft TOFL P
iVAPP
i
iTOFL
Criteria
TOFL and Vapp
1
Analysis Tool
FLOPS
Criterion Values
TOFL
min
= 0 ft
Vapp
min
= 0 kts
TOFL
max
= 10500 ft
Vapp
max
= 154 kts
6
4
Baseline
Uncertainty
(Design) Variables
Table 6.2
Non-Deterministic
Control
Variables
Concept/Table 6.1
Deterministic
Joint Probability
Distribution:
Schemes #I
POS =
Feasibility
2
3
5
8
Figure 6.6: Determination of Feasibility with JPDM
In other words, the chance of finding a feasible design for the Supersonic
Transport within the specified design space, using current technologies, is 1.07%. Only
the infusion of new technologies can increase the designs feasibility. For a visual
representation of the EDF, its graph is depicted in Figures 6.7 and 6.8 in two and three
dimensions respectively. The graph in Figure 6.7 also displays the area of interest, i.e.
area of feasible solutions, limited by the values for TOFL
max
and Vapp
max
.
0.85 0.9 0.95 1 1.05 1.1 1.15 1.2
x 10
4
155
160
165
170
175
180
TOFL
V
a
p
p
Figure 6.7: Joint EDF for Take-Off
Field Length and Approach Speed (2D)
8000
8600
9200
9800
10400
11000
11600
12200
Figure 6.8: Joint EDF for Take-Off
Field Length and Approach Speed (3D)
Since the display of the EDF in Figure 6.7 is not particularly aesthetic,
Scheme #II is employed to produce graphs for the Joint Probability Model, displayed in
Figures 6.9 and 6.10 for two and three dimensions respectively. The regression of the
data yields a Normal-Distribution with N(9691.3, 510.55) for the take-off field length and
a Normal-Distribution with N(160.3, 3.0) for approach speed. The correlation coefficient
is estimated to be 0.7311.
0.9 1 1.1 1.2
x 10
4
155
160
165
170
175
180
185
TOFL
V
a
p
p
Figure 6.9: Joint Normal Distribution for
Take-Off Field Length and Approach
Speed (2D)
Figure 6.10: Joint Normal Distribution for
Take-Off Field Length and Approach
Speed (3D)
Area of
Interest
Area of
Interest
(
k
t
s
)
(ft)
(ft) (kts)
(ft)
(kts)
(
k
t
s
)
(ft)
Optimization of a Supersonic Commercial Transport
In order to demonstrate the use of the Joint Probabilistic Decision Making
technique for optimization in aerospace systems design, the following problem is posed:
An aircraft manufacturer plans to build a new supersonic passenger transport aircraft that
is supposed to be operated by all major airlines around the world. Initial studies indicated
that the aircraft should fly at a cruise speed of Mach 2.4, has an area-ruled fuselage
(maximum diameter of 12 ft.), a double delta planform, and four nacelles below the wing,
housing mixed flow turbofan power plants, similar to the concept shown in Figure 6.11.
However, the final design of the aircraft is not determined and subject to an optimization
problem, maximizing the chance for the manufacturer and the airline to make a profit.
Figure 6.11: Notional Supersonic Transport
Specifically, the optimal values for the number of total passenger capacity,
number of first class passengers, maximum (design) range, thrust-to-weight ratio, and
wing area are to be determined. They need to maximize the probability of the Supersonic
Transport achieving return on investment values larger than 10% for the airline and 12%
for the manufacturer. However, in order to limit the scope of the design problem, limits,
outlined in Table 6.3, are imposed on these design variables.
Table 6.3: Range of Allowable Design Variable Values for Optimization
Thrust-Weight Wing Area Max. Range
# of First Class
Passengers
Total # of
Passengers
Minimum 0.28 8500 ft
2
5000 nm 6 250
Maximum 0.32 9500 ft
2
6500 nm 60 350
Special attention needs to be directed to the anticipated mission the supersonic
transport is intended to fly, i.e. cities it can serve. Due to the severe impact of the sonic
boom on the ground, it was the decided that the aircraft is not allowed to fly
supersonically over largely populated areas. Since many cities are not located adjacent to
a large body of water, a split supersonic/subsonic mission is assumed to be required.
Unfortunately, the requirement of subsonic cruise penalizes the design, since the
aerodynamic shape cannot be optimized for pure supersonic flight, yielding a
compromised design with higher drag during cruise. As a consequence, the aircraft needs
to carry more fuel, which in turn increases the gross weight. On the other hand, to
require a section of subsonic cruise for service to cities that are 6500 nm apart would
penalize the design too much, since the only cities that far apart and serviced by a direct
flight, are Los Angeles and Sydney, both on the Pacific Ocean.[Oxford, 1999] Hence, it
is assumed that an aircraft designed for a maximum range of 6500 nm with no subsonic
cruise has enough room for fuel to fly a mission of 5000 nm which includes up to
1500 nm of subsonic cruise. A linear relationship is developed and employed for the
sizing routine to calculate a feasible subsonic cruise length, given a particular maximum
range. This relationship, Equation 6.1, corresponds particularly well to actual distances
of major world cities and respective lengths of flight over land.[Oxford, 1999]
Subsonic Cruise Distance = 6500 nm Maximum Range (6.1)
Important to note is that the aircraft has to satisfy certain physical and operational
requirements as well as constraints imposed by the Federal Aviation Agency. Namely,
the landing speed has to be less than 154 knots, landing and take-off field lengths need to
be less than 10,500 ft, and the take-off gross weight has to be less than 1 million pounds
in order for the aircraft to land on existing runways.
Since the objective of this optimization problem is to maximize the probability of
achieving return on investments larger than 12% and 10% for the manufacturer and
airline concurrently, special attention needs to be paid to its economic analysis.
Unfortunately, the manufacturer has little to no control over the key economic parameters
and optimizing them is impossible. The approach taken here, as outlined in Chapters III
and V, is to assign probability distributions to the key economic parameters of interest,
which are summarized in Table 6.4. They have been selected from an elaborate
screening process that identified the most significant economic variables.[Mavris,
Bandte, Schrage, 1995]
The following economic parameters affect the airline only: Load Factor, which is
the ratio of the equivalent full fare booked seats to the number of available seats; Average
Yield, which represents the airlines average yield per revenue passenger mile; Fuel
Price; Economic Range, which is the average distance between city pairs the aircraft is
scheduled to connect; Utilization; and %Subsonic Cruise, which is the average
percentage of the Economic Range flown in subsonic cruise. %Subsonic Cruise itself
depends on the Economic Range and decreases with increasing Economic Range values
linearly. For example, if the Economic Range is 6500 nm, %Subsonic Cruise is zero; if
the Economic Range is 3000 nm, %Subsonic Cruise is 50%, based on the argument made
earlier. The linear relationship is again in correspondence with actual distances of major
world cities and respective lengths of flight over land.[Oxford, 1999]
Table 6.4: Economic Parameter Distributions
Economic Parameter Range Unit
Distribution
Type
Parameters
Load Factor 65 - 85 % Beta = 3, = 3
Average Yield (Coach)
Average Yield (First)
0.09 0.15
0.13 0.19
$ Beta = 5, = 3
Fuel Price 0.6 - 0.9 $/gal Beta = 3, = 5
Economic Range 3000 max Range nm Beta = 2, = 4
Utilization 4500 - 5500 hrs/yr Beta = 3, = 3
% Subsonic Cruise 0 - 50 % Beta = 3, = 3
Inflation 1 9 % Beta = 3, = 5
Aircraft Price
250 - 350
300 - 400
350 - 450
M$ Beta = 3, = 3
Learning Curve 0.75 - 0.85 Beta = 3, = 3
Production Quantity 400 - 800 Beta = 3, = 3
Engineering Labor Rate 50 - 100 $/hr Beta = 3, = 5
Tooling Labor Rate 40 - 90 $/hr Beta = 3, = 5
Years Until Production 3 - 5 yrs Discrete
P(3yrs) = 1/3
P(4yrs) = 1/3
P(5yrs) = 1/3
The manufacturer, on the other hand, is affected by the following variables:
Learning Curve, which represents the reduction in unit production cost with increasing
numbers of aircraft produced; Production Quantity; Engineering Labor Rate; Tooling
Labor Rate, which represents the labor rate for all production work; and Years Until
Production, which entails most of the detailed engineering work. Inflation and Aircraft
Price affect the airline as well as the manufacturer. In particular, Aircraft Price is
assumed to be dominated by the market, therefore uncertain, but correlated with the
Maximum Range and Total Number of Passengers. Hence, aircraft with a Maximum
Range of less than 5750 nm and less than 300 passengers are assumed in this example to
reach price levels between $200 and $300 million, aircraft with a Maximum Range of
less than 5750 nm or less than 300 passengers are assumed to reach price levels between
$250 and $350 million, and aircraft with a Maximum Range larger than 5750 nm and
more than 300 passengers are assumed to reach price levels between $300 and $400
million.
To determine the probability of satisfying both the manufacturer and the airline in
their demands for return on investment, the ten step Joint Probability Decision Making
technique for optimization is executed as outlined in Figure 6.12. The first step identifies
the criteria to be the two return on investments for the airline (ROIA) and manufacturer
(ROIM). In Step #2, the distributions are assigned to the variables over their respective
ranges identified in Table 6.4. In the third step then, values are assigned to the remaining
(control) variables that describe the concept. The system analysis tool, identified in
Step #4, is a combination of the aircraft synthesis/sizing code FLOPS [McCullers, 1994]
and the Aircraft Life Cycle Cost Analysis program ALCCA [Galloway, Mavris, 1993].
To determine the joint probability distribution, Scheme #I was chosen in Step #5,
generating 1,000 samples with the system analysis tool for the Empirical Distribution
Function of ROIM and ROIA, based on a Monte-Carlo simulation. The samples are
collected in pairs and are thus distributed jointly.
Analysis Tool
FLOPS/ALCCA
Weights
w
ROIM
= 0.5
w
ROIA
= 0.5
7
4
Baseline
Uncontrollable/
Noise Variables
Table 6.4
Non-Deterministic
Control
Variables
Table 6.3
Deterministic
Joint Probability
Distribution
Scheme #I
POS
2
3
5
Update
Deterministic
Variables in
Optimization
Scheme
Satisfactory
POS ?
Solution with
Highest POS
Y
N
Criteria
ROIM and ROIA
1
9
Optimization Loop
8
10
Criterion Values
ROIM
min
= 12%
ROIM
max
=
ROIA
min
= 10%
ROIA
max
=
6
Figure 6.12: JPDM as Optimization Process
In Step #6, minimum and maximum values for the criteria are established that
limit the area of interest: ROIM
min
= 12 %, ROIM
max
= , ROIA
min
= 10 %, and
ROIA
max
= . In Step #7 equal weights of 1/N = 0.5 are assigned to both criteria, since
no preference of one criterion over the other is identified for the optimization. Finally,
Step #8 combines the area of interest with the joint probability distribution and
determines the Probability of Success, i.e. the chance of satisfying both return on
investment requirements concurrently. As long as the achieved POS can still be
improved upon, the optimization loop continues to update the deterministic design
variables. The optimization routine used for this research is the line search technique for
constrained optimization, which is part of the optimization package in
MATLAB
.[Branch, Grace, 1996] After 71 iterations, the final setting for the control
variables is determined to be Thrust-to-Weight ratio = 0.312021, Wing Area = 8576 ft
2
,
Maximum Range = 5000 nm, Number of First Class Passengers = 14, and Total Number
of Passengers = 350, yielding a Probability of Success of 0.782. The iteration history is
displayed in Figure 6.13. The constraint on approach speed (V
app
) is the most stringent
for this example and has been included in the optimization history of Figure 6.13. Note
that the optimizer first tries to find an area of large objective function values and then
reduces the values for V
app
.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1 11 21 31 41 51 61 71
Iteration
P
O
S
148
150
152
154
156
158
160
162
V
a
p
p

(
k
t
s
)
POS
Vapp
Constraint
Figure 6.13: Optimization Iteration History of POS and Vapp
On the other hand when using an Overall Evaluation Criterion as an objective
function, %) 10 ( %) 12 ( OEC
2
1
2
1
+ ROIA P ROIM P , the best solution is found at
Thrust-to-Weight Ratio = 0.310435, Wing Area = 8676 ft
2
, Maximum Range = 5000 nm,
Number of First Class Passengers = 59, and Total Number of Passengers = 346, yielding
an OEC value of 0.8845. The POS value of 0.773 for this OEC solution is smaller than
the previous solution based on the Probability of Success. Also, finding the OEC
solution took 109 iterations, as indicated in the optimization history of Figure 6.14, while
the POS solution needed only 71. In conclusion, the OEC is a less adequate objective
function for this aerospace systems optimization problem than POS.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106
Iteration
P
O
S

O
E
C
148
153
158
163
168
V
a
p
p

(
k
t
s
)
OEC
POS
Vapp
Constraint
Figure 6.14: Optimization Iteration History of OEC, POS, and Vapp
Comments and Recommendations
While the computer code FLOPS constitutes the best publicly available routine
for synthesis and sizing of aircraft, it imposes a limitation on the sizing of the fuselage
that should be commented on. Fuselage diameter and length are as much inputs to
FLOPS as the number of passengers, i.e. all three are modeled as independent variables.
This is clearly not true for sizing of a real aircraft: fuselage length or diameter will
increase with the number of passengers. Hence, for the demonstrated optimization
exercise the fuselage length and diameter was kept constant, thereby underpredicting the
fuselage skin friction and wave drag of the larger vehicles. As a consequence, their
operating cost is a little higher than predicted by ALCCA, producing a lower return on
investment for the airline. However, this overprediction of ROIA affects POS as much as
the OEC and the conclusion remains the same: POS is the more adequate objective
function for aerospace systems design, yielding a higher Probability of Success in less
iterations for this example.
A last comment should be made about the line search optimization routine used
for this exercise. While this routine is the simplest technique for optimization, it
highlights the benefits of the use of POS over the OEC by requiring fewer iterations with
POS as an objective function. By using more sophisticated optimization routines the total
number of iterations for both objective functions should be decreased, with the potential
for a less pronounced advantage of POS over the OEC in terms of iterations. However,
there is no evidence that a different optimization routine will enable the OEC to find a
better solution. On the contrary, the better the optimization routine, the better the
chances that it finds the true optimum in Probability of Success with POS as an objective
function rather than the OEC. This is due to the fact that customer satisfaction is
described by a joint probability (POS) rather than a summation of independent univariate
probabilities (OEC). However, an investigation into different optimization routines has
not been part of this research and is left for future studies.
Product Selection
The third application example of the Joint Probabilistic Decision Making
technique is a product selection problem: A large international airline intends to extend
their current fleet by purchasing a new system. The new aircraft are supposed to be used
on long haul routes with high passenger demand. The systems considered are the Boeing
B-747-400, Boeing B-777-200, Airbus A340-300, and a notional Supersonic Transport.
The Supersonic Transport seems exotic next to the three subsonic aircraft, but is of
particular interest to an airline. Due to its increased flight speed, the Supersonic
Transport is able to sustain travel times cut down to half of what a subsonic aircraft is
able to offer. This makes the Supersonic Transport, and consequently the airline that
operates it, very attractive for business travelers and people with very short vacation
periods. Both groups are the airlines best paying customers, having sustained their
business for years when air travel was down significantly. It is therefore anticipated, that
a Supersonic Transport would be received very well by the airlines customers. A visual
representation of each system can be found in Figures 6.15 to 6.18 and the characteristics
for each system are listed in Table 6.5, where Fuel Weight measures the amount of fuel
used when traveling the full range at full capacity.
Figure 6.15: B-747 [Boeing, 2000] Figure 6.16: B-777 [Boeing, 2000]
Figure 6.17: A340 [Airbus, 2000] Figure 6.18: Supersonic Transport
Table 6.5: Four Alternatives for Product Selection Problem
Characteristic Boeing B-747 Boeing B-777 Airbus A340
Supersonic
Transport
Aircraft Price M$150 - 200 M$110 - 160 M$100 - 150 M$200 - 300
Total # of Seats 412 375 330 342
# First Class Seats 82 20 32 60
Range 7500 nm 4600 nm 6600 nm 5000 nm
Cruise Speed Mach 0.85 Mach 0.84 Mach 0.8 Mach 2.4
Gross Weight 868013 lbs 535000 lbs 558900 lbs 882807 lbs
Fuel Weight 343710 lbs 156735 lbs 224733 lbs 532946 lbs
# of Engines 4 2 4 4
Following the ten-step process, outlined in Figure 6.19, the criteria based on
which the aircraft system is selected are identified first. For this example, return on
investment (ROI), total operating cost in million dollars per year (TOC) averaged over the
economic life of the aircraft (20 years), and annual revenue in million dollars (REV) are
selected as the three criteria. As a second step, probability distributions are assigned to
the uncontrollable variables, which have been selected from an elaborate screening
process that identified the most significant economic variables.[Mavris, Bandte, Schrage,
1995] These variables are listed in Table 6.6 with their respective distributions that
represent the likelihood of those values occurring during the 20 years of operation.
Criteria
ROI, REV, and TOC
1
Criterion Values
ROI
min
= 15%
ROI
max
=
REV
min
= M$100
REV
max
=
TOC
min
= M$0
TOC
max
= M$90
Analysis Tool
ALCCA
6 Weights
w
ROI
= 1/3
w
REV
= 1/3
w
TOC
= 1/3
7
4
Alternatives
Uncontrollable/
Noise Variables
Table 6.6
Non-Deterministic
Control
Variables
Table 6.5
Deterministic
Joint Probability
Distribution
Scheme #I
POS
2
3
5
Satisfactory
POS ?
Change
z-Values?
Change
Weights?
Solution with
Highest POS
Y
Y
Y
N
N N
Product Selection Process
8
9 10 11
Figure 6.19: JPDM for Product Selection
The Load Factor is the ratio of the equivalent full fare booked seats to the number
of available seats. Due to the decreased travel time, the airline is assumed to be able to
charge a three cent premium for the notional Supersonic Transport in the coach and first
class sections. However, Table 6.6 only lists the Average Yield, representing the
airlines average yield per revenue passenger mile, for the subsonic aircraft. The
Economic Range depicts the average distance between city pairs the aircraft is scheduled
to connect, while the Inflation is the average anticipated inflation rate over the economic
life. The % Subsonic Cruise, which is the average percentage of the Economic Range
flown in subsonic cruise, is obviously only used for the Supersonic Transport. It depends
on the Economic Range and decreases for increasing Economic Range values linearly.
For example, if the Economic Range is 6500 nm, % Subsonic Cruise is zero; if the
Economic Range is 3000 nm, % Subsonic Cruise is 50%. The linear relationship is again
in correspondence with actual distances of major world cities and respective lengths of
flight over land.[Oxford, 1999] The price of the aircraft system obviously varies from
alternative to alternative (see Table 6.5), but the same type of distribution is used in the
analysis of all competing systems. Passenger and Baggage Weight represent average
values for the all passengers and bags.
Table 6.6: Noise Variable Distributions
Noise Variable Range Unit
Distribution
Type
Parameters
Load Factor 65 - 85 % Beta = 3, = 3
Average Yield (Coach)
Average Yield (First)
0.06 0.12
0.10 0.16
$ Beta = 5, = 3
Fuel Price 0.6 - 0.9 $/gal Beta = 3, = 5
Economic Range 3000 max Range nm Beta = 2, = 4
Utilization 4500 - 5500 hrs/yr Beta = 3, = 3
% Subsonic Cruise 0 - 50 % Beta = 3, = 3
Inflation 1 9 % Beta = 3, = 5
Aircraft Price See Table 6.5 M$ Beta = 3, = 3
Passenger Weight 150 - 210 lbs Normal = 180, = 10
Baggage Weight 29 - 59 lbs Normal = 44, = 5
The third step is comprised of identifying and assigning values to the control
variables, i.e. parameters that distinguish the different alternatives from each other.
Table 6.5 provides these values to a large extent. More detailed information is
incorporated in the system description/input files to the system analysis tool, identified in
Step #4 as the Aircraft Life Cycle Cost Analysis (ALCCA) [Galloway, Mavris, 1993].
Step #5 generates a joint probability distribution for each system, using Scheme #I.
10,000 sample points are generated for each aircraft by the system analysis tool through a
Monte-Carlo simulation. All four distributions are superposed in a two-dimensional joint
probability plot displayed for ROI and TOC in Figure 6.20 and for REV and TOC in
Figure 6.21. Since the Supersonic Transport has a significantly larger operating cost and
revenue than the other systems, and consequently skewing the scale, magnifications of
the joint probability plots for the B-747, B-777, and A340 are provided in Figures 6.22
and 6.23.
Figure 6.20: Joint Probability Plot for
ROI and TOC
REV and TOC
Area of
Interest
B-747
B-777
A340
Supersonic Transport
Area of
Interest
B-747
B-777
A340
Supersonic
Transport
(M$/yr)
(
M
$
/
y
r
)
(M$/yr)
Figure 6.22: Magnified Joint Probability
Plot for ROI and TOC
Figure 6.23: Magnified Joint Probability
Plot for REV and TOC
The areas of interest indicated in the joint probability plots of Figures 6.20 to 6.23
are identified in Step #6, defined by maximum and minimum values for each criterion
that need to be satisfied. The return on investment should be as large as possible, i.e.
ROI
max
= , but not smaller than 15% (ROI
min
). The revenue is also desired to be as large
as possible (REV
max
= ), but should not be smaller than $100 million per year (REV
min
).
The total operating cost, on the other hand, is supposed to be as small as possible, i.e.
TOC
min
= $0, and should not be larger than $90 million per year (TOC
max
). No
preferences are identified in Step #7 for the initial estimation of the Probability of
Success, which is finally calculated for each alternative in Step #8. The results for each
aircraft are summarized in Table 6.7, including the univariate probability of satisfying
just one criterion (disregarding the other criteria).
Area of
Interest
B-747
B-777
A340
(
M
$
/
y
r
)
(M$/yr)
Area of
Interest
B-747
B-777
A340
(M$/yr)
Table 6.7: Summary of POS for each Alternative
Alternatives POS P(ROI > 15%) P(REV > M$100) P(TOC < M$90)
Boeing B-747 0.1379 0.7091 0.1391 0.9112
Boeing B-777 0.003 0.8363 0.003 1
Airbus A340 0 0.6960 0 1
SuperS. Transport 0 0.9097 1 0
Based on these results, the Boeing B-747 clearly yields the highest probability of
satisfying all criteria concurrently and must hence be considered the system of choice for
the airline. Had the decision been based on the return on investment or revenue alone,
the Supersonic Transport would be the best system. Had the decision been based on the
operating cost alone, the Airbus A340 would be the system of choice (see Figures 6.22
and 6.23 for discrimination between B-777 and A340). But since the decision is based on
all three criteria concurrently, the univariate probability values yield conflicting results:
choose the Supersonic Transport or the Airbus A340. Note that the Boeing B-747 does
not even appear as a viable alternative when the univariate probability values are used for
product selection, since there is always an alternative with a higher probabilities.
Furthermore, comparing this result to the result an Overall Evaluation Criterion
yields, the B-747 again does not appear to be the best alternative. As a matter of fact, by
using the common formulation with equal preference weights for the OEC,
) 90 $ M ( P ) 100 $ M ( P %) 15 ( P OEC
3
1
3
1
3
1
< + > + > TOC REV ROI , (6.2)
the Supersonic Transport (OEC = 0.6366) turns out to be the best solution, followed by
the B-777 (OEC = 0.6131). The B-747 only comes in third with OEC = 0.5865, while
the A340 is with OEC = 0.5653 supposedly the worst solution for the airline.
Recognizing that the POS models the decision process of the airline more accurately
leads, however, to the conclusion that the OEC is not the best technique for a product
selection problem.
For resolving conflicts in product selection problems, like the one described
before, the Technique for Order Preference by Similarity to the Ideal Solution (TOPSIS)
[Hwang, 1981] has been indicated to be a valuable decision making tool (see discussion
in Chapter II). In order to test the viability of TOPSIS in this product selection problem,
its result, using the univariate probability values from Table 6.7 is compared to the one
derived with the Joint Probabilistic Decision Making technique. First, TOPSIS requires a
normalization of the elements in the decision matrix:
,
`
.
|
0 1 9097 . 0
1 0 696 . 0
1 003 . 0 8363 . 0
9112 . 0 1391 . 0 7091 . 0
D
and
,
`
.
|

0 9905 . 0 5737 . 0
5944 . 0 0 4389 . 0
5944 . 0 0030 . 0 5274 . 0
5416 . 0 1378 . 0 4472 . 0
4
1
2
R
x
x
r
j
ji
ji
ji
. (6.3)
Next the Euclidean Distance to the ideal solution r
i
*
, i.e. maximum value for each
criterion, and negative ideal solution r
i
-
, i.e. minimum value for each criterion, is
calculated for each system. No preferences have been identified for this product selection
problem and all weights are equal to 1/3.
1981 . 0
3 3
3332 . 0
3 3
3295 . 0
3 3
2879 . 0
3 3
3
1
2
*
*
3
1
2
*
340
* 340
3
1
2
*
777
* 777
3
1
2
*
747
* 747
,
`
.
|

,
`
.
|

,
`
.
|

,
`
.
|

i
i SSTi
SST
i
i i
i
i i
i
i i
r r
S
r r
S
r r
S
r r
S
3332 . 0
3 3
1981 . 0
3 3
2003 . 0
3 3
1863 . 0
3 3
3
1
2
3
1
2
340
340
3
1
2
777
777
3
1
2
747
747
,
`
.
|

,
`
.
|

,
`
.
|

,
`
.
|

i
i SSTi
SST
i
i i
i
i i
i
i i
r r
S
r r
S
r r
S
r r
S
(6.4)
Using the two Euclidean Distances, the Closeness Criterion is evaluated for each
system:
j j
j
j
S S
S
C
*
C
747
= 0.3929, C
777
= 0.3781, C
340
= 0.3729, C
SST
= 0.6271.
Based on TOPSIS, the Supersonic Transport is clearly the best system and should
be purchased by the airline. However, this solution is completely ignoring the fact that
the Supersonic Transport has no chance of ever meeting the operating cost requirement.
The JPDM technique had determined a POS of 0 for precisely this reason. Hence, it must
be concluded that TOPSIS is not a suitable technique for this product selection problem.
Its use and a decision based on it would result in high operating costs for the airline,
which its business structure may not be able to bare.
On the other hand, the airline may be attracted to the Supersonic Transport by its
high revenue and return on investment values and willing to make the higher investment
in operating cost. Then again, a smaller airline may not be able to afford such high
operating costs and will seek a system with lower cost and consequently lower revenues,
such as the Airbus A340. Without oversimplifying the problem, it can be assumed that
any airline would seek a system that yields a higher annual revenue than operating cost.
This behavior is signified by the new area of interest displayed in the joint probability
plot presented in Figure 6.24. The system that demonstrates the largest overlap of its
joint distribution with this area of interest yields the highest new Probability of Success
and should be purchased by the airline. This type of area of interest cannot be
accommodated by the current version of the Joint Probabilistic Decision Making
technique and is suggested as an area of future work at this point.
Figure 6.24: Joint Probability Plot for REV and TOC with New Area of Interest
While the Boeing B-747 turned out to be the best system for an airline which
requires the specified return on investment, revenue, and operating cost values, the actual
POS of 0.1124 is relatively small. The investigation into increasing this joint Probability
of Success is called a Requirement Trade-Off analysis and is demonstrated in the next
section.
A340
New Area
Of Interest
B-777
B-747
(
M
$
/
y
r
)
(M$/yr)
Requirements Trade-Off Analysis and Discussion
The main function of a requirement analysis is to produce insight into the decision
making problem by investigating the impact of requirements on the solution to the
decision making problem. As outlined in the flowchart in Figure 6.19, once the POS has
been determined for each system, requirement/criterion values may have to change, if the
POS values turned out to be too small.
17
In order to increase the joint probability,
requirements have to be relaxed, i.e. made less stringent for the system to reach values of
interest. Relaxing the return on investment, however, has no effect on POS for any
system, as demonstrated in Table 6.8 with relaxed ROI
min
values of 10% and 5%. Only
the univariate probability of satisfying ROI increases, when compared to the values for
ROI > 15%. This can be attributed to the fact that the requirements for the revenue and
operating cost are far more stringent than the requirement for the return on investment.
Table 6.8: Comparison of POS for Different ROI
min
Values
Boeing
B-747
Boeing
B-777
Airbus
A340
Supersonic
Transport
POS(ROI > 15%, REV > M$100, TOC < M$90) 0.1124 0.003 0 0
POS(ROI > 10%, REV > M$100, TOC < M$90) 0.1124 0.003 0 0
POS(ROI > 5%, REV > M$100, TOC < M$90) 0.1124 0.003 0 0
P(ROI > 15%) 0.7091 0.8363 0.6960 0.9097
P(ROI > 10%) 0.8763 0.9323 0.8591 0.9693
P(ROI > 5%) 0.9604 0.9796 0.9461 0.9932

17
Note that technology infusion or any measure that changes the system is impossible in a product
selection scenario, since the systems are fixed in their control variable values and noise variable
distributions.
Relaxing the revenue requirement has a much larger impact on the solution to the
product selection problem. Moreover, when relaxing the revenue requirement, the
operating cost requirement has to be tightened, since it is not practical for the airline to
consider systems that have a high chance of generating little revenue but high operating
costs. At the very least, the revenue should be equal or higher than the operating cost,
similar to the joint probability plot of Figure 6.24. Leaving ROI
min
at 15%, the graph in
Figure 6.25 examines how POS values change for the four different systems when
REV
min
and TOC
max
are changing concurrently.
0
0.1
0.2
0.3
0.4
0.5
0.6
50 100 150 200 250 300
REVmin (M$/yr) = TOCmax (M$/yr)
P
O
S
SST
B-747
B-777
A340
Figure 6.25: Comparison of POS as a Function of Revenue and Cost Requirements
For airlines that can only afford a small average total operating cost, $68 million
per year or less, the Airbus A340 is clearly the best solution. For airlines with a slightly
larger pocket book, the Boeing B-777 becomes the best alternative. Note that the B-777
also has the highest absolute joint probability value of all four systems. If an even larger
aircraft is needed, i.e. Boeing B-747, the airline will have to sustain an even higher
operating cost, but is also guarantied a larger revenue to cover the cost. Finally, the
graph in Figure 6.25 denotes that the Supersonic Transport will only be a viable option
for very large airlines that can absorb operating costs of around $200 million per year and
aircraft.
For illustration purposes, Figures 6.26 through 6.29 display the joint probability
distributions for the four systems, in two dimensions (REV and TOC), at their respective
best requirement settings, identified in Figure 6.25. The Airbus A340 is the best solution
with a maximum Probability of Success of 0.4068 at REV
min
= TOC
max
= $66 million.
The Boeing B-777 is the system of choice with a maximum Probability of Success of
0.5713 at REV
min
= TOC
max
= $73 million, while the Boeing B-747 is the best system
with a maximum Probability of Success of 0.4363 at REV
min
= TOC
max
= $89 million.
Finally the Supersonic Transport is the best choice for the airline with a maximum
Probability of Success of 0.4492, if it can sustain operating costs of $202 million.
50 60 70 80 90 100
40
50
60
70
80
90
100
110
120
130
TOC
R
E
V
REV and TOC (A340 is best)
REV and TOC (B-777 is best)
Area of
Interest
B-747
B-777
A340
Area of
Interest
B-747
B-777
A340
(
M
$
/
y
r
)
(M$/yr)
(
M
$
/
y
r
)
(M$/yr)
REV and TOC (B-747 is best)
REV and TOC (SST is best)
Referring back to the flow chart in Figure 6.19, once the area of interest has been
fixed or if no change is desired in the first place, preferences can be changed among the
criteria. A change in preferences, i.e. a change in the preference weights of the criteria,
will change the POS value. However, the probability of achieving the criterion values
specified in Steps #6 and #10 of the flow chart stays the same. The weights merely alter
the target values according to Equations 5.31 and 5.32, which are subsequently used in
the calculation of POS (Equations 5.33 and 5.34). The probability of achieving the
specified criterion values is only equal to the POS value when all weights are equal, i.e.
no preferences are identified among the criteria. In order to understand the impact of
preferences on the solution to the product selection problem for the airline, POS values
for different preference weights are listed and compared in Table 6.9. All POS values are
based on the original criterion values of ROI
max
= , ROI
min
= 15%, REV
max
= ,
REV
min
= $100 million, TOC
min
= $0, and TOC
max
= $90 million per year.
Area of
Interest
B-747
B-777
A340
Area of
Interest
B-747
B-777
A340
Supersonic
Transport
(
M
$
/
y
r
)
(M$/yr)
(
M
$
/
y
r
)
(M$/yr)
Table 6.9: Comparison of POS Values for Different Criterion Preferences
Preference Weights
w
ROI
w
REV
w
TOC
Boeing
B-747
Boeing
B-777
Airbus A340
Supersonic
Transport
1/3 1/3 1/3 0.1124 0.003 0 0
0.4 0.3 0.3 0.4563 0.0660 0.0006 0
0.5 0.25 0.25 0.2986 0.5056 0.1337 0
0.3 0.3 0.4 0.0186 0.0576 0.0006 0
0.25 0.25 0.5 0 0.0015 0.0123 0
A slight increase of the importance or preference of the return on investment does
not have any impact of the product selection outcome: the Boeing B-747 is still the best
system for the airline. However, increasing the preference significantly, e.g. ROI is twice
as important than the revenue and operating cost, changes the outcome and the Boeing
B-777 is the system of choice. Despite the fact that the B-777 is the best system in this
case, its probability of satisfying ROI > 15%, REV > $100 million, and TOC < $90
million is still only 0.003 = 0.3%. An increase in preference of the operating cost has an
even more drastic effect on the solution. A slight increase changes the best solution from
the B-747 to the B-777. If the preference of TOC is increased even further, e.g. twice as
important as the return on investment and revenue, the Airbus A340 becomes the best
solution for the airline. However, as Table 6.9 indicates, the A340 has no chance of
satisfying the specified criterion values. In conclusion, no matter what the preference
structure, the B-747 is the best choice for the airline in this example, given the
requirements of ROI > 15%, REV > $100 million, and TOC < $90 million, since all other
systems indicate no chance of satisfying these values concurrently.
Finally, in order to get a better feel for the sensitivity of POS with respect to
changes in the input distributions, a sensitivity analysis is conducted which determines
the impact of changes in noise variable means and standard deviations on POS. First,
new distributions are defined for the noise variables, which are summarized and listed
together with the old distributions in Table 6.10.
Table 6.10: Old and New Noise Variable Distributions
Noise Variable
Old
Distribution
Old Mean
and Std. Dev.
New
Distribution
New Mean
and Std. Dev.
Load Factor
Beta
= 3, = 3
= 0.75
= 0.037796
Beta
= 2, = 4
= 0.716667
= 0.035635
Average Yield
Beta
= 5, = 3
= 0.1275
= 0.009682
Beta
= 3, = 3
= 0.12
= 0.011339
Fuel Price
Beta
= 3, = 5
= 0.7125
= 0.048412
Beta
= 3, = 3
= 0.75
= 0.056695
Economic Range
Beta
= 2, = 4
= 3666.67
= 356.3483
Beta
= 3, = 3
= 4000
= 377.9645
Utilization
Beta
= 3, = 3
= 5000
= 188.9822
Beta
= 2, = 4
= 4833.3333
= 178.1742
% Subsonic Cruise
Beta
= 3, = 3
= 0.25
= 0.094491
Beta
= 4, = 2
= 0.333333
= 0.089087
Inflation
Beta
= 3, = 5
= 0.04
= 0.01291
Beta
= 3, = 3
= 0.05
= 0.015119
Aircraft Price
Beta
= 3, = 3
= 250
= 18.89822
Beta
= 3, = 5
= 237.5
= 16.13743
Passenger Weight Normal
= 180
= 10
Beta
= 4, = 2
= 190
= 10.69045
Baggage Weight Normal
= 44
= 5
Beta
= 2, = 4
= 39
= 5.345225
Using the ten-step process, outlined in Figure 6.19, new POS values are
determined for each of the four aircraft, by changing the distribution for one noise
variable at a time. The metrics by which the impact is assessed are based on a percentage
change of mean and standard deviation. The change in POS is not normalized, i.e. the
change is not a percentage but rather an absolute value, since a change in probability
seems more insightful than a percentage change with respect to some arbitrary value.
Both metrics are formulated as:
new old new
old new
POS POS POS
/ ) ( %
(6.5)
new old new
old new
POS POS POS
/ ) ( %
(6.6)
Using this formulation for the metrics, the sensitivities are obtained for each of
the four systems. For reasons discussed before, the POS of the A340 and the Supersonic
Transport is equal to zero and not affected by any changes in the noise variable
distributions. Consequently, Table 6.11 only lists the old and new POS values and
sensitivities for the B-747 and B-777. To further visualize the results from Table 6.11,
plots for the sensitivities with respect to the mean and standard deviation are presented in
Figures 6.30 and 6.31 respectively. Both the sensitivity plots and the results from
Table 6.11 clearly indicate that changes in the Load Factor, Average Yield, and
Utilization distributions have the largest impact on POS. This is true for both metrics,
Equations 6.5 and 6.6, as well as for both systems, B-747 and B-777. For example, a 1%
increase in mean for the Load Factor will increase the POS for the B-747 by 0.0176, a
12.8% increase with respect to the original POS value of 0.1379. Changes in POS are
smaller for the B-777, because its absolute POS values are small.
Table 6.11: Old and New POS Values and Sensitivities for B-747 and B-777
Change in
Distribution
POS for
B-747
POS
%
POS
%
POS for
B-777.
POS
%
POS
%
Original 0.1379 N/A N/A 0.0030 N/A N/A
Load Factor 0.0597 0.0176 0.0137 0.0004 0.0006 0.0005
Average Yield 0.0654 0.0123 -0.0042 0.0011 0.0003 -0.0001
Fuel Price 0.1488 0.0021 0.0007 0.0025 -0.0001 -0.0000
Economic Range 0.1496 0.0013 0.0019 0.0037 0.0001 0.0001
Utilization 0.0793 0.0176 0.0102 0.0006 0.0007 0.0004
% Subsonic Cruise 0.1453 0.0002 -0.0013 0.0023 -0.0000 0.0000
Inflation 0.1436 0.0002 0.0003 0.0023 -0.0000 -0.0000
Aircraft Price 0.1458 -0.0016 -0.0005 0.0028 0.0000 0.0000
Passenger Weight 0.1448 0.0012 0.0010 0.0031 0.0000 0.0000
Baggage Weight 0.1385 -0.0001 0.0001 0.0019 0.0001 -0.0002
0.1496
0.0597
0.0037
0.0004
LF YIELD FUEL$ ECR U SUBS INFL PRICE PAXW BAGW
Figure 6.30: Sensitivity Plot for Changes in Mean for the B-747 and B-777
0.1496
0.0597
0.0037
0.0004
LF YIELD FUEL$ ECR U SUBS INFL PRICE PAXW BAGW
Figure 6.31: Sensitivity Plot for Changes in Standard Deviation for the B-747 and B-777
Particularly, the results for the metric based on changes in input means are not
very surprising and could have been obtained form a deterministic sensitivity analysis:
Load Factor, Average Yield, and Utilization are all variables that strongly affect the
revenue of the airline, which is the most stringent requirement for the B-747 and B-777
(see Table 6.7) that drives the value for POS most. Consequently, it is also not very
surprising that changes in POS due to changes in the input variables standard deviation
are mainly contributed to Load Factor, Average Yield, and Utilization. Slightly more
interesting is the direction of change. For example, POS decreases with increasing values
for the standard deviation of the average yield. This can be attributed to the fact that an
increase in Average Yield standard deviation allocates more revenue values outside the
area of interest. In contrast, the reverse is true for the Load Factor and Utilization,
demonstrated by the sensitivity plot in Figure 6.31.
Example Equation System with Ten Criteria
In order to demonstrate the capability of the Joint Probabilistic Decision Making
technique to handle more than two criteria, a system of ten equations is created for this
optimization example. The ten equations yield values for ten criteria that are to be
maximized. They depend on two design variables, x
1
, x
2
, and ten noise/uncertain
variables, Y
1
, Y
2
, Y
3
, Y
4
, Y
5
, Y
6
, Y
7
, Y
8
, Y
9
, Y
10
. All equations have a different optimum,
and therefore present a system of conflicting criteria. Specifically, the ten equations are:
2
1 2
2
10 1 10 1 2 1 1
) ( ) ( 200 ) , , , ( Y x Y x Y Y x x f (6.7)
2
2 2
2
9 1 9 2 2 1 2
) ( ) ( 200 ) , , , ( Y x Y x Y Y x x f (6.8)
2
3 2
2
8 1 8 3 2 1 3
) ( ) ( 200 ) , , , ( Y x Y x Y Y x x f (6.9)
2
4 2
2
7 1 7 4 2 1 4
) ( ) ( 200 ) , , , ( Y x Y x Y Y x x f (6.10)
2
5 2
2
6 1 6 5 2 1 5
) ( ) ( 200 ) , , , ( Y x Y x Y Y x x f (6.11)
2
6 2
2
5 1 6 5 2 1 6
) ( ) ( 200 ) , , , ( Y x Y x Y Y x x f (6.12)
2
7 2
2
4 1 7 4 2 1 7
) ( ) ( 200 ) , , , ( Y x Y x Y Y x x f (6.13)
2
8 2
2
3 1 8 3 2 1 8
) ( ) ( 200 ) , , , ( Y x Y x Y Y x x f (6.14)
2
9 2
2
2 1 9 2 2 1 9
) ( ) ( 200 ) , , , ( Y x Y x Y Y x x f (6.15)
2
10 2
2
1 1 10 1 2 1 10
) ( ) ( 200 ) , , , ( Y x Y x Y Y x x f . (6.16)
The noise variables are normally distributed with a standard deviation of 1 but
different means, namely Y
1
= N(1,1), Y
2
= N(4,1), Y
3
= N(5,1), Y
4
= N(7,1), Y
5
= N(8,1),
Y
6
= N(10,1), Y
7
= N(9,1), Y
8
= N(6,1), Y
9
= N(3,1), Y
10
= N(2,1). A visual representation
of the ten equations, evaluated at the noise variable means, is presented in Figure 6.32,
highlighting the conflict in finding a single point that maximizes all functions. This fact
is supported also by the graph in Figure 6.33, displaying the ten different maxima of the
equations.
Figure 6.32: Surfaces of Ten Equations
0 2 4 6 8 10
0
2
4
6
8
10
x1
x
2
Figure 6.33: Maxima of Ten Equations
Since the ten criteria depend on random variables, they themselves are random
variables as well. That means, the optimization problem of maximizing the ten equations
becomes a probabilistic optimization problem, maximizing the probability of all criteria,
F
i
, being larger than a specified value. The value selected for this example is 170.
Hence, the optimization problem can be formulated as
2 1
,
max
x x
P(F
1
> 170, F
2
> 170,
F
3
> 170, F
4
> 170, F
5
> 170, F
6
> 170, F
7
> 170, F
8
> 170, F
9
> 170, F
10
> 170).
However, this optimization problem can only be solved with the Joint Probability
Decision Making technique, since it requires a joint probability for ten criteria. In order
for the JPDM technique to be compared to other multi-criteria optimization methods, the
optimization problem has to be reformulated to:
) 170 ) ( ) ( 200 ( max ) 170 ) , , , ( ( max
2
1 2
2
10 1
,
10 1 2 1 1
,
2 1 2 1
> > Y x Y x P Y Y x x F P
x x x x
(6.17)
and ) 170 ) ( ) ( 200 ( max ) 170 ) , , , ( ( max
2
2 2
2
9 1
,
9 2 2 1 2
,
2 1 2 1
x x x x
(6.18)
and ) 170 ) ( ) ( 200 ( max ) 170 ) , , , ( ( max
2
3 2
2
8 1
,
8 3 2 1 3
,
2 1 2 1
x x x x
(6.19)
and ) 170 ) ( ) ( 200 ( max ) 170 ) , , , ( ( max
2
4 2
2
7 1
,
7 4 2 1 4
,
2 1 2 1
x x x x
(6.20)
and ) 170 ) ( ) ( 200 ( max ) 170 ) , , , ( ( max
2
5 2
2
6 1
,
6 5 2 1 5
,
2 1 2 1
x x x x
(6.21)
and ) 170 ) ( ) ( 200 ( max ) 170 ) , , , ( ( max
2
6 2
2
5 1
,
6 5 2 1 6
,
2 1 2 1
x x x x
(6.22)
and ) 170 ) ( ) ( 200 ( max ) 170 ) , , , ( ( max
2
7 2
2
4 1
,
7 4 2 1 7
,
2 1 2 1
x x x x
(6.23)
and ) 170 ) ( ) ( 200 ( max ) 170 ) , , , ( ( max
2
8 2
2
3 1
,
8 3 2 1 8
,
2 1 2 1
x x x x
(6.24)
and ) 170 ) ( ) ( 200 ( max ) 170 ) , , , ( ( max
2
9 2
2
2 1
,
9 2 2 1 9
,
2 1 2 1
x x x x
(6.25)
and ) 170 ) ( ) ( 200 ( max ) 170 ) , , , ( ( max
2
10 2
2
1 1
,
10 1 2 1 10
,
2 1 2 1
x x x x
,(6.26)
which is not exactly the same problem, but the closest current multi-criteria optimization
methods can accommodate. The Utility Function and Goal Attainment methods are
selected for comparison with the JPDM technique.
18
The Utility Function method,
similar to the JPDM technique, does not include an optimization algorithm itself. It is
simply comprised of an Overall Evaluation Criterion and is therefore used in conjunction
with a simplex search algorithm. In order to be able to compare the results, no
preferences are identified for the functions. Thus, the Utility Function and JPDM
technique are assigned equal weights, while the Goal Attainment method is assigned
weights of the same value as the goals, which are equal to the highest probability values
obtainable: One. The optimization problem can thus be formulated for the three
techniques as:
) 170 , 170 , 170 , 170 , 170
, 170 , 170 , 170 , 170 , 170 ( max max : JPDM
1 1 1 7 6
5 4 3 2 1
, ,
2 1 2 1
> > > > >
> > > > >
F F F F F
F F F F F P POS
x x x x
(6.27)
)) 170 ( 1 . 0 ) 170 ( 1 . 0 ) 170 ( 1 . 0 ) 170 ( 1 . 0
) 170 ( 1 . 0 ) 170 ( 1 . 0 ) 170 ( 1 . 0 ) 170 ( 1 . 0
) 170 ( 1 . 0 ) 170 ( 1 . 0 ( max max : Function Utility
10 9 8 7
6 5 4 3
2 1
, ,
2 1 2 1
> + > + > + > +
> + > + > + > +
> + >
F P F P F P F P
F P F P F P F P
F P F P OEC
x x x x
(6.28)
Goal Attainment:
2 1
,
min
x x
(6.29)
subject to: 1 ) 170 (
1
+ > F P and 1 ) 170 (
2
+ > F P
and 1 ) 170 (
3
+ > F P and 1 ) 170 (
4
+ > F P and 1 ) 170 (
5
+ > F P
and 1 ) 170 (
6
+ > F P and 1 ) 170 (
7
+ > F P and 1 ) 170 (
8
+ > F P
and 1 ) 170 (
9
+ > F P and 1 ) 170 (
10
+ > F P .

18
The Minimax method was dropped, since it produced similar solutions to the Goal Attainment method in
the previous example in Chapter II.
To illustrate the use of Equations 6.27 and 6.28 as objective functions for
optimization and in order to get a better feel for the nature of the objective functions in
this example, a surface plot is displayed for both equations in Figures 6.34 and 6.35
respectively. Note that the Probability of Success surface has a much more pronounced
maximum than the Overall Evaluation Criterion surface. Even though both surfaces may
have similar optimal points, the OEC surface is much flatter, and hence slower for an
optimization algorithm to search (see previous optimization example).
Figure 6.34: POS Response Surface
Figure 6.35: OEC Response Surface
The solutions to Equations 6.27, 6.28, and 6.29 are presented in Figure 6.36,
illustrating the differences in methods very well. The solutions to the ten probabilistic,
univariate optimization problems of maximizing P(F
1
> 170), P(F
2
> 170), P(F
3
> 170),
P(F
4
> 170), P(F
5
> 170), P(F
6
> 170), P(F
7
> 170), P(F
8
> 170), P(F
9
> 170), or
P(F
10
> 170) are individually added to the graph for reference purposes. The two
solutions generated by the Joint Probabilistic Decision Making technique correspond to
the two optimization techniques employed in conjunction with POS: a sequential
programming/line search and a simplex search technique.[Branch, Grace, 1996] All
solutions are found starting at (x
1
, x
2
) = (5, 5). However, some optimizations cannot find
a solution that improves the objective function value with respect to the starting point.
For these optimizations the starting point is the best solution.
1 2 3 4 5 6 7 8 9 10
1
2
3
4
5
6
7
8
9
10
x1
x
2
Figure 6.36: Optimal Solutions
The true test of superiority for all methods is presented in Table 6.12. All
solutions identified in Figure 6.36 are listed with their corresponding joint probability of
achieving function values larger than 170, an OEC value based on Equation 6.28, as well
as their univariate probabilities. For ease of identification, the box with the objective
function value that was maximized in that optimization is highlighted. Clearly JPDM in
conjunction with the simplex search method yields the solution with the highest joint
JPDM/Simplex Search
(5.4502, 5.4330)
JPDM/SQP/Line Search
(5, 5)
Goal Attainment
(5, 5)
OEC
(5.6422, 5.7019)
max P(F
7
> 170)
(5.3281, 7.8594)
max P(F
6
> 170)
(6.3516, 9.6172)
max P(F
5
> 170)
(9.6172, 6.3516)
max P(F
4
> 170)
(7.8594, 5.3281)
max P(F
1
> 170)
(3.4844, 2.1875)
max P(F
10
> 170)
(2.1875, 3.4844)
max P(F
2
> 170)
(5, 5)
max P(F
3
> 170)
(5, 5)
max P(F
8
> 170)
(5, 5)
max P(F
9
> 170)
(5, 5)
probability. The JPDM/line search solution has a lower joint probability, since the line
search, unlike the simplex search, does not find a point with improved POS with respect
to the starting point (5, 5). As before in previous examples, the OEC finds a solution that
falls just short of the joint probability value of the JPDM/simplex search solution. On the
other hand, it does find the solution with the highest OEC value, which supports the
previous argument that Equations 6.17 to 6.26 are simply not the right objective functions
for a joint probabilistic optimization problem. The Goal Attainment method cannot find
any solution, since it tries to achieve the same values for all univariate probabilities P(F
i
> 170). Consequently, it returns the starting point as its solution and it must be
concluded that this method is clearly not suitable for solving probabilistic multi-criteria
optimization problems. The remaining optimizations in Table 6.12 are merely
demonstrating that indeed maximizing just one objective P(F
i
> 170) yields a small or no
probability of satisfying all criteria concurrently. As a matter of fact, the solutions to
max P(F
2
> 170), max P(F
3
> 170), max P(F
8
> 170), and max P(F
9
> 170) only yield a
significant amount of joint probability, because the starting point already produces the
maximum objective function value of 1. Therefore, no improvement is necessary, and
the starting point is determined the best solution, which happens to yield a joint
probability of 0.1790.
Table 6.12: Comparison of Joint and Univariate Probabilities for Different Objective Functions
Joint
Probability
OEC
Value
Univariate
P(F
1
170)
Univariate
P(F
2
170)
Univariate
P(F
3
170)
Univariate
P(F
4
170)
Univariate
P(F
5
170)
Univariate
P(F
6
170)
Univariate
P(F
7
170)
Univariate
P(F
8
170)
Univariate
P(F
9
170)
Univariate
P(F
10
170)
JPDM/
Simplex Search
0.2170 0.7801 0.4040 0.9970 1.0000 0.9360 0.5640 0.5620 0.9350 1.0000 0.9970 0.4060
JPDM/SQP/
Line Search
0.1790 0.7608 0.6420 1.0000 1.0000 0.8280 0.3340 0.3340 0.8280 1.0000 1.0000 0.6420
OEC 0.1950 0.7898 0.3010 0.9950 1.0000 0.9700 0.6780 0.6840 0.9700 1.0000 0.9940 0.3060
Goal Attainment 0.1790 0.7608 0.6420 1.0000 1.0000 0.8280 0.3340 0.3340 0.8280 1.0000 1.0000 0.6420
max P(F
1
170) 0 0.5885 0.9990 1.0000 0.9480 0.0260 0.0010 0 0.0070 0.9060 1.0000 0.9980
max P(F
2
170) 0.1790 0.7608 0.6420 1.0000 1.0000 0.8280 0.3340 0.3340 0.8280 1.0000 1.0000 0.6420
max P(F
3
170) 0.1790 0.7608 0.6420 1.0000 1.0000 0.8280 0.3340 0.3340 0.8280 1.0000 1.0000 0.6420
max P(F
4
170) 0.0060 0.7208 0.0360 0.6630 1.0000 1.0000 0.9720 0.7600 0.9420 0.9960 0.8250 0.0140
max P(F
5
170) 0 0.5747 0 0.0470 0.9320 1.0000 1.0000 0.9090 0.9460 0.7850 0.1280 0
max P(F
6
170) 0 0.5747 0 0.1280 0.7850 0.9460 0.9090 1.0000 1.0000 0.9320 0.0470 0
max P(F
7
170) 0.0060 0.7208 0.0140 0.8250 0.9960 0.9420 0.7600 0.9720 1.0000 1.0000 0.6630 0.0360
max P(F
8
170) 0.1790 0.7608 0.6420 1.0000 1.0000 0.8280 0.3340 0.3340 0.8280 1.0000 1.0000 0.6420
max P(F
9
170) 0.1790 0.7608 0.6420 1.0000 1.0000 0.8280 0.3340 0.3340 0.8280 1.0000 1.0000 0.6420
max P(F
10
170) 0 0.5885 0.9980 1.0000 0.9060 0.0070 0 0.0010 0.0260 0.9480 1.0000 0.9990
O
l
i
v
e
r

B
a
n
d
t
e

C
h
a
p
t
e
r

V
I

1
6
5
Oliver Bandte Chapter VII 166
CHAPTER VII
CONCLUSIONS
It has always been the intention of systems engineering to invent or produce the
best product possible. Many design techniques have been introduced over the course of
decades that try to fulfill this intention. Some addressed the problem of capturing the
customers desires and needs. Others addressed the inability of accurate deterministic
performance and cost prediction caused by the inherent uncertainty in assumed values for
the parameters the prediction is based on. Unfortunately, no technique succeeded in
addressing both issues concurrently. The work presented in this thesis successfully
overcomes this deficiency by developing a multi-criteria decision making with
probabilistic design. Its core development is a multivariate probability distribution that
serves in conjunction with a criterion value range of interest as a universally applicable
objective function for multi-criteria optimization and product selection. This objective
function produces a meaningful metric, Probability of Success, that allows the customer
or designer to make a decision based on the chance of satisfying the customers goals.
Joint Probabilistic Decision Making Technique in Design
The design technique developed in this thesis is called the Joint Probabilistic
Decision Making (JPDM) technique. It was built cognizant of the fact that systems
design, at its core, is a decision making process in which the individual decisions concern
the allocation of resources and finding the right values for the design parameters. It is
this process that JPDM facilitates when decisions are based on multiple criteria while the
values for particular design or environmental parameters are uncertain.
Most decision processes can be grouped into two types of problems: optimization
and product selection. Both problems can be solved by comparing Probability of Success
(POS) values for the different decision alternatives, i.e. POS becomes the objective
function for multi-criteria optimization or the product selection metric that allows for an
equal basis comparison of the alternatives. POS itself describes the likelihood that the
system will achieve criterion values of interest to the customer. It is calculated by
integrating the joint probability distribution for the criteria the decision is based on over
the area of criterion values that satisfy the customers requirements and desirements.
While the area of interest can be determined from the needs analysis in the early phases
of design, the determination of the joint probability distribution is less obvious and was
the subject of this research.
Algorithms for Determining the Joint Probability Distribution
To incorporate a joint probabilistic formulation into the systems design process,
algorithms need to be created that allow for an easy implementation into a numerical
framework. Rules were established as a guide to which algorithm is best suited for
implementation into systems design:
Algorithms may not be limited in number of random variables, i.e. criteria.
Algorithms need to be flexible with respect to the criterion distributions.
Algorithms have to satisfy the conditions of a joint probability distribution.
Algorithms cannot require numerical integration.
Unfortunately, most algorithms producing a joint probability distribution that can
be found in the literature either attempt to approximate data from a specific process or
accommodate just one type of distribution. Only a joint-normal formulation would be
compatible with the flexibility requirement, since transformations of actual univariate
criterion distributions could yield normal distributions, to be used in a joint-normal
algorithm. Unfortunately, the joint-normal algorithm is not defined for all combinations
of (negative) correlation coefficients and criterion variances. Even worse, since the
integral of the characteristic function for the (joint) Normal-Distribution does not exist
and numerical integration of the algorithm for a large number of criteria is prohibitively
expensive, the joint-normal formulation can only be used for a small number of criteria.
To overcome this lack of adequate algorithms for generating joint probability
distributions, two new algorithms were created for this research: the (multivariate)
Empirical Distribution Function and the Joint Probability Model. The non-parametric
Empirical Distribution Function (EDF) simply estimates the probability of occurrence of
an event by counting how many times that event occurred in a given sample. In
probabilistic systems design, the EDF has commonly been used in estimating the
univariate distributions of system characteristics from samples generated by a sampling
technique such as Monte-Carlo simulation. This thesis outlined an extension to a joint
(multivariate) formulation. The Joint Probability Model (JPM) on the other hand is an
parametric model to estimate the multivariate joint probability. It is comprised of the
product of the univariate criterion distributions, generated by the traditional probabilistic
design process, multiplied with a correlation function that is based on available
correlation information between pairs of random variables. The selection process for the
correlation function has also been outlined in this thesis. Finally, Table 7.1 summarizes
the advantages and disadvantages of the EDF and JPM.
Table 7.1: Advantages and Disadvantages of EDF and JPM
EDF JPM
Advantages
No approximation with
standard distribution needed.
Estimates probability from
data directly.
Exact in the limit.
Fast estimation of joint
probability.
Only limited information
needed.
Can employ expert guesses.
Easy use in conceptual design.
Disadvantages
Large amount of data needed
in order to be accurate.
Requires modeling and
simulation.
Requires approximation with
parametric distribution.
Requires correlation function.
Estimation of joint probability
is time consuming.
JPDM for Optimization
The Joint Probabilistic Decision Making technique is a valuable tool for multi-
objective optimization, due to its ability to transform disparate objectives into a single
figure of merit, the likelihood of successfully meeting all goals, which is a single
optimizable function. This new objective function, called Probability of Success, allows
for the use of any standard single-objective optimization technique available. By
distinguishing between controllable and uncontrollable variables in the design process,
JPDM can account for the uncertain values of the uncontrollable variables that are
inherent to the design problem, while facilitating an easy adjustment of the controllable
ones to achieve the highest Probability of Success possible.
The application examples in this thesis demonstrated the techniques ability to
produce a better solution with a higher Probability of Success than an Overall Evaluation
Criterion (OEC) or Goal Programming approach. The first example was an optimization
problem to determine a supersonic transport aircraft configuration that would maximize
the returns on investment for the manufacturer and the airline concurrently, subject to
landing and take-off field length and approach speed constraints. JPDM yielded a
solution with a higher Probability of Success (0.782) than the OEC (0.773). In addition,
the solution was found in 71 rather than the 109 iterations needed by the OEC.
The other application example employed a set of ten equations, function of two
design/controllable and ten noise/uncontrollable variables. All ten functions were
contrived such that the location of their maximum within the design space was dependent
on the setting of the noise variables. Consequently, the optimization problem of
concurrently maximizing all ten functions presented a conflict that allowed no
deterministic solution. However, it was shown that, upon assignment of probability
distributions to the noise variables, only the JPDM technique is able to solve this
problem, since it is the only technique capable of forming and treating a joint probability
distribution of the responses. A comparison of the JPDM solution to OEC and Goal
Programming on a makeshift problem, that intended to maximize the individual
probabilities concurrently rather than one joint distribution, demonstrated JPDMs ability
to produce the best solution. The Goal Programming technique did not find any better
solution than the starting point while the OEC found a solution better than the starting
point, but with less POS than the JPDM solution.
JPDM for Product Selection
The Joint Probabilistic Decision Making technique is also an outstanding tool for
multi-attribute product selection, due to its ability to transform the likelihood of
achieving certain criterion values into a single metric that allows comparison of all
alternative solutions on an equal basis. This new metric, called Probability of Success
(POS), simplifies a complex multi-criteria selection problem into a simple ordering
problem, where the solution with the highest POS is the best solution. Also, by
distinguishing between controllable and uncontrollable variables in the design process,
JPDM can account for the uncertain values of the uncontrollable variables that are
inherent to the problem while distinguishing between alternatives through different
settings of the controllable ones. After deriving a joint criterion probability distribution
for each alternative, the Probability of Success can be obtained for each alternative by
integrating each distribution over the customer specified area of criterion values.
Subsequently, trade-off studies can be conducted at a very low level of effort that keep
the alternative distributions constant but change the criterion values of interest and
thereby change the Probability of Success for each alternative.
The application example in this thesis demonstrated the techniques ability to
produce a better solution with a higher Probability of Success and different ranking than
the Overall Evaluation Criterion or Technique for Order Preferences by Similarity to the
Ideal Solution (TOPSIS) approach. This product selection problem determined an
airlines best choice among the Boeing B-747, Boeing B-777, Airbus A340, and a
notional Supersonic Transport. The selection was based on values for return on
investment, average annual revenue, and total average annual operating cost. JPDM
determined the B-747 to be the best solution with the highest Probability of Success for
the scenario considered herein, while the OEC and TOPSIS both predicted the
Supersonic Transport to be the best selection for the airline. Considering that the
Supersonic Transport has a zero probability of satisfying all three criteria and their
respective values concurrently, this result has to be deemed erroneous and the JPDM
seems to be the only technique producing a sensible solution.
Upon further deliberation, the return of investment requirement was determined to
have very little influence on the best solution, since varying return of investment
requirement values produced the same result. However, changes in the requirements for
the revenue and total operating cost had a significant influence on the solution to the
selection problem, each aircraft being the best solution at some point, dependent on the
requirement value for revenue and operating cost. One conclusion drawn from these
results was that the current shape of the area of interest with independent requirement
values may not be adequate for all decision problems, and it was suggested to expand the
technique to include areas of interest with dependent criterion requirement values.
Research Questions and Answers
Finally, it is useful to consider again the research questions posed in the
introduction to this thesis and what this thesis has demonstrated in response to them.
Does uncertainty in the systems engineering design obstruct the decision process?
No. When treated properly, through probabilistic assumptions for example,
uncertainty about numerical values of design or operational parameters, model
fidelity, or technology availability does not impede the decision making process.
Can the use of probabilistic design be beneficial in the decision making process?
+ Yes. As demonstrated in the thesis, multi-criteria optimization or product selection
problems that dont have a deterministic solution can be solved through intelligent
use of a joint probability distribution.
Is there a numerical value representing customer satisfaction?
+ Yes. The numerical value established in this thesis is called Probability of Success,
and is obtained by integrating the joint criterion distribution over the area of criterion
values that are of interest to the customer.
Does a technique already exist that can help the decision-maker find a best solution
based on multiple criteria arising from a probabilistic design technique?
No. An extensive search of the current literature brought about no reference that
indicated a particular technique that combines multi-criteria decision making with
uncertain criterion values.
Is it possible to create such a technique and what should it look like?
+ Yes. As the thesis demonstrates, a joint probability distribution can be created for all
criteria of the multi-criteria decision problem, while accounting for the inherently
uncertain values of uncontrollable parameters in the design process.
Can this technique be used for optimization?
+ Yes. As demonstrated in the thesis, the Probability of Success can serve as an
objective function that can be used by any optimizer to find the best solution,
maximizing customer satisfaction.
Can this technique be used for product selection?
+ Yes. As demonstrated in the thesis, the Probability of Success can be used as a metric
that allows for a comparison of alternatives on an equal basis, the alternatives simply
need to be ranked based on their Probability of Success value.
Recommendations
In closing, an outlook for future research is provided at this point. It was
mentioned before and in Chapter VI, that in certain decision making situations
independent criterion value requirements for the area of interest may not be appropriate.
One modeling option to remedy the situation is to create linear equations of the criterion
requirement values z
min
and z
max
of the form:
0
1
max min/ 0
+
N
i
i i
z a a , 0
1
max min/ 0
+
N
i
i i
z b b , and so on. (7.1)
More complex shapes of the area of interest can be proposed as well, but its applicability
is questionable and can only be justified if detailed knowledge about the shape of the area
of interest is available. Some equations could be of the form:
0
1
2
max min/ 0
+
N
i
i i
z a a , 0
1
2
max min/ 0
+
N
i
i i
z b b , and so on, (7.2)
or 0
1
0
max min/
+
N
i
z
i
i
e a a , 0
1
0
max min/
+
N
i
z
i
i
e b b , and so on. (7.3)
Since this thesis borders closely on the area of utility theory in decision making, a
possible and natural extension of this research is to include a criterion value utility in a
new formulation of the Probability of Success. This new formulation may favor design
solutions that produce criterion values far within the area of interest. But it is not clear at
this point whether this may also be achieved by simply adjusting the area of interest such
that solutions with a previously low Probability of Success drop out. Adjustment and use
of the formulation for preference weights as proposed in this thesis may achieve this goal
as well. The benefit of a utility function for criterion values would only be noticeable
when this function has a non-monotonic behavior that favors criterion values in a
particular region. And even in this case, an intelligent way of modeling the area of
interest may yield the same effect.
Finally, a comment on conditional probabilities is in order. The utilization of a
joint probability distribution as the central element in this new decision making technique
suggests the investigation into use of conditional probabilities as well. While this
investigation was not further documented in this thesis, it was found that the use of
conditional probability does not add to the decision making process and is confusing at
best. When using conditional probability to indicate the joint or univariate distribution
for one or more criteria with fixed values for the other criteria, the Empirical Distribution
Function collapses due to the sharply reduced sample size of the conditional probability.
Even opening up the range of the condition, i.e. P(x, y| z
1
< z < z
2
) rather than
P(x, y| z = z
0
), decreases the sample size for the EDF to a point where it may no longer be
accurate. It can also be confusing to use a conditional probability for the plotting of the
(conditional) joint distribution for two criteria. While the Probability of Success for a
particular alternative may be small, conditioning the probability may display the
(conditional) joint distribution well within the area of interest, suggesting a high POS.
This confusing discrepancy is due to the fact that the criteria may be (strongly) correlated
and the process of eliminating the sample points with unfavorable values for the criterion
the distribution is conditioned on may actually eliminate the sample points with
unfavorable values for the criteria displayed. This would yield a probability distribution
that does not show any probability mass or density for unfavorable values (i.e. outside of
the displayed area of interest), yielding a (conditional) distribution well within the area of
interest. For these reasons the concept of conditional probability was not included in this
thesis and is probably not worth further investigation.
Oliver Bandte Appendix A 177
APPENDIX A
DISTRIBUTIONS
This appendix lists the parametric formulation of all univariate distributions
mentioned in this thesis. Formulations for these distributions can be found in any
probability theory reference, but have been taken for this appendix from [Hogg, Tanis,
1993] and [Jones, 1997] specifically. Many distributions are only defined for positive
values of the independent variable x. In order to be able to use the listed distributions in
systems design for criteria with non-positive values, modifications have been made to the
formulations that extend the range of valid x-values. Generally, a Min-Value and a Max-
Value limits this range. The range of validity for x is indicated for each distribution. ,
and are distribution specific parameters that did not need modification from the
standard formulation.
Beta-Distribution, B(, , Min, Max):
1 1
) ( ) (
) , (
) (

,
`
.
|
,
`
.
|

Min Max
x Max
Min Max
Min x
x f , Min x Max (A.1)
Gamma-Distribution, G(, , Min):

Min x
e Min x x f
1
) (
) (
1
) ( , Min x (A.2)
Oliver Bandte Appendix A 178
Lognormal-Distribution, LN(, ):
2
2
2
) ) (ln(
2
1
) (

x
e
x
x f , - x (A.3)
Normal-Distribution, N(, ):
2
2
2
) (
2
1
) (

x
e x f , - x (A.4)
Uniform-Distribution, U(Min, Max):
Min Max
x f
1
) ( , Min x Max (A.5)
Weibull-Distribution, W(, , Min):
) ( 1
) ( ) (
Min x
e Min x x f

, Min x (A.6)
Oliver Bandte Appendix B 179
APPENDIX B
CORRELATION FUNCTIONS
The correlation function introduced in Chapter V, Equation 5.18, is particularly
simple and easy to apply. However, due to the nature of the function, it does not increase
any joint density values around the midpoint of the range it is defined over. It only
significantly increases density values over points located close to the two opposite
corners of that range (see graph in Figure B.1). This may not be much of a disadvantage
for such joint distributions as the one displayed in Figures 5.5 and 5.6. However,
functions with a mode close to the center point of the range are much more effected by
this skewed distribution of density. As a matter of fact, very high values for can
produce a bimodal joint distribution and negative density values, invalidating the function
as a joint probability density function.
Figure B.1: Surface Plot of Equation 5.18 for Two Variables
A more suitable function is identified in Equation B.1 and B.2 for positive and
negative . Both equations are displayed in the graphs of Figures B.2 and B.3.
2 *
2
*
1
2 1
) ( 1
1
) , (
x x
x x g
+
, > 0 (B.1)
2 *
2
*
1
2 1
) ( 1
1
) , (
x x
x x g
+
, < 0 (B.2)
with
min 1 max 1
mid 1 1 *
1
2
x x
x x
x
,
min 2 max 2
mid 2 2 *
2
2
x x
x x
x
,
2
min 1 max 1
mid 1
x x
x
+
,
2
min 2 max 2
mid 2
x x
x
+
,
max 1 1 min 1
x x x < < , and
max 2 2 min 2
x x x < < .
Figure B.2: Surface Plot of Equation B.1 Figure B.3: Surface Plot of Equation B.2
To visualize the impact of Equations B.1 and B.2, two univariate beta
distributions are employed: X
1
~ B(6,6,0,1) and X
2
~ B(6,6,0,1). Their joint distribution,
without any correlation, is displayed in Figure B.4. When employing Equation B.1 as a
correlation function with = 0.09, points close to the x
1
= x
2
line see an increase in
likelihood of occurrence, as indicated in the joint probability plot of Figure B.5. This
trend is increased dramatically, when increasing to 0.9, as demonstrated in the joint
probability plot of Figure B.6.
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
x1
x
2
Figure B.4: Joint Probability Plot for Two Beta-Distributions ( = 0)
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
x1
x
2
Figure B.5: Joint Probability Plot for
Two Beta-Distributions with Correlation
Function B.1 ( = 0.09)
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
x1
x
2
If other shapes are desired for the joint probability distribution, the correlation
function can be constructed from more basic functions. Equations B.3 through B.6, for
example, skew the distribution towards a particular corner. Their effect is demonstrated
with the previously mentioned univariate beta-distributions in the joint probability
contour plots of Figures B.7 through B.10. is a positive scalable parameter that
determines the skewness of the distribution.
g(x
1
, x
2
) = 1 + x
1
*
x
2
*
,
min 1 max 1
min 1 1 *
1
x x
x x
x
,
min 2 max 2
min 2 2 *
2
x x
x x
x
(B.3)
g(x
1
, x
2
) = 1 + x
1
*
x
2
*
,
min 1 max 1
min 1 1 *
1
x x
x x
x
,
min 2 max 2
2 max 2 *
2
x x
x x
x
(B.4)
g(x
1
, x
2
) = 1 + x
1
*
x
2
*
,
min 1 max 1
1 max 1 *
1
x x
x x
x
,
min 2 max 2
min 2 2 *
2
x x
x x
x
(B.5)
g(x
1
, x
2
) = 1 + x
1
*
x
2
*
,
min 1 max 1
1 max 1 *
1
x x
x x
x
,
min 2 max 2
2 max 2 *
2
x x
x x
x
(B.6)
with
max 1 1 min 1
x x x < < ,
max 2 2 min 2
x x x < < , and > 0.
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
x1
x
2
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
x1
x
2
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
x1
x
2
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
x1
x
2
Oliver Bandte Appendix C 184
APPENDIX C
NUMBER OF SAMPLES FOR THE MONTE-CARLO
SIMULATION
The Monte-Carlo simulation is a sampling technique that determines which input
variable values to chose for a given simulation run. These input values are used in an
equation or analysis code, yielding the outputs of interest. Since the outputs depend on
random variables, they are random variables themselves. However, the Monte-Carlo
simulation does not determine the output distribution, but rather creates a random sample
X
1
, X
2
, X
3
, , X
n
from a Bernoulli distribution. In other words, each simulation run, or
sample i, determines, whether the analysis outcome, R, is less, more, or equal to a
specified value a; if so, X
i
= 1, if not, X
i
= 0.[Shooman, 1968]
Summing all samples yields a new random variable, Y = X
i
, which is binomially
distributed with a mean = np and variance
2
= np(1-p). n is the number of samples
run in the Monte-Carlo simulation, and p is the true probability of the specified event, as
in p = P(R < a). The question then becomes how many sample cases are needed to bring
the estimator of p, Y/n, reasonably close to p.
The Central Limit Theorem states that the distribution of
Y
W
(C-1)
is N(0,1), i.e. normal with = 0 and
2
= 1, in the limit where n . Thus, if n is
sufficiently large, the binomial distribution of Y can be approximated with
N(np, np(1-p)). Sufficiently large is determined by experience more than anything else.
Typically, the condition np > 5 and n(1-p) > 5 is a good guideline.[Shooman, 1968] The
95% confidence interval for Y/n being close to p , i.e. W = 0, is
% 95 ) 2 0 2 ( < < W P . (C-2)
Since W is N(0, 1), the interval becomes P(-2 < W < 2) = 95%, or
% 95 ) ) 1 ( 2 ) 1 ( 2 ( < < p np np Y p np P . (C-3)
Dividing by n we have
% 95 )
) 1 (
2
) 1 (
2 (
< <
n
p p
p
n
Y
n
p p
P , (C-4)
which is a 95% confidence interval for the estimator of p, Y/n. If the percent error with
respect to p is defined by
p
p
n
Y

, (C-5)
combining it with Equation C-4 yields
np
p
1
2 . (C-6)
Solving for n yields the equation that determines how many Monte-Carlo
simulation cases need to be run for the error to be within a 95% confidence interval at
the probability level p:
p
p
n

1 4
2
. (C-7)
From Equation C-7 and its depiction in Figure C-1 it follows that the number of
samples depends highly on the probability level of interest and . While the logarithmic
scale is deceiving, the parallel lines for different indicate that the number of samples, n,
increases for smaller values of much more rapidly with decreasing probability levels p.
For probability levels larger than 0.9 the number of samples becomes so small that the
sampling cannot be considered statistically significant anymore.
1
10
100
1000
10000
100000
1000000
10000000
0 0.2 0.4 0.6 0.8 1
Probability Level p
N
u
m
b
e
r

o
f

S
a
m
p
l
e
s

n
1%
2%
5%
10%
15%
Figure C.1: Number of Samples as a Function of Probability Level and Percent Error
% Error
Oliver Bandte Appendix D 187
APPENDIX D
JOINT PROBABILISTIC DECISION MAKING TECHNIQUE
- THE COMPUTER PROGRAM -
This appendix provides a list of the files created for this thesis work. Taken
together they comprise the computer program that calculates the Probability of Success
and creates the graphs displayed in Chapters V and VI. The flow of information within
the program is outlined in the flow chart of Figure D.1.
POS
JPDM.m
inputs.m
EDF or JPM
contourplot.m
surfplot.m
cumprobplot.m
JPM.m
JPMprob.m
2 0 1 4 0 1 6 0 1 8 0 2 0 0 2 2 0 2 4 0 2 6 0 2 8 0 3 0
EDF.m EDF#.exe
passback
POS
Data
passEDF
Data1 Data2 DataM
- 5 0 5 1 0 5
0
5
0
5
calls
reads in
creates
Figure D.1: Flow Chart of the Joint Probabilistic Decision Making Technique
The Joint Probabilistic Decision Making technique is currently coded in Matlab
.
The file that starts the process is called JPDM.m and is invoked from within Matlab
. Its
principal function is to read the input information from inputs.m into the memory and
make the decision based on the inputs whether to execute the Joint Probability Model
(JPM.m) or the Empirical Distribution Function (EDF.m). The Joint Probability Model
calculates the POS based on the distribution parameters and correlation function specified
in inputs.m. For visual checks of the inputs, JPM.m provides the ability to plot the
specified univariate distributions. If inputs.m indicates a request for plots,
JPMprob.m is called from JPM.m to calculate the probability (density) values required
for a two-dimensional contour plot (contourplot.m), three-dimensional surface plot
(surfplot.m), or two-dimensional cumulative probability contour plot
(cumprobplot.m).
In order to use the Empirical Distribution Function, data needs to be provided in
form of columns, one for each criterion, within files named Data1, Data2, , DataM,
with M being the number of alternatives evaluated concurrently. For optimization, one
file, Data1, which changes with each iteration is sufficient. EDF.m will copy the data
for each alternative into a new file called Data, and supply information about the area of
interest, the size of Data, and step size for plotting in a file called passEDF. Next, it
calls EDF#.exe, # is the number of criteria, which in turn reads the information from
passEDF and Data. EDF#.exe is a short FORTRAN program that contains the actual
counting procedure for the Empirical Distribution Function. The decision to code this
procedure in FORTRAN was based on the general availability of FORTRAN and the fact
that the counting takes orders of magnitude longer in MATLAB
. EDF#.exe, then,
supplies joint and univariate probability values for the criteria as well as probability
values for plotting in a file called passback back to EDF.m. This loop is repeated for
all alternatives until EDF.m finally prints out the joint POS and univariate probability
values for all alternatives. If plotting is requested in inputs.m, contourplot.m,
surfplot.m, and/or cumprobplot.m are called to produce a two-dimensional
contour plot, three-dimensional surface plot, and/or two-dimensional cumulative
probability contour plot.
List of Files
JPDM.m
% JPDM.m %
clear all
ff = clock;
% Default Values
crplots = [0 0];
t = 1;
sp = 0;
aalow = [0 0 0 0];
aahigh = [0 0 0 0];
inputs
nfom = length(maxvalue);
if method == 1
points = length(distribution(:,1));
JPM
elseif method == 2
points = length(l);
EDF
end
inputs.m (Joint Probability Model)
% inputs
method = 1;
syms x1 x2 ;
% Choose Distribution from:
% 1 = Beta(a,b,U,L)
% 2 = Exponential(mu)
% 3 = negExponential(mu)
% 4 = Gamma(a,b,L)
% 5 = negGamma(a,b,U)
% 6 = Lognormal(L,mu,sig)
% 7 = negLognormal(U,mu,sig)
% 8 = Normal(mu,sig)
% 9 = Rayleigh(b,L)
% 10 = negRayleigh(b,U)
% 11 = Uniform(U,L)
% 12 = Weibull(a,b,L)
% 13 = negWeibull(a,b,U)
% 14 = Students t(mu,a)
distribution(1,1) = 1;
fom(1) = x1;
a(1,1) = 5;
b(1,1) = 10;
L(1,1) = 135;
U(1,1) = 147 + L(1,1);
lower(1,1) = 135;
upper(1,1) = U(1,1);
plotrange(1,:,1) = [L(1,1) U(1,1)];
ipl(1,1) = y;
fom(2) = x2;
L(1,2) = -5;
U(1,2) = 111.5;
a(1,2) = 8;
b(1,2) = 21;
lower(1,2) = -5;
upper(1,2) = 111.5;
plotrange(2,:,1) = [-10 80];
ipl(1,2) = y;
a(2,1) = 6;
b(2,1) = 7;
L(2,1) = 69;
U(2,1) = 91;
lower(2,1) = 69;
upper(2,1) = 91;
plotrange(1,:,2) = [69 91];
ipl(2,1) = y;
L(2,2) = -5;
a(2,2) = 4;
b(2,2) = 3;
U(2,2) = 36;
lower(2,2) = -5;
upper(2,2) = 36;
plotrange(2,:,2) = [-10 80];
ipl(2,2) = y;
a(3,1) = 6;
b(3,1) = 8;
L(3,1) = 55;
U(3,1) = 81;
lower(3,1) = 55;
upper(3,1) = 81;
plotrange(1,:,3) = [55 81];
ipl(3,1) = y;
L(3,2) = -5;
a(3,2) = 6;
b(3,2) = 5;
U(3,2) = 46.5;
lower(3,2) = -5;
upper(3,2) = 46.5;
plotrange(2,:,3) = [-10 80];
ipl(3,2) = y;
fcorr(1) = 1+0.41*(2*fom(1)-(upper(1,1)+lower(1,1)))/(upper(1,1)-
lower(1,1))*(2*fom(2)-(upper(1,2)+lower(1,2)))/
(upper(1,2)-lower(1,2));
% Requirements
maxvalue = [90 150];
minvalue = [0 15];
% Plot information
nstep = [50 50
50 50
50 50
50 50 ];
crplots = [1 2 ];
condrange = [-10 -10
500 100 ];
probplot2D = y;
probplot3D = y;
cumplot = y;
cp = (0.00001:0.0002:0.5);
cc = (0:0.1:.99);
aalow = [50 -10 ];
aahigh = [250 80 ];
name(1) = {TOC(M$/yr)};
name(2) = {ROI};
contourcolors = [k
b
r
c];
JPM.m
% JPM.m
for n = 1:points;
for i = 1:size(distribution,2)
if distribution(n,i) == 1
func(i) = gamma(a(n,i)+b(n,i))*((fom(i)-L(n,i))/(U(n,i)-
L(n,i)))^(a(n,i)-1)*((U(n,i)-fom(i))/(U(n,i)-
L(n,i)))^(b(n,i)-1)/(gamma(a(n,i))*gamma(b(n,i)));
if ipl(n,i) == y
xsmin(n) = max(lower(n,i),plotrange(i,1,n));
xsmax(n) = min(upper(n,i),plotrange(i,2,n));
sstep(n) = (xsmax(n)-xsmin(n))/nstep(n,i);
xs(n,:) = xsmin(n):sstep(n):xsmax(n);
ys(n,:) = subs(func(i),fom(i),xs(n,:));
figure
plot(xs(n,:),ys(n,:))
xlabel(name(i));
axis tight
end
elseif distribution(n,i) == 2
func(i) = exp(-fom(i)/mu(n,i))/mu(n,i);
if ipl(n,i) == y
figure
xlabel(name(i));
axis tight
end
func(i) = exp(fom(i)/mu(n,i))/mu(n,i);
if ipl(n,i) == y
figure
xlabel(name(i));
axis tight
end
func(i) = (fom(i)-L(n,i))^(a(n,i)-1)*exp(-(fom(i)-L(n,i))/
b(n,i))/(b(n,i)^a(n,i)*gamma(a(n,i)));
if ipl(n,i) == y
figure
xlabel(name(i));
axis tight
end
func(i) = (U(n,i)-fom(i))^(a(n,i)-1)*exp(-(U(n,i)-fom(i))/
b(n,i))/(b(n,i)^a(n,i)*gamma(a(n,i)));
if ipl(n,i) == y
figure
xlabel(name(i));
axis tight
end
func(i)=exp((log(fom(i)-L(n,i))-mu(n,i))^2/(2*sig(n,i)^2))/
((fom(i)-L(n,i))*sig(n,i)*sqrt(2*pi));
if ipl(n,i) == y
figure
xlabel(name(i));
axis tight
end
elseif distribution(i) == 7
func(i)=exp((log(U(n,i)-fom(i))-mu(n,i))^2/(2*sig(n,i)^2))/
((U(n,i)-fom(i))*sig(n,i)*sqrt(2*pi));
if ipl(n,i) == y
figure
xlabel(name(i));
axis tight
end
func(i) = exp(-(fom(i)-mu(n,i))^2/
(2*sig(n,i)^2))/(sig(n,i)*sqrt(2*pi));
if ipl(n,i) == y
figure
xlabel(name(i));
axis tight
end
func(i) = (fom(i)-L(n,i))*exp(-(fom(i)-L(n,i))^2/
(2*b(n,i)^2))/b(n,i)^2;
if ipl(n,i) == y
figure
xlabel(name(i));
axis tight
end
func(i) = (U(n,i)-fom(i))*exp(-(U(n,i)-fom(i))^2/
(2*b(n,i)^2))/b(n,i)^2;
if ipl(n,i) == y
figure
xlabel(name(i));
axis tight
end
func(i) = 1/(U(n,i)-L(n,i));
if ipl(n,i) == y
figure
xlabel(name(i));
axis tight
end
func(i) = b(n,i)*((fom(i)-L(n,i))/a(n,i))^(b(n,i)-1)*
exp(-((fom(i)-L(n,i))/a(n,i))^b(n,i))/a(n,i);
if ipl(n,i) == y
figure
xlabel(name(i));
axis tight
end
func(i) = a(n,i)*b(n,i)*(U(n,i)-fom(i))^(b(n,i)-1)*
exp(-a(n,i)*(U(n,i)-fom(i))^b(n,i));
if ipl(n,i) == y
figure
xlabel(name(i));
axis tight
end
func(i)=gamma((a(n,i)+1)/2)/(sqrt(pi*a(n,i))*gamma(a(n,i)/
2)*(1+(fom(i)-mu(n,i))^2/a(n,i))^((a(n,i)+1)/2));
if ipl(n,i) == y
figure
xlabel(name(i));
axis tight
end
end
end
clear xsmin xsmax sstep xs ys;
F = int(func(1)*fcorr(n),fom(1));
FS = subs(F,fom(1),upper(n,1))-subs(F,fom(1),lower(n,1));
F = int(FS*func(i),fom(i));
FS = subs(F,fom(i),upper(n,i))-subs(F,fom(i),lower(n,i));
end
Ftot = double(FS);
Fp = int(func(1)*fcorr(n),fom(1));
FpS=subs(Fp,fom(1),min(upper(n,1),maxvalue(1)))-subs(Fp,fom(1),
max(lower(n,1),minvalue(1)));
if min(upper(n,1),maxvalue(1)) < max(lower(n,1),minvalue(1))
FpS = 0;
end
Fp = int(FpS*func(i),fom(i));
FpS = subs(Fp,fom(i),min(upper(n,i),maxvalue(i)))-
subs(Fp,fom(i),max(lower(n,i),minvalue(i)));
if min(upper(n,i),maxvalue(i)) < max(lower(n,i),minvalue(i))
FpS = 0;
end
end
POS(n) = double(FpS/Ftot);
if probplot2D == y
JPMprob
elseif probplot3D == y
JPMprob
end
end
POS
cd plot.files
if probplot2D == y
contourplot
end
if probplot3D == y
surfplot
end
if cumplot == y
cumprobplot
end
cd ..
JPMprob.m
% JPMprob.m %
for m = 1:length(crplots(:,1))
plotx = crplots(m,1);
ploty = crplots(m,2);
xmin(n,plotx) = plotrange(plotx,1,n);
xmax(n,plotx) = plotrange(plotx,2,n);
xmin(n,ploty) = plotrange(ploty,1,n);
xmax(n,ploty) = plotrange(ploty,2,n);
aal = 1:nfom;
bll = find(aal ~= plotx);
cll = find(bll ~= ploty);
cond = bll(cll);
xmin(n,cond) = condrange(1,cond);
xmax(n,cond) = condrange(2,cond);
step(n,:) = (xmax(n,:)-xmin(n,:))./nstep(n,:);
x(n,:) = xmin(n,plotx):step(n,plotx):xmax(n,plotx);
y(n,:) = xmin(n,ploty):step(n,ploty):xmax(n,ploty);
[xp,yp] = meshgrid(x(n,:),y(n,:));
invalidxl = find(x(n,:)<lower(n,plotx));
xp(:,invalidxl) = lower(n,plotx);
invalidxu = find(x(n,:)>upper(n,plotx));
xp(:,invalidxu) = upper(n,plotx);
invalidyl = find(y(n,:)<lower(n,ploty));
yp(invalidyl,:) = lower(n,ploty);
invalidyu = find(y(n,:)>upper(n,ploty));
yp(invalidyu,:) = upper(n,ploty);
if length(fom) > 2
Fcond = int(func(cond(1))*fcorr(n),fom(cond(1)));
FScond = subs(Fcond,fom(cond(1)),min(condrange(2,cond(1)),
upper(n,cond(1))))-subs(Fcond,fom(cond(1)),
max(condrange(1,cond(1)),lower(n,cond(1))));
if length(cond) > 1
for i = 2:length(cond)
Fcond = int(FScond*func(cond(i)),fom(cond(i)));
FScond = subs(Fcond,fom(cond(i)),min(condrange(2,cond(i)),
upper(n,cond(i))))-subs(Fcond,fom(cond(i)),
max(condrange(1,cond(i)), lower(n,cond(i))));
end
end
else
FScond = fcorr(n);
end
ff = FScond*func(plotx)*func(ploty)/Ftot;
ff = simplify(ff);
ffpro(:,:,n) = subs(ff,fom(plotx),xp);
for i = 1:length(y(n,:))
ffprob(i,:,n) = subs(ffpro(i,:,n),fom(ploty),yp(i,1));
end
prob(:,:,n) = double(ffprob(:,:,n));
end
inputs.m (Empirical Distribution Function)
% inputs.m %
method = 2;
maxvalue = [90 1000 1000];
minvalue = [0 100 5];
nstep = [100 100 100
100 100 100
100 100 100
100 100 100];
l = [10000
10000
10000
10000];
crplots = [1 2 ];
condrange = [-100 -100
100 100];
probplot2D = y;
probplot3D = y;
cumplot = y;
cp = (0.00005:0.0003:0.05);
cc = [0,0.1,.2,.3,.4,.5,.6,.7,.8,.9];
aalow = [50 40 ];
aahigh = [250 340];
name(1) = {TOC(M$/yr)};
name(2) = {REV(M$/yr)};
contourcolors = [0.4 0.4 0.4
0.6 0.6 0.6
0.8 0.8 0.8
0.0 0.0 0.0];
EDF.m
% EDF.m %
dat = NaN * ones(nfom,max(l),points);
fid(1) = fopen(Data1,r);
for n = 1:points
dat(:,:,n) = fscanf(fid(n),%g,[nfom l(n)]);
data(:,:,n) = dat(:,:,n);
end
status = fclose(all);
for n = 1:points
xmin(n,:) = min(data(:,:,n),[],1);
xmax(n,:) = max(data(:,:,n),[],1);
lower(n,:) = max(xmin(n,:),minvalue);
upper(n,:) = min(xmax(n,:),maxvalue);
fidd = fopen(passEDF,w);
if n == 1
!copy Data1 Data
elseif n == 2
!copy Data2 Data
elseif n == 3
!copy Data3 Data
elseif n == 4
!copy Data4 Data
elseif n == 5
!copy Data5 Data
elseif n == 6
!copy Data6 Data
elseif n == 7
!copy Data7 Data
elseif n == 8
!copy Data8 Data
elseif n == 9
!copy Data9 Data
elseif n == 10
!copy Data10 Data
end
if nfom == 2
step(n,:) = (xmax(n,:)-xmin(n,:))./nstep(n,:);
fprintf(fidd,%g\n,l(n),xmin(n,:),xmax(n,:),step(n,:),
lower(n,:),upper(n,:));
status = fclose(fidd);
!EDF2.exe
x(n,:) = xmin(n,1) + step(n,1)/2:step(n,1):xmax(n,1);
y(n,:) = xmin(n,2) + step(n,2)/2:step(n,2):xmax(n,2);
fidd = fopen(passback,r);
i = fscanf(fidd,%g,[1 1]);
j = fscanf(fidd,%g,[1 1]);
count = fscanf(fidd,%g,[i j]);
C(n) = fscanf(fidd,%g,[1 1]);
countv(n,:) = fscanf(fidd,%g,[1 nfom]);
prob(:,:,n) = count(2:i,2:j)/l(n);
prob(1,:,n) = (count(1,2:j) + count(2,2:j))/l(n);
prob(:,1,n) = (count(2:i,1) + count(2:i,2))/l(n);
plotx = 1;
ploty = 2;
if n == points
cd plot.files
if probplot2D == y
contourplot
end
if probplot3D == y
surfplot
end
if cumplot == y
cumprobplot
end
cd ..
end
elseif nfom == 3
fprintf(fidd,%g\n,l(n),lower(n,:),upper(n,:));
!EDF3.exe
elseif nfom == 4
!EDF4.exe
elseif nfom == 5
!EDF5.exe
end
end
cumulative = C./l;
probtable = zeros(points+1,nfom+2);
probtable(2:points+1,1) = transpose(1:points);
probtable(1,2:nfom+2) = (0:nfom);
probtable(2:points+1,2) = cumulative;
for n = 1:points
probtable(n+1,3:nfom+2) = countv(n,:)/l(n);
end
probtable
EDF2.exe
program EDF2
implicit none
real x, y, xmin, xmax, ymin, ymax, stepx
real minx, maxx, miny, maxy, data(15000,2)
real count(1001,1001), stepy, epsx, epsy
integer k, i, j, l, s, t, C, CX, CY
C = 0
CX = 0
CY = 0
open (8, FILE = passEDF, status = old)
read (8, *) l, minx, miny, maxx, maxy, stepx
read (8, *) stepy, xmin, ymin, xmax, ymax
open (9, FILE = Data, status = old)
epsx = stepx/100.
epsy = stepy/100.
do 10 k = 1,l
read (9, *) data(k,1), data(k,2)
j = 0
do 20 x = minx,maxx+epsx,stepx
i = 0
j = j + 1
do 30 y = miny,maxy+epsy,stepy
i = i + 1
if (data(k,1) .gt. x - stepx) then
if (data(k,1) .le. x) then
if (data(k,2) .gt. y - stepy) then
if (data(k,2) .le. y) then
count(i,j) = count(i,j) + 1
endif
endif
endif
endif
30 continue
20 continue
if (data(k,1) .ge. xmin) then
if (data(k,1) .le. xmax) then
if (data(k,2) .ge. ymin) then
if (data(k,2) .le. ymax) then
C = C + 1
endif
endif
endif
endif
if (data(k,1) .ge. xmin) then
if (data(k,1) .le. xmax) then
CX = CX + 1
endif
endif
if (data(k,2) .ge. ymin) then
if (data(k,2) .le. ymax) then
CY = CY + 1
endif
endif
10 continue
open (10, FILE = passback)
write (10, *) i, j
do 40 s = 1,j
do 50 t = 1,i
write (10, *) count(t,s)
50 continue
40 continue
write (10, *) C
write (10, *) CX
write (10, *) CY
endfile (10)
close (10)
close (9)
close(8)
end
EDF5.exe
program EDF5
implicit none
real min(5), max(5), data(15000,5)
integer k, l, s, D, C(5)
D = 0
do 5 s =1,5
C(s) = 0
5 continue
open (8, FILE = passEDF, status = old)
read (8, *) l, (min(s), s=1,5), (max(s), s=1,5)
open (9, FILE = Data, status = old)
do 10 k = 1,l
read (9, *) (data(k,s), s=1,5)
if (data(k,1).ge.min(1) .and. data(k,1).le.max(1)) then
D=D+1
endif
endif
endif
endif
endif
do 20 s = 1,5
if(data(k,s).ge.min(s) .and. data(k,s).le.max(s))then
C(s)=C(s)+1
endif
20 continue
10 continue
open (10, FILE = passbackEDF)
write (10, *) D
do 40 s = 1,5
write (10, *) C(s)
40 continue
endfile (10)
close (10)
close (9)
close(8)
end
contourplot.m
% contourplot.m %
figure
hold
[xm,ym] = meshgrid(max(min(min(x)),minvalue(plotx))+step(1,plotx)
/2:step(1,plotx):min(max(max(x)),maxvalue(plotx))-
step(1,plotx)/2,max(min(min(y))+step(1,ploty)/2,
minvalue(ploty)):step(1,ploty):min(max(max(y)),
maxvalue(ploty))-step(1,ploty)/2);
plot(xm,ym,g.);
for n = 1:points
[Cont,H] = contour(x(n,:),y(n,:),prob(:,:,n),cp,-);
set(H,Color,contourcolors(n,:))
end
ycp = min(min(x)):step(1,plotx):max(max(x));
xcp = min(min(y)):step(1,ploty):max(max(y));
if minvalue(plotx) > min(min(x))
lo = ones(1,length(xcp))*minvalue(plotx);
plot(lo,xcp)
clear lo
end
if minvalue(ploty) > min(min(y))
lo = ones(1,length(ycp))*minvalue(ploty);
plot(ycp,lo)
clear lo
end
if maxvalue(plotx) < max(max(x))
lo = ones(1,length(xcp))*maxvalue(plotx);
plot(lo,xcp)
clear lo
end
if maxvalue(ploty) < max(max(y))
lo = ones(1,length(ycp))*maxvalue(ploty);
plot(ycp,lo)
clear lo
end
if sum(aalow) == 0
axis tight
else
axis([aalow(plotx) aahigh(plotx) aalow(ploty) aahigh(ploty)])
end
xlabel(name(plotx));
ylabel(name(ploty));
print -djpeg95 prob2D.jpg
hold
surfplot.m
% surfplot.m %
figure
if sp == 0
for n = 1:points
if method == 2
tprob(:,:,n) = fliplr(prob(:,:,n));
graph = bar3(y(n,:),tprob(:,:,n));
view(155,20);
else
graph = surf(x(n,:),y(n,:),prob(:,:,n));
view(-15,30);
end
set(graph,EdgeColor,[0.6 0.6 0.6]);
hold on
end
else
if method == 2
tprob = fliplr(prob(:,:,sp));
graph = bar3(y(sp,:),tprob);
view(155,20);
else
graph = surf(x(sp,:),y(sp,:),prob(:,:,sp));
view(-15,30);
end
set(graph,EdgeColor,[0.6 0.6 0.6]);
hold on
end
axis tight
zlabel(Density);
print -djpeg95 prob3D.jpg
cumprobplot.m
% cumprobplot.m %
if method == 2
step = ones(points,nfom);
end
[ey,ex] = size(prob(:,:,1));
for n = 1:points
for i = 1:round(t):ey
kt = 1+(i-1)/round(t);
for j = 1:round(t):ex
lt = 1+(j-1)/round(t);
if direction(plotx) == l
if direction(ploty) == l
cmp(kt,lt,n) = sum(sum(prob(i:ey,j:ex,n)));
elseif direction(ploty) == s
cmp(kt,lt,n) = sum(sum(prob(1:i,j:ex,n)));
end
elseif direction(plotx) == s
if direction(ploty) == l
cmp(kt,lt,n) = sum(sum(prob(i:ey,1:j,n)));
elseif direction(ploty) == s
cmp(kt,lt,n) = sum(sum(prob(1:i,1:j,n)));
end
end
end
end
cump(:,:,n) = cmp(:,:,n)*prod(step(n,:));
end
figure
for n = 1:points
[Cont,H] = contour(x(n,:),y(n,:),cump(:,:,n),cc,-);
set(H,Color,contourcolors(n,:))
clabel(Cont,H,fontsize,8,color,contourcolors(n,:),labelsp
acing,1000);
hold on
end
if maxvalue(plotx) > xmax(n,plotx)
cx = ones(1,length(y(n,:)))*minvalue(plotx);
if maxvalue(ploty) > xmax(n,ploty)
cy = ones(1,length(x(n,:)))*minvalue(ploty);
elseif minvalue(ploty) < xmin(n,ploty)
cy = ones(1,length(x(n,:)))*maxvalue(ploty);
end
elseif minvalue(plotx) < xmin(n,plotx)
cx = ones(1,length(y(n,:)))*maxvalue(plotx);
if maxvalue(ploty) > xmax(n,ploty)
cy = ones(1,length(x(n,:)))*minvalue(ploty);
elseif minvalue(ploty) < xmin(n,ploty)
cy = ones(1,length(x(n,:)))*maxvalue(ploty);
end
end
plot(x,cy)
plot(cx,y)
if sum(aalow) == 0
axis tight
else
axis([aalow(plotx) aahigh(plotx) aalow(ploty) aahigh(ploty)])
end
print -djpeg95 cum2D.jpg
Oliver Bandte Bibliography 209
BIBLIOGRAPHY
Airbus Industries, Products Technical - Specifications <http://www.airbus.com>
August, 2000.
Ang, A. H.-S., Tang, W. H., Probability Concepts in Engineering Planning and Design,
Volume I, John Wiley & Sons, Inc., New York, 1984.
Asimow, M., Introduction to Design, Prentice-Hall, Inc., Englewood Cliffs, NJ, 1962.
Beakley, G.C., Leach H.W., Engineering - An Introduction to a Creative Profession, 2
nd
Edition, The Macmillan Company, New York, 1972.
Blanchard, B.S., Systems Engineering Management, 2
nd
Edition, John Wiley & Sons,
Inc., New York, 1998.
Blanchard, B.S., Fabrycky, W.J., Systems Engineering and Analysis, 3
rd
Edition,
Prentice-Hall, New York, 1998.
Boeing Company, The, Commercial Aircraft Product Information
<http://www.boeing.com/commercial/> August, 2000.
Box, G.E.P., Draper, N.R., Empirical Model-Building and Response Surfaces, John
Wiley & Sons, Inc., New York, 1987.
Box, G.E.P., Hunter, W.G., Hunter, J.S., Statistics for Experimenters, John Wiley &
Sons, Inc., New York, 1978.
Branch, M.A., Grace, A., MATLAB
- Optimization Toolbox (Users Guide), The

MathWorks, Inc., 1996.
Chen, W., et al., Integration of Response Surface Method with the Compromise Decision
Support Problem in Developing a General Robust Design Procedure, Advances in
Design Automation (Azarm, S., et al. Eds.), Vol. 82-2, ASME, New York, 1995.
Cheng, B., Titterington, D.M., Neural Networks: A Review from a Statistical Perspective,
Statistical Science, Vol. 9, No. 1, 2-54, 1994.
Cleland, D.I., King, W.R., Systems Analysis and Project Management, McGraw-Hill,
Inc., New York, 1983.
DeLaurentis, D.A., A Stochastic Approach to Aircraft Design Incorporating Guidance
and Control Uncertainties, Ph.D. Thesis, Georgia Institute of Technology, 1998.
DeLaurentis, D.A., Calise, A., Schrage, D.P., Mavris, D.N., Reduced Order Guidance
Methods and Probabilistic Techniques in Addressing Mission Uncertainty, 6th
AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and
Optimization, Bellevue, WA, September, 1996, AIAA-96-4174.
DeLaurentis, D.A., Mavris, D.N., Schrage, D.P., System Synthesis in Preliminary Aircraft
Design Using Statistical Methods, 20th Congress of the International Council on the
Aeronautical Sciences, Sorrento, Italy, September, 1996.
Dieter, G.E., Engineering Design, 2
nd
Edition, McGraw-Hill, Inc., New York, 1991.
Dixon, J.R., Design Engineering: Inventiveness, Analysis, and Decision Making,
McGraw-Hill Book Company, New York, 1966.
Ertas, A., Jones, J.C., The Engineering Design Process, John Wiley & Sons, New York,
1993.
Fleming, P.J., Application of Multiobjective Optimization to Compensatory Design for
SISO Control Systems, Electronics Letters, Vol.22, No.5, pp.258-259, 1986.
Fox, E.P., The Pratt & Whitney Probabilistic Design System, 35
th
AIAA/ASME/ASCE/
AHS/ASC Structures, Structural Dynamics, and Materials Conference, Hilton Head,
SC, April, 1994, AIAA-94-1442.
Fox, E.P., Reh, S, On the Accuracy of Various Probabilistic Methods, 41
st
AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials
Conference, Atlanta, April 2000, AIAA-2000-1631.
Galloway, T.L., Mavris, D.N., Aircraft Life Cycle Cost Analysis (ALCCA) Program,
NASA Ames Research Center, September 1993.
Garvey, P.R., Probability Methods for Cost Uncertainty Analysis A Systems Engineers
Perspective, Marcel Dekker, Inc., New York, Basel, Hongkong, 1999.
Garvey, P.R., Taub, A.E., Joint Probability Model for Cost and Schedule Uncertainties,
26
th
Annual Department of Defense Cost Analysis Symposium, September, 1992.
Gembicki, F.W., Performance and Sensitivity Optimization: A Vector Index Approach,
Ph.D. Thesis, Case Western Research University, 1974.
Giunta, A.A., et al., Wing Design for a High-Speed Civil Transport Using a Design of
Experiments Methodology, 6
th
AIAA/ USAF/NASA/ISSMO Symposium on
Multidisciplinary Analysis and Optimization, Bellevue, WA, September, 1996,
AIAA-96-4001.
Hazelrigg, G.A., Systems Engineering: an Approach to Information-Based Design,
Prentice-Hall, Inc., Upper Saddle River, NJ, 1996.
Hogg, R.V., Tanis, E.A., Probability and Statistical Inference, Macmillan Publishing
Company, New York, 1993.
Huber, K.-P., Berthold, M.R., Szczerbicka, H., Analysis of Simulation Models with Fuzzy
Graph based Metamodeling, Performance Evaluation, Vol. 27&28, North-Holland,
October 1996.
Hwang, C.-L., Masud, A.S.Md., Multiple Objective Decision Making Methods and
Applications (Lecture Notes in Economics and Mathematical Systems; 164), Springer
Verlag, Berlin, Heidelberg, New York, 1979.
Hwang, C.-L., Yoon, K., Multiple Attribute Decision-Making Methods and
Applications (Lecture Notes in Economics and Mathematical Systems; 186), Springer
Jones, B., MATLAB
- Statistics Toolbox (Users Guide), The MathWorks, Inc., 1997.

Kaufman, M., et al., Variable-Complexity Response Surface Approximations for Wing
Structural Weight in HSCT Design, 34
th
Aerospace Sciences Meeting & Exhibit,
Reno, NV, January, 1996, AIAA-96-0089.
Khalessi, M.R., Lin, H.-Z., Most-Probable-Point-Locus Structural Reliability Method,
Proceedings of 34
th
Structures, Structural Dynamics, and Materials Conference, April,
1993, AIAA-93-1439.
Kleijnen, J.P.C., Statistical Techniques in Simulation, Part I, Marcel Dekker, Inc., New
York, 1974.
Kleijnen, J.P.C., Statistical Tools for Simulation Practitioners, Marcel Dekker, Inc., New
York, 1987.
Mavris, D.N., Bandte, O., Effect of Mission Requirements on the Economic Robustness of
an HSCT, 18
th
Annual Conference of the International Society of Parametric
Analysts, Cannes, France, June, 1996.
Mavris, D.N., Bandte, O., A Probabilistic Approach to Multivariate Constrained Robust
Design Simulation, World Aviation Congress and Exposition, Anaheim, CA, October,
1997, SAE-97-5508.
Mavris, D.N., Bandte, O., Comparison of Two Probabilistic Techniques for the
Assessment of Economic Uncertainty, Joint Cost Management Proceedings, AACE
International, 1998.
Mavris, D.N., Bandte, O., DeLaurentis, D.A., Robust Design Simulation: A Probabilistic
Approach to Multidisciplinary Design, Journal of Aircraft, Vol. 36, No. 1, January
February, 1999, pp. 298-307
Mavris, D.N., Bandte, O., Schrage, D.P., Economic Uncertainty Assessment of an HSCT
Using a Combined Design of Experiments/ Monte-Carlo Simulation Approach, 17
th
Annual Conference of the International Society of Parametric Analysts, San Diego,
CA, June, 1995.
Mavris, D.N., Bandte, O., Schrage, D., Application of Probabilistic Methods for the
Determination of an Economically Robust HSCT Configuration, AIAA/USAF/
NASA/ISSMO Multidisciplinary Analysis and Optimization Conference, Bellevue,
WA, September, 1996, AIAA-96-4090.
Mavris, D.N., DeLaurentis, D.A., A Stochastic Design Approach for Aircraft
Affordability, 21
st
Congress of the International Council on the Aeronautical Sciences,
Melbourne, Australia, September, 1998, ICAS-98-6.1.3.
Mavris, D.N., Kirby, M.R., Forecasting the Impact of Technology Infusion on Subsonic
Transport Affordability, World Aviation Congress and Exposition, September, 1998,
SAE-98-5576.
Mavris, D.N., Kirby, M.R., Qiu, S., Technology Impact Forecasting for a High Speed
Civil Transport, World Aviation Congress and Exposition, September, 1998,
SAE-985547.
McCullers, L.A., Flight Optimization System, Computer Program and Users Guide,
Version 5.7, NASA Langley Research Center, Hampton, VA, December 1994.
Nas, T.F., Cost Benefit Analysis Theory and Application, SAGE Publications, Thousand
Oaks, CA, 1996.
Ostrofsky, B., Design, Planning, and Development Methodology, Prentice-Hall,
Englewood Cliffs, NJ, 1977.
Osyczka, A., Multicriterion Optimization in Engineering with FORTRAN Programs,
Ellis Horwood Limited, Chichester, 1984.
Oxford Atlas of the World, 7
th
Edition, Oxford University Press, New York, 1999.
Raymer, D.P., Aircraft Design: a Conceptual Approach, 2
nd
Edition, American Institute
of Aeronautics and Astronautics, Inc., Washington, DC, 1992.
Reklaitis, G.V., Ravindran, A., Ragsdell, K.M., Engineering Optimization Methods and
Applications, John Wiley & Sons, Inc., New York, 1983.
Robinson, D.G., CRAX/Cassandra User Manual, Sandia Report SAND99-3162, Sandia
National Laboratories, 1999.
Ross, S.M., Introduction to Probability Models, Academic Press, Inc., San Diego, CA,
1993.
SAE, G-11 Probabilistic Methods Committee, Perceptions and Limitations Inhibiting the
Application of Probabilistic Methods, SAE Standard AIR5086, SAE, Warrendale,
1998.
Saaty, T.L., The Analytic Hierarchy Process: Planning, Priority Setting, Resource
Allocation, McGraw-Hill, New York, 1980.
Sacks, J., Schiller, S.B., Welch, W.J., Design for Computer Experiments, Technometrics,
Vol. 31, No. 1, February, 1989.
Sacks, J., Welch, W.J., Mitchell, T.J., Wynn, H.P., Design and Analysis of Computer
Experiments, Statistical Science, Vol. 4, No. 4, February, 1989.
Sakata, I.F., Davis, G.W., Evaluation of Structural Design Concepts for Arrow-Wing
Supersonic Cruise Aircraft, NASA CR-2667, May, 1977.
Sen, P., Yang, J.-B., Multiple Criteria Decision Support in Engineering Design, Springer
Verlag, Berlin, Heidelberg, New York, London, 1998
Shooman, M.L., Probabilistic Reliability; An Engineering Approach, McGraw-Hill, New
York, 1968.
Siddall, J.N., Optimal Engineering Design, Marcel Dekker, Inc., New York and Basel,
1982.
Silver, G.A., Silver, J.B., Introduction to Systems Analysis, Prentice-Hall, Inc.,
Englewood Cliffs, NJ, 1976.
Southwest Research Institute, FPI Users and Theoretical Manual, San Antonio, TX,
1995.
Stadler, W., Caveats and Boons of Multicriteria Optimization, Microcomputers in Civil
Engineering, Vol. 10, Blackwell, Cambridge, MA, 1995, pp. 291-299.
Stadler, W., Dauer, J., Multicriteria Optimization in Engineering: A Tutorial and Survey,
Structural Optimization: Status and Promise (ed. Kamat, M.P.), Vol 150, Progress in
Astronautics and Aeronautics, AIAA, Washington, DC, 1992, pp. 209-244.
Starr, M.K., Product Design and Decision Theory, Prentice Hall, New York, 1963.
Steuer, R.E., Multiple Criteria Optimization: Theory, Computation, and Application,
John Wiley & Sons, New York, 1986.
Stolarski, T.A., Analytical Design Techniques, Elsevier Science BV, Amsterdam, 1995.
Strang, G., Introduction to Linear Algebra, Wellesley-Cambridge Press, Wellesley, MA,
1993.
Tai, J.C.M., A Multidisciplinary Design Approach to Size Stopped Rotor/Wing
Configurations Using Reaction Drive and Circulation Control, Ph.D. Thesis, Georgia
Institute of Technology, 1998.
Tong, Y.L., The Multivariate Normal Distribution, Springer Series in Statistics, Springer
Unal, R., Stanley, D.O., Joyner, C.R., Parameter Model Building and Design
Optimization Using Response Surface Methods, Journal of Parametrics, Washington,
DC, May, 1994.
Vanderplaats, G.N., Numerical Optimization Techniques for Engineering Design, 3
rd
Edition, Vanderplaats, Research & Development, Inc., 1999.
Websters New World Dictionary of American English, 3
rd
College Edition, Prentice
Hall, New York, 1996.
Welch, W.J., et al., Screening, Predicting, and Computer Experiments, Technometrics,
Vol. 34, No. 1, February, 1992.
Wu, Y.-T., Burnside, O.H., Dominguez, J., Efficient Probabilistic Fracture Mechanics
Analysis, 4
th
International Conference on Numerical Methods in Fracture Mechanics,
Pineridge Press, Swansea, U.K., May, 1987.
Wu, Y.-T., Millwater, H.R., Cruse, T.A., Advanced Probabilistic Structural Analysis
Method for Implicit Performance Functions, AIAA Journal, Vol.28, No.3, September
1990, pp.1663-1669.
Zeleny, M., Multiple Criteria Decision Making, McGraw-Hill, Inc., New York, 1982.
Oliver Bandte Vita 220
VITA
Oliver Bandte was born in Hamburg, Germany on January 24
th
, 1970. He lived in
Hamburg till after graduation from Gymnasium Meiendorf (high school) in 1989. He
subsequently moved to Braunschweig, Germany, where he completed a degree in
mechanical engineering (Diplom-Ingenieur) received from the Technische Universitt
Braunschweig in 1994. Since then he earned a Master of Science degree (no designation)
from the Georgia Institute of Technology (1996).

Aircraft Design Thesis

Transféré par

Informations du document

Description originale:

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Aircraft Design Thesis

Transféré par

Droits d'auteur :

Formats disponibles

A Probabilistic Multi-Criteria Decision Making

Technique for Conceptual and Preliminary

.[Branch, Grace, 1996] The optimization problem is formulated

and well documented in the

, SPSS, etc. Nonetheless, there

- Optimization Toolbox (Users Guide), The

- Statistics Toolbox (Users Guide), The MathWorks, Inc., 1997.

Vous aimerez peut-être aussi