The Kriging Wizard

An internal code (Excel Add-In) for experimental design,

response-surface fitting, variable screening & robust design

Don Jones
GMU class on CAE-Based Robust Design

Global

Performance Integration

General Motors Corp. For internal GM use - Not to be duplicated or disclosed outside GM without approval of the Global Performance Integration
Director.

06/15/15
.

Why fit response surfaces to CAE simulations?

Gain insight

Explore design space globally

Identify important variables
Visualize shape of input-output relationships
Quantify tradeoffs

Optimization
Do quick what-ifs
Make optimization possible on long-running CAE simulations
Facilitates multi-disciplinary balancing since each discipline has bottled
up version of analysis and can react real-time to change proposals

Robust design
Adjust control parameters to desensitize design to variation, shift mean
Explore benefits of tolerance reductions
2

Why use kriging instead of standard methods?

Standard response-surface methods fit model by least squares
Polynomial regression
Fractional factorials, orthogonal arrays, Taguchi designs + ANOVA
Neural networks (back propagation = steepest descent on squared error)

Two reasons to use kriging for computer experiments

Standard methods were developed for physical experiments, which have
very different characteristics and challenges than computer experiments.
Least-squares methods have a little-discussed theoretical assumption that
is patently false in the case of computer experiments.

Characteristics and key challenges of experiments

Physical

Computer

Typically have few variables

(1-10)

Typically have many variables

(10-100)

Variables have only a few levels

Variables usually are continuous

Input-output relationships are
nonlinear, a few levels wont do

(usually 2 or 3)
Can sometimes sample all or
nearly all combinations.
Key challenge is to separate the
effect of the control variables
from noise variables

No way to cover the entire

design space.
Key challenge is to know if you
have enough points to capture
nonlinearity, interactions, etc.,
and can therefore draw valid
conclusions
4

Kriging versus Least-Squares Regression

Kriging and
regression
are actually
sub-cases of
the same
general
statistical
model.
They differ
in how they
treat the
deterministic
part of the
model and
the error
terms.

yi polynomial ( xi1 , xi 2 ,..., xik ) i

DETERMINISTIC PART:

ERROR TERM:

Regression:
Try to capture as much as possible
with many polynomial terms, selected
via stepwise procedure from large
set.

Regression:
Errors random, independent of x,
unpredictable. Can think of as
measurement error.

Kriging:
Dont spend much time guessing
functional form. Use either constant or
linear model.

yi polynomial ( xi1 , xi 2 ,..., xik ) i

Kriging
Errors are treated as left out terms in
x that cannot be precisely predicted
but can be roughly estimated

Practical advantages of kriging for CAE

Captures nonlinearity without the need to assume functional form
If simulation is very nonlinear and if a sufficient number of points are
sampled kriging will be more accurate than regression
Interpolates sampled points
At the sampled points, predicts the sampled value, as it should

Provides confidence intervals with desirable properties

intervals are bigger in the gaps between sampled points
intervals shrink to zero at the sampled points

Provides way to infer if we have enough points

All this will become clear as we go along
6

When do polynomials fit poorly?

14

Polynomial response surfaces

may not fit well if the surface
has sweet spot, asymptotes,
or similar nonlinear shapes.

Hypothetical CAE simulation

Best fitting cubic polynomial

12
10

Kriging can pick up these

shapes from data if one has
enough points because it is
not constrained by an
assumed form.

8
y
6
4
2
0

5
x

10

Kriging is named after a

South African mining engineer
called Krige, who used it to
model the underground
concentration of minerals
based on a few core samples.

Constructing a kriging surface

14

To understand kriging,
lets go back to the
example from the last slide
and imagine that we have
run the simulation at the 6
points shown.

Hypothetical CAE simulation

Sampled Points

12
10
8
y
6
4
2
0

5
x

10

Constructing a kriging surface

14

Of course, we will not see

the true function, but will
only see the results at the
six sampled points.

12
10
8
y
6
4
2
0

5
x

10

Constructing a kriging surface

14

For computer simulations,

most people expect that a
good approximation should

Interpolator 1
Interpolator 2
Interpolator 3
Sampled Points

12
10

(1) go through the

sampled points
(interpolate)

8
y
6

(2) Be continuous and

reasonably smooth.

4
2
0

5
x

10

Clearly this leaves lots of room

for different interpolators (three
of them shown).

10

Constructing a kriging surface

14

Kriging can be interpreted

as a statistically-based
way to generate many
such interpolators.

30 Possible Interpolators
Sampled Points

12
10
8

Here we show 30.

The mechanics of how this

is done are somewhat
complicated, but the
concept is simple.

y
4
2
0

5
x

10

11

Constructing a kriging surface

30 Possible Interpolators
Kriging surface = Mean of Family of Interpolators
Sampled Points
14

The value of the kriging

surface at a given point is
the mean of the values for
all the possible
interpolators.

12
10
8
y

Actually, one doesnt have

to generate many
interpolators and average
them, because the result of
this computation has a
closed form expression.

6
4
2
0

5
x

10

12

Kriging confidence intervals

30 Possible Interpolators
Kriging surface = Mean of Interpolators
99% Confidence Interval = Mean +/- 2.58 StdDev
Sampled Points
14
12
10
8
y
6
4
2
0

5
x

10

At any point, we can not

only compute the mean of
the possible interpolators,
but also compute the
standard deviation. From
this we can compute upper
and lower confidence
intervals.
The plot shows a 99%
confidence interval. If we
run the simulation at any x
value, we can be 99%
confident that the y value
will be in the confidence
interval.

13

Cross-validating the surface

Kriging surface fit with 4th point left out
Lower 99%
Prediction
Upper 99%
Sampled Points

To check if the confidence

intervals can be trusted, we
can refit the surface with one
of the points left outhere
the fourth point.

14

If the surface is working, the

value of the left-out sampled
point should lie within the
confidence interval.

12
10
8
y

6
4
2
0
-2

5
x

10

One can do this for each of

the sampled points. If we are
using a 99% confidence
interval, the true value should
be in the confidence interval
99% of time. The available
software automates this
check.

14

Effect of adding more points

Because kriging interpolates, as
you add points the surface will
begin to snap to the function.
If the surface doesnt change
much when more points are
added, you can assume that you
have converged.

Idea due to Jian Tu, GM R&D

15

Sequenced sampling
First 25 points

First 50 points

Checking for convergence

requires a sequenced
sample, that is, the points
should gradually fill in the
space as they are added.
First 75 points

All 100 points.

The available Kriging

Wizard software provides
such sequenced DOEs.

16

ANOVA functional decomposition

n

i 1

f ( x1 , x 2 ,..., x n ) g i ( xi ) hij ( xi , x j )
i 1

i 2 j 1

A function of n variables is decomposed into the sum of

a constant
functions of
functions of
functions of

(overall mean)
one variable
two variables
three variables

(main effects)
(two-way interactions)
(three-way interactions)

Interpreting the results

Main effects provide way to visualize effect of individual variables
Can assess importance of a main effect or two-way interaction by computing how
much squared error goes down when add that term
If no interactions are important, can optimize one variable at a time and essentially
read the best design from the main effect plots
If a two-way interaction exists between a control and noise variable, you may be able
to adjust control to desensitize design to the noise factor.
Computing decomposition requires performing multidimensional integrals, but for a
kriging surface these reduce to manageable one-dimensional integrals!
17

Example:

Kriging captures nonlinearity

600

Sample data
from a recent
N&V study.

550
500
450

SEATZ
Kriging Surface

400

There was a
clear sweet
spot with
respect to
one of the
mount rates.

350
300
300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500

Transmission Mount Rate, Z Direction (N/mm)

18

Example:

Kriging captures nonlinearity

600

A four-level
design would
have entirely
missed the
sweet spot.

550
500
450
400
350
300
300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500

Transmission Mount Rate, Z Direction (N/mm)

19

Example:

Kriging captures nonlinearity

Fitted polynomials would also
have missed the sweet spot.

20

Example:

Percent contributions & convergence

The available Kriging Wizard software computes
the ANOVA percent contributions.
It then sees how the percent contributions of the
top 10 effects would have been different if one had
used fewer points. If the percent contributions
have stopped changing a lot, one can assume the
surface has converged.

21

Testimonial
Don,
As you know I was involved for some in the GMT 900 pick-up frame development. The issue designing frame and
isolator tuning package to reduce shake.
Task was formulated as optimization problem, with auxiliary model used to modify frame structure and. I ended up
with problem with 52 structural and 15 isolator tuning variables and 6 system model runs to evaluate objective
function and constraints. To solve it using iSIGHT for optimization would've required about 2 days just for one set
of constraints, without any trade-off studies. So I decided to use DOE in combination with kriging response surface
and DIRECT optimization tool in Excel.
A DOE of 1200 points was executed and data used to fit response surface in Excel. Spreadsheet for optimization
based on kriging surface was prepared and combined with another one to calculate frame section as combination of
auxiliary model and underlying baseline (produced by Joe Wong and Marv Zurek). As a result, almost instant
design optimization became possible.
Various combinations of packaging, manufacturing and strength constraints were evaluated during meeting between
supplier, DRE and Development Engineer. At the end, I didn't even have to be present at those meetings! Your
vision and tools you developed made it possible to solve complex trade-off problems in real time by people without
in-depth background in optimization. Even though work still continues, I think impact on the program was quite
significant and proof-of-concept frame will be built by mid July to confirm analytical studies.
Mikhail
22

Demo of Available Software:

Kriging Wizard

In the presentation, we will do an interactive demo.

Here, in the notes, we will do a quick tutorial based on screen shots.
These shots are from version 60 but we are now up to version 101. However,
the basic operation is essentially the same.

Global

Performance Integration

General Motors Corp. For internal GM use - Not to be duplicated or disclosed outside GM without approval of the Global Performance Integration
Director.

06/15/15
.

Kriging Wizard Excel Add-In:

Installation
You can find the Kriging Wizard site by searching on
Kriging Wizard in Socrates. You will get links to Kriging
Wizard the on the homepages of the Robust Synthesis and
Analysis group and the Optimization group.
Go to the Kriging Wizard homepage and load the software
using the instructions given on the website.

24

Kriging Wizard Excel Add-In:

Launch the wizard on the demo

Open the file

c:\kriging\kwdemo.xls
Switch to the DOE tab
in the spreadsheet.
Launch the Kriging
Wizard by clicking
Tools and then
Kriging Beta

25

Kriging Wizard Excel Add-In:

Make an experimental design (DOE)

Switch to the DOE tab of
the Wizard.
Fill in the fields as shown
and click Write Design to
Worksheet.
The tool is pretty self
explanatory and explains
what data is expected in
each field.
Where it says select type
of DOE, pick StrengthTwo Orthogonal Array if
you can afford to do the
suggested number of
points. Otherwise, choose
Best DOE for userspecified number of
points.
26

Kriging Wizard Excel Add-In:

Make an experimental design (DOE)

The resulting DOE will be

written to the worksheet
Assignment 1.

27

Kriging Wizard Excel Add-In:

Fitting a surface

Switch to worksheet Small Data.

Select the data as shown on the left.
The first row indicates what data is
in the column. SKIP means ignore
the column. X means the column
has an independent (x) variable.
Y(interp) means the column has a
output (y) variable that you want to
interpolate.
The next row has the names of the
variables (header row).
The remain rows have the data.

28

Kriging Wizard Excel Add-In:

Fitting a surface
Click Tools, Kriging
Beta to launch
the Wizard.
The tool should
auto recognize the
XY data row with
header and the
Variable Type row.
Enter Small Data
in the field for
Enter prefix for
output worksheets.
Click Fit Surface.
After fitting is done,
click OK and exit
the Wizard.

29

Kriging Wizard Excel Add-In:

Fitting a surface
When fitting is
complete, two new
worksheets will
have been made:
Small Data SURF
Small Data CV
The former has all
the data needed to
make predictions
with the surface.
The latter shows
the cross-validation
(CV) results, which
give you an idea of
the accuracy of the
surface.

30

Kriging Wizard Excel Add-In:

Predicting outputs at new points

Switch to the
worksheet New
Points and
Predictions.
Launch the Kriging
Wizard and switch
to the Predict tab.
Fill in the Predict
tab as shown and
click Predict.
The Wizard will
insert predictions,
with confidence
intervals, for your
new points.

31

Kriging Wizard Excel Add-In:

Inserting functions in a worksheet

Switch to the worksheet
New Points and
Predictions.
Launch the Kriging
Wizard, switch to the
Insert Function tab, fill in
the information as shown,
and click Insert Function.
A function will be inserted
into your worksheet that
does the prediction.
If you type in new x
values, the prediction will
change. You can now
apply a spreadsheet
optimizer to search for
good/robust designs.
32

Kriging Wizard Excel Add-In:

Performing ANOVA
Launch the Kriging Wizard, switch
to the ANOVA tab, fill in the
information as shown, and click
Do ANOVA. Switch to worksheet
Small Data ANOVA [1] and
inspect the results.

33

Kriging Wizard Excel Add-In:

Performing Monte Carlo (clouds)

Switch to the Monte Carlo
worksheet.
Launch the Kriging Wizard,
switch to the Monte Carlo
tab, fill in the information as
shown, and click Create
Cloud, and then exit the
wizard.
A cloud of points will be
created. If you change the
mean or standard deviation
of a variable, the cloud will
adjust.
Predict outputs at the cloud
points using Insert Function
to see how this input
variation translates into
output variation.
34