Multivariate Statistical Process

Control and Optimization

Alexey Pomerantsev & Oxana Rodionova
Semenov Institute of Chemical Physics
Russian Chemometrics Society
Chris Marks

10.02.04

Agenda
1. Introduction
2. SPC
3. MSPC
4. Passive optimization (E-MSPC)
5. Active optimization (MSPO)
6. Conclusions

10.02.04

Statistical Process Control (SPC)

SPC Objective
To monitor the performance of the process
SPC Concept
To study historical data representing good past process behaviour
SPC Method
Conventional statistical methods
SPC Approach
To plot univariate chart in order to monitor key process variables

10.02.04

Historical Process Data (Chemical

Reactor)
Production cycles s1, s2, ... ,s54

Key process variables (sensors) X1, X2, ... , X17

s1
s2
s3
s4
s5
s6
s7
s8
s9
s10
s11
s12
s13
s14
s15
s16
s17

s54

X1
-1.19E-01
-1.37E-01
2.51E-02
-1.14E-01
-7.93E-02
1.51E-02
7.44E-02
3.65E-02
1.36E-01
-2.74E-02
7.47E-02
-1.17E-01
1.06E-01
7.39E-02
-9.87E-03
-1.06E-01
-4.76E-02

X2
7.28E-01
7.28E-01
-9.15E-02
6.70E-01
4.14E-01
-6.38E-02
-5.24E-01
-2.66E-01
-7.06E-01
3.60E-01
-3.31E-01
7.02E-01
-2.82E-01
-5.28E-01
1.02E-01
7.68E-01
2.66E-01

X3
-2.15E-02
-2.89E-02
6.73E-03
-2.18E-02
-1.69E-02
3.74E-03
1.11E-02
5.12E-03
2.89E-02
1.82E-03
1.80E-02
-2.16E-02
3.23E-02
1.07E-02
-3.21E-04
-1.52E-02
-9.52E-03

X4
5.22E-01
6.08E-01
-1.13E-01
5.04E-01
3.51E-01
-6.75E-02
-3.24E-01
-1.59E-01
-6.01E-01
1.12E-01
-3.34E-01
5.13E-01
-4.82E-01
-3.21E-01
4.17E-02
4.62E-01
2.10E-01

X5
7.06E-04
7.09E-04
-9.07E-05
6.50E-04
4.04E-04
-6.28E-05
-5.06E-04
-2.56E-04
-6.88E-04
3.42E-04
-3.25E-04
6.81E-04
-2.85E-04
-5.09E-04
9.75E-05
7.41E-04
2.59E-04

X6
7.32E-01
7.02E-01
-7.58E-02
6.65E-01
3.98E-01
-5.67E-02
-5.45E-01
-2.78E-01
-6.77E-01
4.12E-01
-2.99E-01
7.03E-01
-1.87E-01
-5.50E-01
1.13E-01
8.03E-01
2.61E-01

X7
3.10E-04
6.58E-04
-2.29E-04
3.83E-04
3.96E-04
-1.15E-04
-1.73E-05
1.43E-05
-6.83E-04
-4.31E-04
-5.30E-04
3.40E-04
-1.25E-03
2.49E-06
-8.29E-05
-2.54E-05
1.92E-04

X8
-6.13E-04
-1.22E-03
4.10E-04
-7.34E-04
-7.35E-04
2.07E-04
7.92E-05
-3.95E-07
1.26E-03
7.24E-04
9.62E-04
-6.63E-04
2.21E-03
4.48E-05
1.36E-04
-2.68E-05
-3.61E-04

X9
-5.92E-05
-1.49E-04
5.65E-05
-7.96E-05
-9.05E-05
2.78E-05
-1.07E-05
-1.14E-05
1.56E-04
1.22E-04
1.28E-04
-6.76E-05
3.14E-04
-1.59E-05
2.44E-05
2.88E-05
-4.19E-05

6.61E-02 -5.40E-01 7.19E-03 -2.85E-01 -5.19E-04 -5.78E-01 1.81E-04 -2.67E-04 -6.23E-05

10.02.04

X17
9.74E-03
1.01E-02
-1.43E-03
9.07E-03
5.78E-03
-9.49E-04
-6.79E-03
-3.42E-03
-9.86E-03
4.18E-03
-4.84E-03
9.44E-03
-4.99E-03
-6.81E-03
1.23E-03
9.90E-03
3.65E-03

-6.78E-03

Shewart Charts (1931)

1.2
0.5
1.5
X2
Control
Control
X2 Normal
X1 Control
Normal

0.2
0.5
0.4
0.1

X1 Normal
X1 Control
X1 Normal

0.0
X2 Normal

-0.1
-0.4
-0.5
-0.2

X1 Control
X2 Normal
Normal

-0.3
-1.0
-0.8
-0.4

X1 Control
X2 Normal
Control
X2 Control

s56

s54

s51

s49

s46

s43

s41

s39

s37

s35

s33

s31

s29

s27

s25

s23

s21

s19

s17

s15

s13

s11

s9

s7

s5

s3

-1.5
-0.5
-1.2
s1

Sensors
X1 All
&
X1X2
X2
Sensor
Sensors

0.4
0.8
1.0
0.3

Cycles (time)

10.02.04

Panel Process Control (just a game)

1

9 10 11 12 13 14 15 16 17

0.31 -0.4 0.26 -0.3 -0.3 -0.4 -0.1 0.13 0.08 -0.3 -0.3 #### -0.3 -0.3 #### -0.1 -0.4

3
2
0.26

0.13 0.08

0.01

-0.06

-0.09
-0.36

-0.31 -0.27

-0.37

-0.01
PC2

0.31

-0.35 -0.35

-0.28

-0.35

-0.38

0
-4

-3

-2

-1

0
-1
-2
-3
PC1

Off
Time till the end of shift:

10.02.04

Exit

On
7:59:10

Multivariate Statistical Process Control

(MSPC)
MSPC Objective
To monitor the performance of the process
MSPC Concept
To study historical data representing good past process behavior
MSPC Method
Projection methods of Multivariate Data Analysis (PCA, PCR, PLS)
MSPC Approach
To plot multivariate score plots to monitor the process behavior

10.02.04

Projection Methods

Initial Data
Data Plane
Data Center
PCs
Data Projections

10.02.04

Low Dimensional Presentation

10.02.04

MSPC Charts (Chemical Reactor)

Samples

Key Variables
3

Scores

0.6

PC2

Loadings

PC2

X13
2
s13
s51

-4

-3

-2

s23
s10

X16
X15
X12
X9
X8

s16

s1 s50
s27
s26 s12
s4
s40 s47
s43 s21 s2
s15
s11
s17s5
s41 s3s6
s20
s25
s39
0
s37 s48 s46 s33
s38
s45
s42 s35s44
s9
s34
s22
-1 s24s52 s8
0 s36
s28 1 s32s312
s29
s19
s7
s14
s18
-1 s53
s49
s30

PC1
3

0.3

X5

X4

X3
-0.4

X1

10.02.04

PC1

0
-0.2

-2

-3

X6
X2
X17
X14
X10
X11

0.2

0.4

X7
-0.3

10

Panel Process Control (not just a game)

1

9 10 11 12 13 14 15 16 17

0.31 -0.4 0.26 -0.3 -0.3 -0.4 -0.1 0.13 0.08 -0.3 -0.3 #### -0.3 -0.3 #### -0.1 -0.4

3
2
0.26

0.13 0.08

0.01

-0.06

-0.09
-0.36

-0.31 -0.27

-0.37

-0.01
PC2

0.31

-0.35 -0.35

-0.28

-0.35

-0.38

0
-4

-3

-2

-1

0
-1
-2
-3
PC1

Off
Time till the end of shift:

10.02.04

Exit

On
7:59:10

11

Cruise Ship Control (by Kim Esbensen)

10.02.04

12

Key Process Variables

10.02.04

13

PLS1 Prediction of Fuel Consumption

Predicted vs. Measured
3

Scores

PC2

Slope:
Offset:
Correl:
RMSEP:
SEP:
Bias:

0.95
0.02
0.98
0.23
0.24
-0.005

Predicted Fuel

Samples

PC1

Measured Fuel

0
-4

-3

-2

-1

0
0

-3

-2

-1

-1

-1

-2

-2

-3

-3

Weather conditions
X1, X2, X3, X4

10.02.04

Caps setup
X5, X6, X7

PLS1

Fuel Consumption
Y

14

Passive Optimization
Weather
conditions
Prediction ?

Prediction !

Order!!!
Fuck
censored

X1, X2, X3, X4

X5,X5,
X6,X6,
X7 X7

42
Captain

10.02.04

Computer

Student

15

Active Optimization
Weather
conditions
X5 X6, X7
Advice!!!

X1, X2, X3, X4

Censored
Order?

Optimal
X5, X6, X7

42
Student

10.02.04

Computer

Captain

16

In Hard Thinking about PC and PCs

Forty two
censored

10.02.04

17

Multivariate Statistical Process

Optimization (MSPO)
MSPO Objective
To optimize the performance of the process (product quality)
MSPO Concept
To study historical data representing good past process behavior
MSPO Methods
Projection methods and Simple Interval Calculation (SIC) method
MSPO Approach
To plot predicted quality at each process stage

10.02.04

18

Technological Scheme. Multistage Process

S
S1

S2

S3

W
W1

W2

M1

CW

WR1
WR2

W3

M2

A1 A2
A3 A4
A5 A6

CM

MR1
MR2

M3

CM1 CM2 CM3

CW1 CW2 CW3

II

III

11

10.02.04

IV

VI

VII

14

16

19

25

19

A6

A5

A4

A3

A2

A1

CM3

CM2

CM1

MR2

MR1

M3

M2

M1

CW3

CW2

CW1

WR2

WR1

W3

W2

W1

S3

S2

Training
Set (102)

XI

XII

XIII

XIV XV XVI

XVII

Test
Set (52)

S1

Historical Process Data

XI

XII

XIII

XIV XV XVI

XVII

X preprocessing

10.02.04

Y preprocessing

20

Quality Data (Standardized Y Set)

1.2

Training Set Samples

Highest Quality Y=+1

0.8
0.4
0.0
-0.4
-0.8

Lowest Quality Y=-1

-1.2
1
1.2

21

41

61

81

101

Test Set Samples

Highest Quality Y=+1

0.8
0.4
0.0
-0.4
-0.8

Lowest Quality Y=-1

-1.2
1

11

10.02.04

21

31

41

51

21

XTRAINING

PLS
6 PCs

XTEST
0.3

A6

A5

A4

A3

A2

A1

CM3

CM2

CM1

MR2

MR1

M3

M2

M1

CW3

CW2

CW1

WR2

WR1

W3

W2

W1

S3

S2

S1

General PLS Model

^
Y
^
Y

Calbration

RMSE

Validation
0.2

0.1

PCs
0
PC_0

10.02.04

PC_2

PC_4

PC_6

PC_8

22

SIC Prediction. All Test Samples

1.0

SIC

PLS1

SIC
SIC Prediction
Prediction

Test
16

0.5

22

23

19

6
1
0.0

10
5

2
4

-0.5

30
27

18

14

39

32

35

37

34

28

25

21

15

12

24

17

13

11

31

46

40
38

44

41
43

49
47

45

33

20

26

48

29
36

Test Samples

-1.0

52
50
51

42

1.0

Status
Status plot
plot
26
36

SIC-Residual

0.5

41
52
31
13
13

0.0
43

25

38

50

18
51
35 19
2827
34 10
37
39

1
6

75
15

2
4911
45

21
16
16

-0.5

22
312

14
33

4
29 23
42

24

30
17
4744

46 40

48 32
20

SIC-Leverage

-1.0

10.02.04

23

SIC Prediction. Selected Test Samples

SIC Prediction

Object Status plot

1.0

0.5

0.0

3
1

SIC-Residual

Outsiders
4

0.5

0.0

1.0

Outsiders

-0.5

-0.5

Insiders
5

4
-1.0

Selected Test Samples

Sample No

Quality status

-1.0

SIC-Leverage

SIC Status

Normal

Insider

High

Outsider

Normal

Absolute outsider

Low

Outsider

Normal

Insider

10.02.04

Abs. Outsiders

1.0

24

Passive Optimization in Practice

Objective
To predict future process output being in the middle of the process
Concept
To study historical data representing good past process behaviour
Method
Simple Interval Prediction
Approach
Expanding Multivariate Statistical Process Control (E-MSPC)

10.02.04

25

XI

XII

XIII

XIV XV XVI

xI

xII

xIII

xIV

1.0

Sample 1, Normal Quality Insider

A6

A5

A4

A3

A2

A1

CM3

CM2

CM1

MR2

MR1

M3

M2

M1

CW3

CW2

CW1

WR2

WR1

W3

W2

W1

S3

S2

Training
Set (102)

S1

Expanding MSPC, Sample 1

XVII

x
x

xV

VI

VII

SIC

PLS1

x1

0.5
0.0
-0.5
-1.0

10.02.04

26

Expanding MSPC , Samples 2 & 3

Sample 2, High Quality, Outsider
1.0

x2

SIC

PLS1

0.5
0.0
-0.5
-1.0
1.0

Sample 3, Normal Quality, Absolute Outsider

0.5

0.0

-0.5

-1.0

10.02.04

x3

SIC

PLS1

27

Expanding MSPC , Samples 4 & 5

1.0

Sample 4, Low Quality, Outsider

x4

SIC

PLS1

x5

SIC

PLS1

0.5

0.0

-0.5

-1.0
1.0

Sample 5, Normal Quality, Insider

0.5

0.0

-0.5

-1.0

10.02.04

28

Active Optimization in Practice

Objective
To find corrections for each process stage that improve the future
process output (product quality)
Concept
Corrections are admissible if they are similar to ones that
sometimes happened in the historical data in the similar situation
Method
Simple Interval Prediction and Status Classification
Approach
Multivariate Statistical Process Optimization (MSPO)

10.02.04

29

Linear Optimization
Linear function always reaches extremum at the border.
So, the main problem of linear optimization is not to find a
solution, but to restrict the area, where this solution should
be found.
y=a*x

10.02.04

30

Optimization Problem
Weather
Fixed variables
conditions
X1, X2, XfixX3, X4

Caps
Optimized
setup
X5, X6,
Xopt X7

PLS1

Fuel
Quality
Consumption
measure
YY

Model
Y = X*a = Y0 + Xopt*a2, where Y0 = Xfix*a1 = Const
Task
For given Xfix and a1 to find Xopt that maxi(mini)mizes Y
Solution
max (Y) = Y0 + max (Xopt)*a2, as all a > 0 (by factor)

10.02.04

31

Interval Prediction of Xopt

XI
Xopt

1.2
0.4

XII

XIII

M1

M3

M1

M2

X opt

CW3

CW2

CW1

WR2

WR1

W3

W2

W1

S3

S2

S1

X fix

XIV

PLS2

PLS
PLS2*RMSEP
SIC Prediction
Borders

0.3
0.8
0.2
0.4
0.1
0.0

1
1

2
2

3
3

44

5
55

-0.1
-0.4
-0.2
-0.8
-0.3
-1.2
-0.4
-1.2

10.02.04

Selected
Selected Test
Test Samples
Samples

32

Dubious Result of Optimization

1.5

Optimized

1.0
0.5
0.0
-0.5
SIC
Y
Y
x4 Test
Test
x4

-1.0
-1.5
I

II

III

IV

PLS
x4 Opt
x4 Opt
Limits

VI

VII

Predicted Xopt variables are out of model!

10.02.04

33

Adjustment with SIC Object Status

Concept
Corrections are admissible if they are similar to ones that sometimes
happened in the historical data in the similar situation.
Optimal variables Xopt should be within the model !
M1

SIC Prediction

0.2

0.0

0.5

Insiders
4

0.0

-0.5

-0.2

-0.4

Object Status plot

1.0

SIC-Residual

0.4

51
3

2 2 2

Selected Test Samples

10.02.04

-1.0

SIC-Leverage

34

Sample 1 Normal Quality Insider

Optimized

1.0

SIC

PLS

x1

0.5
0.0
-0.5
Object Status plot

-1.0

SIC-Residual

1.0

0.5

0.0

1.0

4
5

0.5

III

IV

VI

VII

1.0

-0.5

-1.0

II

SIC

PLS

x1

SIC-Leverage

0.0
-0.5
-1.0

Test

10.02.04

II

III

IV

VI

VII

35

Sample 2 High Quality Outsider

1.0

Optimized

0.5
0.0
-0.5
Object Status plot

1.0

SIC

PLS

x2

SIC-Residual

-1.0

0.5

0.0

4
5

II

III

IV

VI

VII

1.0
0.5

1.0

-0.5

-1.0

SIC-Leverage

0.0
-0.5

SIC

PLS

x2

-1.0

Test

10.02.04

II

III

IV

VI

VII

36

Sample 3 Normal Quality Abs. Outsider

Optimized

1.0
0.5
0.0
-0.5

Object Status plot

1.0

SIC

PLS

x3

SIC-Residual

-1.0
0.5

0.0

4
5

0.5

II

III

IV

VI

VII

1.0

-0.51.0

-1.0

SIC-Leverage

0.0
-0.5
-1.0

Test

10.02.04

II

III

IV

VI

SIC

PLS

x3
VII

37

Sample 4 Low Quality Outsider

1.0

Optimized

SIC

PLS

x4

0.5
0.0
-0.5
Object Status plot

-1.0

SIC-Residual

1.0

0.5

0.0

1.0

0.5

II

III

IV

VI

VII

1.0

-0.5

-1.0

SIC

PLS

x4

SIC-Leverage

0.0
-0.5
-1.0

Test

10.02.04

II

III

IV

VI

VII

38

Sample 5 Normal Quality Insider

1.0

Optimized

SIC

PLS

x5

0.5
0.0
-0.5
-1.0

Object Status plot

SIC-Residual

1.0

0.5

1.0

0.0

4
5

II

III

IV

VI

1.0

-0.5

VII
SIC

PLS

x5

0.5

-1.0

SIC-Leverage

0.0
-0.5
-1.0

Test

10.02.04

II

III

IV

VI

VII

39

Philosophy of MSPO. Food Industry

Food Quality

Home-made quality
Intuitive (expert)
control

Home-made quality
MSPO

Restaurant quality
Standard (descriptive)
control

Fast Food quality

ISO-9000

Production Effectiveness
10.02.04

40

Conclusions

Thanks and ...

Bon Appetite!
10.02.04

41