Measuring System Analysis

Case Study for the Automotive

Industry
Alberto A. Yez-Moreno
TMAC/UTA
ASQ Houston 2013

Learning Objectives

Understand why we measure

MSA basic concepts
Components of measurement errors
Understand how to conduct a MSA
Data analysis theory
Automotive case studies

Measurement Systems
Measurement systems are like eyeglasses, when the
lenses are incorrect, the vision is blurred.
A measurement system allows us to see the process.
When a measurement system is poor, we lose the ability to
make good decisions about how to improve the process
In the Measure Phase of the DMAIC process, the MSA
should be conducted on the Y or KPOV
In the Control Phase of the DMAIC process, the MSA
should be conducted on the critical few Xs or KPIV

Basic Concepts
Every process produces a product or service
Every product or service possesses requirements
Every requirement can be measured
The total observed variation is equal to the real product
variation plus the variation due to the measurement system
Note: We want most of the variation coming from the
Product/Part and very little coming from the Measuring
System
2
2
2
Total
= Process/ + Measurement System
Part/
Service

Why Worry About Measurement

Variation?
Consider the reasons why we measure:
Verify product/process conformity to
specifications

How might measurement

variation affect these decisions?

Assist in continuous improvement

activities

What if the amount of

measurement variation is
unknown?
Process

Process

Measurement
Measurement

Measurement variation can make our process

capabilities appear worse/better than they are
5

Components of Measurement Error

The sources of variability for the measurement system
containing continuous data are:

Measurement Properties to Study

The two most common key measures associated with a
measurement system are accuracy and precision
Accuracy and precision are different, independent
properties. You may encounter a data set that is accurate,
yet not precise or precise, yet inaccurate
Sometimes you may encounter a data set that is neither
accurate nor precise. Obviously, we desire to have data
that exhibits both properties

Measurement Properties to Study

Not only do we want our measurement systems to be
accurate and precise, but
We want the measurement system to be able to detect small
changes to the process (good discrimination)
When applied to the same items of interest, the measurement
system should produce the same results in the future that it did in
the past (stability)
We want the system to be linear. Linearity concerns the behavior
of the measurement process across a wide spectrum of
applications

Accuracy vs. Precision Illustrated

Desired
Current

LSL

Target

USL

Accuracy
Current

Desired

LSL
9

Target

USL

Precision

Accuracy and Bias

Accuracy is the extent to which the average of the
measurements deviate from the true value. In simple terms, it
addresses the quesEon, On average, do I get the right
answer? If the answer is yes, then we say that the
measurement system is accurate. If the answer is no, then we
have an inaccurate measurement system
Bias is the term given to the distance between the observed
average measurement and the true value, or right answer

10

Accurate vs. Inaccurate

True
Value

True
Value

Bias

Mean
11

Mean

Potential Bias Problems

Average of measurements are different by a fixed
amount. Consider the manufacture of first-article. Bias
effects include:
Operator Bias Different operators get detectable different
averages for the same value
Instrument Bias Different instruments get detectable different
averages for the same measurement
Others Day-to-day (environment), fixtures, customer and
supplier (sites)

Instrument 1
Instrument 2

12

Precision
In addition to accuracy, it is important for a measurement
system to be precise
Precision
True
Value

Bias
Mean
13

Precision is the extent to

which we are able to get
the same data values
when independent
measurements are made
on the same entity
Using the mean of
repeated measurements
improves precision
because the dispersion of
the averages is always
less than the dispersion of
the individual data points

Discrimination
Discrimination is the capability of detecting small changes in the characteristic
The instrument may not be appropriate to identify process variation or quantify
individual part characteristic values if the discrimination is unacceptable
If an instrument does not allow differentiation between common variation in the
process and special cause variation, it is unsatisfactory
A common cause to MSA failure can be attributed to rounding up or down
measurements
.28
.279
.2794

14

.28
.282
.2822

Ruler
Caliper
Micrometer

.28
.282
.2819

.28
.279
.2791

Stability
If measurements do not change or drift over time, the
instrument is considered to be stable

Time One

Time Two

15

Linearity
A measure of the difference in bias (or offset) over the range of
the sample characteristic the instrument is expected to see
determines linearity. If the bias is constant over the range of
measurements, then linearity is good
Over what range of values for a given characteristic can the
device be used?
When the measurement equipment is used to measure a wide range of values, linearity is a
concern

Measurement
Variation

Low
End
16

Measurement Scale

High
End

Basic Model of Precision

2

2
2
Measurement = Repeatability

+ Reproducibility
System

2
MS

2
rpt

2
rpd

The Measurement System Variation is equal to the variation due to

Repeatability plus the variation due to the Reproducibility and represents
the common cause variation in the Measurement System

17

Repeatability (Gage precision)

Repeatability is the inherent variability of the
measurement system. Used as an estimate of short term
variation. It is the variation that occurs when successive
measurements are made under the same conditions:
Same part
Same characteristic
Same person
Same instrument
Same set-up
Same environmental conditions

Repeatability: It takes one to repeat

18

Reproducibility (operators precision)

Reproducibility is the variation in the average of the
measurements made by different operators using the
same measuring instrument when measuring the identical
characteristic on the same part
Operator A

= +
2

rpd
Operator B

Operator C

Reproducibility: It takes two to reproduce

19

% Repeatability and Reproducibility

Observed Part/Process Variation

%R&R = 20%

Measurement
System Variation

%R&R = 75%

%R&R = 100%

20

Rule of Thumb:
We want
%R&R < 30%

Capability of the Measurement

System
Once precision has been quantified, we now need to
assess if the measurement system is precise enough. To
do this we must compare the measurement variation to
the production process. There are many capability indices
but we will be focusing on the following two indices:
% R&R
Discrimination Index

21

Gage R&R: % R&R

MS
%R & R =
100
Total

Usually expressed
as percent

Assesses what percent of the Total Variation is taken up by

measurement error
Includes both repeatability and reproducibility
Care must be taken to use samples representing full, but typical,
process variation

22

Acceptability Summary

% Contribution

% Study
Variation

% Tolerance

10%

30%

30%

1%

10%

10%

10

Process
Control

Product
Control

Desirable to Have All 4 Indicators Say Go

23

Number of
Distinct
Categories

Gage R&R
The objective of a Gage R&R study is to learn as much as possible
about the measurement process in a short amount of time
The strategy is to include equipment, operators, parts, and other factors
that will usually be elements of the measurement process
Make careful selection of parts (or samples) representing the entire
range of process variation. (Good and Bad to the entire specification)
The parts should be labeled in such a way to preclude operator
identification and therefore remove possible operator bias
Each part will then be measured multiple times in random sequence by
each operator using the same equipment. This can be replicated for
each equipment set
Verify that an on-going calibration, maintenance, and metrology
program exists and is current on the measurement system
24

AIAG Gage R&R Standards

The Automotive Industry Action Group (AIAG) has two
recognized MSA standards for Gage R&R:
Short Form Five parts measured two times by two
different operators
Long Form Ten parts measured three times each by three
different operators

For additional insights into Gage R&R, go to

www.aiag.org
Remember that the Measurement System is acceptable if
the Gage R&R variability is small compared to the
Process or Study Variation seen

25

Running the Gage R&R

Each sample should be measured 2 - 3 times by each operator
(2 Times is the Short Test)
Make sure the parts are marked for ease of data collection but
remain blind (unidentifiable) to the operators
Be there for the study. Watch for unplanned influences
Randomize the parts continuously during the study to preclude
operators influencing the test
The first time evaluating a given measurement process, let the
process run as it would normally run
Enter data in appropriate software like: Minitab, QI Macros,
SigmaXL, Engine Room, Jump, etc.

26

Minitab Gage R&R Sixpack

Lets look at these six
charts one at a time

27

Gage R&R Relationships

A measurement process is said to be consistent when the results for
operators are Repeatable and the results between operators are
Reproducible
A gage is valid to detect part-to-part variation when the variability of operator
measurements is small relative to process variability or the tolerance range
The percent of process variation consumed by the measurement (% R&R) is
then determined once the measurement process is consistent and can detect
part-to-part variation

28

2. R Chart by Operator
Repeatability is checked by using a special Range Chart where the
differences in the measurements by each operator on each part is
charted. If the difference between the largest value of a measured part
and the smallest value of the same part does not exceed the UCL,
then that gage and operator are considered to be Repeatable

Repeatability is indicated when virtually all of the range points lie

under the upper control limit on the range chart. Any points that
fall above the limit need to be investigated
29

3. Xbar Chart by Operator

It is desirable to see plots that consistently go outside the UCL and LCL because
limits are determined by gage variance and these plots should show that gage
variance is much smaller than variability between the parts
If the samples chosen do not represent the total variability of the process, the
gage (repeatability) variance may be larger than the part variance and invalidate
the distinct categories calculation
If the patterns of the operators are not comparable, there may be significant
operator and part interactions (discussed on another slide)

30

On this chart you want At Least 50% of the points to be

Outside the Control Limits

4. Response by Part

This graph shows the data for all ten parts for all operators plotted
together. It should show plots that vary from the smallest dimensions
for the parts made by the process to the largest dimensions for the
same parts. Parts should be both in tolerance and out of tolerance if the
process makes them
If a part shows a large spread, it may be a poor candidate for the test
because the feature may not be clear on that part
31

5. Response by Operator

This graph shows the data for all ten parts for plotted by each operator.
The line connecting the averages (shown as the circle with crosshairs)
of all 10 parts measured by each operator should be horizontal
Any significant slope is an indication that this operator has a general
bias to measure large or small when compared to the other operators
32

6. Operator * Part Interaction

Operator Influence: If the lines connecting the plotted
average points diverge significantly, then there is a
relationship between the operator making the
measurements and the part that the operator is measuring.
This is not good and needs to be investigated

33

Minitab Sixpack

34

Questions on the graphical output?

Gage R&R: The Numerical Output

For %Contribution the guidelines are different:

<1% : Acceptable
1 to 9% : Adequate
>=10% : Unacceptable
The first column represents the source of variation, the second column is an estimate
of the actual variation for each source (factor). The third column is the linear % that
each represents of the total variation. It is depicted as the red bar (left-most) on the
Pareto in the six-pack diagram
35

Gage R&R: The Numerical Output

These should all
be
Less Than 30%

This value should be

5 or More
This tabulation from Minitab builds the % of Study Variation that each source contributes to a calculated
potential Total Variation seen in the study. The 6.0 * SD is how statistically 99.73% of the Total Variation
is calculated and this is assumed to equal 99.73% of the true process variation unless the Historical
Sigma is input
The %s are used to grade the validity of the measurement system to perform measurement analysis
using %s already taught. If the process is performing well, the % Tolerance is then important. The sum
of the %s may add to more than 100% due to the math
The Number of Distinct Categories represents the number of non-overlapping measurement groups that
this measurement system can reliably distinguish in the Study Variation. We would like that number to be
5 or higher. Four is marginal. Fewer than 4 implies that the measurement system can only work with
attribute data
36

Case Study # 1
Selection of parts to be tested
10 parts randomly selected
Good, bad & average parts used

Instructions to operators
Perform tests as normal
Report variance

Tests conducted
10 parts tested
2 operators
3 checks on each of the 10 parts
given to the operator in random order

37

MSA Tree
MSA

Part 1

Part 2

Part 3

Operator
1

Repeat 1

38

Repeat 2

Part 4

Part 5

Part 6

Part 7

Operator
2

Operator
1

Repeat 3

Repeat 1

Part 8

Part 9

Operator
2

Repeat 2

Repeat 3

Part 10

MSA Gage R&R for Value

39

MSA Session - Gage R&R

<10%

<30%

>5
40

Case Study # 2
MSA Bolt Torque

41

Electronic Torque Wrench

Bolt Torque Wrench

42

Operator

Measuring System Analysis

Tree Structure
MSA

Part 1

Part 2

Part 3

Part 4

Part 5

Part 6

Part 7

Part 8

Part 9

Part
10

Operator
1

Sample
1

43

Part
11

Part
12

Operator
2

Sample
2

Sample
1

Sample
2

Part
13

Part
14

Part
15

Part
16

Minitab Data

44

Minitab Data Session Data

45

Minitab Data

46

Minitab Data

47

Minitab Data

48

Q & A Session