Measurement Systems Analysis For Variables Data:

Measurement Precision Calculator

Version 1.0, August 2002

Introduction
Welcome to the Measurement Precision Calculator. This workbook will help
perform the calculations for Measurement Precision Studies as described
in the IPC test method IPC-TM 1.9.
This software is useful in cases where the measurement data is recorded
as variables data and recoreded on a continuous scale.
Please refer to the test method before beginning the measurement precision study.
The test method defines the terms and equations used in detail.
Also refer to the User Guide for detailed instructions..
This workbook includes the following
- A blank spreadsheet to enter measurements
- A spreadsheet with example data
- A form for manual data entry, if desired.
- Further information on the K factors used in the calculations
The title cells on the spread sheet have been color coded to aid understanding.
The following is the color key:
Color

Usage
General title
Cells for user to enter data
Title cells for test condition related data
Title cells for data related to repeated readings
Title cells for sample related data
Title cell for calculated data

Study:___________________

Measurement Precision Study: Information Sheet

Version 1.0, August 2002
Enter data into yellow areas.
Test Method
Parameter Measured
Company
Name of Study Organizer
Study Completion Date
Instrument
Measurement Units
Lower Specification Limit, LSL
Upper Specification Limit, USL
Number of Test Conditions, m

K2

3.65

Number of Samples, n

K3

3.65

Number of Repeat Readings, k

K1

4.57

Entering Raw Data:

Raw measurement data is to be entered on the following area of this worksheet.
The sheet is set up for up to 10 different test conditions, up to 5 repeated measurements on up to 10 samples.
The actual number of conditions, measurements, and samplesshould be determined for each study.
Test conditions could include different operators, test equipment or test locations, etc.
The greater the number of conditions, measurements, and samples, the more meaningful the results.

Study:___________________

Measurement Precision Study: Data Entry Sheet

Condition
1

Reading
1
2
3
4
5

1
2
3
4
5

1
2
3
4
5

1
2
3
4
5

1
2
3
4
5

1
2
3
4
5

1
2
3
4
5

1
2
3
4
5

1
2
3
4
5

10

1
2
3
4
5

Sample
1

Sample
2

Sample
3

Sample
4

Sample
5

Sample
6

Sample
7

Sample
8

Sample Sample
9
10

Study:___________________

Measurement Precision Study: Calculation Sheet

Ranges:
Sample
Condition
1
2
3
4
5
6
7
8
9
10

10

Average of the range averages

Averages:
Condition
Readings
Set 1
Set 2
Set 3
Set 4
Set 5

10

Op Avg
Absolute difference of condition averages

Sample
Avg

10

Average of sample averages

Range of sample averages

Average
Cond
Ranges

Study:___________________
`

Measurement Precision Study: Scorecard

Variability Summary

Remarks

Repeatability
Reproducibility
Measurement R&r
Process std dev
Total Variability

Standard deviation of the within condition variation

Standard deviation of the between condition variation
Standard deviation of the measurement process
Standard deviation of the product variation
Standard deviation of the total observed variation

Product Average

Average of all product measurements taken during study

Comparison to Specifications
(Only used when both specifications exist)
% Repeatability
% Reproducibility
% Measurement R&r

Percent of specification width used by repeatability error

Percent of specification width used by reproducibility error
Percent of specification width used by measurement error
Compared to the specification limits, the precision of
this measurement:

Comparison to Total Variation

% Repeatability
% Reproducibility
% Measurement R&r

Percent of total variability used by repeatability error

Percent of total variability used by reproducibilityerror
Percent of total variability used by measurement error
Compared to the total variability, the precision of
this measurement:

Measurement Tolerance
Measurement Tolerance
+/-

Units
0 On the basis of this study, there is a 99% chance that
this interval around the measured product performance
will contain the actual product performance level.

Study:___________________

Measurement Precision Study: Information Sheet

Version 1.0, August 2002
Enter data into yellow areas.
Test Method
Parameter Measured

Gasket thickness

Company
Name of Study Organizer
Study Completion Date
Instrument
Measurement Units

mm

Lower Specification Limit, LSL

0.6

Upper Specification Limit, USL

Number of Test Conditions, m

K2

2.7

Number of Samples, n

10

K3

1.62

Number of Repeat Readings, k

K1

4.57

Entering Raw Data:

Raw measurement data is to be entered on the following area of this worksheet.
The sheet is set up for up to 10 different test conditions, up to 5 repeated measurements on up to 10 samples.
The actual number of conditions, measurements, and samplesshould be determined for each study.
Test conditions could include different operators, test equipment or test locations, etc.
The greater the number of conditions, measurements, and samples, the more meaningful the results.

Study:___________________

Measurement Precision Study: Data Entry Sheet

Sample
1
0.62
0.66

Sample
2
0.99
1

Sample
3
0.82
0.81

Sample
4
0.85
0.89

Sample
5
0.59
0.48

Sample
6
1.02
1.03

Sample
7
0.97
0.97

Sample
8
0.85
0.82

1
2
3
4
5

0.53
0.53

1.05
0.93

0.8
0.77

0.83
0.76

0.39
0.4

1.04
1.08

0.97
0.91

0.73
0.7

0.98
0.95

0.54
0.6

1
2
3
4
5

0.51
0.55

1.05
1.02

0.8
0.77

0.79
0.81

0.45
0.49

1.01
1.05

0.96
0.96

0.8
0.81

1.04
1.05

0.84
0.81

Condition
Reading
1
1
Station 1
2
3
4
5
2
Station 2

3
Station 3

1
2
3
4
5

1
2
3
4
5

1
2
3
4
5

1
2
3
4
5

1
2
3
4
5

1
2
3
4
5

10

1
2
3
4
5

Sample Sample
9
10
1
0.61
1
0.7

Study:___________________

Measurement Precision Study: Calculation Sheet

Ranges:
Sample
Condition
1
2
3
4
5
6
7
8
9
10

10

0.04
0
0.04

0.01
0.12
0.03

0.01
0.03
0.03

0.04
0.07
0.02

0.11
0.01
0.04

0.01
0.04
0.04

0
0.06
0

0.03
0.03
0.01

0
0.03
0.01

0.09
0.06
0.03

Average of the range averages

Averages:
Condition
1
Readings
Set 1
0.83200
Set 2
0.836
Set 3
Set 4
Set 5
0.834
Op Avg

0.786
0.763

0.825
0.832

0.7745

0.8285

10

Absolute difference of condition averages

Sample
Avg

1
2
0.56667 1.00667

3
0.795

4
5
6
7
0.82167 0.46667 1.03833 0.95667

Average
Condition
Ranges
0.034
0.045
0.025
0
0
0
0
0
0
0
0.03467

8
9
10
0.785
1.00333 0.68333
Average of sample averages
Range of sample averages

0.0595

0.81233
0.57167

Study:___________________
`

Measurement Precision Study: Scorecard

Variability Summary

Remarks

Repeatability
Reproducibility
Measurement R&r
Process std dev
Total Variability

0.03076
0.03038
0.04324
0.17983
0.18495

Standard deviation of the within condition variation

Standard deviation of the between condition variation
Standard deviation of the measurement process
Standard deviation of the product variation
Standard deviation of the total observed variation

Product Average

0.81233 Average of all product measurements taken during study

Comparison to Specifications
(Only used when both specifications exist)
% Repeatability
39.6067 Percent of specification width used by repeatability error
% Reproducibility
39.1143 Percent of specification width used by reproducibility error
% Measurement R&r
55.6652 Percent of specification width used by measurement error
Compared to the specification limits, the precision of
this measurement:
Needs improvement
Comparison to Total Variation
% Repeatability
% Reproducibility
% Measurement R&r

2.76653 Percent of total variability used by repeatability error

2.69817 Percent of total variability used by reproducibilityerror
5.46469 Percent of total variability used by measurement error
Compared to the total variability, the precision of
this measurement:
Is adequate

Measurement Tolerance
Measurement Tolerance
+/-

0.1112439379

Units
mm

On the basis of this study, there is a 99% chance that

this interval around the measured product performance
will contain the actual product performance level.

Measurement Precision Study - Information Sheet

Version1.0, August 2002
Test Method
Parameter Measured
Company
Name of Study Organizer
Study Completion Date
Instrument
Measurement Units
Lower Specification Limit, LSL
Upper Specification Limit, USL
Number of Conditions, m
Number of Samples, n
Number of Repeat Readings, k

Measurement Precision Study - Data Sheet

Condition
A
Reading
1
2
3
4
5
Average
Range
Condition
B
Reading
1
2
3
4
5
Average
Range
Condition
Reading
C
1
2
3
4
5
Average
Range
Condition
D
Reading
1
2
3
4
5
Average
Range
Condition
E
Reading
1
2
3
4
5
Average
Range

Sample
5
6

10

Average

XA=
R A=
1

Sample
5
6

10

Average

X B=
RB =
1

Sample
5
6

10

Average

XC=
RC =
1

Sample
5
6

10

Average

X D=
R D=
1

Sample
5
6

10

Average

X E=
R E=

Measurement Precision Study - Data Sheet (Cont)

Condition
F
Reading
1
2
3
4
5
Average
Range
Condition
G
Reading
1
2
3
4
5
Average
Range
Condition
Reading
H
1
2
3
4
5
Average
Range
Condition
I
Reading
1
2
3
4
5
Average
Range
Condition
J
Reading
1
2
3
4
5
Average
Range

Sample
5
6

10

Average

X F=
R F=
1

Sample
5
6

10

Average

XG=
RG=
1

Sample
5
6

10

Average

X H=
RH=
1

Sample
5
6

10

Average

X I=
RI=
1

Sample
5
6

10

Average

X J=
RJ =

Measurement Precision Study - Calculation Sheet

Sample Data

Sample

Average part
average

10

Part
average

X=

Condition Data
A
Condition

Max Part Avg

X Max =

Min Part Avg

Range of part
averages

X Min=

Average

R P=
Average Condition
Range

R=

Average
Range
Max Condition
Avg

X Max =

Min Condition
Avg

X Min=

Range of
Condition Avgs

RX =

Variability Calculations
Topic
Repeatability

Reproducibility
Total R&r
Poduct Variation
Total Variation
K Factors

Equation

Calculation

Answer

S r =RK 1 /5.15

S R=

Sr=

28 . 1*S2r

( R XK 2 )2 n*k

S R=

5. 15

S R&r = S 2r +S2R

S R&r =

S P=R PK 3 /5 .15

S P=

S T = S2r +S 2R+S 2P

ST=

Readings
2
3
4
5
6
7
8
9
10

K1
4.565603
3.041937
2.501214
2.214101
2.03236
1.904586
1.808922
1.734007
1.673164

Operators
2
3
4
5
6
7
8
9
10

K2
3.652482
2.696335
2.299107
2.076613
1.928839
1.819788
1.739865
1.672078
1.619497

Parts
2
3
4
5
6
7
8
9
10

K3
3.652482
2.696335
2.299107
2.076613
1.928839
1.819788
1.739865
1.672078
1.619497

Measurement Precision Study - Scoreard Calculations

Comparisons to specifications (Used when both specifications exist)

Topic
Equation
Calculation
% Repeatability
% Reproducibility
% Measurement
R&r

Answer

5 .15*Sr
100
USL-LSL

5. 15*S R
USL-LSL

100

5 . 15*SR&r
100
USL-LSL

Comparisons to total variation

Topic
% Repeatability
% Reproducibility
% Measurement
R&r

Equation

Calculation

Answer

Calculation

Answer

2
Sr
100
S 2T
S 2R
100
S 2T

S 2R&r
S 2T

100

Measurement Tolerance
Topic
Measurement
tolerance

Equation
+
2 . 57*SR&r

K Factors for the GRR Worksheet

(Taken fron the AIAG MSA book, 2nd Edition, Feb 95, page 29)
K1 = 5.15 / d2*
K2 = 5.15 / d2*
K3 = 5.15 / d2*
The sigma spread used in the AIAG book is 5.15 units, because that
forms a 99% confidence interval. The K factors are
calculated from d2*.
d2* is adjusted for G (number of parts*appraisers). The equation below
uses the AIAG assumptions for G.
Std Dev Spread

Trials
2
3
4
5
6
7
8
9
10

5.15 sigma

d2*
G (parts*appraisers)
1
>15
1.41
1.128
1.91
1.693
2.24
2.059
2.48
2.326
2.67
2.534
2.83
2.704
2.96
2.847
3.08
2.97
3.18
3.078

Readings
2
3
4
5
6
7
8
9
10

K1
4.57
3.04
2.50
2.21
2.03
1.90
1.81
1.73
1.67

K Factors
Operators
K2
2
3.65
3
2.70
4
2.30
5
2.08
6
1.93
7
1.82
8
1.74
9
1.67
10
1.62

Parts
2
3
4
5
6
7
8
9
10

K3
3.65
2.70
2.30
2.08
1.93
1.82
1.74
1.67
1.62