249

ASNT Fall Conference and Quality Testing Show 2009 [Columbus, OH, October 2009]: pp 249-256. Copyright 2009, 2011,
American Society for Nondestructive Testing, Columbus, OH.
Probability of Detection Analysis for Eddy Current Inspection Systems
ABSTRACT
Jennifer Herberich Brown
Pratt & Whitney Rocketdyne
M/S 731-84, P.O. Box lO9600, West Palm Beach, FL 33410-9600
(561) 796-2230; fax (860) 622-3453;jennifer.brown@pwr.utc.com
Eddy current (EC) inspection is a non-destructive evaluation (NDE) method commonly used for flight quality
assurance of commercial, military, and rocket engine hardware. EC inspection is used to detect surface or near-
surface anomalies, such as cracks, in metallic hardware by sensing changes in coil impedance. Understanding the
relationship between crack size and system response is the basis for determining the detection capability of an EC
system and establishing find/no-find decision rules. For a given EC system, the relationship between the EC
response and crack size can be evaluated by scanning a set of standard specimens with a known number and
distribution of crack sizes. The results are analyzed using the Ii versus a analysis method to quantity the relationship
between the EC response (i.e., Ii) and crack size (i.e., a) and develop a Probability of Detection (POD) curve, which
defines the crack size that can be detected with a specified level of reliability. Available POD or statistical software
packages have the ability to perform an Ii versus a analysis. However, the validity ofthe analysis, resulting POD
curve, and any parameters of interest (e.g., a90, a90/9S) can be significantly impacted by the choice of analysis inputs.
Hence, it is critical that the practitioner have a basic understanding of the statistical methods behind the "black box"
so as to ensure a valid analysis and avoid drawing misleading conclusions. This paper provides a practical overview
of the Ii versus a methodology and illustrates how some of the analysis inputs impact an Ii versus a analysis.
BACKGROUND
Eddy current (EC) inspection is a non-destructive evaluation (NDE) method commonly used for flight quality
assurance of commercial, military, and rocket engine hardware. One application is detection of surface
near-surface anomalies, such as cracks, in metallic hardware. When the EC probe is placed on metallic hardware, it
induces an electrical current, called an eddy current, in the hardware. When the probe scans over a surface crack or
near surface crack, the induced current is altered. The change in the coil impedance is sensed by the EC instrument,
which processes the signal and displays it as a measurable quantity. The measured EC signal is expected to increase
as defect size increases. Understanding the relationship between crack size and EC signal response is the basis for
determining the detection capability of an EC system and establishing find/no-find decision rules.
Provided that key EC system parameters are well-understood and controlled, a probability of detection (POD)
demonstration test and analysis is the best available method for quantitying the relationship between crack size and
system response and determining the crack size that the EC system is capable of detecting with a specified level of
reliability. A POD demonstration test is typically conducted in an environment similar to the anticipated production
EC environment using a set of standard specimens of the same geometry and material that have a known number
and distribution of crack sizes. Data on EC response for a given crack size is generated by scanning the specimens.
The Ii versus a analysis method, which is the recommended method per MIL-HDBK-1823, is used to quantitying the
relationship between the EC response (i.e., Ii) and known crack size (i.e., a). A POD curve can then be developed
based on the established relationship to define the capability of the EC system. Several POD software packages are
available with the capability to perform an Ii versus a analysis. However, just because the POD software produces a
POD curve does not mean that the POD curve is valid. The validity of the analysis, resulting POD curve, and any
parameters of interest (e.g. , ~ O , a90/95) can be significantly impacted by the choice of user-defined analysis inputs.
Hence, it is critical that the practitioner have a basic understanding of the inner workings of the "black box" and how
the analysis inputs affect the results so as to ensure a valid analysis and avoid drawing misleading conclusions.
A VERSUS A ANALYSIS INPUTS AND UNDERLYING STATISTICAL METHODS
Standard inputs to Ii versus a analysis that are required by POD software include the specified Ii versus a model
formulation, the saturation threshold level, the noise threshold level, the decision threshold, and the POD
250
ASNT Fall Conference and Quality Testing Show 2009 [Columbus, OH, October 2009]: pp 249-256. Copyright 2009, 2011,
American Society for Nondestructive Testing, Columbus, OH.
demonstration test data. To understand how the analysis inputs can affect the results of an a versus a analysis, it is
necessary to have a basic understanding of the underlying statistical methods on which it is based.
The a versus a analysis method involves two parts, which are executed sequentially: (1) quantifYing the relationship
a and a, and (2) developing a POD curve. Both parts are based on established statistical methods. Part 1 is based on
advanced regression techniques. Part 2 is based on normal probability theory. An overview of each part is
presented in the following sections.
Quantifying the Relationship between Ii and a
The underlying assumption in EC inspection is that EC signal response is assumed to increase with flaw size. The
relationship between EC signal response (denoted a) and known crack size (denoted a) is assumed to be best
described by a simple linear regression model.
A simple linear regression model is characterized by having a single continuous predictor variable and a single
continuous response variable, where the relationship between them is assumed to be linear. The theoretical model is
expressed as Jii = /30 + /31 * Xi + Cj , where Xi represents the value of the predictor variable, /-li is the expected mean
response for a given value of the predictor variable, ~ o is the y-intercept, ~ I is the slope, and Ci is the error term. Ci is
included in the theoretical model to represent the naturally occurring variation around /-li for a given Xi. In linear
regression it is assumed that the error terms Ci are independent and follow a normal distribution with mean 0 and
constant variance. In other words, the observed variability in the data for a given value of the predictor variable Xi is
expected to be centered around /-li and should not increase or decrease as the value of Xi increases or decreases.
The predictive model derived from sample data is expressed as ilj = bo + bl * Xj where A is the predicted mean
response for a given value of the predictor Xi, and b
o
and b
l
are estimates of o and ~ I respectively. Note that the
error term Ci is not an expression in the predictive model. This is because, in theory, the value ofci is expected to be
o on average. However, Ci plays an important role in assessing model validity, which will be discussed in detail
later.
In practice, when analyzing EC inspection POD demonstration test data, the predictive model is typically expressed
as In(a) = b
o
+ b
l
* In(a) rather than a = b
o
+ b
l
* a . This formulation of the model falls in the class of simple linear
regression models and makes practical sense since it eliminates negative values. Flaw size and measured EC signal
response are positively valued data. Using the natural log restricts both the predictor variable and the response
variable to be greater than O. Hence, predicted EC signal response will be greater than O.
When performing a simple linear regression analysis, b
o
and b
l
are derived using the method of least squares. The
method of least squares selects values for ~ o and ~ I that minimize the sum of the squared distances between each
observed value and predicted value. In order to use this method, each data point must have an exact observed value.
However, due to inherent system noise and measurement capability, the exact EC signal response for a given flaw
size may not be known. When the response value is not known exactly, it is referred to as a censored data point.
In general, there are three types of censored data: right-censored, left-censored, and interval-censored.
Right-censored and left-censored data are typically encountered in EC inspection POD demonstration test data.
Right-censoring occurs when the signal generated by a large flaw exceeds the electronic limit of the EC system. For
example, suppose that the maximum amplitude that can be reported by an EC system is 25. If the measured signal
from a large flaw exceeds 25, all that is known is that a is some value to the "right" of 25. Left-censoring occurs
when the EC system cannot distinguish the signal generated by a small flaw from EC system and/or material noise.
For example, suppose that the noise threshold is 1 division. That is, any signal below 1 division is indistinguishable
from noise. If the measured signal from a small flaw falls below 1, all that is really known is that a is within the
noise, or to the "left" of 1.
Censoring can be random or predetermined (Meeker, et aI. , 1998). In this context, it is predetermined by the
limitations of the EC instrument electronics. A predetermined saturation threshold level and noise threshold level,
251
ASNT Fall Conference and Quality Testing Show 2009 [Columbus, OH, October 2009]: pp 249-256. Copyright 2009, 2011,
American Society for Nondestructive Testing, Columbus, OH.
denoted a
sat
and a noise respectively, are used to identify censored data. Hence, a
sat
and a noise are both required inputs
in an a versus a analysis. The saturation threshold a
sat
is used to identify right-censored data points. That is, EC
signal responses that saturate the system are treated as right-censored data in the analysis. The noise threshold anoise
is used to identify left-censored data points. That is, any measured EC signal response following below a noise is
treated as a left-censored data point in the analysis. (Note that MIL-HDBK-1823 (1999) refers to a noise as d
h
or the
inspection threshold.) An illustration of right-censored (i.e., electrical saturation) and left-censored data (i.e. , noise)
is shown in Figure 1. Note that in this example a
sat
= 25 and anoise = 1.
Measured EC Signal Response vs. Crack Size
Illustration of Right-censoring (Electrical Saturation) and Left-censoring (Noise)
I I I I I I I I I I
30 -T---------------------------i-------T--r---i i-irrrfiHiH 25
- ------------------ ------- ---
I I I I I I I I I 111111 ....
15 -+-------------------1--------1----..
I I I I I I I I I IIIIII1111
Z' I I I . I I I I I I I I I I I I IIII
10 - r--------------------r --------t - -1--,---r --r- T- -r-ii -r if i
.. I I I I I I I I I I I I I II I I
I I I ' I I I I I I I IIII1 I I
&: I I I I I I I I I IIIII1
8. 5
:n : I I I' I I I 1'1 I I
'" I I I Data Type
i 0 Left-censored (Noise)
iii : Measured Response
&l : 0 Right-censored (Electrical Saturation)
I
I

I
CD
<p

<0

is
o
8
<0
o
a
<0
Crack Size (a)
,
I I
I
I
:
I I I
I I I I I I
I I I III I
I I I I I I
I I I I I I
: : :: : :
s 8 8 g g
o c:i 0 c:i c:i 0 oooc:ic::X::ri:::Xi::
Figure 1: Illustration of right-censored data (electrical saturation) and left-censored data (noise).
Because censored data are commonly encountered in EC POD demonstration test data, a more advanced regression
technique for estimating and is needed in order to properly handle the censored data. Rather than using the
method of least squares, the a versus a analysis uses the method of maximum likelihood to estimate the model
coefficients. The method of maximum likelihood selects values for b
o
and b
l
by maximizing the likelihood function.
The likelihood function is analogous to a probability distribution function (MIL-HDBK-1823, 1999). However, a
probability distribution function describes how likely a data point is given the distribution parameters, whereas the
likelihood function describes how likely the parameters are given the actual observed data. In other words, the EC
signal data are considered fixed (since they are already observed), and it is the model parameters b
o
and b
l
that can
vary. The method of maximum likelihood finds the values of b
o
and b
l
for which the value of the likelihood
function is the largest. That is, the method of maximum likelihood searches for values ofb
o
and b
l
that are the most
consistent with the actual observed data. In general, the method of maximum likelihood is viewed as a more
versatile method for fitting a model to data since it can be applied to a wide variety of data types, including censored
data, as well as to a wide variety of statistical models (Meeker et a1., 1998). (It should be noted that when the values
obtained for b
o
and b
l
using the method of maximum likelihood are the same as those obtained using the method of
least squares when no censored data are present (Neter et a1. , 1996).)
Developing a POD Curve
A POD curve defines the crack size that the EC system is capable of detecting with a specified level of reliability.
The POD curve resulting from an a versus a analysis is based on the established relationship between a and a, the
defined decision threshold, and normal probability theory.
The established relationship between a and a defines the average EC signal response for a given crack size.
However, the measured EC signal response for cracks of the same size will vary due to other physical characteristics
of the flaw, such as depth, and the inherent variability in the EC inspection process. As illustrated in Figure 2, the
variation around the average EC signal response for a given crack size is assumed to be normally distributed.
252
ASNT Fall Conference and Quality Testing Show 2009 [Columbus, OH, October 2009]: pp 249-256. Copyright 2009, 2011,
American Society for Nondestructive Testing, Columbus, OH.
Measured EC Signal Response with a-hat vs. a Model At
Measured EC Signal Response
WO -- Predicted EC Response (Maximum Lik elihood)
o Censored Data
I
I
I
I I I
W
o

a
I
I
I

8
a
s
o
a
, I I I I I I IIIII II
I I I I I I I IIIII II
I I I I I I I IIIII I
I I I I I I I IIIII I' i!
--1---+--+-,--;'-+-1-+-++++.1- ,-H+t-
I I I I I I I , ' II I I II
I I I I I I I r IIIII II
I I I r I " I I I II
I I I I I II I I I II
I 'I I IIIII II
! ! ! ! !!!! ! I 25
I IIIII II
I IIIII II
I IIIII II
I I I I I I I IIIII II

s
ci c:i c:i ci ci ci ci c:i ci cicicX::i::::i:::i:
Crack Size (a)
Figure 2: Actual measured EC signal response is assumed to be normally distributed around the average.
In order to calculate POD, a decision threshold level is a required analysis input. The decision threshold, denoted
adec. represents the value of a above which the EC signal response is interpreted as a find. POD is defined as the
fraction of flaws of a nominal size that are expected to be found (MIL-HDBK-l823, 1999). Probability equates to
the area under the normal curve. Hence, POD equates to the area under the normal curve above a dec . Suppose, for
example, that any measured EC signal above the noise threshold level is considered a find. That is, a dec = anoise = 1.
Hence, POD equates to the area under the normal curve above adec =1 as illustrated in Figure 3.
100
j

ill
c
10
8.
VI

;;
c
'" iii
u
III
Measured EC Signal Response with a-hat vs. a Model At
POD = area under the normal curve
: Measured EC Signal Response
-, -- Predicted EC Response (Maximum Lik elihood)
: 0 Censored Data
I
I I I I I I I IIIII II
I I I I I I I IIIII II
I I I I I I I IIIII I
I I I I I I I 111111' 11
- -1- - - + - - + - -t - -t - + -1- t- + + + + ,-H +t-
I I I I I I I I' rill II
I I I I I I I J IIIII II
I I I I r I IIIII II
I I I 'I I IIIII II
I I' I I IIIII II
I . I I IIIII II

I
I
I
I I
I I I I I I I I IIIII II
I I I I I I I I11111 II
I I I I I I I IIIII II


11111111111111
.-
I I
I I
I I
I
I
I I I I I I I II
I (
AI) I b I 1b,l lhl l )
n a
"
=; 0 -Ii \ )a,
I
I
I
I
I
I

I
I
I
I
I
, ,

co
S ?<l
co 0 co
8
N M M
0 0 0 0 0 0
a a a a a a a a ci ci ci ci cicicic:i:i:i:i:O
Crack Size (al
Figure 3: POD is the area under the normal curve above Ii dec'
When performing an a versus a analysis, POD is calculated as
253
ASNT Fall Conference and Quality Testing Show 2009 [Columbus, OH, October 2009]: pp 249-256. Copyright 2009, 2011,
American Society for Nondestructive Testing, Columbus, OH.
where <I> is the cumulative standard normal distribution function, ai is the expected mean EC signal response for a
given flaw size a;, b
o
and b
l
are the estimated model coefficients, and a- is the estimated standard deviation around
the expected mean response. Since ai is a function of crack size, POD is a function of crack size. Given the model
coefficients, a-, and a dee, POD can be calculated for each flaw size and plotted to establish a POD curve as
illustrated in Figure 4.
Probability of Detection (POD) Curve
Model : In(a-hat) = bO +bl * ln(a), bO and bl estimated using Maximum Ukelihood
1.0
TTl I I I I
I I I I I I I
0.9
I I I I I I I
i i 'j' '.,.- i i
I I I I I I I I
I I I I I I I I
0.8 - - - - - - - - - - - - - - - ~ - - - - - - - - - ~ - - - - ~ - - - ~ - - ~ - - ~ - ~ - ~ - +
I I I I I I I I
I I I I I I I I
0.7
I I I I I I I I I
------------------T-----------T------T----T---T--T--T-T-,-T
I I I I I I I I I
I I. I I I I I I I
0.6 ------------------+---------- - + - - - - - - + - - - - + - - - + - - + - - ~ - + - ~ - +
I I I I I I I I I I
Q
0 0.5
0-
I I I I I I I I I I
---------------+------ . + - - - - - - ~ - - - - ~ - - - + - - ~ - - ~ - ~ - ~ - +
I I I I I I I I I I
I I I I I I I I I I
0.4
I I I I I I I I I I
T T------T----T---T--T--T-T-,-T
I I I I I I I I I I
I I I I I I I I I I
0.3 - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - + - - - - - - + - - - - + - - - + - - + - - ~ - + - ~ - +
I I I I I I I I I I
I I I I I I I I I I
0.2
__________________ L ___________ L ______ ____ ___ L __ __ _ ~ _ ~ _ L
I I I I I I I I I I
I I I I I I I I I I
0.1
I I I I I I I I I I
------------------r-----------r------T----T---r--T--r-T-i-r
I I I I I I I
0.0
III l lJ.!.
8
ci
8
ci

ci
8 ~ 8 ~ ~ ~
c:i 0 c:i
Crack Size (a)
Figure 4: Resulting POD curve.
The resulting POD curve represents the typical (or nominal) POD curve and can be used, for example, to estimate
the flaw size that can be detected with 90% probability, denoted a90. This is accomplished by finding the flaw size
that intersects the curve where POD is 0.90. (Note: In order to estimate the flaw size that can be detected with 90%
probability and 95% confidence, denoted a90/95 , a confidence bound is put on the established a versus a relationship
which can then be translated into a confidence bound on the POD curve in a manner similar to that which was used
to derive the typical POD curve.)
POD CURVE VALIDITY
Just because the POD software produces a POD curve does not mean the POD curve is valid. Validity of the POD
curve depends on the identification of censored data, selection of the decision threshold a den and the verification of
the underlying a versus a model assumptions.
Identifying Censored Data in an Ii versus a Analysis
Censored data are commonly encountered in EC POD demonstration test data and contain valuable information
about the capability of the inspection system. Hence, values for a sa! and a naise are required inputs in an a versus a
analysis to identity right-censored and left-censored data respectively. Not specitying values for a Sal and a naise is
equivalent to treating the censored data as known values in the analysis, which can have a significant impact on the
a and a analysis and resulting POD curve.
For the simulated EC data in Figure 1, not specitying values for a
Sal
and a naise treats measured EC signals responses
that fall below the noise level (i.e., a na;se = 1) as exact values and treats measured EC signals above the electrical
saturation as equal to 25 since the EC system cannot output values past the saturation threshold level. When no
censored data are present, a simple linear regression analysis can be performed. Figure 5 illustrates the effect on
both the a versus a model and POD calculations when censored data are treated as exact known values. The dashed
line represents the model resulting from a simple linear regression analysis, and the solid line represents the model
resulting from using the method of maximum likelihood to estimate the model coefficients.
254
ASNT Fall Conference and Quality Testing Show 2009 [Columbus, OH, October 2009]: pp 249-256. Copyright 2009, 2011,
American Society for Nondestructive Testing, Columbus, OH.
100

.!!.
3!
c
10

/!J.
iii
c
'" iii
u
...
Measured EC Signal Response with Model Fit
Treating Censored Data Correctly vs. Incorrectly
Variable
o Censored Data
Predicted EC Signal Response (Method of Maximum Ukelihood)
--- Predicted EC Resp (Simple Unear Reg., Method of Least Squares)
Measured EC Signal Response
, , ,
, , ,
25


d
Crack Size (al
Figure 5: Effect of treating censored data as known values on the Ii versus a model and POD calculations.
In Figure 5, it is clear there is a difference between the two models. The model resulting from a simple regression
analysis appears to over-predict the EC signal response for small flaw sizes and under-predict large flaw sizes. In
this example, when the censored data are treated as failures, they playa large role in determining the position of the
best-fit line. This is because the method of least squares attempts to find values of and I that minimize the sum
of the squared distances between each observed value and the predicted value, including the exact values used for
the censored data. The method of maximum likelihood correctly assumes the right-censored data would have been a
higher value and the left-censored data would have been a lower value, which is a more accurate representation of
the data. In general, the regression line should run through the data, passing as close as possible to all points.
Hence, accurately accounting for censored data not only produces a better model fit with respect to the exact
measured EC signal response data but also with respect to the censored data. Figure 5 also illustrates the effect on
a90. In this example, a simple linear regression analysis results in an optimistic (i.e., smaller) a90. Since the best-fit
line has a significant impact on POD calculations, the POD curve is affected as well as shown in Figure 6.
Probability of Detection (POD) Curve
Treating Censored Data Correctly vs. Incorrectly
1.0,--------------=--"' T----------"' T
: J, : : :
0.9 +-----------------+- , ----+-- , --.---. --+, --+, --+1 1 0.9
I , I I I I I I I
0.8 - ------------------ t------7-- ------+ - - - - t ---t --t--t -t -i-t
0.7 ---------------r- 1- 1+ ---T----T---r--T--r-T-,-r
: , : :: : : : : :
0.6 ---------------t- 1 t ----+----t---t--t--j-t-j-t
Q I I I I I I I I I I
o QS -------------------+--------- +------t----f---+--f--}-f-4-+
a. I I I I I I I I I I
0.4 - ------------------ _l __ ----- -t ------+ - - - - - - - - -
: I : :: : : : : :
0.3 -------------------+.'-------- --+------+----+---+--f--+-f--1-+
., I I I .. I I I
0.2 .'+ t t
/ 1 11-- POD (Maximum Ukelihood)
., : : 1--- POD (Least Squares)
O.O-L ____ .... ."".joJ '
a90if90
0.1
8
ci

ci
8 8
c:i 0 c:i
Crack Size (al
Figure 6: Effect on POD curve of treating censored data as known values.
255
ASNT Fall Conference and Quality Testing Show 2009 [Columbus, OH, October 2009]: pp 249-256. Copyright 2009, 2011,
American Society for Nondestructive Testing, Columbus, OH.
Incorrect specification of the values for a
Sal
and a no;se can also result m an inaccurate representation of the
relationship between EC signal response and crack size.
Selection of the Decision Threshold
The value selected for a dec is a critical input in an a versus a analysis. A POD curve cannot be produced if a dec is not
specified. If a dec is incorrectly specified, the result could be a seriously misleading POD curve.
By definition, a dec is associated with the flaw size that can be detected with 50% probability, denoted aso. Hence
changing a dec shifts the POD curve along the x-axis. Increasing the value of a dec> for example, will shift the POD
curve to the right. Figure 7 and Figure 8 illustrate the effect of increasing a dec on the POD calculation and resulting
POD curve. (Note that the saturation threshold and noise threshold remain the same in the analysis, e.g., a
Sal
= 25
and a no;se = 1. All that is changing is the value of a dec. )
j
.e
iii
c
8.
<II

i
'" iii
u
...
100
10
8
o
Measured EC Signal Response with a-hat vs. a Model Fit

Measured EC Signal Response
-- Predicted EC Response (Maximum Likelihood)
0 Censored Data
8 eJ eJ 8
o d d d d d d d d d ddc:X:i:i:i:i5
Crack Size (a)
Figure 7: Effect on POD of increasing the decision threshold.
Probability of Detection (POD) Curve
1.0 .' I I I
1-- POD (Maximum Likelihood) ' .:... : : .
0.9 -1 ___ Increased Decision Threshold --, --,----r---T---T--r-r-T-,-T
I : : : : : : :
o.S -- -----------------+ ---------- - - ( - - - - t----1" ---t --t--j -t -j-+
0.7
0.6
---------------+-------
J
,
I
I I I I I I I
I I I I I I I
-T----T---T--T--r-T-i-T
I I I I I I I
I I I I I I I

Q I : : : : : : :
o 0. 5
Q. , : : : : : : :
Q4
Q3
I I I I I I I I
0.2 ---------------+---- I
I ::::: : :
0.1 _ I ____ ____
I : : :
o.o-L---""'''''''''''''''''''''''''''''''''i'''''''''''''''':::!!!!!.!!! ...
8 8

8888888
o o o ci ci 0 0 ci ci ci
Crack Size (a)
Figure 8: Effect on POD curve of increasing the decision threshold.
256
ASNT Fall Conference and Quality Testing Show 2009 [Columbus, OH, October 2009]: pp 249-256. Copyright 2009, 2011,
American Society for Nondestructive Testing, Columbus, OH.
Verification of the Underlying a and a Model Assumptions
The validity of the POD curve is directly related to the validity of the a versus a model. The analysis results should
not be considered valid until the following underlying model assumptions are evaluated and shown to hold true: (1)
the error terms Ei are independent and follow a normal distribution with mean 0 and constant variance, and (2) the
relationship is in fact linear. Recall that the error term Ei is not an expression in the predictive model since in theory
it is expected to be 0 on average. However, it can be estimated as the difference between the observed value and the
model prediction. This difference is more commonly referred to as the residual. In general, analysis of the residuals
should be performed after any regression analysis to verify that the underlying model assumptions hold. This can be
accomplished by a few simple plots and/or more formal statistical methods. For example, the normality assumption
can be assessed using a histogram and/or Anderson-Darling test for normality. The POD software may produce
analysis results even if the underlying model assumptions do not hold. If the assumptions do not hold, any
conclusions drawn about the EC system capability based on the analysis are likely to be erroneous and,
consequently, misleading (MIL-HDBK-1823, 1999). If the POD software does not perform or does not have the
capability to perform a residual analysis, the practitioner should still perform a residual analysis to verify that all
underlying model assumptions hold.
The validity of the a versus a model also depends whether the maximum likelihood convergence criterion is met.
When censored data are present, the likelihood equation from which the model coefficients are derived does not
have a closed solution. Hence, an iterative numerical search procedure is used to find values of b
o
and b
l
that
maximize the likelihood. The procedure iterates until a convergence criterion is met, at which point estimates of the
model coefficients are obtained from the last iteration. If no solution exists, then the procedure will not reach
convergence. Some POD software may produce estimates of b
o
and b I even though the convergence criterion has
not been met. However, they are likely to be erroneous and should not be used. A POD software package should
provide some indication (though it may be subtle) as to whether the convergence criterion was met. It is the
responsibility of the practitioner to verify that convergence was achieved.
SUMMARY
Standard inputs to a versus a analysis that are required by POD software include the specified a versus a model
formulation, the saturation threshold level (a
sol
), the noise threshold level (ana;se), the decision threshold (adee), and
the POD demonstration test data. If anyone of these inputs is incorrectly specified or if the established relationship
between a and a is invalid, then the analysis and resulting POD curve (as well as any associated confidence bound)
is invalid. POD software performs the calculations based on user-defined inputs. The practitioner is responsible for
ensuring that the analysis inputs are correctly specified and that the underlying model assumptions hold. POD
software should not be treated as a "black box". It is critical that the practitioner have a basic understanding of the
underlying statistical methods and how the analysis inputs affect the results to ensure a valid analysis and avoid
drawing seriously misleading conclusions about the inspection capability of an EC system.
ACKNOWLEDGMENTS
This paper would not have been possible without the support and encouragement ofKon Haake. Thanks also to
Islay Rodriguez and David Raulerson for taking the time to educate a statistician in the physics of eddy current
inspection and providing opportunities for hands-on experience; to Tommie Watkins for imparting his knowledge of
POD methodology; and to Chuck Annis for happily entertaining all my questions.
REFERENCES
1. MIL-HDBK-1823 (30 April 1999). Non-Destructive Evaluation System Reliability Assessment.
2. MIL-HDBK-1823 (28 February 2007 DRAFT). Non-Destructive Evaluation System Reliability Assessment.
3. Meeker, W.Q. and L.A. Escobar, Statistical Methods for Reliability Data, John Wiley and Sons, New
York. 1998.
4. Neter, Kutner, Nachtsheim, & Wasserman, Applied Linear Statistical Models, 4th Ed., The McGraw-
Hill Companies, Inc. 1996.