Validation

by Dr. Michael Scheutwinkel

International Federation for Consulting GmbH IFC

1

 Governmental

regulations

Books . Recommendations

of associations

References
EURACHEM Guide The Fitness for Purpose of Analytical Methods A Laboratory Guide to Method Validation and Related Topics (12/1998) www.eurachem.com

CITAC / EURACHEM Guide Guide to Quality in Analytical Chemistry An Aid to Accreditation (2002) www.eurachem.com
3

Which chapter of ISO/IEC 17025 is dedicated to validation?

Do not ask me, I am only a simple guy from Alemania!

Definition in clause 5.4.5.1 of ISO/IEC 17025

Validation (ISO/IEC 17025)

Validation is the confirmation by examination and the provision of objective evidence that

the particular requirements for a specific

intended use are fulfilled.

.fit for purpose

7

Validation of an analytical test method

Validation is the process of establishing the

Performance characteristics Limitations of a method Identification of the influences which may change these characteristics and to what extent.

Which analytes can be determined in which matrix in the presence of which interferences?
Within these conditions what levels of precision and accuracy can be achieved?
8

Equipment and method valdiation

Equipment validation
DQ IQ

Soft and hardware

OQ

PQ

Method validation

different items
9

Validation in order to analytical method

Standardized-methods

Verification

Modified standardized and in-house methods

Validation
10

Verification of an analytical test method

Verification is the process to demonstrate the

competence in test performance

of an already validated standardized test method.

The tools are the same as for validation,

the laboratory needs only to verify that the documented performance characteristics

can be met (e.g. accuracy).

11

When should be validated?

Changes in established methods New method developed for a particular problem Established method used in different laboratories, different equipment or different staff Out-of-control situations within internal quality assurance Non-successful participation in PTs Demonstration of equivalence between two methods (e.g. a rapid new test against a standard method)
12

Validation Strategy

Specific test requirements defined by customer or market

C Comparison of method charcacteristics with requirements

D Verification to give proof that the requirements are fulfilled

Characterization of the test method

13

Description of test method (clause 5.4.4)

Identification / Scope Parameters in which matrix

Reagents and materials, CRMs

Equipment and environmental conditions Sampling and sample preparation

Quality assurance Uncertainty or procedure for its estimation
14

Calibration

Test performance including safety instructions

Documentation , reporting evaluation criteria

Please show us the way

15

Tools

Determination of method characteristics

Characteristics of the range Characteristics of calibration Characteristics of accuracy Robustness / Ruggedness

Selectivity or Specifity

16

Tools

Determination of method characteristics

Perform more point calibration Characteristics of calibration Linearity Sensitivity Correlation coefficient: r > 0.99 ?? Linearity is defined by the correlation coefficient

Sensitivity is defined by the slope of the calibration graph

17

Commission Decision 2002/657/EC

18

Validation of testing methods Calibration

Calibration values should be applied within the working range. Lowest point should be the limit of quantification. For calibration certified standard solutions shall be used. In addition, also standard solutions have to be added to blank materials, analysed over all process steps.

For linear calibration functions at least five concentrations are

necessary. Test calibration functions for linearity and highlight failing linearity.
19

Validation of testing methods

Sensitivity (resolution)

What is sensitivity?

It is the difference in an analytical concentration that corresponds to the smallest difference of a signal in a method which is still detectable.

Sensitivity can be extracted from the calibration curve or defined by using samples with different concentrations.
20

Validation of testing methods Linearity

Definition based on samples with varying concentrations and the calculation of the regression of results.

Signal and concentration do not have to be fully related. Five standards suffice to produce a calibration curve if linearity is o.k. More standards are necessary once the linearity is unsatisfactory.

Examine repeat samples and standards over a particular

working range to experience whether a reliable line can be drawn between proof and detection limit.
21

Homogeneity of variances
Characteristic values for the comparison of standard variations for differing concentration. In case of great differences between standard deviations (in-homogeneity of variances),

the working range needs to be split.

22

Tools

Determination of method characteristics

Characteristics of the range Limit of detection Limit of quantification Characteristics of accuracy Robustness / Ruggedness

Characteristics of calibration

Selectivity or Specifity

23

Validation of testing methods Working range

Finding out the appropriate working range using different matrices varying concentrations.

Concentration range within the achievement of acceptable

accuracy and
precision is possible.

In general the working range is broader than the linear range.

24

Validation of testing methods

Prerequisites for calibration

Standards must be faultless.

Precision has to be similar throughout the entire working range. The model function is applicable: either linear or curved. Errors only may occur randomly within signals. Errors have to follow the normal distribution.

25

LOD

Characteristics of the range

Limit of detection LOD

by calculation

xLOD = 3 . sL / b (Blank value method) xLOD = 4 . sxo (Calibration function method)

xLd = Limit of detection sL = Standard deviation of the blanks Sxo = Standard deviation of the calibration function b = Slope of calibration function

Various conventions

XLOD = Mean of blanks + 3 x SL

26

LOQ

Characteristics of the range

Limit of quantification LOQ

by calculation

xLOQ = 9 . sL/ b (Blank value method) xLOQ = 11 . sxo (Calibration function method)
xLOQ = Limit of quantification sL = Standard deviation of the blanks Sxo = Standard deviation of the calibration function b = Slope of calibration function

Various conventions

XLOQ = 5 or 6 or 10 x SL
27

Commission Decision 2002/657/EC

28

LOD

Characteristics of the range

Limit of detection

From 2 : 1 to 5 : 1

by signal to noise ratio

Limit of quantification

From 5 : 1 to 10 : 1
29

Effect of peak shape on LOD / LOQ

30

Tools

Determination of method characteristics

Characteristics of the range Characteristics of calibration Characteristics of accuracy Robustness or Ruggedness

Selectivity or Specifity

31

Selectivity / Specifity

Selectivity Specifity

Extent to which particular analytes can be determined in complex mixtures.

A method which is selective for an analyte is said to be specific.

Selective detectors in instrumental analysis

Use of antibodies

Cross-reactions ?

Selective media in microbiology

32

The difference between selectivity and specificity

Selectivity A selective method gives correct results for all interesting analytes whereas a specific test method gives correct results for the interesting analyte whereas other analytes might interfere each other. Specifity Outlines the analytical extent to which an analytical substance or substance group can be determined without interference from sample related components.

No problem, usually both words are mixed

33

Validation of testing methods

Selectivity

Outlines the extent to which an analytical substance can be determined without interference from other components. Method selective to one analytical substance specific.

Assess selectivity while application to pure solution up to complex matrices.

Document disturbances and restrictions of the method.
34

Tools

Determination of method characteristics

Characteristics of the range Characteristics of calibration Characteristics of accuracy Robustness / Ruggedness

Selectivity or Specifity

35

Validation of testing methods Robustness

Degree of proneness of a method to

conscious and unconscious alterations of a working instruction.
Part of the routine.
Learn from other analytical institutes, as the method-establishing laboratory will have proved robustness before publishing a new method.

36

Robustness

Sensitivity degree of the test method against small deviations in experimental conditions.
Examples: Times within process steps Environmental conditions (e.g. temperatures) Minor process changes (e.g. pH, flow rates in HPLC)

37

Ruggedness, according to USP

Ruggedness is the degree of reproducibility obtained under a variety of conditions, expressed as relative SD, e.g.:
different analysts, different equipment, different trade marks of reagents, etc.

Internal reproducibility
38

Tools

Determination of method characteristics

Characteristics of the range Characteristics of calibration Characteristics of accuracy: Correctness Uncertainty Precision (Repeatability/ Recovery) Reproducibility Selectivity or Specifity
39

Robustness or Ruggedness

Accuracy = Precision and correctness I Exactness of an analytical method

Accuracy

Precision
Repeatability

Degree of repeatability of an analytical method

Reproducibility Uncertainty

Correctness / Trueness

Systematic error / Correct value

Recovery / Bias
40

Random and systematic deviations

precise and correct

not precise but correct

precise but wrong

not precise and wrong

41

Accuracy = Precision and correctness II

Characteristics of accuracy
Correctness
Measure of correctness, covering systematic and non-systematic mistakes (use of CRM, comparison to a well characterized method)

Precision

Measure of the degree of repeatability, covering systematic mistakes

Repeatability: one laboratory; Better: internal reproducibility Reproducibility: more than one laboratory
42

Validation of testing methods Repeatability

Use at least one of the following procedures for determination:

Measurement of reference materials.

Fortification of a blank sample with the analytical substance.

Comparison to a reference procedure.

Participation in a co-operative (interlaboratory) test in which the true content is known.
43

Validation of testing methods Precision

Information on accuracy of two coinciding and independent analytical results. Precision depends on the concentration of the analyte. Repeatability is part of the precision with respect to repeated measurements: same material, same method, same analyst,

same laboratory and

short time-span in-between the analyses.
44

Validation of testing methods

Correctness / Trueness
Use the one of the following methods to assure correctness:
Analysis of certified reference material. Participation in interlaboratory comparisons. Comparison to a known procedure.

If not available:
Document any data that prove correctness. Make an approximation as a first approach to correctness. Purchase reference material. Use a similar method in parallel.
45

Validation of testing methods Recovery

Performed by addition of the analytical substance to the matrix. A matrix shall be free of the analytical substance. If not the sample will be enriched with a weak concentration of the analytical sample. Or a simulated matrix will be used. Recovery can be determined only when the analytical sample is available in pure form. List the recovery rate (in %) and the standard deviation when recovery rates are constant. Otherwise issue recovery rate as a function.
46

Validation of testing methods Frame conditions for the determination of the recovery rate

The analytical substance needs to be added in that form in which it occurs in nature.
The samples requires good homogenisation. The native content shall be below the determination limit.

47

Validation of testing methods

Reproducibility
internally Prove reproducibility

externally
Statistically

Comparative standard deviation Replicate standard deviation

Through participation in proficiency testing schemes

or participation in interlaboratory comparisons.
48

Tools

Determination of method characteristics

Characteristics of the range Limit of detection Limit of quantification
Characteristics of accuracy Correctness Uncertainty Precision (Repeatability/ Recovery) Reproducibility
49

Characteristics of calibration Linearity Sensitivity

Robustness or Ruggedness

Selectivity or Specifity

Validation of testing methods

Classification of the test method
Distinguish between

Analysis of contaminants (e.g. pesticides, heavy metals, food additives) Analysis of food constituents (salt and minerals)
Methods which are determining true values

Test methods which have to be calibrated with instruments

Physical measurement (pH value, conductivity) Qualitative determinations Microbiological methods
50

Validation of testing methods

Parameters for validation

Parameter Analysis of trace contaminants Description of test method Calibration Working range Detection limit Determination limit Recovery Repeatability Correctness Specificity Reproducibility X X X X if possible X X X X if possible if possible X X X X if possible if possible X X X if possible X if possible X X X X X X X X X X X X X X X X X X X X X X Analysis of constituents Conventional methods Methods not necessary to calibrate X X X X Physical methods Qualitative methods Bacteriological methods

51

Validation of testing methods

Common validation needs of known methods
Method AAS, AES, ICP, RF Polarography/ Voltametry Titration Water assay Chromatography/ Capillary Electrophoresis X Spectroscopy Release TOC Enzyme Immuno Assays X X X X X X X X X X X X X X X X X X X X X X X X X Accuracy Precision X X X X X X Specificity/ Selectivity (X) Linearity X X X X X Detection/ Determination limits X X Robustness X X X

52

Procedures for method validation

The Analysis of Standard Reference Materials

Generally accepted method for validation. Such standards are provided with guarantee on the market. It may be necessary to contract the preparation of a unique sample in particular matrix in order to utilise this procedure for method validation. The analyst must demonstrate that the method provides accuracy and precision.

53

Procedures for method validation

The participation in PT-schemes or in a laboratory Collaborative Study

The most widely accepted procedure for method validation.

Serious practical draw backs: Costly and time consuming Effort in co-ordination Shipping of samples and data Statistical analysis and interpretation of results. This method is rarely used for the first description of a method in the literature.

54

Procedures for method validation

Comparison with a currently accepted method

Usually done by one analyst or two using a split-sample. using results from the currently accepted method for verification. Agreement suggests validation.

Disagreement could also suggest that the currently accepted method is invalid.
In such case, another procedure has to be employed for the method validation.

The more samples are analysed and the wider the range of concentration the higher the credibility of the validation method.
55

Procedures for method validation The Zero-Blind Method

One analyst.
Using samples at known levels of analyte to demonstrate recovery, accuracy and precision. Fast, simple and useful but

it leads to subjective results.

Suitable for a first approximation requiring minimal time, manpower, samples and cost. In general, a good start for the overall validation process.

If this methods fails there is no reason to proceed with further validation of the method.
56

Procedures for method validation The Single-Blind Method

One analyst at the start.

Samples are given to a 2nd analyst to whom levels of analyte are unknown. Results are compiled and compared by the 1st analyst. Comparison by the first analyst makes objectivity questionable. This method still is biased on behalf of the 1st analyst. Suitable after the zero-blind method has been successful and before additional analysts or the management will be involved.

57

Procedures for method validation The Double-Blind Method

Three analysts.
1st analyst prepares samples at known levels. 2nd analyst does the actual analysis. 3rd analyst (or administrator) compares both data from the first two analysts. Only the 3rd analyst has access to these data.

Most effective approach to objectiveness

58

Validation of testing methods Clearance of validated test procedures

Quality Manager

Validation results

Customer requirements

59

Validation of testing methods

Decision criteria for the extent of a validation

Technical feasibility and staff disposition? Analytical viewpoints
facts

Measuring principle, method complexity? Occurrence of the sample, wide-spread?

Risk potential for client/ company? Political decisions

super-ordinated criteria

Actual aim of the validation? Consideration of normative and other requirements?

60

Validation of testing methods Extent of validation in analytical sciences

Purpose
Suitability of the method Marketability of the method Analytical sensibility Costs Requirements from the outside

61

ISO/IEC 17025 states:

Validation is always a balance between costs, risks and technical possibilities.

The validation shall be completed by a statement by the laboratory that the method is fit for the intended use.

62

Validation is always a balance betwee costs, risks and technical possibilities.

63

Do not forget: Self developed software or excel sheets have to be validated.

Records for that must be available.

64

Muito obrigado!

65

66

Bring them together .

67

68

69

70

Trabajo en grupos Humedada en Alimentos

Valor [Unidades] Incertidumbre estndar [unidades]

Fuente de incertidumbre

Smbolo

Distribucin de probabilidad

Divisor

Calibracin de la balance

uEst

+ 0.003 g

Tipo B / rect

0.7%

0.4%

3
Calibracin de la estufa
uPip 0% Tipo B / rect

Tiempo

uBurInd

0%

Repetibilidad

uRep

+ 0.1 %

Tipo B

0.1%

ucombinada = 3.43 %

Uexpandida = 7 %
71

IfEP Workshop: Proficiency Testing - Background and general aspects

Why proficiency testing?

PT schemes are just one important tool in quality control and quality management.

Participating in Proficiency Tests

Use of certified reference material

Validation of methods

QM
72