Troubleshooting Failed QC

Dr Douglas Chesher
Clinical Biochemistry, NSW Health Pathology, Royal
North Shore Hospital
Northern Clinical, School University of Sydney
Outline
• Types of errors
• Approach to troubleshooting
• Problem fixed, now what?
• Lot number changes
Experimental Errors
• Systematic Error
• Random Error
• Blunders
Systematic Errors
• Due to identified causes and can, in principle, be
eliminated. Errors of this type result in measured
values that are consistently too high or
consistently too low.
• Types
– Instrumental
– Observational. For example, parallax in reading a
meter scale.
– Environmental
– Theoretical. Due to simplification of the model system
or approximations in the equations describing it.

http://www.physics.nmsu.edu/research/lab110g/html/ERRORS.html
Random Errors
• Random errors are positive and negative
fluctuations that cause about one-half of the
measurements to be too high and one-half to be
too low.
– Sources of random errors cannot always be identified.
• Types of random error
– Observational. For example, errors in judgment of an
observer when reading the scale of a measuring
device to the smallest division.
– Environmental. For example, unpredictable
fluctuations in line voltage, temperature, or
mechanical vibrations of equipment.

http://www.physics.nmsu.edu/research/lab110g/html/ERRORS.html
Blunders
• An outright mistake.
• Should stick out like sore thumbs if we
make multiple measurements or if one
person checks the work of another.

http://www.physics.nmsu.edu/research/lab110g/html/ERRORS.html
“My QC Failed”
Response to a failed QC
• Stop
• Do not just rerun the QC
– Unless part of your rule (eg. 1 2s rerun ‘once’)
– Regression towards the mean
Response to failed QC
• Do not just recalibrate as a routine.
– Calibrations add noise and increase
imprecision.
150

145

140

135

130

125

120
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59

148
146
144
142
140
138
136
134
132
130
128
126
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59
Points of failure
• Instrument Failure:
– Check for error messages (printed or displayed). Refer to the
troubleshooting section of the instrument manual.
• Reagent, Quality Control Materials, Calibrators:
– In correct reconstitution, i.e. wrong or contaminated diluent or
use of the wrong volume of diluent.
– In correct storage, e.g. left at room temperature for excessive
amount of time.
– Prepare fresh (allow enough time to dissolve)
– NOTE: Bulk liquid reagents may be contaminated.
• Human:
– Reread the method - Check whether the correct sample volumes
or reagents were used. Make sure that a step in the procedure
was not missed.
– Were the correct parameters entered?
Have other tests also failed?
• Blunders
– Swapped QC levels
• Short sampled
Are there any instrument flags?
What rule has failed?
• Systematic error
– 1 3s, 2 2s, 4 1s, 10x etc
• Random error
– 1 3s, R4s
Review your QC Chart
Systematic
148
error

147 Serum X
-3SD
146
-2SD
-1SD
145
Mean
1SD
144
2SD
3SD
143

142
Has anything changed?
• Just calibrated
• New bottle of reagent
• New lot of reagent
 Quarterly
New reagent lot
maintenance
Review points of failure
• Reagents
– Correct reagent?, sufficient volume, shelf expiry
date, on-board expiry date
• Quality Control Material
– Material, lot no. & assigned value, preparation,
shelf expiry date, in-use expiry date, storage
conditions.
• Calibrators
– Correct material, lot no. & assigned values,
preparation, storage, expiry date. Inspect
calibration trace of last calibration.
Review points of failure
• Instrument
– Is maintenance up to date?
– Recheck flags, probes, lamps, cuvettes, water
bath.
 Corrective
action Monitor QC performance
QC Failure required and correct non-urgent
immediately faults as appropriate
?
Y

Instrument N
Correct instrument fault
flags?

Maintenanc Y Perform all outstanding

e up to
maintenance
date?

Reagents N
Correct reagent fault
OK?

N Make up and run correct

QC OK
QC

Calibrators N Make-up and run correct

OK? calibrator

Instrument N Fix identified instrument Rerun QC to confirm

Call for Help!
OK? fault problem fixed
Problem Solved!
Problem solved, now what?
• Run QC for evidence that problem has been
solved.
• Document what you have done.
Troubleshooting logs, QC annotation.
• Address patient results from previous good
QC to when QC failure occurred.
– Repeat testing
• Exclude / Inactivate failed QC from data
analysis if cause of the outlier is clearly
identified.
Reagent lot changes
• CLSI EP26-A User Evaluation of Between-
Reagent Lot Variation; Approved Guideline.
• “The protocol attempts to balance the need to
reliably detect clinically significant change in
reagent performance that may affect patient
results with the recognition that reagent lot
verification is a relatively frequent task that
puts demands on the laboratory’s limited
resource”
QC material is not always
commutable
• Shift in QC may not reflect a similar shift in
patient results
• Just because QC does not show a shift, does
not mean the patients will not show a shift.

• Verify all new lots before they are put into

use.
– Need only perform once for group of labs if using
same lot QC and reagent.
Test patient samples with
current and candidate
reagent lots

Estimate average
difference between lots

Was QC
Average Yes acceptabl
Yes Candidate lot acceptable
diff < CD? e with new for patient testing
lot?

No No

Investigate lot difference. Update QC

Do not report patient targets
results with this lot

CD: Critical Difference

Defining the critical difference
• Evaluation of comparability based on clinical
outcomes.
• Evaluation of comparability based on clinical decisions
– derived from either biological variation or
– from data based on clinicians’ opinions.
• Published professional recommendations.
• Performance goals set by regulatory bodies or
organizers of External Quality Assessment Schemes.
• Goals based on the current state of the art as
demonstrated by data from EQA or from current
publications on methodology.
 • Alpha (Type I error, false positive)
• Probability of detecting an error
Determine the statistical when there is no error.
power
• Typically set at 0.05
• Beta (Type II error, false negative)
• Probability of not detecting a
difference when there is a
Determine the number of difference
target concentrations
• Typically set at 0.20 to 0.05
• Statistical Power = 1 – Beta.
Generally measure at clinical decision
Determine the number of
samples to be tested and
points, often similar to the QC levels.
the rejection limit
Statistical tables may suggest that only one
patient is required but it is recommended
that at least three samples are used to
Determine the number of avoid random errors.
replicates per sample
If difficult to get sufficient samples,
replicate analysis may provide the
necessary additional statistical power.
However, under normal circumstances
measure in singlicate
Conclusions
• Do not automatically re-run QC or recalibrate
an assay.
• Type of QC failure and Quality Control Charts
may give a clue to the nature of the problem.
• Have a structured approach to
troubleshooting.
• Have a process for managing your patient
results
• Prevention is better than cure
– Manage introduction of new lots to the laboratory