Downloaded from bmj.

com on 10 June 2008

Understanding sensitivity and specificity with

the right side of the brain
Tze-Wey Loong

BMJ 2003;327;716-719
doi:10.1136/bmj.327.7417.716

Updated information and services can be found at:

http://bmj.com/cgi/content/full/327/7417/716

These include:
References
5 online articles that cite this article can be accessed at:
http://bmj.com/cgi/content/full/327/7417/716#otherarticles
Rapid responses 46 rapid responses have been posted to this article, which you can access
for free at:
http://bmj.com/cgi/content/full/327/7417/716#responses

You can respond to this article at:

http://bmj.com/cgi/eletter-submit/327/7417/716
Email alerting Receive free email alerts when new articles cite this article - sign up in the
service box at the top left of the article

Topic collections Articles on similar topics can be found in the following collections

Other Statistics and Research Methods: descriptions (612 articles)

Correction A correction has been published for this article. The contents of the correction
have been appended to the original article in this reprint. The correction is
available online at:
http://bmj.com/cgi/content/full/327/7422/1043

Notes

To order reprints follow the "Request Permissions" link in the navigation box
To subscribe to BMJ go to:
http://resources.bmj.com/bmj/subscribers
 Downloaded from bmj.com on 10 June 2008

Clinical review

Understanding sensitivity and specificity with the right

side of the brain
Tze-Wey Loong

Can you explain why a test with 95% sensitivity might identify only 1% of affected people in the
general population? The visual approach in this article should make the reason clearer

Department of I first encountered sensitivity and specificity in medical

Community,
Occupational, and
school. That is, I remember my eyes glazing over on
Family Medicine, being told that “sensitivity = TP/TP+FN, where TP is
National University the number of true positives and FN is the number of
of Singapore,
Singapore false negatives.” As a doctor I continued to encounter
Tze-Wey Loong sensitivity and specificity, and my bewilderment turned
clinical teacher to frustration—these seemed such basic concepts; why
(part time)
were they so hard to grasp? Perhaps the left (logical)
Correspondence to: side of my brain was not up to the task of
T-W Loong, King
George’s Medical comprehending these ideas and needed some help
Centre, Block 803 from the right (visual) side. What follows are diagrams
King George’s that were useful to me in attempting to better visualise
Avenue, [01-144,
Singapore 200803, sensitivity, specificity, and their cousins positive predic-
Singapore tive value and negative predictive value. Fig 2 Hypothetical population
tzewey@
singnet.com.sg
Sensitivity and specificity
BMJ 2003;327:716–9
I will be using four symbols in these diagrams (fig 1).

....is a well person

....is a person with a disease

....is a negative test result

....is a positive test result

and therefore....

....is a well person who tests negative (a true negative) Fig 3 Results of diagnostic test on hypothetical population
....is a person with a disease who tests positive (a true positive)

....is a well person who tests positive (a false positive)

....is a person with a disease who tests negative (a false negative)
Fig 1 Key to symbols
Sensitivity refers to how good a test is at correctly
identifying people who have the disease. When
calculating sensitivity we are therefore interested in
only this group of people (fig 4). The test has correctly
identified 24 out of the 30 people who have the

Let us start by looking at a hypothetical population

(fig 2). The size of the population is 100 and the
number of people with the disease is 30. The
prevalence of the disease is therefore 30/100 = 30%.
Now let us imagine applying a diagnostic test for
the disease to this population and obtaining the results
shown in figure 3. The test has correctly identified
most, but not all of the people with the disease. It has
also correctly labelled as disease free most, but not all,
of the well people. Calculating sensitivity and
specificity will allow us to quantify these statements. Fig 4 Sensitivity of test

716 BMJ VOLUME 327 27 SEPTEMBER 2003 bmj.com

 Downloaded from bmj.com on 10 June 2008
Clinical review

disease. Therefore the sensitivity of this test is

24/30 = 80%.
Specificity, on the other hand, is concerned with
how good the test is at correctly identifying people who
are well (fig 5). The test has correctly identified 56 out
of 70 well people. The specificity of this test is therefore
56/70 = 80%.

Fig 8 Positive predictive value

On the other hand, negative predictive value is

concerned only with negative test results (fig 9). In our
example, 56 out of 62 negative test results are correct,
giving a negative predictive value of 56/62 = 90%.

Fig 5 Specificity of test

Having a high sensitivity is not necessarily a good

thing, as we can see from figure 6. This test has
achieved a sensitivity of 100% by using the simple
strategy of always producing a positive result. Its
specificity, however, clearly could not be worse, and the
test is useless. By contrast, Figure 7 shows the result a
perfect test would give us.

Fig 9 Negative predictive value

The interesting thing about positive and negative

predictive values is that they change if the prevalence
of the disease changes. Let’s assume that the
prevalence of disease in our population has fallen to
10%. If we were to use the same test as before, we would
obtain the results in figure 10. The sensitivity and

Fig 6 Test with 100% sensitivity

Negative predictive value

Fig 7 Perfect test

Predictive values
Now let us consider positive predictive value and nega-
tive predictive value. We will again use the population
introduced in figure 3. Positive predictive value refers
to the chance that a positive test result will be correct.
That is, it looks at all the positive test results. Figure 8
shows that 24 out of 38 positive test results are correct. Positive predictive value

The positive predictive value of this test is therefore

24/38 = 63%. Fig 10 Results of testing population with disease prevalence of 10%

BMJ VOLUME 327 27 SEPTEMBER 2003 bmj.com 717

 Downloaded from bmj.com on 10 June 2008
Clinical review

specificity have not changed (sensitivity = 8/10 = 80%

and specificity = 72/90 = 80%), but the positive predic-
tive value is now 8/26 = 31% (compared with 63% pre-
viously) and the negative predictive value is
72/74 = 97% (compared with 90% previously).
In fact, for any diagnostic test, the positive
predictive value will fall as the prevalence of the disease
falls while the negative predictive value will rise. This is
not really so mystifying if we consider the prevalence to
be the probability that a person has the disease before
we do the test. A low prevalence simply means that the
person we are testing is unlikely to have the disease
and therefore, based on this fact alone, a negative test
result is likely to be correct. The following real example
should make this clearer.

A real example
So far we have been discussing hypothetical cases. Let us
now take a look at the use of the antinuclear antibody
test in the diagnosis of systemic lupus erythematosus. I
have massaged the numbers slightly to make them
easier to illustrate, but they are close to reported figures
in both the United Kingdom and Singapore.1 2 The
prevalence of systemic lupus erythematosus is 33 in
100 000, and the antinuclear antibody test has a sensi-
tivity of 94% and a specificity of 97%. To visualise this we
need to imagine 1000 of the 10 by 10 squares used in
the earlier figures (fig 11). Only one of these squares
contains some patients with the disease.

No of true negatives = 96 900

No of false negatives = 2
Negative predictive value = 96 900
96 900 + 2
= 99.99%

No of true positives = 31

No of false positives = 3067

Positive predictive value = 31

999 of squares consist Only this square contains some
entirely of well individuals patients with systemic lupus 31 + 3067
erythematosus (33 of them) ≅ 1%

Fig 11 Prevalence of systemic lupus erythematosus
Figure 12 shows the result of applying the
antinuclear antibody test to this population. There are
many more true negative results than false negative
results and many more false positive than true positive
results. The test therefore has a superb negative predic-
tive value of 99.99% and a depressingly low positive
predictive value of about 1%. In practice, since most
diseases have a low prevalence, even when the tests we
Fig 12 (top) Results of antibody nuclear test in systemic lupus
erythematosus; (bottom) negative and positive predictive values
use have apparently good sensitivity and specificity we
may end up with dismal positive predictive values.
Knowing that the positive predictive value of this
test is 1%, we may then ask: does a positive test result in
a female patient with arthritis, malar rash, and
proteinuria really mean that she has only a 1% chance
of actually having systemic lupus? The answer is no.

718 BMJ VOLUME 327 27 SEPTEMBER 2003 bmj.com

 Downloaded from bmj.com on 10 June 2008
Summary points
No of true positives No of true negatives
Sensitivity Divided by
Specificity Divided by
No of people with the disease
No of well people
No of true positives No of true negatives
PPV Divided by
NPV Divided by
No of people with the disease
No of negative results
● For a given test, the lower the prevalence of the disease, the lower the positive predictive value
● Since most diseases have a low prevalence in the general population, even a test with an
apparently good sensitivity and specificity (>90%) may have a very low positive predictive value
● However, if this test is applied to a person with symptoms or signs of the disease, the positive
predictive value will be higher, as that person is from a population with a higher prevalence of
the disease
Who invented that bleeping thing?
While preparing a talk on Thomas Fogarty, of balloon catheter
fame, I stumbled on information about a different gentleman
who was a joint winner with Fogarty of the much coveted
MIT-Lemelson prize. This person is someone who affects nearly
all doctors every day. Indeed, if he had not recently died, I am
sure many of us would love to get our hands on him. However, as
you read on and discover what a truly remarkable man he was,
you may see him and his invention in a different light.
Al Gross was born in 1918 in Toronto but grew up in
Cleveland, Ohio. He had a childhood interest in amateur radio
and went on to study for a diploma in electronics. He was a bright
student, and his area of interest lay in unexplored radio
frequencies above 100 MHz. He wanted to invent a small, mobile,
two way radio, and by1938, two years into his diploma, he had
invented the first handheld radios (“walkie-talkies”), which could
communicate for up to 30 miles. These caused quite a stir with
the military, who deemed their invention as “top secret.” They
quickly commandeered the idea and furthered its use to
introduce ground to air communication for fighter pilots and to
detonate bombs at a distance, such as for blowing up bridges.
After time, these long range radios were made public
knowledge, and, as a result, in 1946 citizen band (CB) radio was
invented, the familiar mode of communication of taxi drivers and
truckers.
BMJ VOLUME 327 27 SEPTEMBER 2003 bmj.com
Clinical review
Look at it this way—the patient is not a member of the
general population. She is from the population of
people with symptoms of systemic lupus erythemato-
sus, and in this population the prevalence is much
higher than 33 in 100 000. Hence the positive pre-
dictive value of the test in her case is going to be much
higher than 1%.
Using both sides of the brain
I hope that having worked through sensitivity and
specificity from scratch you will be wondering why it
initially seemed so confusing. It may be because of our
dependence on the left (linguistic) side of the brain.
When told that a test has a sensitivity of 94% and a
positive predictive value of 1%, our left brain has
difficulty grasping how a test can be 94% sensitive and
yet be correct only 1% of the time. It is partly misled by
the huge difference between prevalence, on the one
hand, and sensitivity and specificity on the other.
The prevalence of systemic lupus erythematosus is
0.033% while the sensitivity and specificity of the test
are about 95%; this difference is of several orders of
magnitude. If, for example, we developed a test with
sensitivity and specificity of 99.999% rather than 95%,
we would be able to boast of a positive predictive value
of 97%.
Competing interests: None declared.
1 Johnson AE, Gordon C, Palmer RG, Bacon PA. The prevalence and inci-
dence of SLE in Birmingham, England. Relationship to ethnicity and
country of birth. Arthritis Rheum 1995;38:551-8.
2 Boey ML. Systemic lupus erythematosus. Singapore Med J 1992;33:291-3.
A more sinister twist in Gross’s career occurred in 1949, when
he invented the telephone pager system. However, his first large
scale attempt to sell pagers to doctors did not meet with the
success he had anticipated. “In Philadelphia, there was a hospital
convention, and we set up the pager there. We demonstrated the
pager to all the hospital administrators, doctors, and nurses, and
they absolutely refused to go along with the idea,” said Gross.
“They claimed it would disturb the patient, the nurses wouldn’t
want to carry it, and the doctors would be disturbed in their game
of golf.”
Although the idea initially failed to catch on, New York’s Jewish
Hospital did install his paging system in 1950, and the Federal
Communications Commission officially approved it in 1958,
marking the era of mass production. The name “pager” is derived
from the Motorola Pageboy 1, one of the first commercially
available models.
As we are daily reminded, pagers are here to stay. Recent
estimates suggest that there are now over 60 million pagers in use
worldwide. All I can say in their defence is that, when you are out
on a Saturday night in the rain, and taxi control gets a cab to you
in five minutes using CB radio communication, perhaps you will
see your pager in a different light.
Fraser Smith research registrar, St James’s Hospital, Dublin, Republic
of Ireland
719
 Downloaded from bmj.com on 10 June 2008
undertake their research activities and continue to par-
ticipate in routine patient care within the NHS.
Existing measures of success and recognition for
those working in other areas of science are often
inappropriate for clinical research. Data emerging
from clinical studies is seldom published in the high
impact journals Nature, Cell, and Science, and the time
required to move through the development and
implementation of a single set of protocols is such that
productivity can easily be perceived to be low. Recog-
nition must be found for individuals undertaking
clinical investigation that acknowledges the challenges
associated with developing and instituting protocols
in patients.
New funding should be made available
The biggest limitation to expansion of clinical research
once an appropriate infrastructure is in place would be
programme funding. Extra funds should be available
through the Medical Research Council to support
clinical trials and provide for a funding stream for
experimental medicine and training clinical scientists.
This money should be ring fenced. Support is also
required to develop new methods for studying chronic
disease, where randomised controlled trials are often
inappropriate.
In response to this increase, major charities need to
commit to properly resource the aspect of clinical
research relevant to their interests. Attempts should
also be made to ensure that the biotechnology industry
and pharmaceutical companies recognise this oppor-
tunity and increase their investment in UK research.
Collaboration between funders, although difficult to
achieve, will be essential to fund studies that are likely
to become bigger and more complex as standards of
care improve.
Educate the public about merits of clinical research
Expansion of clinical research will be successful only if
the public recognises its value and is willing to partici-
pate. Serious attempts must be made to ensure that
people understand the benefits of clinical research, not
just for those participating in studies but also for future
patients who will benefit from the insights gained. In
exchange, the NHS should make it possible for any
patient who wishes to participate in a clinical study to
have the opportunity to do so.
Conclusion
The United Kingdom is not alone in facing a decline in
research. Many other countries are experiencing simi-
lar problems. However, the NHS is perhaps more
dependent on a healthy research environment than
other healthcare systems. Any attempt to energise
clinical research will require the joint efforts of the
Department of Health, the Department of Trade and
Industry, the Medical Research Council, and the major
medical charities. The success or failure of their efforts
will have serious implications for the effective manage-
ment of the NHS, for patients who require new
treatments for their disease, and for those attempting
to develop new medicines in the biotechnology and
pharmaceutical industries.
We thank Robin Fears, the academy’s senior policy adviser, for
support in producing the report. We also thank Patrick Vallance,
BMJ VOLUME 327 1 NOVEMBER 2003 bmj.com
Education and debate
Summary points
Clinical research is in decline in the United
Kingdom
The main problems are in experimental medicine
and clinical trials
A national network for clinical research is needed
to help coordinate funding and research
programmes
Better career and reward structures are needed
for clinical researchers
Funding must be increased from all sources
members of the review group, the academy fellowship, and all
the respondents to the call for evidence for instructive
comments and support.
Contributors and sources: Members of the working group, sup-
ported by the research capacity of the secretariat, provided evi-
dence, analysis of issues, and prioritisation of strategic directions
and met to collate themes and prepare inputs. The data were
supplemented by a general call for evidence on the academy’s
website and emailed to all fellows.
Funding: Kohn Foundation and GlaxoSmithKline.
Competing interests: JB has been employed by the NHS
and MRC for many years, both of whom could benefit from this
report. He was a member of MRC Council until July 2002. He
has modest equity positions in a pharmaceutical investment
fund and has shares in Roche AG. He is a non-executive
member of Oxagen, Avidex, and Roche AG. He has had
many speaking engagements funded by industry and the MRC
and has served on numerous scientific advisory boards for
universities and medical schools. He is on the board of
Oxford Genetic Knowledge Park funded by the Department of
Health.
1 Doll R, Hill AB. Smoking and carcinoma of the lung. BMJ 1950;i:739-48.
2 Heart Protection Study Collaborative Group. MRC/BHF heart
protection study of cholesterol lowering with simvastatin in 20 536 high
risk individuals: a randomised placebo-controlled trial. Lancet
2002;360:7-22.
3 Chalmers I, Rounding C, Lock K. Descriptive survey of non-commercial
randomised controlled trials in the United Kingdom, 1980-2002. BMJ
2003;327:1017-9.
4 Nuffield Trust for Research and Policy Studies. Learning from experience:
privacy and the secondary use of data in health research. London: Nuffield
Trust, 2002.
Corrections and clarifications
Understanding sensitivity and specificity with the right
side of the brain
We introduced a typographical error when we
redrew the summary figure for this article by
Tze-Wey Loong, and unfortunately this was not
noticed at the proof stage (27 September,
pp 716-9). The bottom orange block should be
labelled “Number of positive results” [not “Number
of people with the disease”].
A history lesson
In this filler by Catriona Rundle (6 September,
p 545) a bizarre editorial error that we have not yet
been able to unravel led to the author’s institution,
Perth Royal Infirmary, being wrongly assigned to
Perth in Western Australia (whose main hospital is
Royal Perth Hospital) rather than to Perth in
Scotland.
1043