Vous êtes sur la page 1sur 9

Meta-analysis of Bacterial Meningitis Score

Validation Studies
Lise E Nigrovic, Richard Malley, Nathan Kuppermann

Disclosures
Arch Dis Child. 2012;97(9):799-805.

Abstract and Introduction

Methods

Results

Discussion

Conclusions

References

Sidebar 1

Sidebar 2

Print

Abstract and Introduction


Abstract
Objective The Bacterial Meningitis Score, a derived and validated clinical decision rule, identifies
children with cerebrospinal fluid (CSF) pleocytosis who are at very low risk of bacterial meningitis.
Low-risk features include the following: negative CSF Gram stain, CSF absolute neutrophil count
(ANC) <1000 cells/l, CSF protein <80 mg/dl, peripheral blood ANC <10 000 cells/l and no seizure
at or prior to initial presentation. The study objective of the present work was to calculate the
performance of the Bacterial Meningitis Score by performing a meta-analysis of all published
validation studies.
Methods A meta-analysis of all studies published between 2002 and 2012 was performed,
evaluating the performance of the Bacterial Meningitis Score in children with CSF pleocytosis. Study
quality was assessed using the assessment of diagnostic accuracy studies instrument and then the
test performance of the prediction rule was calculated.
Results From 8 studies, 5312 patients were identified, of whom 4896 (92%) had sufficient clinical
data to calculate the Bacterial Meningitis Score. Bacterial meningitis was diagnosed in 1242 children
(23% of study patients). The combined sensitivity of the Bacterial Meningitis Score for bacterial
meningitis was 99.3% (1224/1233; 95% CI 98.7% to 99.7%), specificity 62.1% (2274/3663; 95% CI
60.5% to 63.7%) negative predictive value 99.7% (2274/2283, 95% CI 99.3% to 99.9%), positive
likelihood ratio 2.6 (95% CI 2.5 to 2.7) and negative likelihood ratio 0.01 (95% CI 0.01 to 0.02).

Conclusions The Bacterial Meningitis Score is a highly accurate clinical scoring system that could
be used to assist clinical decision making for the management of children with CSF pleocytosis.

Introduction
In regions of the world with high vaccination rates, the incidence of bacterial meningitis has declined
substantially due to highly effective conjugate vaccines.[15] However, children with cerebrospinal fluid
(CSF) pleocytosis are frequently hospitalised and given broad-spectrum antibiotics while awaiting
the results of bacterial cultures, which may take 48 h to reliably exclude bacterial growth. [68]
Children at very low risk for bacterial meningitis can be considered for outpatient management if they
are otherwise well appearing and have adequate clinical follow-up. The Bacterial Meningitis Score
clinical prediction rule was derived and internally validated from a retrospective cohort of 696
children with CSF pleocytosis hospitalised at a single institution. [9] Although the Bacterial Meningitis
Score performed with very high accuracy, clinical application was limited by the small study sample
size, single-centre design with a highly referred population, lack of external validation, as well as
ongoing changes in the epidemiology of bacterial meningitis related to the introduction of bacterial
conjugate vaccines.[5]
The 'real-world' performance of a clinical prediction rule is most accurately assessed by its
application in a variety of clinical settings. We initially tested the Bacterial Meningitis Score in a large
multicentre US study of children with CSF pleocytosis.[10] The Bacterial Meningitis Score has also
been evaluated in six additional studies by independent investigators. [1116] In this study, we sought to
measure the accuracy of the Bacterial Meningitis Score by aggregating the patients from the eight
validation studies and to report the performance of the prediction rule in the combined population.

Methods
Study Design
We performed a fixed-effects meta-analysis of the published Bacterial Meningitis
Score validation studies.[17 18] We searched the Medline and Embase electronic
databases for eligible articles published between October 2002 and March 2012. We
used the following search terms: Bacterial Meningitis Score, bacterial meningitis
prediction and meningitis validation study. Additionally, we reviewed all publications
that referenced the derivation study.[9] One publication that is currently in press was
identified by a published abstract and we subsequently communicated with the
corresponding author. We reviewed potentially eligible studies to identify those that

included children younger than 18 years of age as well as sufficient information to


calculate the Bacterial Meningitis Score. We excluded patients used for the
prediction model derivation conducted by the study investigators. [9]

Data Collection
We reviewed eligible studies to determine study design, inclusion and exclusion
criteria, patient population and case definitions. Because antibiotic pretreatment can
render bacterial cultures falsely negative[19 20] and also impact CSF profiles,[21] we
excluded studies in which patients had received antibiotics prior to lumbar puncture
(LP) (defined as 'antibiotic pretreatment'). We contacted the corresponding authors
of the published studies to clarify study methods, as necessary. We excluded studies
for which we could not verify study procedures. We assessed the quality of the
included studies using the Quality Assessment of Diagnostic Accuracy Studies 2
(QUADAS-2) instrument.[22]

Bacterial Meningitis Score Performance


For each of the included studies, we determined the number of patients with
bacterial and aseptic meningitis using study-specific case definitions. Children with
none of the five high-risk Bacterial Meningitis Score predictors were classified at
'very low risk' for bacterial meningitis (Table 1).[9 10]Children with one or more high-risk
predictor were classified as 'not low risk' even if other predictors were missing.
Otherwise, children missing predictors included in the Bacterial Meningitis Score
were excluded from the prediction rule validation.

Statistical Analysis
We abstracted the Bacterial Meningitis Score as calculated by the study
investigators. We then calculated the performance of the dichotomised Bacterial
Meningitis Score ('very low risk' vs 'not very low risk') in the aggregated patient
population. Our primary outcome was the prediction rule sensitivity, which we
calculated for each of the included studies. Our secondary outcomes were
specificity, negative predictive value (NPV), positive predictive value (PPV) as well
as positive and negative likelihood ratios. We used patients from case-control and
cohort studies for calculation of sensitivity and specificity, but not for PPV and NPV.

We used children from cohort studies for the calculation of sensitivity, specificity,
NPV, PPV as well as likelihood ratios using standard techniques. [23]
We present the pooled effects as a point estimate with 95% CI using binomial
methods with a Forest plot for the primary outcome measure. We calculated the Q
statistic (2-distributed with one less than the number of included studies degrees of
freedom) to assess the heterogeneity between studies as well as the I2 statistic to
assess for consistency of the results across studies.
As a prespecified subgroup analysis, we also calculated the performance of the
Bacterial Meningitis Score in the subgroup of study patients not included in either
the internal[9] or the multicentre[10]validation studies conducted by this study's
investigators.
We performed statistical calculations using Stata statistical software. [2

Results
We identified 405 published studies using our search strategy of which 10 met our
inclusion criteria. We excluded an adult cohort study (111 patients) [25] and a
paediatric cohort study (91 patients)[26] because insufficient details were provided to
allow us to assess the patient population or the antibiotic pretreatment status. We
included the remaining eight studies in this analysis ( Table 2 ).[916] We assessed the
quality of the included studies using the QUADAS-2 instrument (Table 3). [22]

(Enlarge Image)

Table 3.
Quality assessment of diagnostic accuracy studies 222

The eight included studies had the following study designs: case series (one study),
[13]
case-control (one study),[12] retrospective cohort (five studies)[911 14 16] and
prospective cohort (one study).[15]Study patients presented for emergency care in the
USA,[9 10] Western Europe[11 12] and South America (Argentina).[15] Of the 5312 included

children, 1242 (23%) had bacterial meningitis and 4070 (77%) had aseptic
meningitis.
The patient populations for each validation study varied slightly ( Table 4 ). One
study was limited to patients with bacterial meningitis [13] and another had available
procalcitonin results.[14] Five studies were of hospitalised children, including patients
referred for management of meningitis.[9 11 12 1416]One other study was conducted in the
emergency department[10] and the other was a nationwide meningitis registry (216
participating institutions).[13 27] While Haemophilus influenzae type B vaccination rates
were high in all study populations, only one population (USA) had widespread
seven-valent pneumococcal conjugate vaccination available during the study period.
[10 28]
None of the populations had routine meningococcal vaccination during the study
period. All studies excluded children with immunosuppressive medical conditions or
therapies, while the other exclusion criteria varied by study: critical illness, [9 10 14
16]
recent neurosurgery or presence of a ventricular shunt, [9 10 1216] purpura,[9 10] focal
bacterial infections requiring parenteral antibiotic treatment, [9 10] traumatic LP,[12 13
16]
Lyme meningitis[16] or transferred patients.[16]
While in all studies the bacterial meningitis case definition included patients with
CSF culture positive for a bacterial pathogen, they varied in whether patients with a
positive CSF Gram stain,[12] latex agglutination test,[12 1416] CSF bacterial PCR test[15
16]
or CSF pleocytosis with a positive blood culture but negative CSF culture [911 13 15
16]
were considered to have bacterial meningitis. Although patients pretreated with
antibiotics were excluded, the time between antibiotic administration and diagnostic
LP defined as antibiotic pretreatment was not standardised.
The Bacterial Meningitis Score could be calculated for 4896 (92%) of the 5312
patients in the aggregated patient population ( Table 5 ). Patients from all studies
contributed to the calculation of test sensitivity. Patients from seven studies were
included in the calculation of test specificity and from six studies in the predictive
value and likelihood ratio calculations. The Bacterial Meningitis Score had an overall
test sensitivity of 99.3% (95% CI 98.7 to 99.7%) (figure 1). For the primary outcome
measure (sensitivity), we found no evidence for heterogeneity between studies
(p=0.88) or results (I2=0.0) in our meta-analysis. Children categorised as 'not very
low risk' by the Bacterial Meningitis Score had a positive likelihood ratio for bacterial
meningitis of 2.6 (95% CI 2.5 to 2.7) and those categorised as 'very low risk' by the

Bacterial Meningitis Score had a negative likelihood ratio of 0.01 (95% CI 0.01 to
0.02).

(Enlarge Image)

Figure 1.
Forest plot showing Bacterial Meningitis Score sensitivity with 95% CI for individual studies (and a
pooled estimate).

Among the 1783 children included in the 6 validation studies not conducted by this
study's investigators, 1083 (61%) had bacterial meningitis and 700 (39%) had
aseptic meningitis. In this subgroup, the Bacterial Meningitis Score had a sensitivity
of 99.3% (95% CI 98.6% to 99.7%), specificity of 61.0% (95% CI 57.3 to 64.7%),
NPV of 98.3% (95 CI 96.6% to 99.3%) and PPV 28.1% (95% CI 22.6% to 33.9%).
We could not exclude the possibility that children with bacterial meningitis from 1 of
the 216 centres in the bacterial meningitis registry[13] may also have been included in
a more recent multicentre retrospective cohort study (personal communication, M.
Chalumeau, Hpital de Paris). However, when we excluded this cohort study, the
Bacterial Meningitis Score performed similarly (data not shown).
Nine patients with bacterial meningitis were classified as 'very low risk' by the
Bacterial Meningitis Score ( Table 5 ). Of these, three were younger than 2 months
of age (an age at which we previously recommended the Bacterial Meningitis Score
not be applied)[10] and three others presented with petechiae or purpura on
examination.[13] The three misclassified patients with bacterial meningitis who were
older than 2 months and who did not have a petechial or purpuric rash on presenting
examination are described in Table 6 (patients 3, 6 and 9). All three of these children
had meningococcal meningitis.

Discussion
The Bacterial Meningitis Score, a clinical prediction rule to identify children with CSF
pleocytosis who are at very low risk of bacterial meningitis, has been validated in

eight published studies.[916] In this meta-analysis, the score had a sensitivity of


99.3% (95% CI 98.7% to 99.7%) for bacterial meningitis. The included studies were
of high quality and did not have significant study heterogeneity. Although the study
designs, patient population, inclusion and exclusion criteria as well as bacterial
meningitis case definition differ between studies, the Bacterial Meningitis Score
performed with a high degree of accuracy. When the validation studies conducted by
this study's investigators were excluded, the results did not change.
A clinical prediction rule is a decision-making tool that combines history, physical
examination and laboratory results to predict the probability of an outcome for an
individual patient. Clinical prediction rules must be developed and validated
according to rigorous methodological standards prior to widespread implementation.
[2931]
While prospective rather than retrospective validation is typically preferred,
because of the rarity of bacterial meningitis in high-income countries, [5] such a
validation is not readily feasible; accordingly, seven of the eight included studies
were retrospective. We do not believe that the retrospective validation introduced
important biases since the predictors are objective. Four of the five factors are
laboratory values and one clinical factor (seizure at or prior to the time of initial
presentation), should be reliably recorded in the medical records. [32 33]
For children younger than 18 years of age in the USA, the incidence of bacterial
meningitis caused by H influenzae, Streptococcus pneumoniae, group B
streptococcus (GBS), Neisseria meningitidis or Listeria monocytogenes declined
31% over the past decade.[5] Despite these substantial declines in the overall
incidence, the bacterial meningitis case death rate has remained unchanged at 7%;
most affected children do not have predisposing medical conditions. [5] Given the high
mortality and morbidity, clinicians must still maintain a high index of suspicion for
bacterial meningitis. Looking ahead, the increasing availability of the 13-valent S
pneumoniae and quadrivalent N meningitidis conjugate vaccines in high-income
countries will further decrease the incidence of bacterial meningitis. The Bacterial
Meningitis Score can support clinician decision making by providing a highly
accurate initial assessment of the risk of bacterial meningitis for children with CSF
pleocytosis.
We recommend that the Bacterial Meningitis Score, as with other clinical prediction
rules, be used to assist rather than replace clinical decision making. [34] Although the
score performs extremely well, it is highly unlikely to develop a clinical prediction rule

with 100% accuracy in all patient populations.[35] Of note, the misclassified patients
who presented without petechia or purpura all had meningococcal meningitis which
has been previously described to present without CSF pleocytosis. [36] Nevertheless,
children with a very low-risk Bacterial Meningitis Score are at such low risk of
bacterial meningitis that they may be considered for outpatient management,
potentially after the administration of a long-acting parenteral antibiotic. Currently,
most children with CSF pleocytosis are hospitalised and given parenteral antibiotics
in order to avoid missing the very few with bacterial meningitis. Application of the
Bacterial Meningitis Score could substantially reduce unnecessary hospitalisation of
children with aseptic meningitis, while still providing a safety margin by the
administration of long-acting antibiotic prior to culture results. As we have previously
recommended,[10] however, the Bacterial Meningitis Score should not be applied to illappearing children, infants 2 months and younger, in whom the risk of bacterial
meningitis is highest[5] and to those with physical examinations suggestive of
invasive bacterial infection (eg, those with petechiae or purpura). [37] This approach
will further reduce the risk of misclassification of children with bacterial meningitis.
Furthermore, biologic markers of inflammation have been investigated for their
ability to discriminate between cases of bacterial and aseptic meningitis. For
example, CSF lactate[38 39] and serum procalcitonin[40 41] are higher in children with
bacterial than aseptic meningitis, although there is overlap in levels of these
biomarkers. Diagnostic assays that use RNA expression to identify host response to
specific pathogens are currently being studied in the clinical setting. [4244] In the future,
these novel assays might allow rapid identification of specific meningitis pathogens,
or host responses associated with bacterial meningitis.
Our study has some limitations. First, as most of the validation studies were
retrospective, we could not evaluate the general appearance of the patients, which
plays an important role in clinical decision making. Certain clinical factors such as
the presence of petechiae or purpura could not be evaluated in all studies.
Nevertheless, the accuracy of the Bacterial Meningitis Score remained very high in
each of the evaluated studies. Second, we were only able to include children who
had sufficient clinical data to apply the Bacterial Meningitis Score from published
validation studies. Given the number of patients evaluated in these studies and the
objective nature of the variables in the score, it is unlikely that the model would have
performed substantially differently in patients with missing variables. Third, we could

not exclude the possibility that a few children with bacterial meningitis from a single
centre may have been represented in two included studies. [13 14] However, the
prediction model performed similarly if this recent study was excluded (and therefore
eliminating the possibility of patient redundancy). Fourth, we were unable to include
patients from the case-control or registry studies in the calculations of NPV, PPV or
likelihood ratios as these calculations require population prevalence. The variability
between experimental definitions and methods may have reduced our ability to
accurately combine the study patients. However, the similar performances of the
model across a wide variety of clinical settings and patient populations as well as the
lack of heterogeneity in the sensitivity and NPV estimates increase the
generalisability of our findings. Furthermore, we recognise that there is a
preponderance (almost two-thirds) of patients from a single validation study.[10] To
address a potential skewing effect, we performed an additional subgroup analysis
after excluding studies conducted by this study's investigators, and found similar
results to the main analysis. Finally, the Bacterial Meningitis Score does not predict
the likelihood of other central nervous system infections such as herpes simplex
virus, Lyme[4547] or tuberculous meningitis. Therefore, this clinical prediction rule
should be used in concert with careful clinical assessment of the patient, which
would include consideration of these other important treatable infections.

Conclusions
In summary, the Bacterial Meningitis Score performed with a high degree of
diagnostic accuracy in eight validation studies. This score, in conjunction with clinical
judgment can identify children with CSF pleocytosis who are at very low risk for
bacterial meningitis. To minimise misclassification of children with bacterial
meningitis, we recommend that the Bacterial Meningitis Score only be applied to
non-ill-appearing children older than 2 months, who do not have either petechiae or
purpura on examination and have not been pretreated with antibiotics. For those
children at very low risk, who have adequate clinical follow-up, clinicians could
consider outpatient treatment after administration of a long-acting parenteral
antibiotic. Future studies should focus on the implementation of the Bacterial
Meningitis Score to prospectively identify children who are at very low risk of
bacterial meningitis.