Académique Documents
Professionnel Documents
Culture Documents
DOI: 10.1902/jop.2016.150554
GENERAL COMMENTS
Statistical analyses are routinely used for the management and interpretation of data from
interventional studies in clinical periodontal research. In situations where two clinical
interventions are compared (i.e., test vs. control) and a statistically significant difference is
found in the outcome variable of interest, the null hypothesis is rejected and it is often
concluded that one treatment procedure is better or superior to another.1-3 However, a
statistically significant difference, per se, does not necessarily mean that the difference is
clinically important. In evaluating the data from clinical studies the clinician must ask
themselves the following question, Is the statistically significant difference clinically
relevant and does this finding allow me to advocate and use this therapy as the most
effective and safe way to manage the clinical problem? The issue of statistical significance
versus clinical relevance is extremely important and all areas of medicine have been
struggling with it for a long time.1, 2, 4-13 Indeed, several authors have stressed that statistical
findings are often inappropriately considered as clinically relevant.1, 2, 4-13
DOI: 10.1902/jop.2016.150554
nonsurgical periodontal interventions (i.e., test vs control therapies), it was found that the
test therapy led to a statistically significant reduction (p = 0.01) in the mean number of
bleeding sites with periodontal probing depths [PD] 6 mm than the control therapy (e.g. 5
sites vs 15 sites). As a result, it might be concluded that the test treatment reduced the
number of sites that were candidates for surgical therapy (e.g., open-flap debridement).
DOI: 10.1902/jop.2016.150554
CLINICAL RELEVANCE
Different definitions can be applied to the terms clinical relevance or clinical significance,
such as, a way to determine the practical value of a treatment, as opposed to the
statistical significance,4 the minimally clinically important difference between
groups,7 or a returning to normal functioning.6 Jacobson et al.6 have suggested two
criteria that are needed to establish the minimum requirements of a clinically significant
change: (1) The magnitude of the change has to be statistically reliable and (2) by the end
of therapy, clients/patients have to end up in a range that renders them indistinguishable
from well-functioning people.6 In terms of periodontal therapy, the minimum requirements
of clinically significant outcomes include those associated with potential clinical
changes/improvements after therapy (e.g., gains in CAL and reduction in PD, BOP and
mobility) as well as patient-reported outcomes (e.g., comfort, better aesthetics and ability to
chew).
To reach these objectives, the following should be considered:
1 Clinicians judgment on whether a new therapy seems applicable in terms of clinical
outcomes (Are the clinical improvements obtained with the tested therapy as effective as the
gold-standard established by the periodontal literature?);
2 Patient-centered outcomes and their importance in the implementation of novel
treatment approaches in the future (e.g., adverse effects [such as. discomfort, pain],
functional limitations [e.g., limitations on the chewing and deglutition of food], costs, and
patients preferences);5, 13, 20, 21
3 Long-term impact of treatment (Do the short-term results of therapy remain longterm?);
4 The interpretation of the base of evidence from efficacy and effectiveness trials.
The use of statistical approaches has been also described to determine clinical relevance.
For instance, the Cohens effect size22 computes the degree of the relationship between
outcome measures as well as the magnitude of the difference of the groups based on two
mean scores (one from each group) and the pooled standard deviation of the groups
(meanTest group mean Control group/SDpooled). However, the issue of clinical relevance is more
involved with aspects related to an ample group of conditions that have been studied by
evidence-based medicine/dentistry.
DOI: 10.1902/jop.2016.150554
certainly a critical issue due to inherent conditions related to the protocol of the study [i.e.,
adherence to the study design, sample of patients treated, expertise of clinician(s) providing
treatment], and the number of studies included in the analysis.15, 20, 21 Meta-analyses
reporting, or not reporting, statistical differences assist the clinician in rendering a treatment
plan, but they do not certify that one therapy is much better than the other. It should be noted
that factors associated with the periodontal diagnosis, sample size, and the applied inclusion
criteria may make the pooling of data from different trials a critical issue. For instance, the
inclusion of studies with mean PD > 6 mm can increase the impact of outcome change by
promoting greater differences between baseline and follow-up means (i.e. baselinemean followupmean = outcome change), an element that may impact the estimates of meta-analyses.
Thus, it is expected that studies should be combined into meta-analyses according their
baseline characteristics in order to generate more accurate evaluations.18, 20, 21 Moreover,
both RCTs and systematic reviews may not answer key questions regarding uncommon
conditions where an adequate sample of patients may not be easily available for analysis, or
for clinical outcomes of interest that could not be reproduced in humans due to ethical
aspects. Thus, information gathered from private-practice studies should assist in answering
some research questions.
Effectiveness is a very significant question, which has not been explored to the same
extent that efficacy has, and the idea of effectiveness is based on outcome of therapy by
average clinicians in the community and not by research experts.21 Most practitioners are not
used to designing research protocols or conducting/running clinical studies, and only a few
will ever present the results of their work as case reports/series.21 These factors are
important and deserve distinction, because often there is a gap between protocols
investigated by experts and those carried out in clinical practice.21 As a result, other
assessment tools/criteria that take into account the available clinical evidence and expert
opinions might be considered. An example of such an assessment tool is the one defined by
the U.S. Preventive Services Task Force (USPSTF) adapted by the American Dental
Association in 2013,24 with the intent that it may guide the strength of recommendation of a
procedure as follows: a) strong evidence strongly supports providing this intervention; b)
in favor evidence favors providing this intervention; c) weak evidence suggests
implementing this intervention after alternatives have been considered; d) expert opinion for
evidence is lacking; the level of certainty is low (expert opinion guides this
recommendation); e) expert opinion against evidence is lacking; the level of certainty is
low (expert opinion suggests not implementing this intervention); and f) against evidence
suggests not implementing this intervention or discontinuing ineffective procedures.24
In other words, new or alternative procedures should be used when definitive
information is scarce and evidence based on clinical experience must be used to fill the gap
in knowledge until strong evidence becomes available.
CONCLUDING REMARKS
The findings of a study should allow for the balanced interpretation of both the statistical
significance and clinical relevance of the data followed by a thoughtful assessment of their
clinical applicability in a practical setting.1 The use of P-values is important, but other useful
estimates/tools such as confidence intervals (CIs) can quantify the amount of differences
between groups. For instance, an honest way to evaluate the balance between significance
and relevancy relies on the following question proposed by Saiani11: Are any of the values
within the 95% CI big enough to care about? If the answer is no, then the effect is clinically
insignificant and statistical significance is immaterial.
DOI: 10.1902/jop.2016.150554
Without a doubt the use of statistical analysis is certainly important and necessary for
clinical research. However, statistically significant differences alone between two
procedures may not be sufficient to support the use of a new therapy in private practice. An
evidence-based clinical practice clearly requires the data generated by clinical research, and
statistical analysis of the data provides a basis for the development of treatment strategies
and helps in the decision-making process. However, cost-benefit analyses and magnitude-ofchange determinations of any treatment approach depend on how much of the statistical
differences can be translated into clinically useable tools for daily practice.
CONFLICT OF INTEREST
The authors report no conflict of interest related to this commentary.
REFERENCES
1.
Bhardwaj SS, Camacho F, Derrow A, Fleischer Jr AB, Feldman SR. Statistical significance and clinical
relevance the importance of power in clinical trials in dermatology. Arch Dermatol 2004; 140: 1520-1523.
2.
Fethney J. Statistical and clinical significance, and how to use confidence intervals to help interpret both.
Aust Crit Care 2010; 23: 93-97.
3.
Tu Y-K, Gilthorpe MS. Key statistical and analytical issues for evaluating treatment effects in periodontal
research. Periodontol 2000 2012; 59: 75-88.
4.
Jacobson NS, Follette WC, Revenstorf D. Psychotherapy outcome research: methods for reporting
variability and evaluating clinical significance. Behav Ther 1984; 15: 336-352.
5.
Todd KH. Clinical versus statistical significance in the assessment of pain relief. Ann Emerg Med 1996;
27: 439-441.
6.
Jacobson NS, Roberts LJ, Berns SB, McGlinchey JB. Methods for defining and determining the clinical
significance of treatment effects: description, application, and alternatives. J Consult Clin Psychol 1999;
67: 300-307.
7.
Beaton DE, Boers M, Wells GA. Many faces of the minimal clinically important difference (MCID): a
literature review and directions for future research. Curr Opin Rheumatol 2002; 14: 109-114.
8.
Campbell TC. An introduction to clinical significance: an alternative index of intervention effect for group
experimental designs. J Early Intervent 2005; 27: 210-227.
9.
Baicus C, Caraiola S. Effect measure for quantitative endpoints: Statistical versus clinical significance, or
how large the scale is? Eur J Internal Med 2009; 20: e124e125.
10. Thirthalli J, Rajkumar RP. Statistical versus clinical significance in psychiatric researchAn overview for
beginners. Asian J Psychiatr 2009; 2: 7479
11. Sainani KL. Clinical versus statistical significance. Phys Med Rehabil 2012; 4: 442445.
12. Jakobsen JC, Gluud C, Winkel P, Lange T, Wetterslev J. The thresholds for statistical and clinical
significance a five-step procedure for evaluation of intervention effects in randomised clinical trials.
BMC Medical Res Methodol 2014; 14: 34.
13. Sedgwick P. Clinical significance versus statistical significance. BMJ 2014; 348: g2130.
14. Hujoel PP, Armitage GC, Garcia RI. A perspective on clinical significance. J Periodontol 2000; 71: 15151518.
15. Chambrone L, Chambrone D, Lima LA, Chambrone LA. Predictors of tooth loss during long-term
periodontal maintenance: a systematic review of observational studies. J Clin Periodontol 2010; 37: 675
684.
16. Tonetti MS, Claffey N. Advances in the progression of periodontitis and proposal of definitions of a
periodontitis case and disease progression for use in risk factor research. Group C consensus report of the
5th European Workshop in Periodontology. J Clin Periodontol 2005; 32 (Suppl. 6): 210213.
DOI: 10.1902/jop.2016.150554
17. Matuliene G, Pjetursson BE, Salvi GE, Schmidlin K, Brgger U, Zwahlen M, Lang NP. Influence of
residual pockets on progression of periodontitis and tooth loss: Results after 11 years of maintenance. J
Clin Periodontol 2008; 35: 685695.
18. Esposito M, Grusovin MG, Coulthard P, Worthington HV. Enamel matrix derivative (Emdogain) for
periodontal tissue regeneration in intrabony defects. Cochrane Database Syst Rev 2005; Issue
4:CD003875
19. Needleman IG, Worthington HV, Giedrys-Leeper E, Tucker RJ. Guided tissue regeneration for
periodontal infra-bony defects. Cochrane Database Syst Rev 2006; Issue 2:CD001724.
20. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0
[updated September 2011] The Cochrane Collaboration 2011. Available from www.cochranehandbook.org (accessed in May 15, 2015).
21. Chambrone L, Tatakis DN. Periodontal soft tissue root coverage procedures: a systematic review from the
AAP Regeneration Workshop. J Periodontol 2015; 86(2 Suppl): S851.
22. Cohen J. The concepts of power analysis. In: Cohen J. editor: Statistical power analysis for the behavioral
sciences. Hillsdale, New Jersey: Academic Press, Inc: 1998. p. 1-17.
23. Pihlstrom BL, Curran AE, Voelker HT, Kingman A. Randomized controlled trials: what are they and who
needs them? Periodontol 2000 2012; 59: 1431.
24. ADA Clinical Practice Guidelines Handbook [updated November 2013]. American Dental Association
available at: http://ebd.ada.org/contentdocs/ADA_Clinical_Practice_Guidelines_Handbook__2013_Update.pdf (accessed and downloaded in 29 January 2014).
Contact : Leandro Chambrone, Rua Antonio Pinto Guedes, 626, Mogi das Cruzes, SP,
08820-430 Brazil e-mail: leandro_chambrone@hotmail.com
Submitted September 10, 2015; accepted for publication December 22, 2015.