Research Report

Quantitative Analysis Research

Physical Therapy

Background and Purpose. Some physical therapists argue that research will Valma J Robertson
justrh and improve practice in the discipline. Whether these outcomes can be
realized is partly a function of the nature of the research, the research methods
used, and the extent to which clinical treatments are investigated.Subjects
and Metbods. This study analyzed aspects of a sample of 272 articles pub-
lished in Physical Therapy during 1954 through 1 9 3 . Each article was read
and categorized according to the method used and to whether it investigated
clinical treatments. Results. An increasing percentage of the articles sampled
described studies that used group designs, and most of the studies did not in-
vestigate clinical treatments. ConcCusfonand Dfsnrssfon TheseBndings a m
paradoxical given existing criticisms of group designs and eqctations in phys-
ical therapy of 1-esearch.More discussions and evaluations of current research
practices in physical therapy are needed. [Robertson VJ.A quantitative analysis
of research in Physical Therapy. Phys Ther. 1995;75:315327.1

Key Words: Physical therapy profession, Publications and audiovisual material,

Research, Research methods.

Research and interest in research have apy Inc in the United States). Journal Physical therapists appear to anticipate
increased in physical therapy over the editors and ex-editors have high- that research will benefit the profes-
past two decades. Many articles and lighted an increased publication of sion and practice of physical therapy.
books on how to do research have research in the profession.12 New Among the benefits expected from
been published.'-5 Physical therapy English-language journals, including research are scientific credibility1' and
databases have been developed (eg, the Journal of Orthopaedic and S p m the growth and development of the
Physiotherapy Index). Editorials and Physical Therapy and Physiotherapy profession.14J5Research is also ex-
journal policy statements about re- Theory and Practice, have appeared. pmed by some to validate current
search have been pubhhed in most At the same time, established physical practice16 and to satisfy increasing
physical therapy Articles therapy journalsAustralian Joumal demands for the accountability of
encouraging the formation of special of Physiotherapy, Physical Thmpy, professional gr0ups.~71~~ Various
interest groups on research and on Physiotherapy Canada, and Physio- sources expect treatments to be of
obtaining funding for research have t h e r a p have increased the volume high quality and cost effective: and
been published.lO,"In some countries, of research contributions they publish. research is expected to demonstrate
special funds for research in physical Together, these developments demon- that physical therapy interventions are
therapy have been instituted (eg, the strate the increasingly active interest in cost effeai~e~9~~0 and of hlgh quality.21
Physiotherapy Research Foundation in research within physical therapy over Research is also expected to demon-
Australia and in the United Kingdom, the past two decades. strate and improve the effectiveness of
and the Foundation for Physical Ther- physical therapy techniques, interven-
tions, and modalitie~.7.~~.~~.~3The em-
phasis on the benefits of research for
clinical practice reflects the centrality
VJ Robertson, PhD, BAppSdPhty), BA(Hons), is Senior Lecturer, School of Physiotherapy, La
Trobe University, Locked Bag 12, Carlton South, Victoria, Australia 3053.
of practice in physical therapy. An
important question is whether re-
7bis article was submittedjanuury ZG, 1994, and was accepted November 23, 1994.

Physical Therapy / Volume 75, Number 4 / April 1995 313 / 69

search is providing physical therapy
with the expected benefits.
Research outcomes are in large part a
function of the suitability and produc-
tivity of the research method used. A
research method most conspicuously
reconlrnended in physical therapy,
both explicitly and hlplicitly, is group-
experimental designs. This is an ex-
perimental method that compares
oulcomes for groups, rather than out-
comes for individuals or series of
individuals (ie, single-subject experi-
mental designs), following the applica-
tion of controlled interventions.l(~lR*
The method is also known as group-
companson experimend(p1'14)and
randomized clinical trials.24-27 If
some requirements of the group-
experimental method are made less
rigorous, it is then called a quasi-
group-comparison design or a clin-
ical trial. Explicit support is appar-
ent in the positive advocacy of the
investigatory powera~27,znand gen-
cia1 dcccptdbility of the group-
expuhilental irlethtd.26 Inlplicit sup-
port for group-experimental designs is
apparent in the extent to which arti-
cles and books aimed at the physical
therapy market focus on these de-
signs. Some books and articles that
describe how to do research 1 2 4 28-33
I 3
and how to evaluate resear~h.3~83~
focus almost exclusively on group-
experimental designs.
Published criticisms of group-
experimental designs suggest that the
method might not be optimal for gen-
eral use in the clinical ~ o n t e x t . l ~ ~ - ~therapy
hnong other things, criticisms of
group-experimental designs have
referred to difficulties in obtaining
suficiently large and appropriately
homogeneous samples of subjects
within the required time period,%-3"n
providing and maintaining the re-
quired experimental control within the
clinical context,l.39-" in using statisti-
cal procedures correctly and appropri-
ately (eg, in medicine43and orthope-
d i c ~ ,psychology
~" and sociology,*5
and social worka), in interpreting
findings based on Inferential statii-
ti~s,"~,4'-5land in validating findings
and cumulating knowledge from re-
search using group-experimental de-
s i g n ~ . %Further,
, ~ ~ when used to inves- does research in physical therapy
tigate the relatively small effects investigate clinical treatment? Does
typically obtained in clinical practice, research focusing on clinical treat-
group designs (experimental and ments use methods different from
quasi) produce findings with low those used in nonclinical research?
power and a reduced probability of
correctly rejecting null hypotheses.52 Answers to these questions about
published physical therapy research
The relevance of many findings from are not available, although some rele-
group designs for practitioners is ques- vant studies69J0have been conducted.
tionable. The findings are based on These studies indicate that the relative
averaged group performance when frequency of studies using group-
clinicians focus on indi~iduals.~~~+55 experimental designs ranges from
Another criticism suggests that the 10.7% in Physiotherapy Canada, Phys-
choice of problems investigated is ical Z%erapy, and Physiotherapr com-
constrained by amenability to statisti- bined (data reworked70) to 41% in
cal analysis48s5%ther than by clinical, Physiotherapy Canada.@ Another
theoretical, or practical significance study71showed that the relative fre-
and relevance. Other criticisms of quency of "experimental trials" in-
group-experimental designs concern creased from 19??over the period
the ethical dilemmas raised by the 1980 to 1984 to 27% over the period
requirements of the research method. 1985 to 1989 in articles from the jour-
Such ethical issues can arise in the nals Physiotherapy,Physical nerapy,
random allocation of subjects to treat- Physiotherapy Canada, Australian
ment and control g r o ~ p s ~ and
~ z 5in~ Journal of Physiotherapy,and South
the replacement of a therapeutic situa- African Journal of Physiotherapy.The
tion responsive to the changing needs category definitions provided in that
of the patient by one in which treat- study do not make clear how clinical
ment is based on previously specified studies using group designs would be
experimental procedures.36 Some of classified and apparently exclude
these problems may be obviated by methods such as single-subject experi-
using the less rigorous quasi-group- mental designs and series of case
comparison designs. Some of the studies. The definitions provided in
practical problems inherent in clinical other stud1esM,7~~~3 do not differentiate
research, such as the recruitment of a group-experimental designs from
sufficient number of appropriate sub- quasi-group-comparisondesigns and
jects, however, still tend to remain for other data-based research methods.
users of group designs. The conclusions of some studies of
research methods are limited by the
The popularity of group- lack of details of the source or the
experimental designs in physical method of selection of the articles
~ research and the critiques exarninedM~69,~~ and by the reliance on
of these designs in clinical research a convenience sample of
raise some questions. To what published over a period of relatively
extent does research in physical short duration (10 years maximum) in
therapy depend on group- a context in which the relative fre-
experimental designs? Conversely, quency of the different types and
to what extent are alternative re- topics of articles can vary markedly
search methods used? Possible from year to year.
alternative research methods dis-
cussed in physical therapy journals My study had three speclfic aims: (1)
include the following: case studies to establish the frequency of use of
and series of ca~es,~,5~-59 single- group-experimental designs in a longi-
subject experimental de~igns,3~,60-63 tudinal sample of articles from the
ethnographic studies,64l65sequential physical therapy literature, (2) to ex-
trials design,36,66 and surveys.67 amine the extent to which the same
Books intended for physical thera- sample of articles investigated clinical
pists discuss these and other re- treatments, and (3) to investigate the
search methods.',68 To what extent extent to which clinical research was
Physical Therapy / Volume 75, Number 4 /April 1995
based on methods differing from those
used to examine nonclinical issues.
Method
Literature Sample
In this study, I analyzed a sample of
272 articles published from 1754 to
1 9 3 . The sample was derived from
Physical i'Berapy, a physical therapy
journal described in its purpose state-
ment as "a scholarly, refereed journal
that contributes to and documents the
evolution and expansion of the scien-
fit and professional body of knowl-
edge in physical therapy."74
Physical Therapy was for many years
the sole physical therapy journal p u b
lished monthly and indexed in Index
Medicus and the Science Citation
Index as well as in physical therapy
indexes (eg, Physiotherapy Current
Awareness Topic Search, Physiother-
apy Index). Physical irherapy also has
a growing impact factor75and is cited
more frequently within the profession
than any other physical therapy
j0urna1.7~3~
Articles published in Ph.ysica1 fierapy
prior to 1954 were not included in this
sample for three reasons. First, the
number of articles published per year
prior to 1954 was small, especially
during World War 2. Second, by 1754,
the Journal had changed to its current
monthly publication rate (in 1747).
Third, a four-decade period was ex-
pected to provide a suficiently long
time frame to reliably indicate trends
in the aspects of the articles investi-
gated in this study.
The sample of 272 articles consisted of
all 106 articles indexed in PF?ysical
fierapy on the topic "knee," all 79
articles indexed on the topic "back,"
and all 87 articles indexed on the
topic "electrical stimulation" between
1754 and 1993. The topics were cho-
sen by establishing which indexed
topics included the largest numbers of
articles, excluding commentaries,
author responses to commentaries,
and letters to the editor, published in
Pbwical7herapy from 1954 to 1 9 3 .
When the topics "professional issues,"
Physical Therapy /Volume 75, Numbe
"professional education," and "neuro-
physiology/neuroanatomy" were ex-
cluded, the topics of direct relevance
to clinical practice and with the largest
numbers of articles were the knee, the
back, and electrical stimulation. Given
the system of cross-referencing used
by Physical Theram, each of these
topics included articles on subjects as
diverse as anatomy, equipment, and
evaluations of treatments. Having two
anatomically defined areas and a treat-
ment modality broadened the basis for
the study and optimized clinical rele-
vance. The prominence of an electrical
modality among the Merent manual
and electrical treatment techniques
and types of exercise indexed was
consistent with the findings of an
earlier survey of the treatment most
commonly provided by physical then-
p i ~ t sIf. ~a ~particular article was in-
dexed under two or more of these
three topic areas (ie, back, knee, or
electrical stimulation), it was included
in each area for analysis of the topic.
Classification Systems
Two systems were used to categorize
each of the 272 articles. The first clas-
sification system was based on the
type of method used in each article.
The eight method categories used,
shown in Table 1, were (1) group
experimental, (2) quasi-group compar-
ison, (3) single-subject design, (4)
practical, (5) descriptive, (6) survey,
(7) case study, and (8) correlational.
The classification of the method used
in each article was based on a system
described by Ottenbacher and Short,79
who surveyed 674 research articles
published from 1970 to 1780 in the
Occupational 7herapyJoumal of
Research. These authors subsequently
tested the reliability of their categoriza-
tions and obtained an interobserver
agreement of .74 using a coefficient of
agreement. The wording of their
seven-category definitions was modi-
fied in my study in an attempt to in-
crease the precision and the relevance
to physical therapy. I also included an
eighth category called "single-subject
design" to enable the differentiation of
single-subject experimental designs
from case studies, group-experimental
designs, and quasi-group-comparison
designs.
The second classification system was
developed to drstinguish articles deal-
ing with treatment from those that did
not deal with treatment (Tab. 2). The
criteria were based on existing distinc-
tions between aspects of clinical and
nonclinical practice and research
drawn w i t h the physical therapy and
psychology literature^.^^^^^^^^ The start-
ing point was categories used by Hol-
liday70 in her analysis of research
contributions made by physical thera-
pists. The common element in the
categories Holliday called " c h c a l
surveys" and "clinical trials" was
"treatment in a clinical context."70The
term "in a clinical context" was too
general, however, to provide a suffi-
cient drstinction between clinical treat-
ment and nonclinical studies, as it did
not require that the treatment was
used to effect a beneficial outcome for
a patient. A treatment could equally
well have been tested on asymptom-
atic subjects.
Based on distinctions drawn by Gold-
i a m ~ n dclinical
,~ treatrnent studies
were differentiated from nonclinical
studies by the requirement that they
reported an aim of maintaining or
improving the function of patients
receiving the treatment. Support for
this distinction is found in a guest
editorial in PhLysica17herapyentitled
"Reliability Discussion Required.""'
The editorial emphasized the impor-
tance of selecting a sample that repre-
sents the population that the treatment
being investigated is intended to affect
and that using unimpaired volunteers
when clinically useful information is
being sought is inappropriate. That is,
subjects can only be said to be partici-
pating in clintcal studies of treatment if
they represent at least some part of
the population that treatment is in-
tended to benefit.
Category Assignment
I read each of the 272 articles in the
sample and assigned each article to
one of the eight method categories.
Following a second reading, I then
categorized each article as either clii-
cal treatment or nonclinical.
-
Table 1 . Method Categories

Category

Group experimental
Description

In group-experimental articles, there was a comparison between an experimental or treatment group and a control or
nonexperimental group, or between two or more alternative experimentalor treatment groups, in which the subjects
were randomly assigned to each group. The article explicitly reported these features and presented a statistical
analysis of the results.
Quasi-group comparison In quasi-group-comparison articles, there were group-comparisons but there was no indication of the use of random
group assignment or nontreatment groups. Both dependent and independent variables were generally clearly
identified, and there was an attempt to manipulate the variables involved. A statistical analysis of the results was
included.
Single-subject design In single-subject design articles, there was use of a single-subject experimental design in which a baseline was
established, repeated measures of patient performance were made, and a reversal to baseline conditions was
reinstated. A graphic presentation of the data was included with or without an accompanying statistical analysis of the
data.
Practical In practical articles, there were reports of the development of new apparatuses; modifications to existing equipment; or
new assessment, measuring, or treatment techniques. The apparatuses, equipment, and techniques reported were
applicable in the clinical environment. Articles included in the practical category did not fit into any of the other
categories above.
Descriptive In descriptive articles, there was a description of a situation, a theory, a condition, a treatment regimen or technique, a
historical perspective, or a philosophical approach, or there was a discussion of anatomy, physiology, and
biomechanics. Review articles were inciuded in this category. Articles containing numerical data or statistical analyses,
or both, were excluded from this category.
Survey In survey articles, there was use of some type of measuring instrument or interview technique to gather information of
either an attitudinal or observational nature. Included were articles about patients, conditions, or treatments and
articles that established norms for different performances (eg, articles surveying the isometric or isokinetic strength of
the quadriceps femoris muscles in college students or evaluating the use of different pain scales for patients following
surgery). No attempt was made to control or manipulate variables of interest.
Case study In case-study articles, there were reports of an intensive investigation of one or more persons. These reports would
usually, although not always, include a description of a therapeutic intervention and the relevant medical history. The
case study was carried out over two or more successive consultations, interviews, or sessions. In these articles,
numerical data may, but need not necessarily, have been presented.
Correlational In correlational articles, an attempt was made to determine the relationship between variables. These articles reported
the use of correlational statistics (eg, Pearson's coefficient of correlation, Goodman and Kruskal's coefficient of ordinal
association, and other such statistics). No attempt was made to vary or control one of the variables, as is done in
experimental research.

D
Table 2. Description of ClmiJicatiott System: Clinical Treatment and Nonclinical Categories

Category Description

Clinical treatment In clinical treatment articles, there were reports of the implementation and the outcomes of treatment techniques applied to
individual patients or to groups of patients. There was an intention of maintaining or improving the function of those patients
receiving the treatment. Reports of methods of treatment were not inciuded in this category unless they involved a report of
the results of the application of those methods.
Nonclinical In nonclinical articles, there was no report and evaluation of treatment techniques intended to improve or maintain the function
of those receiving the treatment. Such articles included descriptions of (1) medical, surgical, and physical therapy techniques
and apparatuses, (2) procedures conducted using unimpaired volunteers not requiring the treatment to improve their
functioning, (3)norms established for different capabilities, (4) theories, and (5) conditions. These articles also included reports
of anatomy, biomechanics, and physiology.
-

Physical Therapy /Volume 75, Number 4 /April 1995

Reliability
I checked the reliability of both cate-
gory assignments of the part of the
sample dated 1954 to 1 W at least 1
year after the initial analyses. Twenty-
seven of these articles, slightly over
10%of that section of the sample,
were checked. Using a table of ran-
dom numbers, articles were selected
fmm each topic subset in turn. The
initial assignments of each article to a
particular method category and to
either the clinical category or the non-
clinical category were then compared
with the blinded reassignments of
each article. A coefficient of agreement
was used. The resultant level of agree-
ment was 1.00 for the classification of
articles as clinical or nonclinical and
.93 for the methods. This level of
agreemtmt for the methods is com-
parable to the .94 value obtained
by Ottenbacher and Short," the
originators of the methods classifi-
cation system used in this study. I
categorized each article in the re-
mainder of the sample (ie, articles
published 1991-1993) and then
recategorized each article after a
1-week interval to confirm the reli-
ability of the classification.
bined, in each 10-year period exam-
ined (ie, 1954-1963, 1964-1973,
1974-1 983, 1984-193). Figure 1
shows that an increased percentage of
articles in the sample described the
use of group designs in each succes-
sive 10-year period. In the most recent
period (ie, 1984-1!993), group designs
were used in the studies repolted in
53% of aU articles in the sample. By
contrast, group designs were used in
studies reported in only 12% of the
articles from the period 1954 to 1963.
Figure 2 shows the number of clinical
treatment articles (top graph) and
nonclinical articles (lower graph) in
each of the successive 10-year periods
examined. Figure 2 also shows the
methods used in both the clinical
treatment and nonclinical categories.
40
35
30
25
The figure shows that increases have
occurred in the total numbers of arti-
cles published and indexed per de-
cade in each topic area since 1954.
Figure 2 shows that of the 272 articles
in the sample, 187 (69%) were catego-
rized as nonclinical and 85 (31%) as
clinical treatment. In the initial period
examined, 1954 to 1963, more than
half (55940) of the articles described
studies that investigated clinical treat-
ment. Since then, increasingly more
nontreatment articles have been pub-
lished. In 1964 to 1973, 57% of the
articles were nonclinical; in 1974 to
1983, 80% were nonclinical. In the
most recent period examined, 19% to
1333, 70% of the articles were classi-
fied as nonclinical. In 1973, the cumu-
lative number of nonclinical articles in

20

15 Experimental
Figure 1 shows the percentage of
articles describing studies using group 10 Quasi-group
designs, group-experimental and
quasi-group-comparisonarticles com- 5 Correlation
cn
a,
7

-
.u
4
0
1954- 1963 1964- 1973 1974- 1983 1984- I 9 9 3
Practical

2
&
. . Single subject
0
Survey

case

[I Descriptive

Figure 1. Percentage of articles in

the sarnple using group designs (group-
experimental and quasi-group-
comparison designs) for four successive
10-yearpennods.
1954- 1963 1964- 1973 1974- 1983 1984- 1993
Figure 2. Number of clinical (clinical treatment) and nonclinical articles in each
method categoryfor four successive 10-yearperiods.

Physical Therapy / Volume 75, Number 4 /April 1995

 examined, 1954 to 1963, group
designs were used in studies re-
ported in 11% of the clinical arti-
cles and in 13% of the nonclinical
40 articles. The use of group designs
35 increased steadily in the following
Experimental
cn two decades, and, in the most re-
a,
'.-
;5 30 ~uasi-group cent period examined (ie,
4
(ti Correlation 1984-19931, 61% of the studies
2
v-
25 reported in the clinical treatment
0 20 ~ractical articles and 50% of the studies
L
2 15 ... Single subject
reported in the nonclinical articles
used group designs.
E
2 10
survey
The studies reported in the nonclinical
5 case
articles, as shown in Figure 2, used
0 Descriptive both groupexperimental and quasi-
Knee Back Electrical
groupcomparison designs in addtion
Stimulation
to descriptive, survey, practical, and
correlational methods. Nonclinical
articles did not report the use of case
f ig~l'e3. Number of articles in each to@ categoy (ie, knee, back, electrical stim- studies or single-subject experimental
ulation) using each method. designs. By contrast, clinical articles
described the use of case studies,
the sample finally equaled that of the Figure 2 also shows a marked in- single-subject experimental designs, or
treatment articles. Since 1973, the crease has occurred in the use of either of the two types of group
cumulative number of nonclinical both types of group designs (ie, designs.
alricles in the sample has always ex- group-experimental and quasi-
ceeded the cumulative number of group-comparison designs) in both Figure 2 shows changes over time in
clinical treatment articles. the clinical treatment and nonclini- the number of articles reporting the
cal categories. In the initial period use of alternative methods. The use of
case studies was reported in 49% of all
sampled articles in 1954 to 1963 and
in 10% of the articles from 1984 to
193. The percentage of descriptive
studies dropped steadily from 27% in
1954 to 1963 to 6% in 1984 to 1 9 3 ,
except during the decade 1974 to 1983
when it rose to 42%. Special issues on
both the back (August 1979) and the
knee (December 1980) were pub-
lished during this decade and included
a large number of articles classified as
descriptive. These special issues re-
sulted in a marked inflation for the
decade of the number of descriptive
articles, which are, by definition, non-
clinical articles. Two methods, single-
subject designs and correlational stud-
ies, were used only in the most recent
decade surveyed, 1984 to 1 9 3 . Small
numbers of articles were involved; for
example, only two articles described
the use of single-subject designs. No
clear trends are apparent in the use of
the remaining alternative methods.
Figure 4. Percentage of articles categorized as clinical treatment in each topic
category (ie, knee, back, and electrical stimulation) tn each of four successive 10-year Figures 3 and 4 both illustrate the
periods. contribution of the individual topics

Physical Therapy / Volume 75, Number 4 / April 1995

(ie, knee, back, and electrical stimula-
tion) comprising the sample of articles
examined in this study. Figure 3
shows that group designs (ie, group-
experimental and quasi-group-
comparison designs) were used more
frequently than any other methods in
the articles on electrical stimulation
(56%) and second most frequently in
articles on the knee (30?/0) and the
back (27%). More articles on the back
(29?/0)and the knee (27%)were de-
scriptive. Figure 3 also shows that the
articles on electrical stimulation re-
ported the use of few surveys and
more case studies than articles on
either the back or the knee.
Figure 4 shows the percentage of all
articles on each topic categorized as
clinical treatment in each of the suc-
cessive 10-year periods investigated.
The figure shows that in only five
instances. (ie, for the knee and back in
1954-1943, for the back and electrical
stimulation in 1W-1973, and for
electrical stimulation in 1974-1983)
has the percentage of clinical treat-
ment articles in any topic area reached
or exceeded 50%. In the most recent
period examined (ie, 1984-1993),
fewer than 30% of the articles on the
knee and back and 40% of the articles
on electrical stimulation were cli~lical
treatment articles.
Discussion
Over the four-decade period exam-
ined (ie, 1954-19!93), an increasing
percentage of articles on all three
topics analyzed used group designs,
and the majority of articles on each
topic were nonclinical. The context of
this finding is the literature of a clinical
professjon that actively and positively
promotes research. The profession
expects research to validate16 and to
further develop practice, and to in-
crease its own scientific credibility."
These findings, however, suggest that
the direct validation of practice via
research may not be occurring as
frequently as many believe and would
like. These findings also suggest that a
concurrent narrowing, rather than a
widening, has occurred in the choices
of acceptable research methods.
Physical Therapy/ Volume 75, Numbe
These findings must be put in per-
spective. Three of a large number of
indexed topics were analyzed (ie,
knee, back, electrical stimulation), and
all articles on these topics came from
the same physical therapy journal. The
results showed some differences
among the three topics in both the
methods used and the percentage of
clinical treatment arricles. The results,
however, were similar across all three
topics: a general increase in the use of
group designs and a clearly lower
percentage of clinical treatment articles
than nonclinical articles. Further, the
number of articles published in and
since 1954 on each of these particular
topics shows they are of major interest
to physical therapists and of major
importance in the profession. Each of
these three topics has direct relevance
for practitioners in what is essentially a
clinical profession. Further, all articles
came from Physical 7berapy, arguably
the most influential and enduring
English-language journal in the profes-
sion as, among other things, it is the
only journal in the profession that
both is indexed in major medical,
scientific, and physical therapy in-
dexes and has been published on a
monthly basis for decades.
Arguably, the broader relevance of
these findings appears to rest on fur-
ther analyses of other major topics
within the different parts of the physi-
cal therapy literature. That is, the gen-
eralizability of the findings of this
study would appear to be limited until
replications of the same procedures on
articles indexed on other topics and
from other parts of the physical ther-
apy literature are undertaken. If subse-
quent investigations show the findings
of thls study apply to only the three
topics examined and to the journal
Physical ~ e m p ythese
, findings
would appear to be of limited interest.
These findings, however, would re-
main relevant and diicult to easily
dismiss. The findings of this study are
generally consistent with what is said
about research in physical therapy.
Further, the articles examined in this
study comprise a central portion of an
important and influential body of the
physical therapy literature and concern
topic areas of major importance in the
profession. These three sets of articles,
therefore, likely reflect the more gen-
eral trends and changes in the re-
search pamdigms acceptable within
the profession, and possibly also influ-
ence them. Articles on topics of lesser
general interest and in some of the
smaller bodies of physical therapy
literature are less likely to reflect gen-
eral trends, or to influence them. Fur-
ther studies of both different topics in
and different parts of the physical
therapy literature would indicate the
extent of generalizability of these
findings. Further studies would also
extend the degree of reliable reflection
possible within the profession on
trends in relevant published research.
The finding of an increasing reliance
on group designs in all t h e topic
areas is consistent with what I believe
is the extent of support in the profes-
sion for the method. Discussions of
research and research methods in the
different journals and books produced
for the physical therapy market tend
to advocate use of group designs.
Implicit support for group designs is
found in the greater space given them
in many of the books and articles that
describe how to do1,2,4,28-33 and to
evaluateM.35 research and in the un-
stated presumptions in many discus-
sions of research that a group design
will be used. Whether these findings
are primarily a function of editorial
policies in the different physical ther-
apy journals, or whether they directly
reflect the beliefs and practices of
readers of and contributors to those
journals, is beyond the scope of this
study. This finding is, however, consis-
tent with an apparently high level of
support for a particular approach to
research in much of the physical ther-
apy literature.
The finding of an increasing reliance
on group designs in both clinical treat-
ment and nonclinical studies is, how-
ever, paradoxical given existing criti-
cisms of the method. Physical therapy
is a clinical profession. Many criticisms
of group designs apply particularly
when the method, not originally de-
signed for clinical research, is used in
a clinical ~ o n t e ~ t . For ~ ~exam-
~ 5 ~ ~ ~
ple, criticisms of group designs con-
cem the difFiculties in obtaining SUB-
ciently large and appropriate patient
samples and the risks of compromises
that may be made.l.36-41 Other criti-
cisms concern difficulties in maintain-
ing experimental control in the clinical
environment and in overcoming the
ethical problems consequent upon the
requirement that treatment is allocated
according to the research design rather
than to the patients' needs. Yet other
criticisms question the relevance to
practitioners, concerned with individu-
als, of findings based on averaged
group data.36.42The relevance for
physical therapy.of these criticisms of
group designs becomes apparent
when the clinical nature of the profes-
sion, the context of the research, and
the heavily clinically oriented benefits
expected from research are
remembered.
Little attention has been paid in the
physical therapy literature to the role a
research method may play in research
outcomes. Some contributors to the
physical therapy literature have sug-
gested that the choice of problems
investigated might be constrained by
amenability to statistical analysis.48~56
Inferences based on existing criticisms
of group designs would support this
view. The diaculties inherent in inves-
tigating questions where sufficiently
large and homogeneous patient treat-
ment and nontreatment or alternative
treatment groups are a requirement of
the method quite likely do have impli-
cations for what is researched in phys-
ical therapy. That is, many questions
from within the practice domain are
possibly not amenable to research
using group designs. Further, prob-
lems arise when the likely outcomes
of the research using group designs in
the clinical context are considered. A
consequence of the demonstrated low
power of many clinical studies using
group designs may be the unwar-
ranted acceptance of the null hypothe-
sis in~estigated.~~ Research conducted
with larger groups of more homoge-
neous patients, or research using dif-
ferent measures or even a quite Mer-
ent research method, might produce a
quite different outcome. That is, the
selection of a particular research
method has implications for the find-
ings that may be obtained as well as
for the questions that can be
investigated.
Alternative research methods to group
designs are available. Among those
methods explicitly designed for use in
the clinical context are single-subject
experimental designs and series of
case studies. In this study, however, I
found few single-subject experimental
designs and increasingly fewer case
studies in the topics examined. This
finding is not explained by editorial
policy. Editors of all major physical
therapy journals have published re-
quests for57359 or advice on how to
use@-63 these methods. A range of
different methods have also been used
in the physical therapy literature. The
question of why proportionately so
few studies in the sample used alter-
native methods remains. The impor-
tance of this question lies in the role a
research method can play in establish-
ing which questions are investigated,
in the possible research outcomes, and
in the expectations in physical therapy
that research will benefit practice.
The second major finding of this study
was that most studies reported in the
sample of articles examined did not
investigate clinical treatments. Clinical
practice is central to physical therapy,
and physical therapists expect research
to benefit clinical practice. Most sur-
veyed articles, however, did not report
or evaluate the treatment techniques
or regimens used in physical therapy
to maintain or improve patient func-
tion. Instead, most articles described
procedures, apparatuses, or tech-
niques used on unimpaired subjects
who explicitly did not require treat-
ment to improve their hnctioning or
provided information or discussed
issues of interest to physical therapists.
The prevalence of nonclinical articles
is not explained by the sample se-
lected. All three constituent topics (ie,
knee, back, and electrical stimulation)
are highly relevant to practice. All
articles are cross-indexed and include
either areas often treated or methods
often used for treating patients. Fur-
ther, Physical nerapy actively solicits
articles on topics of clinical relevance
Physical Therapy /Volume 75, Number 4 /April 1995
and interest. For example, in the pe-
riod investigated, the Journal's highest
priorities included the publication of
clinically relevant research57 and stud-
ies on the eficacy of therapeutic inter-
ventions.*l Additionally, during that
period, the current editor of the Jour-
nal on more than one occasion explic-
itly solicited research articles and case
reports relevant to clinicians.7 Al-
though requests by the current editor
for articles could not have affected
most of the articles analyzed here,
they are indicative of the emphasis the
Journal has placed over time on clini-
cal articles.
The trend toward relatively fewer
clinical treatment articles cannot be
explained by the advent of "new"
journals. The newer physical therapy
journals, those introduced since 1970,
commenced publication on a relatively
infrequent basis, and the earliest of
them, the Journal of Orthopaedic and
Sports Physical 7berapy,was not pub-
lished until 1979. By contrast, the
vend toward a higher relative fre-
quency of nonclinical articles was well
established by the 1980s, to the extent
that their cumulative frequency in the
sample always exceeded that of the
clinical articles published after 1973.
That is, the trend away from chnical
treatment research was well estab-
lished prior to the advent of the spate
of "new" physical therapy journals.
Questions arise about the clinical
treatment and nonclinical classification
system. In particular, given the lack of
precedents for such an analysis in
physical therapy, does the classifica-
tion system used enable meaningful
and reliable distinctions? Distinctions
between clinical treatment and non-
clinical articles were based on those
existing within the physical therapy
and psychological literatures, appear
valid, and were shown to be reliable.
Care, however, must be taken in inter-
preting the findings. In particular, the
division between clinical treatment
and nonchcal articles does not imply
that nonclinical articles have no rele-
vance for clinical practice. Clearly,
many nonclinical articles describe
techniques used by clinicians or the
anatomy of a region, or they report
the testing of measurement or treat-
ment procedures subsequently used
and even investigated clinically, mese
articles, however, do not report on the
im~lementationof treatments. Their
subsequent or potential clinical usehl-
ness would be very diacult to estab-
lish and would involve a number of
very questionable assumptions. The
distinction between nonclical and
treatment articles was specfi-
cally made in this study to establish
the exrent to which a carefully se-
lected sample of physical thelapy
literature explicitly investigated clinical
treatments. As even though
the profession is primarily clinical and
most ex~ectedoutcomes from re-
search relate to improving and validat-
ing practice, most articles examined
were nonclinical.
The findings of this study, together
with the increasing body of criticism
of group designs and the awareness in
the ~rofessionof the need for research
into clinical treatments, suggest more
extensive considelation of and reflec-
tions on the research published within
the physical therapy literature are
needed. For example, is the reliance
on group designs found in this study
also found in the more extended
physical therapy literature?Why are
apparently so few researchers using
alternatives to group designs, or hav-
ing them published? Do the findings
in this study of apparently so few
studies of clinical treatments also ap-
ply to other topic areas? If so, why,
and how can more research into clini-
cal treatments be actively promoted
and published within the profession's
literature?
Conclusions
The findings of this study question
whether the benefits expected to ac-
crue from research to physical thelapy
can be realized, given current research
practices and emphases. More physical
therapists need to be aware of the
importance of researching our clinical
treatments and the range of research
methods available.
physical Therapy / Volume 75, Number 4 / ~ ~ r1995
References
I Currier DP. E l m a t s of Reseanh in Ph-ysi-
cal 7berapy. 3rd ed. Baltimore, Md: Williams
a Wilkins: 1 m .
2 Makrides L, Richman I. Research methodol-
o m and applied statistics. part 1: general prin-
ciples and basic concepts. Physiotherapy Can-
ada. 1980:32:135-139.
3 Michels E. Design of Research and Analysis
of Data in the Clinic. Alexandria, Va: Ameri-
can Physical Therapy Association; 1985.
4 Paycon OD. Research: 7he Validation of
Practice. 2nd ed. Philadelphia, Pa: FA Davis
co; 1988.
5 Schlaobach PB. Gerher N1, eds. Phvsiother-
apy: ~ o ~ t r o l l eTrials
d and Facts. ~ a & lSwit-
,
zerland: S Karger AG, Medical and Scientific
publishers:
, -
6 Twomey L. A growing commitment to re-
search and evaluation. Australian Journal of
Physiotherapy. 1W,36:83.
7 Rothstein JM. Editor's note: The Journal-
past, present, and future. Phys 7ber. 1989;69:
795-796.
8 Physiotherapy research. Physiotherapy.
1332;78:356-357. Policy statement.
9 Dean E. A vision of Physiotherapy Canada:
All things to all people. Physiotherapy Can-
ada. 1993:45:5-6.
10 The Association of Chartered Physiothera-
pists in Research. Physiotherapy. 1983;69:317.
11 Miles-Ta~~ing C, Dyck A, Brunham S,
et al. Canadian therapists' priorities for clinical
research: a Delphi study, phys ner,
19~;70:448-454.
12 Rose SJ. Editor's note: Gathering storms.
P h y ~7ber. 1989;69:354-355.
13 Gielen F. Discussion of placebo elfect in
physiotherapy based on a noncritical review
of the literature. Physiotherapy Canada.
1989;41:210-216.
14 List M. Physiotherapy: a mobile profession
in health care. Physiotherapy. 1986;72:
122-124.
15 Momssey MC, Kanda LT, Brewster CE.
Development of a clinical physical therapy
research program. Phys 7ber. 1987;67:
1110-1114.
16 Cooper BA, Saarinen-Rahikka HK. Interne-
lationships of theory, clinical models and re-
search. Physiotherapy Canada. 1986;
38:97-100.
17 Ottenbacher KJ. Emluating Clinical
Change: Strategiesfor Occupational and
Physical 'Therapists.Baltimore, Md: Williams L
Wilkins; 1986.
18 Zimny NJ, Tandy CJ. Problem-knowledge
coupling: a tool for physical therapy clinical
practice. Phys 7ber. 1989;69:155-161.
19 Mathews JS. The 1987 Presidential
Address: Towards a profession of suhstance.
Phys n e r . 1987;67:1723-1725.
20 Wood R. Twenty-third Mary McMillan
Lecture: Footprints. Phys 7her. 1989;69:
975-980.
21 Stockham MJ. Are physiotherapists neces-
sary? Physiotherapy. 1988;74:499-501.
22 Grieve E. A fair trial for physiotherapy: the
controlled trial. Physiotherapy. 1993;79:304.
23 Jackson DA. Where is the physiotherapy
profession going? Physiotherapy. 1987;73:
590-591.
i l
24 Norton BJ, Strube MJ. Some cautionary
comments on the use of retrospective designs
to evaluate treatment efficacy. Phys n e r .
1988;68:1374-1377.
25 Norton BJ, Strube MJ. Making decisions
based on group designs and meta-analysis.
Phys 7bm. 1989;69:594-600.
26 Sim J. Methodology and morality in phys-
iotherapy research. Physiotherapy. 1989;75:
237-243.
27 Van der Linden S, Bouter L, Tugwell P.
What are the minimal methodological and
statistical requirements of a good trial? In:
Schlapbach PB, Gerber NJ, eds. Physiothwapy:
Controlled Trials and Facts. Basel,
Switzerland: S Karger AG, Medical and Scien-
tific Publishers; 1991:2.
28 Makrides L, Richman J. Research method-
ology and applied statistics, part 6: ethics in
human research. Physiotherapy Canada.
1981;33:89-94.
29 Prince B, Makrides L, Richman J. Research
methodology and applied statistics, part 2: the
literature search. Pbysiotherapy Canada.
1980;32:201-206.
30 Richman J, Makrides L, Prince B. Research
methodology and applied statistics, part 3:
measurement procedures in research. Physio-
therapy Canada. 1980;32:253257.
31 Richman J, Makrides L. Research method-
ology and applied statistics, part 4: overcom-
ing statistics anxiety. Physiotherapy Canada.
1980;32:321-329.
32 Makrides L, Richman J. Research method-
ology and applied statistics, pan 5: experi-
mental design. Physiotherapy Canada.
1981;33:6-14.
33 Makrides L, Richman J. Research method-
ology and applied statistics, part 7: writing the
research proposal. Physiotherapy Canada.
1981;33:16%168.
34 Domholdt FA, Malone TR. Evaluating re-
search literature: the educated clinician. Phys
'Ther. 1985;65:487-491.
35 APTA Committee on Research: Guidelines
on critically considering research papers. Phys
7ber 1988;68:555-556.
36 Gomella C. Designs for clinical research.
Pbys 7ber. 1973;53:1276-1283.
37 Lehmkuhl LD. Mixing one part common
sense with each part statistics in planning the
design and reporting the results of clinical
research in Ph.vsica1 7berapy. Phys 'Ther.
1987;67:1851-1853.
38 Riddvch J, Lennon S. Evaluation of
practice: the single-case study approach. Phys-
iotherapy 7beory and Practice. 1331;7:%11.
39 Shepard KF. Qualitative and quantitative
research in clinical practice. Phys 7ber.
1987;67:1891-1894.
40 Deyo RA,Walsh NE, Schoenfeld LS,Ra-
mamurthy S. Can trials of physical treatments
be blinded? Am J Phys Med Rehabil. 1990;
69:6-9.
41 Partridge C. The myth of the medical
model. Physiotherapy. 1982;68:329.
42 Rosnow RL, Rosenthal R. Statistical proce-
dures and the justification of knowledge in
psychological science. Am Psychol. 19W,
44:1276-1284.
43 Gore SM, Jones IG, Rytter EC. Misuse of
statistical methods: critical assessment of arti-
cles in BMJfrom January to March 1976. EM].
1977;1:85-87.
 44 Vrbos LA, Lorenz MA, Peabody EH,
McGregor M. Clinical methodologies and inci-
dence of appropriate statistical testing in or-
thopaedic spine literature. Spine. 1993;18:
1021-1029.
45 Grichting WL. Psychology and sociology in
Australia: the published evidence. Australian
Psychologfi. 1989;24:115-126.
46 Huxley P. Statistical errors in papers in the
British Journal of Social Work. Soc Work.
1986;16:64>658.
47 Carver RP. The case against statistical sig-
nificance testing. Haruard Educational Re-
dm. 1978;48:378-399.
48 French S. How significant is statistical sig-
nificance? A critique of the use of statistics in
research. Physiotherapy. 1988;74:266-268,
49 Harcum ER. The highly inappropriate cali-
brations of statistical significance. Am Psychol.
1989;44:964-965.
50 Oakes M. Statistical Inference;A Commen-
tary for the S o c ~ and
l Behauiouml Sciences.
Chichester, England: John Wiley & Sons Ltd;
1986.
51 Rothstein JM. Editor's note: Much ado
about probability. Phys Ther. 1990;70:535-536.
52 Matyas TA, Ottenbacher KJ. Confounds of
insensitivity and blind luck: statistical conclu-
sion validity in stroke rehabilitation clinical
trials. Arch Phy.s Med Rehabif. 1993;74:
559-565.
53 Barlow DH. On the relation of clinical re-
search to clinical practice: current issues, new
directions. J Consult Clin Psychol. 1981;2:
147-155.
54 Ottenbacher KJ. The significance of power
and the power of signhcance: recommenda-
tions for occupational therapy research. Occu-
pational Therapy- - Journal of Research. 1984;
4:37-50.
55 Ottenbacher KJ. Clinically relevant designs
for rehabilitation research: the idiographic
Invited Commentarv
Dr Robertson provides us with much
useful information about selected
publicationsincluding their methods
and clinical content--that have ap-
peared in Physical merapy over the
last 40 years. She and I differ, how-
ever, in our views of the framework
into which these data should be
placed and in the conclusions that we
draw from them. In this commentary,
then, I will place a different spin on
the data that Dr Robertson has care-
fully collected and reported.
model. Am J Phys Med Rehabil. 1990;69:
286-292.
56 Tamrnivaara J, Shepard KF. Theory: the
guide to clinical practice and research. Phys
Ther. 1990;70:578-582.
57 Lister MJ. Editor's note: Farewell. Phys
Ther. 1988;68:183-184.
58 Rothstein JM. Editor's note: Clinical litera-
ture. Phys Ther. 1989;69:895-8%.
59 Rothstein JM. Editor's note: The case for
case reports. Phys Ther. 1993;73:492-293.
60 Cole P. How to . . . single-subject designs
for clinical and research purposes. Australian
Journal of Physiotherapy. 1991;37:127-128.
61 Gonnella C. Single-subject experimental
paradigm as a clinical decision tool. Phys
Ther. 1989;69:601-609.
62 Riddoch J, Lemon S. Single-case research
designs. Physiotherapy Theory and Practice.
1991;7:1. Editorial.
63 Robertson VJ, Lee VL. Some misconcep-
tions about single-subject designs in physio-
therapy. Physiotherapy. 1994;80:762-766.
64 Jensen GM. Qualitative methods in physi-
cal therapy research: a form of disciplined
inquiry. Phys Ther 1989;69:492-500.
65 Schmoll BJ. Ethnographic inquiry in clini-
cal settings. Phys mer. 1987;67:1895-1897.
66 Stonecipher DR, Catlin PA. The effect of
a forearm strap on wrist extensor strength.
J M o p Sports Phys Ther. 1984;6:184-189.
67 Michels E. Using and Understanding
Surveys; An Introductory Manual. Alexandria,
Va: American Physical Therapy Association;
1985.
68 Portney LG, Watkins MP. Foundations of
Clinical Research. East Norwalk, Conn: Apple-
ton & Lange; 1993.
ClinicaVNonclinical Conclusions
One of Dr Robertson's major findings
is that only 31%of the articles in her
sample investigated clinical treatments.
My first point related to the clinical/
nonclinical conclusions drawn in this
study concerns Dr Robe~tson'sview of
the purpose of research. The finding
of only 31% of clinical articles was
viewed as problematic by Dr Robert-
son because such a low proportion of
clinical treatment articles seems un-
likely to fulfill a mission of research
that is "expected to demonstrate and
improve the effectiveness of physical
Physical Therapy /Volume 75, Number 4 / April 1995
69 Aufdemkarnpe G. Some comments on
single-case studies. Pbysiotherw Theory and
Practice. 1991;7:63-71.
70 Holliday PJ. A survey of research contribu-
tions of physiotherapists. Physiotherapy Can-
ada. 1981;33:372-376.
71 Mitchell J. Research in the 1980s: world
trends in physiotherapy research. Physiother-
apy Theov and Practice. 1%3;9:171-176.
72 Dean E. Physiotherapy Canada survey
(1973-19821: a trend to research. Physiother-
apy Canada. 1985;37:158-161,
73 Jones D. M~nichielloV. A survey of geron-
tology articles in physiotherapy journals. Aus-
tralian Journal of Physiotherapy. 191;37:
155-161.
74 Instructions for authors. Phys Ther. 1993;
73:58.
75 Bohannon RW. Importance of Physical
Therapy grows. Phys Ther. 1988;68:584.Letter
to the Editor.
76 Bohannon RW. Physiotherapy Practice.
1989;5:99-100. Letters page.
77 Bohannon RW. Core journals of physio-
therapy. Physiotheram Practice. 1987;3:
126-128.
78 Dennis JK. Decisions made by physio-
therapists: a study of private practitioners in
Victoria. AustrallanJoumal of Physiotherapy.
1987; 33:181-191.
79 Ottenbacher KJ, Short MA. Publication
trends in occupational therapy. Occupational
TherapyJournal of Research. 1982;2:80-88.
80 Goldiamond I. Prvtection of human sub-
jects and patients. Behauiorism. 1976;l:l-41.
81 Lamb RL, Bohannon RW, Craik RL., et al:
Guest editorial: Reliability discussion required.
Phys Ther. 1987;67:501.
therapy techniques, interventions, and
modalities." I h d thts to be a limited
and somewhat unrealistic view of
research. Because I have not exam-
ined the citations that Dr Robertson
used to substantiate the above quote,
I am uncertain whether my debate is
with Dr Robertson's interpretation of
these sources or with the sources
themselves. In any event, this view
that research will "demonsvate effec-
tiveness" presupposes that physical
therapy is effective and that a major
purpose of research is to document
for others what we, as physical thera-
pists already convinced of our woah