Medical Education, 1977, 11, 341-346

An analysis of the use of problem oriented medical records (POMR) by medical and surgical house officers: factors affecting use of this format in a teaching hospital
L. C A R O L F E R N O W , I. M c C O L L , C H R I S T I N E M A C K I E
AND

M. R E N D A L L

Kings Fund Record Project, Department of Surgery, Guys Hospital, London

Summary One set of case notes a week for 12 successive weeks was selected randomly for each of twenty-eight house officers and scored for use of the POMR format in the data base, initial plans, and progress notes. Analysis was based on 336 sets of records. The purpose of the study was to obtain information which might improve the techniques of teaching POMR to junior hospital staff. The significant findings were: marked differences in scores among house officers; those who performed well or badly did so in all three components of the system; high scorers in the medical group identified more problems with no evidence that their patients were more ill; scores were better for house officers with favourable attitudes toward POMR and scores were better in the innovative sections if senior staff used the system; house officers with BA or BSc degrees scored better with initial plans and progress notes. No positive effect of feedback was demonstrated.

Guys by medical and surgical house officers and to examine the ways in which feedback and attitudes affect this use. The specific questions we wished to answer were the following: (1) how well do the house o f f i c e r s use the three major components of the POMR format for which they are primarily responsible, i.e. data base (basic admission information), initial plans, and progress notes? (2) Are the performances in each of these three components uniform for a single house officer? (3) Does feedback of scores affect performance? (4) Are scores affected by the house officers attitude towards POMR? ( 5 ) Are scores associated with the house officers perception of the attitudes of the firms consultants and registrars towards POMR? (6) Do scores for the house officers with BA or BSc degrees differ from scores of house officers who have not had this additional academic experience?
Background

The plan for this study originated in the need to stimulate better and wider use of the POMR Key words: *POMR; *HOSPITALS, TEACHING; recording method at Guys after it became official hospital policy in 1974. A booklet entitled GuideINTERNSHP AND RESIDENCY ; EDUCATION, MEDICAL, GRADUATE; ATTITUDE OF HEALTH PERSONNEL; EVALU- lines for the Preparation of the Problem Orientated Medical Record had been distributed to all staff. ATION STUDIES; LONDON Although additional enquiries were welcomed by the Steering Committee it was soon apparent that Introduction these measures were not enough to effect the desired conversion to the new method of recording. The The objective of this study was to evaluate the use of experience of the University of Connecticut School the Problem Oriented Medical Records format at of Medicine had been that a score of format reCorrespondence: Dr L. Carol Fernow, Kings Fund f f i c e r could be an turned to the intern or house o Record Project, Department of Surgery, Guys Hospital, important adjunct to improved performance in the St Thomas Street, London SEI 9RT. 341

342

L. Carol Fernow et al.

house officer was selected and reviewed during the second month by the project staff. They discussed these results with the house officers, explained the study, and assured them that their identity would be kept in confidence. Beginning in the twelfth week of each six-month house appointment, one case a week for each house officer was selected randomly for review. Notes were scored by a trained clerk using explicit written instructions and an objective evaluation questionnaire that had been pre-tested on forty-eight notes of six house officers. Partial or full credit was awarded precisely, which accounts for scores to one decimal place. These scores were used both for feedback and for this analysis. In addition to scoring the POMR format, data were collected on the physical, psychological, and social problems identified by the house officer. The following information was returned through the hospital post to each house officer. (1) A photocopy of the review with errors in format identified was sent weekly. It was accompanied by appropriate explanation and offers of assistance. (2) Every month for three months each general medical and general surgical house officer received the statistical results of his monthly scores. I n addition to the mean and median scores, the Kruskal-Wallis one way analysis of variance by ranks was performed to test whether there was a statistically significant difference among the scores of the house officers. The Kruskal-Wallis test seemed particularly suitable because the display of the data on which the test is based provided an intuitive estimate of each house officers performance relative to his colleagues and did not require a knowledge of statistics. Each house officers scores were arranged in a column with the rank of each in the total array of scores below it in brackets (1 = low). The houseman with the highest sum of ranks was labelled A with the ranks of B, (7,and D in descending order. The column of his scores was identified for each individual house officer. The size of the statistic and the interpretation of significance were also given. In the final two weeks of the appointment, an attitudinal questionnaire was completed to determine satisfactions and dissatisfactions with POMR, as well as perceptiom of the attitudes towards POMR of the consultants and registrars on the firm. Housemens responses were classified independently into very enthusiastic, moderately enthusiastic, and not enthusiastic.

use of POMR (Roberts et al., 1973). With this in mind a method of scoring the notes for POMR format was devised. Studies involving review of patients records have contributed to our hypotheses. Ciocco et al. (1950) corroborated the findings o f Huntley that change in one portion of the clinical record . . . would be a reliable indicator of the effect of changes in the chart conference procedure on the overall quality of the clinical record. Huntley et al. (1961), Morehead (1970) and Peterson et al. (1965) also identified a consistency in the different medical recording tasks. We anticipated that the performance of housemen would be influenced by the attitudes and behaviour of senior staff toward POMR, and we expected to find that those who had used the system best were more thorough and thus identified more problems. We also thought that house officers who obtained BA or BSc degrees prior to graduation from medical school might use POMR better. An analogy made between a chess game and POMR (Tufo et al., 1973a), describes a conceptual structure that the traditional record does not have. Users of POMR describe two inter-related types of feedback which result from examination of (1) procedural performance, in the use of this system, i.e. review of format, and (2) clinical performance, i.e. review of the thoroughness and reliability of the medical logic. We began with the former because correct use of the format must precede effective review of clinical performance. It has been shown that case notes can be reviewed for procedure by non-medical personnel (Tufo et al., 1973 b).

Method This report is based on the reviews of 336 case notes af twenty-eight housemen*, fourteen with appointments in general medicine and fourteen in general surgery. One set of case notes was selected at random every week for 12 weeks for each house officer. The first eleven weeks of each house officers appointment were not studied. During this time the introduction to POMR consisted of giving each house officer a copy of the Guidelines and a brief description of the study during the house officers induction course. One in-patient record of each

* House officers whose consultants actively discouraged the use of POMR were excluded from the analysis.

Use of POMR by house oficers

343

Analytical method
Non-parametric rank order sta'istics were used to summarize the data and to assist in the evaluation of our hypotheses. We adopted the conventional probability level of P<0.05 to reject the null hypothesis and to suggest 'significance' in the statistical sense of a finding that is unlikely to be a chance occurrence. In research of this kind it would be unwise to disregard totally the information obtained from a summary statistic that does not meet this criterion. The probabilities are stated to allow the reader to draw his own conclusions.

Data analysis
The distribution of median scores in each segment of POMR showed some important contrasts between scores for data base and scores for initial plans and progress notes (Fig. 1). The range of data base scores was proportionately narrower than the ranges in the other areas, despite the fact that data base had a possible 42 points, while initial plans had a possible range of 14 and progress notes 24

points. Most of the data base scores clustered around the median, whereas there was a wide spread of scores for initial plans and progress notes. This was not surprising, in so far as Guy's has decided to accept the traditional method of recording the history and examination. The POMR format for initial plans and progress notes, by contrast, is a radical departure from conventional note-taking. These sections are therefore better indicators of the use of POMR. The first question that we wished to answer was whether or not there were differences in total scores among the housemen in each group. The twelve scores of each of the fourteen house physicians were arrayed and tested for differences, as were the scores of the fourteen house surgeons*. The test statistics (P<O.OOl) for both groups left little doubt that there were significant differences in performance within each group. These differences persisted when the monthly median scores were analysed. High or low scorers in the first month tended to remain so in the months that followed, a pattern that continued when

* Kruskal-Wallis

one way analysis of variance by ranks.

physicians
0. 0.0. 0 . 0 0 .

House

I
1

House Dhvsicions

MedionT8.2
~. 0

.
:I4
1 I I 1

House surgeons
I 1
1 1 1 1 1

1
1 1

Medion
1 1

1
House physicians

A 10.2
1 0

I I

15

20 Prdgress notes

25

0. .

House surgeons
1 1 1 1 1 1 1 1 1 1 1 l 1 , 1 1 1 I l l " 1 , I -

344

L. Carol Fernow et al.

TABLE 2. Median scores (12 week) and ranks (1 = low) for house surgeons on use of POMR data base, initial plans, and progress notes: June 1975-May 1976 House Data base surgeon Score Rank
A'

the three independent components of the score were analysed separately. These test statistics were all significant with a probability of less than 0.01 for the data base scores of house surgeons. All other probabilities were less than 0.005, indicating that there were significant differences for both groups of house officers in their grasp of different parts of the POMR format. We observed furthermore that individuals who did well on one part of the format generally did so on the others. While we had hoped to find that our feedback techniques improved performance in each successive month we were unable to demonstrate this for the group as a whole.
Relationship of scores on data base, initial plans and progress notes

Initial plans Score Rank 14.0 14.0

11.0

Progress notes Score Rank 21.0 21.0 22.2 18.6 24.0 20.4
18.0

B' C'

D'
El

F' G' H' I'

J '
K' L'

M'
N'

34.8 29.5 31.3 33.5 30.5 30.5 31.8 29.3 29.8 32.5 27.5 33.0 27.0 28.8

(14) (5) (9) (13) (7.5) (7.5) (10) (4) (6) (11) (2) (12) (1) (3)

(13.5) (13.5) (9)

(11)

11.5 12.0 11.0 11.0 7.5 8.5 7.0

8.0 8.0 5.0

(12) (9) (9) (4) (7) (3) (5.5) (5.5) (2)

(1)

(11.5) (11.5) (13) (9) (14) (10)

(8)

4.0

17.4 14.4 14.7 11.1 6.0 14.1 9.4

(7) (5) (6) (3) (1) (4) (2)

We observed a correspondence in performance in each segment of the format, particularly between initial plans and progress notes, and this was borne out by tests on the medians of the twelve scores (see Tables 1 and 2). The Kendall coefficient of concordance, W, measured the relationship of the ranks of the three sets of scores for data base, initial plans and progress notes. The probability of a statistic as high or higher than the W's found was P<0.02, confirming our previous observation that house officers who scored well or badly on one part of the record did so on the other two, Rank crossover occurred primarily with scores on data base. Looking only at
TABLE1. Median scores (12 week) and ranks (I = low) for house physicians on use of POMR data base, initial plans and progress notes: June 1975-May 1976 House physician Data base Score Rank 3 2 . 0 (9.5) 3 3 . 0 (13) 32.5 (12) 3 1 . 5 (6-5) 3 1 . 5 (6.5) 32.0 (9-5) 3 2 . 0 (9.5) 3 1 . 0 (5) 3 2 . 0 (9.5) 2 5 . 3 (1) 3 5 . 0 (14) 2 5 . 5 (2) 2 9 . 8 (4) 2 6 . 0 (3) Initial plans Score Rank 14.0 11.0 13.0 11.5 11.0 11.0 7.5 8.3 7.0
8.0

Data base, initial plans, progress notes: Kendall coefficient of concordance: W = 0 . 7 0 ; P<O.O2. Initial plans, progress notes: Spearman rank correlation coefficient: rs=0.81; t = 4 . 7 2 ; P<0.0005 (one tailed).

the correlation between scores on initial plans and progress notes, we obtained a probability of Pt0.0005 in each group of house officers*, which allows us to say that housemen who formulate initial plans correctly will probably also do well with progress notes and the other way around.
Scores and the number of problems identified

Progress notes Score Rank 24.0 22.8

18.8

A B C D E F
G H I

(14) (10) (13) (12)

(10)

(14) (13)
(8) (11.5) (10)
(11-5)

(10) (6)
(8)

J K L M
N

6.0 6.0 4.5 5.5

(5) (7) (3.5) (3.5)

(1)

(2)

20.4 19.8 20.4 19.2 18.6 14.4 15.9 7.2 10.8 9.3 10.2

(9) (7) (5) (6) (I) (4) (2) (3)

One of the major principles underlying POMR is that, in promoting a systematic thoroughness, more of the problems that affect patient care will be brought to the attention of clinicians. For house surgeons, no associations between number of problems and scores were found*. This may reflect the tendency for surgeons to identify only the specific surgical problem presented by a routine admission for operation. The initial plan scores of house physicians were significantly related to the number of problems identified (P<O-Ol). The statistics for data base and progress notes were so close to the 5% level (P<0.055)that we must not dismiss the possibility of an association here. It is important to point out that no correlation was found between the number of problems identified and case severity.
Factors associated with performance

Data base, initial plans, progress notes: Kendall coefficient of concordance: W = 0 . 6 9 ; P<O.O2. Initial plans, progress notes: Spearman rank correlation coefficient: r s = 0 . 8 8 ; t = 6 . 3 0 ; P t 0 . 0 0 0 5 (one tailed).

After observing differences among house officers in their use of the POMR format, the next step in the

* Spearman rank correlation.

Use o f POMR by house officers

3 4 5

analysis was to attempt to identify factors which may have influenced these differences. Dealing with them as separate samples it was apparent that the scores and probabilities for medical and surgical housemen did not differ* with the exception of the number of problems identified. Since we had no reason to think that one group would use the format better than the other, the similarity of the findings suggests that the method is at least reproducible in the setting of Guys Hospital. The two groups have been combined in the remaining analysis in order to take advantage of the improvement in estimation which is afforded by a larger sample.
Relationship between scores and attitudes

TABLE 3. Median total scores (12 week) and ranks* in relation

to general medical and general surgical house officers attitudes toward POMR: June 1975-May 1976 Very enthusiastic Moderately enthusiastic Not enthusiastic

Table 3 shows the median total scores on format assigned according to attitude category. When the Kruskal-Wallis one way analysis of variance by ranks was performed, the results (Pt0.01) showed a significant differencein the scores on format among the house officers with different attitudes. The results o f the same test on data base, initial plans and progress notes (P<0.05) similarly indicated a correspondence between attitude and use of the separate segments of the POMR system. It is not possible, however, to state whether favourable attitudes contributed to high scores or high scores encouraged the house officers in their attitudes towards the system, since the attitudinal questionnaire was administered at the end of the study period.
Influences of senior staff

63.5 (22.5) 49.9 (10) 46.9 (8) 46.4 (7) 43.9 (4) 41 .O (3) 40.2 (2)

*Kruskal-Wallis one way analysis of variance (1 H=9.76; P < O . O I .

low).

Analysis of scores according to use of POMR by senior staff showed that house officers who said their consultants or registrars used the system had significantly higher total scores than the housemen whose senior staff did not (P = 0 . 0 4 1 ) . There was no significant difference in the use of data base, but the probabilities for progress notes (P = 0.036) and initial plans (P = 0.007) suggest to us that those house officers who had the example of their seniors used the new format better.
Differences according to academic background

tested the hypothesis that house officers with additional academic qualifications have backgrounds which predispose them to appreciate the logic of the system and therefore to use it well, particularly the initial plan and progress note sections of POMR. The Mann-Whitney U test was also used here to examine the performances of house officers with BA or BSc degrees in contrast to house officers holding only the MB degree. The total scores of the BA/BSc group were generally better and the test statistic (P = 0.05) indicated that there was probably a difference between the two groups. Scores on data base, once again, were not significantly different. The initial plan scores of the BA/BSc group were suggestive of differences (P = 0-088) and the probability of 0.006 for progress notes encouraged us to conclude that house officers with these additional degrees do indeed use the innovative sections of the format better.
Discussion

Because POMR has a conceptualized structure, we

* Using the median score for each houseman the MannItest was used to estimate possible differences in Whitney ( the two groups. N o differences were found.

There was such variation among house officers in the

346

L. Carol Fernow et a[. evidence of the determination of house officer A to retain his position and of house officer B to surpass him on the next months statistics. This project promoted awareness of POMR at all levels of staff throughout the different departments of the f hospital. We believe that continual evaluation o the use of POMR procedure is essential, certainly in the transitional stage between adopting POMR as hospital policy and full conversion to the system. The main value of the POMR system to clinicians is that it allows the reconstruction and review of medical logic for educational purposes. Until such time as the format is generally used correctly the full advantages of the system will not be realized.

skill of recording using the POMR format, that we must make the obvious statement that some used it well, some moderately well, and others badly. Individuals tended to maintain the same standard of recording in each segment of the record. The data discussed here have suggested to us that once the logical principles which underlie the system are understood, performance in all areas of use is generally high. Arriving at that level of understanding would seem to require more frequent and formal tuition than potential users of POMR might anticipate. Recording in this format is not so simple that it can be mastered in an evenings reading of a well prepared booklet, in an occasional meeting with a clinician skilled in its use, or solely by the feedback techniques that we have used. The most obvious place to teach POMR is in the medical school, not only to avoid the difficulties and antagonisms of unlearning one way of recording and learning another, but to take advantage of the conceptual emphasis in this environment. The better scores in initial plans and progress notes of housemen holding BA or BSc degrees lends support to the belief that POMR has a satisfying logic that traditional recording does not. We think that, once having learned POMR, most clinicians would agree with the three house officers who volunteered the comment in the attitudinal questionnaire: Now that I have learned to use it I would never go back to the old method of recording. The association between the attitudes of the housemen and their use of POMR was not fully explained. Only two house officers claimed to have learned it in medical school, and all said they had read the Guidelines before the review period began. The differences in scores for initial plans according to whether consultants and registrars use the system suggest to us that performance in this difficult area is undoubtedly influenced by the example and interest of the senior staff. We have reached the conclusion, therefore, that a formal effort should be made to advise senior staff in the use of the system. While we realize that the problem of converting to POMR is magnified by years of experience with the traditional method, we would hope to promote sufficient understanding among clinicians in senior positions, even those who were reluctant to change themselves, to encourage juniors to use the system well. Although we could not demonstrate that our method of feedback improved performance among the housemen as a whole, we had ample anecdotal

Acknowledgments This research was supported by a grant from the King Edwards Hospital Fund for London. The authors wish to thank Rosemary Dalton of the Department of Community Medicine at Guys Hospital for her careful and constructive reading of this manuscript.

Readers who would like more details of the study should write to the Department of Surgery, Guys Hospital.

References
ROBERTS, M.P., VILINSKAS, J. & DAVIDS, R.S. (1973) Reporting medical record quality. Hospital Progress, 54, 36. I . (1950) Statistics on CIOCCO, A., HUNT, H. & ALTMAN, clinical services to new patients in medical groups. Public Health Reports, 65, 99. K.L., WILLIAMS HUNTLEY, R.R., STEINHAUSER, R., WHITE, T.F., MARTIN, D.A. & PASTERNAK, B.S. (1961) The quality of medical care: techniques and investigation in the out patient clinic. Journal of Chronic Diseases, 14, 630. MOREHEAD, M. (1970) Evaluating quality of medical care in the neighbourhood health centre. Programme of the Office of Economic Opportunity. Medical Care, 8, 118 PETERSON, O.L., ANDREWS, L.P., SPAIN, R.S. & GREENBERG, B.G. (1965) An analytical study of North Carolina general practice, 1953-1954. Journal of Medical Education,

31, 1 .
TUFO,H.M., EDDY,W.M., VAN BUREN, H.C., BOUCHARD, LOUISE (1973a) ZmpIeR.E., TWITCHELL, J.C. & BEDARD, menting a Problem Oriented Practice (Ed. by K. Walker, J. Willis Hurst and Mary F. Woody), p. 27. Medcom Press, New York. TUFO, HENRY M., EDDY, W.M., VAN BUREN, H.C., BOUCHARD, R.E., TWITCHELL, J.C. & BEDARD, LOUISE (1973b) Audit in a practice group. Applying the Problem Oriented System (Ed. by K. Walker, J. Willis Hurst and Mary F. Woody), p. 32. Medcom Press, New York.