.

J Clh EpibmblVol.43,No. 12,pp. 1327-1335,

1990
Printcd
in GrcatBritain

0895-4356/90
s3.00
+ 0.00
Pergamon
Prcwpk

VALIDATION OF A SELF-ADMINISTERED
DIE? HISTORY
QUESTIONNAIRE
USING MULTIPLE DIET RECORDS
GLADYS BLOCK,** MARGO WOODS,~ARNOLD POTOSKYand CAROLYN CLIFRXD
Division of Camxr Prevention and Control, National Canccr Institute, Bethesda, MD 20892 and
Wommunity Health/Nutrition Unit, Tufts University School of Medicine, Boston, Massachusctts,
U.S.A.
(Recehed

in reuisedform

31 Muy 1990)

Abstract-The validity of a self-administered diet history questionnaire has been

estimated using as the reference data the mean of three 4-day diet records collected over
the year prior to the administration of the questionnaire, in 19851986. Subjects were
women ages 45-70 years, participants in the Womens Health Trial Feasibility Study,
a multi-center clinical trial in which some women were randomized to be taught to adopt
and maintain a low-fat diet, while ethers maintained their usual diet. The questionnaire
produced group mean nutrient estimates closely approzimating the values obtained by
three 4-day records, e.g. in the usualdiet group, 37.7% of calories from fat by both food
records and by questionnaire, and in the low-fat, group, 21.3% of calories from fat by
food records and 23.7% of calories from fat by questionnaire. Correlations between
questionnaire and diet records for per cent of calories from fat were 0.67 and 0.65
respectively in the two groups; most correlations were in the 050.6 range, and were
similar to those achievable by a single 4-day food record.
Epidemiologic methods

Diet

Nutrient intake

INTRODUCMON

To study the role of diet in disease prevention,

investigators need nutrent estimates that accurately reflect an individuals long-term usual
intake. This can be achieved through multiple
non-consecutive diet records or recalls, but the
number of records required reslts in great
expense even for the macronutrients. For many
of the micronutrients, the number of records
required makes this approach completely infeasible [ll. Consequently, numerous investigators
have developed diet history or frquency questionnaires for use in epidemiologic research.
We report here the performance of the selfadministered questionnaire developed by Black

Assessment

Fat

Vitamins

et al. [2], among the two groups of women, one

group maintaining their usual North American
diet and the other group consuming a low-fat
diet. The reference or gold standard data
consist of three Cday diet records collected over
the prior 1 year period. The development of the
questionnaire and its nutrient database have
been presented earlier [2], and an interview and
analysis program for personal computers is
available [3]. Previous published validations of
this questionnaire at the individual leve1 have
included its ability to assess calcium intake in
elderly women, by interview 141,and its ability
to assess dietary intake 10-15 years in the past,
by interview and mail [5].
METHODS

*Reprint requests should be addressed to Dr Gladys Black,

NCI, Executivc Plaza North; Rm 313, 9000 Rockville
Pike, Bethesda, MD 20892, U.S.A.

Subjects were participants in the Womens

Health Trial (WHT) Feasibility Study [CS],

1327

GLADYS BLOCK etal.

1328

which was designed to teach women to maintain

a low-fat diet. For that study, 303 women
aged 45-70 years were enrolled from three
geographic sites, Seattle, Wash., Houston, Tex.,
and Cincinnati, Ohio. Participants were randomly assigned to a low-fat or a usual-diet
group. The usual-diet group maintained their
usual diets, which contained approximately
38% of calories from fat; the low-fat group
received intensive education to bring their diets
to approximately a 20%-fat level. For the present validation study, 277 women provided
both the reference dietary data described
further below, and also completed an extensive
self-administered
diet history questionnaire
(Black questionnaire) 1 year after enrollment
into the Feasibility Study; 260 met edit criteria
described below, and comprise the validation
study population. The age, race and education
characteristics of these study participants are
shown in Table 1.
The reference data

The reference data, the measure of truth

against which the questionnaire estimates are
compared, consist of multiple 4-day diet
records. Four-day records were obtained from
al1 subjects at the time of their entry into the
study, and again 6 months and 1 year later. In
addition 4-day records were obtained from lowfat group subjects at 3 months.
For analyses involving correlations, the average of three 4-day diet records is used as
the reference data. This is necessary since the
correlation coefficient is sensitive to the intraTable 1. Characteristics of 260 female participants in a
randomized low-fat diet trial, whose data were used to
validate the diet history questionnaire*

Characteristic
Age
4549 yr
50-59 yr
60-70 yr

Usual-diet
group (%)
n = 102

Low-fat
group (%)
n = 158

17.7
51.0
31.4

20.9
53.2
26.0

99.0
1.0

96.8
3.2

Race

White
Nonwhite
Educa tion

High school or less

28.4

39.9

Some college
College graduate

26.5

26.0

45.1

34.2

*By design, the Womens Health Trial Feasibility Study

enrolled more participants into the low-fat group. There
are no statistically significant (p c 0.05) differences between usual-diet and low-fat groups in any of these
characteristics.

individual variability in the reference data, and

three 4-day diet records provide a better and
more stable estimate of usual diet than do two.
For analyses involving group means, the questionnaire estimates are compared with food
record estimates from the same -month period
covered by the questionnaire.
Subjects were instructed by trained nutritionists in careful recording of their food intake for
the 4-day records, with emphasis on accurate
measurement of food portions and documentation of brand names, cuts of meat, and food
preparation. For mixed dishes, detailed recipes
were to be recorded. The 4-day records were
checked by study nutritionists, and questionable
responses clarified. Coding and nutrient analysis
was performed by nutritionists who had completed a comprehensive training course, and
who used a computerized coding system developed by the Nutrition Coordinating Unit at
Tufts University. The Tufts Nutrient Data Bank
consists of approximately
500 food items,
which allows for specificity in coding [9], and
the primary nutrient data source is the USDA
Nutrient Data Base for Standard Reference,
Release 5, 1984.
A few errors were detected in nutrient estimates from the 4-day records. For three subjects, the 4-day record containing the error was
ignored, and the estimate was based on the
remaining two 4-day records.
The diet history questionnaire

The questionnaire, a modification of the

Black questionnaire [2], was self-administered
by the respondents 1 year after enrollment
into the Feasibility Study. Respondents report
how often they usually consume each food, as
number of times per day, week, month or year,
and whether their usual portion size of that
food is small, medium or large with respect to
a stated medium portion. The questionnaire
also contains the following ancillary questions:
frequency of consumption of foods from six
types of restaurants; whether skin is usually
eaten on poultry, or fat on meat; frequency
of use of fat or oil in cooking, and types used;
two genera1 questions on the overall frequency
of consumption of fruits and of vegetables.
Age-sex-specific portion size assumptions are
used in the calculations. The impact of using
age-sex-specific and individually variable as
opposed to standard medium portion sizes is
examined below. Vitamin supplement use is
also collected, and results below report nutrient

Validationof a Diet Qu&onnaire

estimates both excluding and including vitamin
supplements as estimated by the questionnaire.
For the WHT Feasibility Study, the questionnaire was modified as follows. Nine food items
were omitted from the standard Black list,
and 5 were added as relevant to low-fat group,
resulting in a food list of 94 food items. The
open-ended section where a respondent may
volunteer additional foods was omitted. Five
ancillary questions were added, regarding leanness of beef or hamburger, dark or light meat
poultry, water-pack or oil-pack tuna, brand
name of yogurt usually eaten, and type of lunch
meats or hot dogs usually eaten. These questions were added in the expectation of improving the precision of the fat intake estimates,
particularly for the low-fat group. Finally, the
respondent was asked to report on her diet over
the past 6 months rather than the past year. The
added and omitted items and the analysis program options in effect for this analysis are listed
in the Appendix. The foods comprising the
standard food list may be found in Ref. [2].
Coding and keying of the questionnaires was
conducted by a centra1 Statistical Coordinating
Unit at Fred Hutchinson Cancer Research
Center. Complete rechecking of the coding as
wel1 as double-keying was performed for 100%
of the questionnaires. In addition to standard
range-edits, computerized edits identified questionnaires with errors suggesting careless responses, such as 16 or more foods skipped. Of
the 277 questionnaires, 17 or 6% were identified
by a standard computer algorithm as having
serious response errors, and were omitted from
further analyses, leaving a study group of 260
individuals. Errors identified by the program are
specified in the Appendix.
Nutrient analyses were calculated using the
computer program and nutrient database developed for the questionnaire [2,3]. NO attempt
was made to adjust the standard Black database to make it correspond to the nutrient
database or default portion sizes used for analysis of the Cday records. Correlation between
estimates based on the two different databases
must therefore be an underestimate of the correlations which would be observed if the two
databases were identical.
Statistical methods
Means, standard deviations and Pearson correlation coefficients are reported for the nutrients estimated by the questionnaire and by
multiple 4-day records. Except for percent of

1329

calories Erom fat, al1 nutrient estimates were

skewed, and were therefore transformed to
using the transformation
achieve normality,
which most reduced skewness and improved
normality.
RFSULTS
In Table 2, al1 nutrient estimates by questionnaire are within f 20% of the estimates produced by the mean of the four-day records.
Indeed, in the usual-diet group 14 of 18 dietary
intake estimates by questionnaire differ by 10%
or less from the diet record estimates, and the
same was true for 13 of 18 in the low-fat group.
This isparticularly notable in view of the fact
that the low-fat and usual-diet groups differed
dramatically in the absolute value of some of
their macronutrient intakes. For example, fat
intake in the low-fat group was half that in
usual-diet group, and this was reflected very wel1
by the questionnaire. Similarly, the major difference in percent of calories from fat between
low-fat and usual-diet are reflected wel1 by the
questionnaire. For vitamins A and C, both with
and without supplements, the questionnaire
estimates are not significantly different from the
mean of the food records, in both groups. For
several other micronutrients, however, the questionnaire estimates are statistically significantly
different from the food record estimates,
although not as much as 20% lower or higher,
as indicated above.
Table 3 shows the mean percent of calories
from fat, as estimated by the food records,
within quintiles of the questionnaire estimate. In
the usual-diet group the mean intake among
those in the lowest quintile of the questionnaire
is 3 1.5% of calories from fat. By contrast, in the
lowest quintile of the low-fat group the mean
intake is only 16.6%, and the highest quintile in
that group consumed only 26.3% of their calories from fat.
Correlations betwecn questionnaire and the
mean of three 4-day records are shown in
Table 4. Correlations for nutrients excluding
supplements range from r = 0.47 (for vitamin
A) to r = 0.67 (for percent of calories from fat)
in the usual-diet group, and from r = 0.37 to
r = 0.66 in the low-fat group, with an average
correlation of about r = 0.55 in both groups.
Correlations were algo examined among al1
responders, including those who were excluded
because they had skipped too many foods, etc.
(Data not shown.) In this population, their

GLADYSBLKICK
et al.

1330

Table 2. Group mean (and SD) as estimated by diet history questionnaire and by mean of two
4-day diet records collected at 6 months and 1 year, among 260 women ages 45-70 years
Usual-diet group
n = 102
Questionnaire
Mean
(SD)
% Calories from fat
Calories (kcal)
Fat (g)
Saturated fat (g)
Monounsaturated
Polyunsaturated

fat (g)
fat (g)

Cholesterol (mg)
Protein (g)
Carbohydrate

(g)

Vitamin A (KJ)
with supplements
Vitamin C (mg)
with supplements
Calcium (mg)
with supplements
Iron (mg)
with supplements

37.7
(8.9)
1509.7
(544.2)
63.8
(33.5)
21.7
(11.6)
22.9
(12.8)
ll.4
(8.0)
239.6
(144.3)
63.4
(21.6)
162.4
(58.0)
7458.6
(3523.0)
10827.7
(4921.8)
123.5
(58.6)
431.5
(480.4)
819.0
(382.9)
1191.6
(455.0)
10.6
(3.5)
18.7
(10.3)

Low-fat group
n = 158

Foo&znords
(SD)
37.7
(6.8)
1559.7
(326.9

Questionnaire
Mean
(SD)
23.7
(7.1)
1304.2
(351.2)
33.5
(15.0)
11.3
(5.8)
ll.7
(5.8)

21.1*
(5.6)
1318.8
(287.7)
31.1
(10.9)
10.1*
(4.1)
10.4*
(4.1)

(3::)
145.8
(z;.;)

(0::)

,:::;
155.0
(66.0)
64.0
(14.7)
195.6*
(50.6)
7273.4
(3566.1)
9891.9
(6200.1)
142.1
(59.5)
372.1
(485.5)
756.4*
(314.1)
1117.5
(509.5)
12.5
(3.6)
28.1
(52.9)
1.3*
(0.4)

(2:::;

(E)

(1%;

(0::;

(i$

,:::;
6.6
(15.1)
1064.5*
(313.4)
2627.4
(690.5)
2179.8*
(536.4)

(Z3
23.1
(8.5)
23.5
(8.2)
12.1
(4.6)
267.2
W;)
(13.3)
175.9
(40.8)
6931.5
(3232.4)
10495.8
(7704.3)
128.5
(56.4)
379.9
(378.0)
704.9*
(254.0)
1063.4
(438.8)
12.2*
(2.9)
21.5
(12;)

(18:4)
179.2
(55.3)
7316.4
(3025.4)
10769.5
(7120.6
142.8
(69.1).
486.2
(674.5)
889.2
(411.7)
1136.1
(413.3)
11.1
(3.2)
18.7
(9.8)

Thiamin (mg)
(Z)

Food records
Mean
(SD)

(0.3;

with supplementst
({.$
Riboflavin (mg)
(2
with supplementst
Phosphorus (mg)
Potassium (mg)
Sodium (mg)

(4:3)
1141.9
(423.5)
2518.3
(875.9)
2554.6
(895.8)

(1:::)
1051.1
(247.2)
2407.9
(567.9)
2430.6
(613.8)

(5.4)
1183.6
(408.2)
2761.4
(788.6)
2375.6
(722.2)

*p c 0.05 for differente between the food record and questionnaire estimates in the group shown.
tFive individuals reported on their diet records that they took over 75 mg of those B vitamins on
the day of the record, resulting in very high group means, but did not indicate on the questionnaire
that they took them every day over the 6-month period. The median values estimated for those
B vitamins with supplements are quite similar by questionnaire and food record (1.9 VS2.1 mg
for thiamin with supplements, and 2.7 VS 2.4 for riboflavin with supplements.)

exclusion resulted in only slight improvements

in the correlations.
Correlations
using calorie-adjusted
data
[10] were also examined. The average of the
correlation coefficients was increased by only
0.02 (i.e. from 0.55, excluding calories, to 0.57)
in the usual-diet group, and was unchanged in

the low-fat group. Calorie-adjusted data are not

presented in Table 4 nor in subsequent tables.
Agreement between questionnaire and reference data as reflected in cross-classification by
quintiles corresponds extremely closely, empirically, to what would be predicted from their
correlation coefficients [ 111, for most nutrients

Validation of a Diet Questionnaire

Table 3. Group mean (and SD) of percent of calories from
fat as estimated by the mean of two 4day diet records
collected at 6 months and 1 year, .within quintiles of the diet
history questionnaire estimate of percent of caloriw from
fat; 260 women ages 65-70 years
Quintile of
diet history
questionnaire
estimate of
percent fat

Food records percent of calories from fat

Mean (SD)
Usual-diet group
n = 102

Low-fat group
n = 158

31.5
(7.1)

16.6
(3.9)

35.7
(3.8)

19.5
(3.8)

37.1
(5.6)

20.1
(4.5)

40.6
(5.9)

23.1
(4.8)

43.4
(4.8)

26.3
(5.4)

(data not shown). However, when the contribution of supplements is included in the vitaminmineral estimates, the agreement
of the
questionnaire with 4-day records is considerably
better than would be expected from their correlation coefficients. For vitamin C, 57% are in the
exactly correct quintile, and 87% are either in the
exact or & one quintile. When only the top and
bottom quintiles are considered, exact agreement ranged from 40% (for iron including
supplements) to 65% (for vitamin C including
supplements); and from 68 to 84% for exact or
adjacent agreement. It should be noted, however, that vitamin supplement use varies in different demographic subgroups, and therefore not
al1 groups wil1 have this degree of agreement.
Table 5 addresses the question, How good
are the correlations seen in Table 4, in comparison with what could be achieved if a single Cday
record were used as the estimator of dietary
intake? For this analysis, the first two 4-day
records (baseline and 6 months in usual-diet,
and 3 months and 6 months in the low-fat
group) are used as the reference data. Against
this slightly different measure of truth, the performance of the questionnaire administered at
1 year is compared with the performance of the
4-day record administered at 1 year. In the
usual-diet group, use of a 4day record to
measure dietary intake would have resulted in
correlations in exactly the same range as those
achieved by the questionnaire. Nine of the 18
correlations in the usual-diet group are higher
by questionnaire, 9 are higher by 4-day record,
and the median correlation is the same by both
methods.

1331

Table 4. Correlations between nutrient eatimates by diet

history questionnaire and by mean of three 4-day diet
records, among 260 women ages 45-70 years*
Usual-diet
group
n = 102

Low-fat
group
n =158

0.67
0.51

Percent calories from fat

Calories (kcal)
Fat (B)
Saturated fat (9)
Monounsaturated fat (g)
Polyunsaturated fat (9)
Cholesterol (mg)
Protein (g)
Carbohydrate (g)
Vitamin A (IU)
including supplements
Vitamin C (mg)
including supplements
Calcium (mg)
including supplements
Iron (mg)
including supplements
Thiamin (mg)
including supplements
Riboflavin (mg)
including supplements
Phosphorus (mg)
Potassium (mg)
Sodium (mg)

z.53
0:59
0.48
0.55
0.48
0.51
0.47
0.55
0.56
0.71
0.56
0.55
0.47
0.54
0.57
0.47
0.63
0.57
0.59
0.55
0.47

0.65
0.51
0.57
0.59
0.58
0.46
0.56
0.56
0.55
0.37
0.58
0.48
0.74
0.62
0.64
0.44
0.55
0.47
0.62
0.62
0.63
0.66
0.60
0.43

Mean
Median

0.545
0.555

0.540
0.560

*Data were transformed using log or square root to reduce

skewness and improve normality, as required by the statistical assumption of tests related to the cormlation coefficient. Al1 correlations shown are significantly greater
than zero at p < 0.01.

The same is not true in the low-fat group,

where the single Cday food record produced
consistently
higher correlations
than the
questionnaire. This was not primarily because
the questionnaire produced lower correlations
in the low-fat group, but because the 4-day
record in that group achieved higher correlations with the earlier food records (median
r = 0.605 compared with median r = 0.505 in
the usualdiet group). This is not due to greater
consistency of dietary intake in the low-fat
group, since standard deviations were similar in
the two groups. It is likely that it is a result of
the low-fat groups intense daily training and
practice, over the previous year, in the exact
foods, brand names and portion sizes they
consumed.
Thus, it would appear that for a population
consuming a usual American diet, this questionnaire is approximately as good as a 4-day
record at ranking people with respect to their
usual diet. On the other hand, for a population
such as the low-fat group here, intensely
intervened on and taught to watch and record

1332

GLADYS BLOCK et al.

Table 5. Comparison of correlations achieved by a diet history questionnaire and those
achieved by a single 4-day record, when both are measured against the same reference data,
the mean of two prior diet records; women ages 45-70 years
Usual-diet group
n = 97t

Low-fat group
n = 145t

Food record

Questionnaire1

Food record

Questionnaire1

Percent calories from fat

Calories (kcal)
Fat (g)
Saturated fat (g)
Monounsaturated fat (g)
Polyunsaturated fat (g)
Cholesterol (g)
Protein (g)
Carbohydrate (g)
Vitamin A (KJ)
Vitamin C (mg)
Calcium (mg)
Iron (mg)
Thiamin (mg)
Riboflavin (mg)
Phosphorus (mg)
Potassium (mg)
Sodium (mg)

0.53
0.50
0.44
0.48
0.42
0.41
0.49
0.58
0.57
0.52
0.48
0.63
0.32
0.43
0.51
0.61
0.56
0.56

0.56
0.47
0.50
0.51
0.49
0.45
0.40
0.40
0.52
0.41
0.57
0.54
0.42
0.57
0.58
0.53
0.54
0.42

0.66
0.65
0.63
0.59
0.57
0.62
0.52
0.57
0.69
0.45
0.58
0.73
0.51
0.48
0.66
0.69
0.70
0.41

0.56
0.43
0.48
0.51
0.47
0.37
0.49
0.50
0.48
0.36
0.40
0.57
0.34
0.37
0.53
0.61
0.56
0.31

Mean
Median

0.50
0.505

0.49
0.505

0.60
0.605

0.46
0.48

tOnly those women who provided al1 three diet records are included.
SCorrelations are lower than those seen in Table 4 because reference data comprised of two
4-day records constitutes a less adequate estimate of usual intake than does three 4-day
records; and because the baseline record was weighted more heavily in a two-record average
than in a three-record average.

their intake, a 4-day record appears to produce

somewhat better correlations than the questionnaire.
Methodologie issues

The effect of omitting the questions added for

the WHT version (e.g. leanness of meat usually
eaten) is shown in Table 6, for macronutrients
and fats only. (The correlations for the remaining nutrients were unchanged.) The added questions had essentially no effect on correlations

with reference data in the usual-diet group. In

the low-fat group, for whom such questions
were relevant to their low fat diet, the added
questions improved the correlations only to a
trivial extent.
The standard questionnaire contains two
questions on the overall frequency of consumption of fruit and of vegetables. These permit
the adjustment of total fruit and vegetable
consumption downward to reduce the overestimates resulting from the use of a long list of

Table 6. Correlations between nutrient estimates by diet history questionnaire and

by mean of three 4-day diet records: effect of including supplementary fat-relevant
questions*; 260 women ages 45-70, participants in a randomized low-fat diet trial
Usual-diet group
n = 102
~~Without
With
extra
extra
questions
questions
Percent calories from fat
Calories (kcal)
Fat (g)
Saturated fat (g)
Monounsaturated fat (g)
Polyunsaturated fat (g)
Cholesterol (mg)
Protein (g)
Carbohydrate (g)

0.68
0.50
0.59
0.62
0.59
0.48
0.55
0.46
0.52

*Sec Appendix for supplementary questions.

0.67
0.51
0.60
0.63
0.59
0.48
0.55
0.48
0.51

Low-fat groups
n = 158

_~
Without
extra
questions
0.63
0.50
0.55
0.56
0.56
0.45
0.55
0.54
0.55

With
extra
questions
0.65
0.51
0.57
0.59
0.58
0.46
0.56
0.56
0.55

Validation of a Diet Questionnaire

foods, while retaining the reported distribution

of these foods. These adjustments reduced the
vitamin A overestimate by about 600 IUs (e.g.
from 7970.9 IUs to 7458.6 IUs in usual-diet),
and improved
the correlation
coefficients
slightly [e.g. from 0.47 to 0.49 for vitamin A in
usual-diet, and from 0.37 to 0.40 in low-fat
(data not shown)].
The effect of portion-size assumptions was
investigated by rerunning the analysis, setting
the program to use standard rather than age-sex
specific portion sizes, and to ignore the respondents smal1-medium-large
and use only
medium. In the low-fat group, 16 of 18 nutrients
had higher correlations when individually variable and age-sex-specific portion sizes were
used, and the average correlation was improved
from 0.49 to 0.54. In several cases the improvements resulting from use of variable portion size
were non-trivial; e.g. the correlation for calories
improved from r = 0.40 to 0.51, and that for
sodium improved from r = 0.29 to 0.43. In the
usual-diet group, 12 of 18 nutrients had higher
correlations
using variable, age-sex-specific
portion sizes, although the improvement in the
average correlation was minor. (Data not
show@.
Standard medium produces serious overestimates of nutrient intake for vitamins A, C
and fiber: for example, for vitamin A,
11,172 IUs VS 6840 by food records, in the
usual-diet group. In the low-fat group, as well,
the means produced by the use of variable
portion size were considerably closer to the diet
record values, while use of a standard medium
produced substantial overestimates of virtually
al1 nutrients. In an interview context, variable
portion size as formatted here constitutes only
20% of interview time; in a self-administered
format it is less.
Finally, a program option may be enabled,
which recalculates the dietary estimates for
respondents who are outliers with respect to
calorie intake. In the usual-diet group, 16 of 18
correlations were improved, increasing the average of the correlations from 0.55 to 0.58. In the
low-fat group, however, 15 of 18 correlations
were decreased, and the average of the correlations decreased from 0.54 to 0.53. (Data not
shown.) Thus this option should be used with
caution in study groups in which the true dietary
intake may be shifted towards the low or high
end of the intake distribution, but appears to
improve nutrient estimates slightly, in genera1
population groups.

1333
DISCU!3!3iON

This study has examined the relationship

between nutrient estimates produced by a particular diet history questionnaire and its associated nutrient analysis program, and those
produced by the mean of three 4-day diet
records collected over a 1-year period. Group
means suggested a good ability of the questionnaire to distinguish between population groups
at substantially different levels of true intake.
The correlations observed, generally in the
range of r = 0.5-0.6, suggest a moderate to
good ability of the questionnaire to place
individuals along the distribution of intake.
[Corrections for variability in the reference
data generally produce correlations which are
considerably higher than observed correlations
[5, 121, and did so in this study (data not
shown.)] The questionnaire results also appear
to be in the same range as the correlations
achievable by a single 4day record, at least in
the usual-diet grot@.
These correlations are also in the same range
as those seen for various physiologic measures.
For example, repeated measures of the sume
variable have produced correlations such as
the following: serum cholesterol,
r = 0.59
v31; adult blood pressure, r = 0.67-0.70
[14]; uric acid, r =0.53 [14]; and even, for
24hour urine sodium, r = 0.24 [15]. Furthermore, two well-accepted measures of obesity,
body mass index and skinfold measurements,
correlate only in the range of r = 0.49 to
r = 0.65 [16]. Thus, while attention should certainly be given to increasing sample sizes in
etiologic studies, so as to improve statistical
power in the face of the real misclassification
inherent in al1 diet data, it also would appear
that dietary measures of the sort shown here
should prove at least as useful in health research
as have the physiologic measures mentioned
above.
It is worth emphasizing that the observed
correlations are with the reference data, not with
true intake?; Because of day-to-day and weekto-week variability, even three 4-day records are
an imperfect estimate of true intake: most of the
nutrients had a ratio of intra-to-interindividual
variante approaching or exceeding 1.O, even on
transformed data (Black, unpublished data.) It
has been shown elsewhere [ 171that correlations
between a questionnaire and reference data
increase as the adequacy of the reference data
(i.e. the number of diet records) increases.

1334

GLADYS BLQCK et al.

With that caveat, the reference data constitute

one of the major strengths of the study. Among
others who have conducted validation studies of
diet questionnaires, few [18, 191 have collected
more days of diet records. In the present study,
three 4-day records were collected over a l-year
period, in two different seasons, with a representative inclusion of weekdays and weekends. On
the other hand, although members of the usualdiet group did not receive special diet instruction, al1 respondents were highly motivated
participants in a diet study. Unquestionably,
these results can only be generalized to respondents who are motivated to complete the questionnaire carefully, and to investigations which
have engaged in careful data quality control,
double-keying, edit-checking, and querying of
questionable responses.
The means shown in Table 2 indicate a
surprising ability of this questionnaire to estimate absolute values of usual nutrient intake
with considerable accuracy. This was true even
in two study groups with widely differing
macronutrient intake.
Other investigators have conducted validations in which diet records or recalls were
used as the reference method [18-211. Willett et
al. [lg] used as reference data the mean of four
7-day records collected over a l-year period, in
validating a self-administered frequency questionnaire in a population of nurses; correlations
averaged approximately 0.45 without adjustment for calories. In a different population,
including men as wel1 as women, Willett er al.
[22] validated a longer version of the questionnaire against 365 days of diet records, and
obtained correlations averaging 0.60. Pietinen
et al. [ 181 had 12 2-day records collected over
a ti-month period; an average correlation coefficient of 0.59 was observed for comparable
nutrients, in their population of Finnish men.
Balogh et al. [21] validated a brief frequency
interview against eight or more nonconsecutive
24-hour recalls collected at random over 1 year,
in 71 male subjects. An average correlation
coefficient of 0.73 was obtained for nine comparable macronutrients;
micronutrients
were
not reported.
Jain et al. [20] conducted a validation study of
a frequency-based diet history interview, in 16
male subjects. The interview was focused on diet
in the past month, and the reference data consisted of 30-day records maintained by their
spouses. The average correlation coefficient,
based on nine comparable nutrients, was 0.45.

Stuff et al. [23] compared a quantified frequency

questionnaire with a 7-day record in a group of
lactating women, but only obtained correlations
ranging from 0 to 0.24.
These studies have differed in many respects,
and the observed correlations are not directly
comparable. In general, self-administered questionnaires would be expected to do more poorly
than interviewer-administered
questionnaires,
because of respondent errors; study groups including both sexes would tend to have a wider
range of nutrient intake and therefore higher
correlations; and questionnaires targeted to and
validated against a brief recent period such as
the past month would be expected to produce
higher correlations than those which ask about
a longer and more variable period such as the
past year. Finally, the more days of diet records
comprising the reference data, the better the
correlations are likely to be [17].
The questionnaire examined here appears to
have a reasonable ability to assess current diet,
over a wide range of nutrients and levels of fat
intake. In addition to producing correlations in
the same range as others have achieved using
comparable reference data, it yields average
nutrient estimates very similar to those obtained
from multiple diet records. Thus, it may be
useful both in dietary research settings and in
clinical or counselling situations where nutrient
estimates for a particular individuals usual diet
are needed. The questionnaire and accompanying nutrient analysis software are available from
the authors on request.
Acknowledgemenrs-The
authors wish to acknowledge the
thouehtful advice of Susan Goldman in the develonment
and Lsting of the questionnaire.

REFERENCES
1. Black G, Hartman AM. Issues in reproducibility and
validity of dietary studies. Am J Clin Nutr 1989; 50:
1133-1138.
2. Black G, Hartman AM, Dresser CM, Carroll MD,
Gannon J, Gardner L. A data-based approach to diet
questionnaire design and testing. Am J Epidemiol 1986;
124: 453-469.
3. Smucker R, Black G, Coyle L, Harvin A, Kessler L.
A dietary and risk factor questionnaire and analysis
system for personal computers. Am J Epidemiol 1989;
129: 445449.
4. Cummings SR, Black G, McHenry K, Baron RB.
Evaluation of two food frequency methods of measuring dietary calcium intake. Am J Epidemiol 1987; 126:
796-802.
5. Sobell J, Black G, Koslowe P, Tobin J, Andres R.
Validation of a retrospective questionnaire assessing
diet 10-15 years ago. Am J Epidemiol 1989; 130:
173-187.

Validation of a Diet Questionnaire

6.

1.

8.

9.
10.

ll.

12.

13.
14.

15.
16.

17.
18.

19.

20.

Clifford CK, Butrum RR, Greenwald P, Yates JW.

Clinical trials of low fat dists and breast cancer
pmvention. In: Ip C, Birt DF, Rogers AE, Metthn C,
Eds. DIatary Fat aud Caneer. New York: Alan R. Lim,
Inc.; 1986: 93-115.
Prentice RL, Kakar F, Hunting S, Sheppard L, Klein
R, Kushi LH. Aspects of the rationale for the
Womens Health Trial. J NatI Caneer Inat 1988; 80:
802-814.
Insull W, Henderson MM, Prent& RL, Thompson
DJ, Clifford CK, Goldman S, Gorbach S, Moskowitx
M, Thompson R, Woods M. Results of a randomixed
feasibility study of a low-fat diet. Ar& Intern Med
1990; 150: 421427.
Catsos PD, Woods M, Selles W, Gorbach S. The Tufts
nutrient calculation system. 1988; Unpublished.
Willett WC, Stampfer MJ, Underwood BA, Speixer
FE, Rosner B, Hennekens CH. Validation of a dietary
questionnaire with plasma carotenoid and alpha-tocopherol levels. Am J CBa Nutr 1983; 38: 631639.
Walker AM, Blettner M. Comparing imperfect
measures of exnosure. Am J Esddemiol 1985: 121:
183-190.
Liu K, Stamler J, Dyer A, McKeever J, McKeever P.
Statistical methods to assess and minimize the role of
intra-individual variability in obscuring the relationship between dietary lipids and serum cholesterol.
J Chrou Dis 1978; 31: 399-418.
Morris JN, Marr JW, Heady JA, Mills GL, Pilkington
TRE. Diet and plasma cholesterol in 99 bank men. Br
Med J 1963; March: 571-576.
Rhoads GG. Reliability of diet measures as chronic
disease risk factors. Am J CBn Nutr 1987; 45:
1073-1079.
Liu K, Dyer AR, Cooper RS, Stamler R, StamIer J.
Can ovemight urine replace 24hour collection to
assess salt intake? Hypertersrdou 1979; 1: 529-536.
Stokes 111J, Garrison RJ, Kannel WB. The independent contributions of various indices of obeaity to the
22-year incidence of coronary heart disease: The
Framingham heart study. In: Vague J, Bjomtorp P,
Guy-Grand B, Rebuffe-Scrive M, Vague P, Eds.
MetahoI& CumpBcatIous of Hmnan Ohe&&sr Proc 6th
Iut Meeting of FadoerlwLogy, Marseille, 30 May-l
June 19851 Amsterdam: xcerpta Medica/Elsevier
Science Publishers: 1985: 49-57.
Potosky A, Black G, Hartman AM. The effect of the
referenc-e data in diet questionnaire validations. J Am
DIet Asaoe 1990: J Am DIet Amue 1990: 90: 810813.
Pietinen P, Hartman AM, Haapa E; Rasanen L,
Haapakoski J, Palmgren J, Albanes D, Virtamo J,
Huttunen JK. Reproducibility and validity of dietary
assessment instruments: 1. A self-administered food
use questionnaire with a portion size picture booklet.
Am J Epidemiol 1988; 128: 655-666.
Willett WC, Sampson L, Stampfer MJ, Rosner B, Bain
C, Witschi J, Hennekens CH, Speizer FE. Reproducibility and validity of a semiquantitative food frequency
questionnaire. Am J EpIdemIoI 1985; 122: 51-65.
Jain M, Howe GR, Johnson KC, Miller AB. Evahration of a diet history questionnaire for epidemiologic
studes. Am J EpIdemIol 1980; 111: 212-219.

1335

21.

Balogh M, Kahn HA, Medalie JH. Random repeat

24hour dietary recalls. Am J CBn Nutr 1971; 24~
304-310.
22. Wiiett WC, Reynolds RD, Cottrell-Hoehner
S,
Sampson L, Browne ML. Validation of a semiquantitative food frequency questionnaire: comparison
with a 1-year diet record. J Am Dkt Anme 1987; 87;
43-47.
23. Stuff JE, Garxa C, OBrian Smith E, Nichols
BL, Montandon CM. A compatison of dietary
methods in nutritional studies. Am J CIin Nutr 1983;
37: 300-306.

APPENDIX
The following infonnation is provided so that other
investigators could replicate the questionnaire nutrient
analysis exactly as it was performed for this investigation.
(1) Questions and food items omitted from this questionnaire version: open-ended section; brand of dry cereal
used; cantaloupe out of season; Tang; mixed vegetables;
winter squash, chilli peppers; livenvurst; pmnpkin pie;
decaffeinated toffee (combined with toffee); tea (combined
with toffee).
(2) Questions and food items added to this questionnaire
version: When you eat poultry, is it usually light meat, dark
meat, both?; When you eat hamburger or beef, is it
usually ngular, lean, extra lean? When you eat tuna, it is
usually oil pack, water pack, either one, dont know? If
you eat yogurt, what brand do you usually eat?, coded as
regular, low-fat or non-fat; If you eat lunch meats or hot
dogs, what kind do you usually eat?, coded as regular or
low-fat; diet margarine and whipped butter were added as
possible responses to the kind of fat usually added to
vegetables; popcorn; diet salad dressing sherbet or jello;
low-fat cottage cheese or low-fat cheeses; plain yogurt.
(3) Errors which caused respondents to be dropped:
skipped more than 15 food items; total foods per day adds
to fewer than 4; total foods from lunch dishes, soups,
breads, breakfast foods, dairy products and desserts/sweets
adds to fewer than two per day; medium or omitted
portion sim for 100% of foods; once per unit time
responses for more than 75% of foods; more than two foods
had unreasonable frequenties (e.g. carrots six times a
day).
(4) The following represent options in the nutrient
analysis software which affect the calculations. Users of
this software who wish to duplicate the conditions of
the present analysis would use the following option settings:
PGRTIONS = AGESEX,
MEDGNLY = NO,
RESTADJ = YES, ADDFATS = YES, BIGFAT = YES,
LEANQUES = YES, TUNAQUES = YES, VEGADJ =
YES, FRTADJ = YES, YEARCOL = 0.0384599. For the
DietEdit program, FIX = YES. These settings create a
nutrient analysis with the characteristics described under
Methods, including use of age- and sex-specitc portion
sizes, use of the respondents portion sim information
(small, medium, large), etc. For further information
about calculation options and other characteristics of the
nutrient analysis system, consult the manual which accompanies the software, available from tbe authors.