BEHAVIOR THERAPY30, 17-30, 1999

A Consumer Satisfaction Measure for Parent-Child Treatments and Its Relation to Measures of Child Behavior Change
ELIZABETH V. BRESTAN

JENIFER R. JACOBS
ARISTA D. RAYFIELD
SHEILA M . EYBERG

Universi~ of Florida
The Therapy Attitude Inventory (TAIL a brief consumer satisfaction measure of parent training, parent-child treatments, and family therapy, was psychometrically evaluated. Participants were 62 mothers of clinic-referred children who met diagnostic criteria for a disruptive behavior disorder and participated in parent-child interaction therapy. Cronbach's alpha for the TAI was excellent (.91) and the stability coefficient across a 4-month period was also high (.85). External validity was demonstrated by moderate correlations (.36 to .49) between TAI scores (total and factor) and changes during treatment measured by pre- to posttreatment difference scores on the Eyberg Child Behavior Inventory and on behavioral observations of compliance. Satisfaction with process was related to changes in parent behavior ratings, and satisfaction with outcome was related to changes in observed compliance. Correlations between TAI scores and behavior problem severity after treatment were not significant. Results suggest that parent satisfaction ratings on the TAI are more closely linked to symptom changes during treatment than to the absolute level of child behavior problems after treatment, and they support the psychometric strength of the TAI. Elizabeth Brestan is now at the Department of Pediatrics, University of Oklahoma, Health Sciences Center, Oklahoma City; Jenifer Jacobs is now at the University of California, San Diego; Arista Rayfield is now at the Department of Pediatrics, Kansas University Medical Center, Kansas City. This work was supported in part by grants from the National Institute of Mental Health (ROIMH-46727) and from the University of Florida Division of Sponsored Research. Participants in these studies were obtained from the databases of two prior, independent studies (Eisenstadt et al., 1993; Schuhmann et al., 1998). Portions of this paper were presented at the annual meeting of the Florida Psychological Association, Key West, June 1995, and the annual meeting of the American Psychological Association, New York, August, 1995. We would like to thank Kenneth Tercyak for his critical reading of an earlier draft of this manuscript. Correspondence concerning this article should be addressed to Sheila Eyberg, Department of Clinical and Health Psychology, University of Florida, PO Box 100165, Gainesville, FL 32610; e-mail: seyberg@hp.ufl.edu. This article was originally accepted under the editorship of Frank Andrasik, Ph.D. 17 005-7894/99/0017-003051.00/0 Copyright 1999 by Associationfor Advancementof BehaviorTherapy All rightsfor reproductionin any formreserved.

18

BRESTAN ET AL.

Due to recent changes in mental health delivery and funding, and an increasing emphasis on treatment effectiveness in outcome research, consumer satisfaction is becoming recognized as an essential component in the outcome assessment of psychological treatments, including parent training (Plante, Couchman, & Diaz, 1995; Seligman, 1995). Consumer satisfaction in the context of psychosocial treatments refers to the extent to which patients or clients liked the process and the outcome of the treatment they received, including the treatment format, the techniques used, and the effects of treatment. As such, measures of consumer satisfaction are considered the most subjective measures of treatment effectiveness (Eyberg, 1993). Indeed, studies of consumer satisfaction have shown moderate positive correlations between treatment satisfaction scores and trait scores on measures of general life satisfaction (Levois, Nguyen, & Attkisson, 1981, as cited in Lebow, 1983) and interpersonal trust (Distefano, Pryer, & Garrison, 1981) among adult clients. Although this degree of bias does not necessarily invalidate the measures (Lebow, 1983), it does make it imperative to demonstrate the construct validity of satisfaction measures using more behavioral, independent criteria of the quality of care and the success of treatment in achieving its goals. In the behavioral literature, consumer satisfaction is considered an index of social validity. Many outcome studies that have included assessment of consumer satisfaction have suggested that greater satisfaction is related to greater efficacy of treatment. However, these studies have not typically used measures of satisfaction or other outcome measures with established reliability or validity (e.g., Bradley & Clark, 1993; Charlop, Parrish, Fenton, & Cataldo, 1987; Kopec-Schrader, Rey, Plapp, & Beumont, 1994). In fact, few existing consumer satisfaction measures have psychometric evidence of reliability, validity, or generalizability (Bornstein & Rychtarik, 1983; Lebow, 1982; McMahon & Forehand, 1983; Sorenson, Kantor, Margolis, & Galano, 1979). Further, as explained by McMahon and Forehand, exclusive use of treatment completers as participants in most consumer satisfaction studies has been a major source of sampling bias, producing high baseline scores and limited response variability, which reduces the value of the results. One consumer satisfaction measure with adequate psychometric support is the Client Satisfaction Questionnaire (CSQ; Larsen, Attkisson, Hargreaves, & Nguyen, 1979). This measure of consumer satisfaction has been used extensively with adult consumers, clients with diverse racial/ethnic backgrounds (Barrera, MacFarlane, Paredes, & Brown, 1985; de Brey, 1983; Perreault, Leichner, Sabourin, & Grendreau, 1992), and, to a limited extent, with parents. Specifically, Byalin (1993) assessed parent satisfaction with adolescent inpatient psychiatric treatment. As a global measure, the CSQ has been used with a variety of populations, but it was not designed or standardized to assess parent satisfaction with parent training, parent-child treatments, or behavioral family treatments. These treatments typically involve specific goals, such as increasing child compliance or enhancing parent-child corn-

THERAPYATTITUDEINVENTORY

19

munication, that are not reflected in the items of the CSQ or other global scales. The need for additional, standardized measures of treatment satisfaction is a priority, particularly for the child treatments. In behavior therapy research, it is advantageous to have several measures of the same or similar constructs, as these measures can tap different aspects of the same construct, such as consumer satisfaction of adults with their personal therapy versus parent satisfaction with their child's treatment. The TAI 1 (Eyberg, 1993; Eyberg & Johnson, 1974) is one of the earliest measures of consumer satisfaction in current use. It is distinct from measures such as the CSQ in that it was designed specifically for use with parent training, parent-child treatments, or behavioral family therapy treatment. In addition, scores on the TAI are not inordinately hampered by the ceiling effects that restrict the range of scores on many measures of consumer satisfaction. The TAI assesses satisfaction in such areas as the parenting skills learned, the child's behavior changes, and the type of treatment program used. The items of the TAI were rationally derived to reflect the goals of parent-child treatments (Foote, Eyberg, & Schuhmann, 1998) to promote parent-child relationship skills and to decrease negative behaviors in the interaction. The reliability and discriminative validity of the TAI were demonstrated by Eisenstadt, Eyberg, McNeil, Newcomb, and Funderburk (1993) in a study that compared two protocols of treatment delivery in parent-child interaction therapy. Cronbach's alpha for the TAI in the sample was .88. Significant differences in posttreatment scores between the groups (Ms -- 45.8 and 49.6, respectively) showed sensitivity within the highly satisfied range of the TAI. Significant differences in TAI scores have also been shown between different child treatments (Bernal, Klinnert, & Schultz, 1980; Taylor, Schmidt, Pepler, & Hodgins, 1998). For example, Eyberg and Matarazzo (1980) found higher scores after a 5-week program of individual parent-child treatment (M = 44) than after a 5-week group parent training program (M = 35). Taken together, these studies suggest that the TAI is sensitive to the variability in satisfaction among different parent training models. Several studies that have used the TAI to document consumer satisfaction have suggested that the TAI assesses satisfaction with two aspects of treatment: process and outcome (Christensen, Johnson, Phillips, & Glasgow, 1980; Johnson & Christensen, 1975; Schuhmann, Foote. Eyberg, Boggs, & Algina, 1998). Whether these aspects of treatment are actually distinct in the TAI, however, has not been empirically examined. In addition, past studies have not documented an assumed relationship between treatment satisfaction, as measured by the TAI, and other outcome measures that have less inherent subjectivity, such as direct observation. If the TAI has distinct process and outcome factors, their separate relationships with child behavior outcome The TAI may be obtained from Sheila M. Eyberg at the Department of Clinical and Health Psychology, University of Florida, PO Box 100165, Gainesville, FL 32610. Electronic mail may be sent via Internet to seyberg@hp.ufl.edu.

20

B R E S T A N ET AL.

could inform treatment. Finally, although the internal consistency of the TAI has been documented previously, no study to date has examined the external reliability, or stability, of the measure. To address stability in a posttreatment outcome measure, it is important to study its performance following a treatment that has itself demonstrated stable outcomes following treatment. For this reason, we examined TALl scores following an empirically supported treatment with evidence of treatment maintenance (Brestan & Eyberg, 1998; Funderburk et al., 1998). The purpose of Study 1 was to establish further the reliability and internal validity of TAI. We examined the factor structure of the TAI using a heterogeneous sample of parent-child interaction therapy completers and a small number of dropouts. We also examined the stability of the TAI with families who completed both a posttreatment and a follow-up assessment. The purpose of Study 2 was to establish the convergent validity of the TAI by examining its relationship to standardized outcome measures of child behavior change at posttreatment assessment. We hypothesized that higher levels of parent satisfaction with treatment, as measured by the TAI, would correlate negatively with changes in maternal ratings of child behavior problems and positively with changes in child compliance as measured by direct observation during mother-child interactions.

STUDY 1
Method

Participants The participants were 62 mothers and their children (53 boys and 9 girls), drawn from patient files of two earlier treatment studies (Eisenstadt et al., 1993; Schuhmann et al., 1998). The children's mean age at treatment intake was 4.5 years (SD = .97). All children had been diagnosed with Oppositional Defiant Disorder (ODD), Attention-Deficit/Hyperactivity Disorder (ADHD), Conduct Disorder (CD), or some combination of these disruptive behavior disorders based on the DSM-III-R Structured Interview for Disruptive Behavior Disorders (McNeil, Eyberg, Eisenstadt, Newcomb, & Funderburk, 1991). The mean family income was $24,348 (SD -- 16,903; median: $19,000). Families were 77% White, 13% African American, 4% Hispanic American, 4% Asian American, and 2% mixed. The families in Study 1 included 57 treatment completers and 5 treatment dropouts who had attended a posttreatment assessment 4 months alter treatment intake and completed a TAI (see Procedures). Measures DSM-III-R Structured hlterview for Disruptive Behavior Disorders (McNeil etal., 1991). This interview was conducted with the mothers to determine whether the referred child met diagnostic criteria for a disruptive behavior disorder. High levels of interrater percent agreement were obtained

T H E R A P Y ATTITUDE INVENTORY

21

from this interview (>.98 for presence of specific symptoms, duration of symptoms, and presence of disorder) in both studies from which the participants in the present study were drawn (Eisenstadt et al., 1993; Schuhmann et al., 1998).

Therapy Attitude Inventory. (TAI; Eyberg, 1993; Eyberg & Johnson, 1974).
The TAI is a brief measure of parent satisfaction with parent training or family therapy. The TAI includes 10 items addressing the impact of therapy on parenting skills and child behavior. Parents rate items on a scale from 1 (indicating dissatisfaction with treatment, or a worsening of problems) to 5 (indicating maximum satisfaction with treatment or improvement of problems). The item ratings are summed to yield a total score between 10 and 50.

Procedures
Data for both Study 1 and Study 2 were collected retrospectively from the files of participants in two prior treatment outcome studies of conductdisordered children (Eisenstadt et al., 1993; Schuhmann et al., 1998) in which the assessment and treatment protocols were similar. All participant families in the original studies had been referred to the Psychology Clinic at the University of Florida Health Science Center for treatment. During the intake assessments, the DSM-III-R Structured Interview had been conducted with the child's mother, and the parents completed several measures, including an Eyberg Child Behavior Inventory (ECBI). The outcome study had been described to the parents at the intake assessment, and they had signed an informed consent allowing their data to be used for research purposes. A posttreatment assessment session had been conducted 4 months after the treatment intake, and had included a TAI and readministration of the ECBI. In those studies, both the treatment completers and the treatment dropouts had been invited to return for the 4-month posttreatment assessment and had been offered payment for participating in the assessment. Of the 30 families who had dropped out of treatment in those studies, 5 dropout families returned for the posttreatment assessment along with the 57 treatment completers. A subset of the families (from Schuhmann et al.) had also been invited to return for an 8-month posttreatment assessment and were again offered payment for participating. Twenty-one families (treatment completers) returned for the 8-month assessment and were included in an analysis of TAI stability. The treatment program used in these studies was parent-child interaction therapy (Eyberg & Robinson, 1982; Hembree-Kigin & McNeil, 1995). This is an empirically supported treatment (Brestan & Eyberg, 1998) designed for parents and young children with behavior problems. During treatment, graduate student-lead therapists (n = 9) and their co-therapists taught and coached parents to use relationship-enhancement skills (e.g., praise, active listening), teaching skills (e.g., information descriptions), and behavior management skills (e.g., direct commands, time-out) to change the problem behaviors of their child. At the beginning of each treatment session, therapists coded the

22

BRESTAN ET AL.

mother and child behaviors during standard parent-child interactions to assess progress and guide the course of treatment. The mean number of 1-hour treatment sessions for the treatment completers was 14. Families were considered treatment dropouts if they attended at least 1 treatment session and stopped attending treatment sessions before meeting the termination criteria. Families who did not complete treatment attended between 1 and l0 sessions.

Analyses
Initially, TAI score distributions were examined and descriptive statistics were calculated. The mean score for the treatment completers (n = 57) was compared to the mean score for the treatment dropouts (n = 5) using an independent sample t-test. A principal components analysis was conducted on the TAI items. A varimax rotation was used to determine the number of factors necessary to describe the data. Cronbach's alphas were calculated for the TAI total and factor scores. Pearson correlations were used to assess the stability of the TAI.

Results
The sample distribution of TAI scores in Study 1 was unimodal and had a slight negative skew (skewness = - 1,70; kurtosis = 3.48) in this sample of treatment completers and treatment dropouts. The mean TAI total score was 44.58 (SD = 5.13). The TAI total scores ranged from 28 to 50 for the treatment completers and from 30 to 50 for the treatment dropouts, demonstrating the ability of the TAI to reflect variability in consumer satisfaction and providing a heterogeneous sample for factor analysis of the inventory. The TAI total score was significantly higher for the 57 treatment completers (M = 45.12, SD = 4.57) than for the 5 treatment dropouts (M -- 38.4, SD = 7.53), t(60) = 2.98, p = .004. A principal components analysis of posttreatment TA1 data (n = 62) yielded two factors with eigenvalues greater than 1. Together, the two rotated factors accounted for 70% of the variance. Factor 1 consisted of six items that appeared to represent the parent's satisfaction with child behavior changes following treatment. This rotated factor containing six items, labeled satisfaction with outcome, accounted for 44% of the variance. Factor 2 consisted of the remaining four items, which appeared to represent the parent's satisfaction with components of the treatment~ such as the parenting skills that were taught and the general approach to treatment. This factor, labeled satisfaction with process, accounted for an additional 26% of the variance. The items and their factor loadings are presented in Table 1. The mean score for Factor 1 was 27.32 (SD = 3.61), and the mean score for Factor 2 was 17.42 (SD = 2.20). Cronbach's alphas for the TAI total score and the Factor 1 score were excellent (.91 and .93, respectively). Coefficient alpha for the Factor 2 score was satisfactory (.76). The stability correlations for the TAI, assessed by correlating the mothers' scores at posttreatment and 4-month follow-up, were

THERAPY ATTITUDE INVENTORY TABLE 1 FACTOR LOADINGS ON THE THERAPY ATTITUDE INVENTORY Item Content 1 learned useful techniques for disciplining 2 learned useful techniques for teaching 3 status of relationship with child 4 ability to discipline my child 5 major behavior problems of child 6 child's compliance to commands 7 general progress of child 8 problems unrelated to child 9 type program used to improve child behavior 10 general feeling about the program
Note.

23

Factor 1 .39 .11 .81 .80 .82 .85 .83 .31 .72 .39

Factor 2
.72 .87 .34 .32 .33 .29 .33 .69 .15 .57

Numbers in italic represent the strongest factor loading for each item.

r(21) = .85, p < .01, for the TAI total score; r(21) = .65, p < .01, for the satisfaction with outcome factor; and r(21) = .52, p = .015, for the satisfaction with process factor.

STUDY 2 Method Participants

For the second study, the five dropout families were excluded so as not to artificially inflate correlations among the outcome measures. In this sample, there were 48 boys and 9 girls, with a mean age of 4.5 years (SD = .99) at the initial assessment. The mean family income was $25,200 (SD = $16,500; median = $20,000). The racial/ethnic background of the families was 75% White, 13% African American, 3% Hispanic American, 3% Asian American, and 3% mixed.

Measures ECBI (Eyberg & Pincus, in press). The ECBI is a 36-item parent-report
measure of disruptive child behaviors that contains two scales: the Intensity Scale, which assesses the frequency of each behavior problem from never (1) to always (7); and the Problem Scale, which assesses the number of behavior problems that the parent finds difficult. These scales have been restandardized with 798 children between the ages of 2 and 16 (Colvin, Eyberg, & Adams, 1998). Additional studies have demonstrated the construct validity (Boggs, Eyberg, & Reynolds, 1990; Webster-Stratton & Eyberg, 1982) and sensitivity to change during treatment (Schuhmann et al., 1998; Webster-Stratton, 1984) of the ECBI.

24

BRESTAN ET AL.

Dyadic Parent-Child Interaction Coding System-H (DP1CS-H; Eyberg, Bessmep; Newcomb, Edwards, & Robinson, 1994). This coding system assesses
the quality of parent-child social interaction during standard situations observed in the clinic. Adequate interrater reliability and discriminative and concurrent validity for the live coding version have been reported for all behavior categories that comprise child compliance to parent commands (Eyberg et al., 19941). For the present study, we used the live coding of child compliance during the first and last treatment sessions, drawn retrospectively from the treatment files of the participants in the Schuhmann et al. (1998) study, to examine outcome on the behavioral observation measure of child behavior. Because the data recorded during treatment sessions are primarily used for clinical purposes, reliability data are not routinely collected and were not available for this study.

Analyses
For Study 2, Pearson correlation coefficients were used to evaluate the concurrent validity of the TAI total and factor scores and child behavior outcomes assessed by similar (parent report) and different (observational) methods of measurement. In addition, two different methods of judging outcome for an individual child were used: (a) the single score representing the posttreatment level of child behavior on the measures; and (b) the difference score representing change on the measures from pre- to posttreatment. The latter change scores were expected to correlate with the TA1. The parent rating scale (ECBI) provided consideration of a broad spectrum of behaviors in the home, and the observational measure provided a cross-informant measure of a single, salient behavior (compliance to the mother/consumer). To control for Type I error among the 12 correlations with parent report and the 6 correlations with observational data, Bonferroni corrections were used requiring significance levels of .004 and .01, respectively.

Results
Posttreatment scores for the outcome measures are shown in Table 2, and the correlations among the outcome measures are shown in Table 3. The correlations between all TA1 scores and the posttreatment scores on the child behavior measures are not significant at protected alpha levels. In contrast, the difference scores for the child outcome measures, which represent the change between pre- and posttreatment on these measures, are all significantly correlated with the TAI total score. Observation of scatter plots of the difference scores on the child measures showed normal distributions for all scores, although there were two individuals whose change scores were markedly in the negative direction on the DPICS-II compliance score ( - . 3 3 and - . 13). The TAI Factor 1 scores show significant, positive correlations of moderate magnitude with the difference score measures on the ECBI parent rating scales, but not on the observational measure of compliance. The TAI Factor 2 score shows a significant, positive correlation of moderate magnitude with

THERAPY ATTITUDE INVENTORY TABLE 2 POSTI"REATMENT MEANS OF CHILD BEHAVlOR OUTCOME MEASURES Outcome Measures Eyberg Child Behavior Inventory Intensity score Problem score Intensity difference score Problem difference score Dyadic Parent-Child Interaction Coding System % Compliance % Compliance difference score
n M

25

SD

57 57 57 57 39 39

126.89 10.92 55.75 14.96 88 56

38.31 9.96 32.78 9.08 21 27

the DPICS-II compliance measure, but not with the parent rating scale measure of behavior problems (see Table 3).

Discussion
Over a decade ago, treatment researchers called for psychometrically sound measures of consumer satisfaction with treatments for children. The need to document the reliability and to examine the relationship of treatment satisfaction to other outcome measures and eventually to generalization and maintenance of child behavior change was recognized (McMahon & Forehand, 1983). The two studies described in this article were designed to address this need. Study 1 extended earlier reports of the psychometric adequacy of the TAI by replicating findings of high internal consistency and establishing empiri-

TABLE 3 THERAPY ATTITUDE INVENTORY CORRELATIONS WITH OUTCOME VARIABLES AT POSTTREATMENT TAI Scores Outcome Measures Eyberg Child Behavior Inventory Intensity score Problem score Intensity difference score Problem difference score Dyadic Parent-Child Interaction Coding System % Compliance % Compliance difference score
a

Total Score

Factor 1

Factor 2

-.28 -.35 .46 a .46 a .29 .36 b

-.26 -.32 .49 a .46 a .26 .24

-.10 -.16 .38 .41 .26 .41 b

Correlation significant at p < .004. b Correlation significant at p < .01.

26

BRESTAN ET AL.

cally its two previously assumed factors, satisfaction with outcome and satisfaction with process. Study 1 also demonstrated stability in the TAI scores. We believe that TAI scores reflect parental "states," as opposed to "traits," and thus the stability reflected in these scores is in some sense a reflection of the stability of the treatment effects and the therapeutic alliance. Still, these scores suggest that after an effective treatmenL the TAI scores represent reliable attitudes that may contribute to the maintenance of parenting skills and child behaviors. Because the Study 1 sample included a small number of treatment dropouts as well as completers, these data are more likely to be representative of mental health service consumers than those from earlier studies (Lebow, 1982). The dropouts also allowed us to examine in a very preliminary way the validity of the TAI to distinguish dropouts from completers for one type of child treatment. However, the small number of dropouts (5) obtained in our study represents only 8% of the sample, in contrast to the estimated 40% to 60% dropout rate typically found for treatment with conduct-disordered children (Prinz & Miller, 1994) and the approximate 34% dropout rate in the studies from which the present data were drawn. Further study of the TAI and other measures of treatment satisfaction with more representative intent-totreat samples will be useful for estimating the true value of measuring satisfaction with treatment. Study 2 supports our hypothesis that maternal satisfaction with treatment is related to both parent report and observational measures of child behavior outcome following parent-child treatment. Specifically, results demonstrated that the TAI total score is moderately correlated with change in mothers' ratings of their child's behavior and in observational data of child behavior change. The moderate relation between the TAI total score and more objective methods of measuring change (behavioral rating scale and observation) provides support for the concurrent validity of the TAI as a measure of treatment outcome. Study 2 also establishes the concurrent validity of the TAI factor scores. Significant correlations between the TAI Factor l (satisfaction with outcome) score and the pre-post difference scores on the Intensity and Problem Scales of the ECBI indicate that mothers appear to be sensitive to changes in the impact and frequency with which their child displays misbehavior. The greater the perceived change, the greater is the mothers' satisfaction with treatment outcome. The TAI Factor 2 (satisfaction with process) score is significantly related to the observational measure of change. This association may be due to mothers' attributing the change in compliance, which is a primary behavior targeted during clinic training sessions, most directly to their own efforts in treatment. Thus, the association between treatment satisfaction as measured by the TAI and the improvements resulting from parent training suggest that mothers are sensitive to the degree of change in their child's behavior and that this change is reflected in their attitudes about therapy as it is reported on the TAI.

T H E R A P Y A T T I T U D E INVENTORY

27

As predicted, the pattern of correlations among the measures also suggests that the amount of change in the child's behavior problems during treatment, rather than the absolute level of their severity after treatment, contributes to the level of maternal satisfaction with treatment. Said another way, these data provide evidence supporting the sensitivity of the TAI to treatment outcome. In addition, the TAI factors correlate somewhat differently with outcome measures, demonstrating that they reflect distinct dimensions of satisfaction with treatment. Strengths of these studies include the use of a diverse, clinic-referred sample representative of typical clinical populations of young children with conduct-disordered behavior. The studies address a critical need in treatment outcome research for psychometrically sound measures of consumer satisfaction. Without evidence of reliability, validity, and sensitivity to treatment effects, clinicians and researchers cannot know whether results describing parent satisfaction represent actual treatment satisfaction versus measurement error. It is only with accurate measurement that researchers can explore the presumed links between parent satisfaction with treatment and the maintenance of change in parenting skills and child behavior. The moderate correlations obtained in this study between treatment satisfaction and symptom change, although consistent with findings published in the adult psychotherapy literature (Ankuta & Abeles, 1993; Attkisson & Zwick, 1982), do not imply that satisfaction can be used as a proxy for change in psychopathology. There has been far too little research on consumer satisfaction measures to understand their precise relationship to meaningful, durable change. The surge in popularity of satisfaction as the preferred index of outcome by managed care organizations (Bilbrey & Bilbrey, 1995) is a significant concern, and the need for further research on measures of consumer satisfaction is a public health need. A major limitation in the present studies is the absence of inter-observer reliability data for the behavioral observation of compliance used in Study 2. The clinic version of the coding system was shown in the standardization study to have acceptable interobserver agreement for the behavioral categories used to record child compliance (Eyberg et al., 1994), and the therapists in the original treatment studies were well trained in the coding system. However, reliability data were not coded for the data used in this study, and the therapist/coders were not masked to whether families were at pre- or posttreatment. Still, the consistency in the pattern of validity correlations among the measures in Study 2 suggests that the reliability may not have been unduly compromised. Other limitations include the small number of treatment dropouts in the present research, discussed earlier, and the small number of participants (21) in the stability analysis due to attrition between posttreatment and follow-up. Attrition in treatment and follow-up is a major concern with conduct-disordered youngsters, and we have elsewhere reviewed a number of strategies to improve treatment and follow-up attrition (Eyberg, Edwards, Boggs, & Foote, 1998).

28

BRESTAN ET AL.

Future Directions

A useful direction for future study of the TAI is to examine whether demographic variables are associated with consumer satisfaction. Although there is a much larger proportion of boys than girls in treatment outcome studies targeting conduct problem behavior (5: l), there is little information available on whether boys and girls respond differently to treatment (Brestan & Eyberg, 1998) and whether these differences would extend to parental satisfaction with treatment. Studies have shown that scores on the TAI are often dissimilar for mothers and fathers who have both participated in their child's treatment (Berhal et al., 1980; Taylor et al., 1998). Investigation of factors contributing to the differential satisfaction of mothers and fathers is warranted as well. The TAI was designed to assess satisfaction with parent training or family therapy from the parent's perspect;,ve. Although we examined satisfaction from only the mothers' point of view in the studies reported here, we recognize that each family member in treatment has a unique vantage point and can contribute information to inform future treatment. In addition to exploring the role of fathers' satisfaction with treatment, it will be useful to develop brief measures to assess the child's satisfaction with treatment. The role of child satisfaction will likely differ at different developmental stages, and may mediate the relation between parent satisfaction and other indices of outcome differently at different ages as well. A final direction for future research we will note is to examine changes in the relation between therapy progress and parent satisfaction at multiple points during treatment and follow-up. Study 2 links scores of child behavior change and parent satisfaction obtained at posttreatment, but does not indicate whether the links are causal, whether they are direct or indirect, or the direction of possible influence. Answers to these questions may provide information to guide treatment planning, and may extend to planning maintenance of treatment gains.

References
Ankuta, G. Y., & Abeles, N. 11993). Client satisfaction, clinical significance, and meaningful change in psychotherapy. Professional Psychology: Research and Practice, 24, 70-74. Attkisson, C. C., & Zwick, R. (1982). The client satisfaction questionnaire: Psychometric properties and correlations with service utilization and psychotherapy outcome. Evaluation and Program Planning, 5, 233-237. Barrera, M., MacFarlane, S.. Paredes, R., & Brown, C. (1985). Assessing client satisfaction in English and Spanish: Replication and extension. Hispanic Journal of Behavioral Sciences, 7, 261-271. Bernal, M. E., Klinnert, M. D., & Schultz, L. A. (1980). Outcome evaluation of behavioral parent training and client-centered parent counseling for children with conduct problems. Journal of Applied Behavior Analysis, 13, 677-691. Bilbrey, J., & Bilbrey, E (1995, July/August). Judging, trusting, and utilizing outcomes data: A survey of behavioral health care payers. Behavioral Health Care Tomorrow~ pp. 62-65. Boggs, S. R., Eyberg, S., & Reynolds, L. A. (1990). Concurrent validity of the Eyberg Child Behavior Inventory. Journal of Clinical Child Psychology, 19, 75-78.

THERAPY ATTITUDE INVENTORY

29

Bornstein, R H., & Rychtarik, R. G, (1983). Consumer satisfaction in adult behavior therapy: Procedures, problems, and future perspectives. Behavior Therapy, 14, 191-208. Bradley, E. J., & Clark, B. S. (1993). Patients' characteristics and consumer satisfaction on an inpatient child psychiatric unit. Canadian Journal of Psychiatry, 38, 175-180. Brestan, E., & Eyberg, S. (1998). Effective psychosocial treatments for children and adolescents with disruptive behavior disorders: 29 years, 82 studies, and 5,275 kids. Journal of Clinical Child Psychology, 27, 180-189. Byalin, K. (1993). Assessing parental satisfaction with children's mental health services. Evaluation and Program Planning, 16, 69-72. Charlop, M. H., Parrish, J. M., Fenton, L. R., & Cataldo, M. F. (1987). Evaluation of hospitalbased outpatient pediatric psychology services. Journal of Pediatric Psychology, 12,485503. Christensen, A., Johnson, S. M., Phillips, S., & Glasgow, R. (1980). Cost effectiveness in behavioral family therapy. Behavior Therapy, 11,208-226. Colvin, A., Eyberg, S. M., & Adams, C. (1998). Restandardization of the Eyberg Child Behavior Inventory. Manuscript submitted for publication. de Brey, H. (1983). A cross-national validation of the client satisfaction questionnaire: The Dutch experience. Evaluation and Program Planning, 6, 395-400. Distefano, M. K., Pryer, M. W., & Garrison, J. L. (1981). Clients' satisfaction and interpersonal trust among hospitalized psychiatric patients. Psychological Reports, 49, 420-422. Eisenstadt, T. H., Eyberg, S.M., McNeil, C. B., Newcomb, K., & Funderburk, B. (1993). Parent-child interaction therapy with behavior problem children: Relative effectiveness of two stages and overall treatment outcome. Journal of Clinical Child Psychology, 22, 42-51. Eyberg, S. M. (1993). Consumer satisfaction measures for assessing parent training programs. In L. VandeCreek, S. Knapp, & T. L. Jackson (Eds.), Innovations in clinical practice: A source book (Vol. 12). Sarasota, FL: Professional Resource Press. Eyberg, S., Bessmer, J., Newcomb, K., Edwards, D., & Robinson, E. (1994). Dyadic ParentChild Interaction Coding System 11: A manual (Social and Behavioral Sciences Documents, Ms. No. 2897). San Rafael, CA: Select Press. Eyberg, S. M., Edwards, D., Boggs, S. R., & Foote, R. (1998). Maintaining the treatment effects of parent training: The role of booster sessions and other maintenance strategies. Clinical Psychology: Science and Practice, 5, 544-553. Eyberg, S. M., & Johnson, S. M. (1974). Multiple assessment of behavior modification with families: Effects of contingency contracting and order of treated problems. Journal of Consulting and Clinical Psychology, 42, 594-606. Eyberg, S. M., & Matarazzo, R. G. (1980). Training parents as therapists: A comparison between individual parent-child interaction training and parent group didactic training. Journal of Clinical Psychology, 42, 594-606. Eyberg, S. M., & Pincus, D. (in press). Eyberg Child Behavior Inventory and Sutter-Eyberg Student Behavior Inventor': Professional Manual. Odessa, FL: Psychological Assessment Resources. Eyberg, S. M., & Robinson, E. A. (1982). Parent-child interaction training: Effects on family functioning. Journal of Clinical Child Psychology, 49, 245-250. Foote, R., Eyberg, S., & Schuhmann, E. (1998). Parent-child interaction approaches to the treatment of child conduct problems. In T. Ollendick & R. Prinz (Eds.), Advances in clinical child psychology (pp. 125-151 ). New York: Plenum. Funderburk, B., Eyberg, S. M., Newcomb, K., McNeil, C., Hembree-Kigin, T., & Capage, L. (1998). Parent-child interaction therapy with behavior problem children: Maintenance of treatment effects in the school setting. Child and Family Behavior Therapy, 20, 17-38. Hembree-Kigin, T., & McNeil, C. M. (1995). Parent-Childbzteraction Therapy. New York: Plenum. Johnson, S. M., & Christensen, A. (1975). Multiple criteria follow-up of behavior modification with families. Journal of Abnormal Child Psychology, 3, 135-154.

30

BRESTAN ET AL.

Kopec-Schrader, E. M., Rey, J. M., Plapp, J. M., & Beumont, R J. V. (1994). Parental evaluation of treatment in an adolescent service. Journal of Adolescence, 17, 239-249. Larsen, D. L., Attkisson, C., Hargreaves, W. A., & Nguyen, T. D. (1979). Assessment of client/ patient satisfaction: Development of a general scale. Evaluation and Program Planning. 2, 197-207. Lebow, J. (1982). Consumer satisfaction with mental health treatment. Psychological Bulletin.

91,244-259.
Lebow, J. (1983). Client satisfaction with mental health treatment: Methodological considerations in assessment. Evaluation Review, 7, 729-752. McMahon, R. J., & Forehand, R. L. (1983). Consumer satisfaction in behavioral treatment of children: Types, issues, and recommendations. Behavior Therapy, 14, 209-225. McNeil, C. B., Eyberg, S. M., Eisenstadt, T. H., Newcomb, K., & Funderburk, B. W. (1991). Parent-child interaction therapy with behavior problem children: Generalization of treatment effects to the school setting. Journal of Clinical Child Psychology, 20, 140-151. Perreault, M., Leichner, E, Sabourin, S., & Grendreau, E (1992). Psychometric properties of a French-Canadian version of a client satisfaction questionnaire for psychiatric inpatients. Canadian Journal of Ps~'chiato', 37, 221-227. Plante, T. G., Couchman, C. E., & Diaz, A. R. (1995). Measuring treatment outcome and client satisfaction among children and families. The Journal of Mental Health Administration, 22, 261-267. Prinz, R. J., & Miller, G. E. (1994). Family based treatment for childhood antisocial behavior: Experimental influences on dropout and engagement. Journal of Consulting and Clinical Psychology, 62, 645-650. Schuhmann, E., Foote, R., Eyberg, S. M., Boggs, S. R., & Algina, J. (1998). Parent-child interaction therapy: Interim report of a randomized trial with short-term maintenance. Journal of Clbtical Child Psychology; 27, 34-45. Seligman, M. E. E (19951. The effectiveness of psychotherapy: The Consumer Reports study. American Psychologist, 50, 965-974. Sorenson, J., Kantor, L., Margolis, R., & Galano, J. (1979). The extent, nature, and utility of evaluating consumer satisfaction in community mental health centers. American Journal of Community Psychology, 7, 329-337. Taylor, T. K., Schmidt, E. Pepler, D., & Hodgins, C. (1998). Comparison of eclectic treatment with Webster-Stratton's Parents and Children Series in a children's mental health center: A controlled trial. Behavior Therapy; 29, 221-240. Webster-Stratton, C. (1984). Randomized trial of two parent-training programs for families with conduct disordered children. Journal of Consulting and Clinical Psychology, 49. 633-645. Wehster-Stratton, C., & Eyherg, S. M. (1982). Child temperament: Relationships with child behavior problems and parent-child interactions. Journal of Clinical Child Psychology, 11, 123-129. RECEIVED: February 23, 1998 ACCEPTED: July 20, 1998