Vous êtes sur la page 1sur 7

Journal of Affective Disorders 212 (2017) 110–116

Contents lists available at ScienceDirect

Journal of Affective Disorders


journal homepage: www.elsevier.com/locate/jad

Research paper

Attachment, dysfunctional attitudes, self-esteem, and association to MARK


depressive symptoms in patients with mood disorders

Kristina Fuhra, , Ivanina Reitenbachb, Jan Kraemera, Martin Hautzingera, Thomas D. Meyerc
a
Clinical and Developmental Psychology, University of Tübingen, Tübingen, Germany
b
Euro-FH University of Applied Sciences, Hamburg, Germany
c
Department of Psychiatry and Behavioral Sciences, University of Texas, Houston, USA

A R T I C L E I N F O A BS T RAC T

Keywords: Background: Cognitive factors might be the link between early attachment experiences and later depression.
Attachment style Similar cognitive vulnerability factors are discussed as relevant for both unipolar and bipolar disorders. The
Cognitive vulnerability goals of the study were to test if there are any differences concerning attachment style and cognitive factors
Depression between remitted unipolar and bipolar patients compared to controls, and to test if the association between
Bipolar disorder
attachment style and depressive symptoms is mediated by cognitive factors.
Self-esteem
Methods: A path model was tested in 182 participants (61 with remitted unipolar and 61 with remitted bipolar
Dysfunctional attitudes
disorder, and 60 healthy subjects) in which adult attachment insecurity was hypothesized to affect
subsyndromal depressive symptoms through the partial mediation of dysfunctional attitudes and self-esteem.
Results: No differences between patients with remitted unipolar and bipolar disorders concerning attachment
style, dysfunctional attitudes, self-esteem, and subsyndromal depressive symptoms were found, but both groups
reported a more dysfunctional pattern than healthy controls. The path models confirmed that the relationship
between attachment style and depressive symptoms was mediated by the cognitive variables ‘dysfunctional
attitudes’ and ‘self-esteem’.
Limitations: With the cross-sectional nature of the study, results cannot explain causal development over time.
Conclusions: The results emphasize the relevance of a more elaborate understanding of cognitive and
interpersonal factors in mood disorders. It is important to address cognitive biases and interpersonal
experiences in treatment of mood disorders.

1. Introduction The importance of interpersonal factors for the occurrence and the
course of affective disorders have consequently been emphasized in
In the last years, cognitive factors have been discussed as one of the research (e.g. Alloy et al., 2006a; Brown and Harris, 1978; Gotlib and
major mechanisms underlying the development and maintenance of Hammen, 1992). Studies have provided evidence for the role of
mood disorders (see for an overview Alloy et al., 2006b). There is a attachment with respect to unipolar depression, with insecure attach-
large amount of empirical evidence for the validity of cognitive factors ment and the experience of an emotionally unavailable parent being
and their role for depression (Abramson et al., 1989; Alloy et al., 2000; associated with a risk for psychosocial problems (e.g. Cummings et al.,
Beck, 1976). Cognitive vulnerability factors such as dysfunctional 1990) or with a higher risk for depression (e.g. Roelofs et al., 2011;
attitudes or self-esteem are not just relevant for unipolar depression, Whisman and McGarvey, 1995). The evidence for the role of the
but also for bipolar disorders (Alloy et al., 2006b; Blairy et al., 2004; attachment style for bipolar disorders is ambiguous. Some studies
Lam et al., 2004; Scott and Pope, 2003; Scott et al., 2000). Considering found no link between attachment style and bipolar disorders (Joyce,
the origins of cognitive vulnerability, most theories focus on negative 1984), (Gilbert et al., 2007). Other studies provided evidence for
experiences in early childhood (Beck, 1967; Ingram, 2003). Within this insecure attachment in bipolar disorders, with bipolar patients in
context, attachment style as based on the early relationships with a remitted, manic, or depressive episodes showing higher anxious
caregiver is discussed as a potential causal risk factor for developing attachments compared to healthy controls (Kökcü and Kesebir, 2010;
psychopathology and can be seen as an interpersonal vulnerability Morriss et al., 2009). Researchers were able to show associations
factor (Bowlby, 1980, 1988). between family environments and the severity of bipolar disorder (for


Corresponding author.
E-mail address: kristina.fuhr@uni-tuebingen.de (K. Fuhr).

http://dx.doi.org/10.1016/j.jad.2017.01.021
Received 25 October 2016; Received in revised form 16 January 2017; Accepted 19 January 2017
Available online 21 January 2017
0165-0327/ © 2017 Elsevier B.V. All rights reserved.
K. Fuhr et al. Journal of Affective Disorders 212 (2017) 110–116

an overview see Alloy et al., 2006a). High ‘Expressed Emotions’ Depression disorder, and 60 individuals without any lifetime history
displayed by relatives, i.e. expressing critical, hostile attitudes and of a mood disorder. The average age was M=42.31 (SD=13.14). The
showing overinvolvement, predict relapse in several mental health sample consisted of 82 men (45%) and 100 women (55%). Most of the
problems including mood disorders (Butzlaff and Hooley, 1998; participants graduated from high school (n=134, 74%). Regarding age,
Miklowitz et al., 1995, 1988). sex, and level of education, no significant differences between the three
Cognitive factors are discussed as being the potential link between groups were found, age, F(2, 181)=2.55, p=.08; sex, Χ 2(2)=1.29,
negative experiences in early childhood and depressive symptoms later p=.53; level of education, Χ 2(2)=1.40, p=.50.
in life (Morley and Moran, 2011). Results from different studies Unipolar individuals had experienced about 11 (M=10.54,
suggest a path model in which dysfunctional attitudes and low self- SD=21.72) depressive episodes. For patients with BD the respective
esteem mediate the relation between attachment style or other inter- numbers were 13 for depressive (M=13.27, SD =19.29) and 10 for
personal factors and depressive symptoms (e.g. Alloy et al., 2006a; manic (M=10.31, SD=19.18) and 16 hypomanic episodes (M=15.63,
Hankin et al., 2005; Reinecke and Rogers, 2001; Roberts et al., 1996; SD=33.81). In the control group, 13 participants had a history of a
Roelofs et al., 2011). The role of cognitive factors such as self-esteem, mental health problem (eight had a lifetime diagnosis of alcohol abuse,
attributions, or negative schemata as a link between attachment style two were diagnosed with a specific phobia, two had a lifetime history of
and depressive symptoms was also confirmed for adolescents (Kamkar bulimia nervosa, and one a lifetime history of social phobia).
et al., 2012; Roelofs et al., 2011). In summary, a theoretical model
postulating a link between early attachment and later depression 2.2. Materials
influenced by cognitive factors can be seen as empirically supported
by the literature (Morley and Moran, 2011). However, the mediation 2.2.1. Structured clinical interview for DSM-IV (SCID-I; First et al.,
model was not tested yet in a clinical sample with remitted patients 1995; German version: Wittchen et al., 1997)
experiencing unipolar and bipolar disorders. The SCID-I was used to assess the lifetime history of any mental
disorders in all participants. Inclusion and exclusion criteria for the
1.1. Aims of the study three different groups were tested. In the current study ‘remission’ was
defined as a minimum of 8 weeks without any episode (DSM-IV,
The goal of the present study was twofold: (1) we wanted to test if American Psychiatric Association, 1994; Frank et al., 1991). Cohen's
there are differences between patients with remitted unipolar and Kappas for a diagnosis of a mood disorder was κ=1.00 and for
bipolar disorder compared to controls in adult attachment style and diagnostic criteria κ=.92.
cognitive factors; (2) we examined if the cognitive factors - i.e.
dysfunctional attitudes and self-esteem - also mediate the association 2.2.2. Young mania rating scale (YMRS, Young et al., 1978)
between attachment style and depressive symptoms in these patients. The YMRS is a clinician-administered rating of current hypomania
symptoms with 11 items for the last week. The cut-off for remission
2. Methods was defined at 5 points (Bauer et al., 1991; Masand et al., 2008). The
Intra-Class-Correlation for YMRS in our study was excellent (ICC
2.1. Participants =.91).

Individuals with history of unipolar or bipolar depression were 2.2.3. Hamilton rating scale for depression (HRSD, Hamilton, 1960)
recruited from the psychology clinic at the university and from self- With the clinician-administered rating of the HRSD consisting of
help support groups. Healthy control subjects were recruited by a the short version with 17 items current depressive symptoms for the
newspaper article introducing the study and via an email to all the last week were administered. The cut-off for remission was defined at 7
members of the university. All participants were interviewed with the points (Kennedy et al., 2006; Lojko and Rybakowski, 2007; Ramel
Structured Clinical Interview for DSM-IV (SCID-I; First et al., 1995). et al., 2007; Zimmerman et al., 2006). The Intra-Class-Correlation for
Any individual with a current substance dependency (in the last 12 HRSD in our study was good (ICC=.83).
months) or a history of psychotic disorders was excluded. Furthermore
as well patients with a current mood episode (defined as within the 2.2.4. Beck depression inventory II (BDI-II; Beck et al., 1996;
previous eight weeks) were excluded from the study. Participants German version: Hautzinger et al., 2006)
signed an informed consent form and agreed to the interview being The BDI-II assesses severity of depressive symptoms as a self-rating
recorded. Presence and severity of current symptoms were assessed scale with 21 items for the last two weeks. In the present study, internal
with the Young Mania Rating Scale (YMRS, Young et al., 1978) and the consistency was .84. The BDI-II was used to measure residual
Hamilton Rating Scale for Depression (HRSD, Hamilton, 1960). depression with a cut-off at 20 points. The BDI-II score of the second
Patients with scores of YMRS > 5 and/or HRSD > 7 and/or Beck session (without any cutoff) was relevant for the outcome of the
Depression Inventory II (BDI-II, Beck et al., 1996) scores > 20 were mediation model.
excluded to ascertain current remission. Structured interviews were
completed by clinical psychologists who had been trained to ensure 2.2.5. Relationship scales questionnaire (RSQ; Griffin and
reliable assessment. Bartholomew, 1994; German version: Mestel, 1994)
In total, 318 people were interested to participate in the study. The RSQ contains 30 items, answered on a 5-point Likert scale. It
Ninety-four people, however, cancelled their participation for personal measures adult attachment style in close relationships with the four
reasons (e.g. time constraints), and 14 participants were excluded scales secure, dismissive, fearful, and preoccupied. The RSQ is based
because they fulfilled criteria for a current mood episode, ten people on the theoretical approach of Bartholomew (Bartholomew, 1990;
had a non-affective psychotic disorder, and eight people showed Bartholomew and Horowitz, 1991) with the two dimensions ‘internal
symptom scores in YMRS or HRSD above the defined cutoffs. Ten model of self’ and ‘internal model of others’. A factor analysis of the
were excluded because their scores were above the predefined cutoffs of German translation, revealed four factors: ‘Closeness’, ‘Separation’,
self-reported depressive symptoms in BDI-II. ‘Autonomy’, and ‘Confidence’ (compare Steffanowski et al., 2001). High
Of those 192 participants who took part in the study, the final scores in fear of closeness (FCloseness), fear of separation
sample consisted of 182 participants: 61 patients met DSM-IV (FSeparation), and lack of confidence in other people (FConfidence),
(American Psychiatric Association, 1994) lifetime criteria for Bipolar as well as very low or high scores concerning wish of autonomy and
I disorder, 61 patients met DSM-IV lifetime criteria for Major independency (Autonomy) are therefore indicating a more insecure

111
K. Fuhr et al. Journal of Affective Disorders 212 (2017) 110–116

attachment style. Internal consistencies of the four German scales in Table 2


the present study were good (Cronbach's alphas between .77 and .85). Means and standard deviations of variables.

Unipolar Bipolar Controls


2.2.6. Dysfunctional attitudes scale (DAS; Weissman, 1979; German (n=61) (n=61) (n=60)
version: Hautzinger et al., 2005) Variables M (SD) M (SD) M (SD)
The DAS measures a person's maladaptive beliefs with 40 items on
RSQ-FSeparation 2.61 (.73) 2.68 (.73) 2.42 (.50)
a 7-point Likert scale. It is based on the cognitive triad for depression RSQ-FCloseness 2.42 (.89) 2.19 (.68) 1.95 (.61)
by Beck and assesses cognitive vulnerability. Internal consistency of the RSQ-FConfidence 2.38 (.74) 2.25 (.74) 1.80 (.55)
present study was high (Cronbach's alpha=.93). RSQ-Autonomy 4.10 (.70) 3.93 (.87) 3.72 (.74)
DAS 116.84 (35.86) 115.03 (30.73) 100.42 (21.58)
RSES 31.56 (5.18) 31.61 (4.31) 34.72 (3.46)
2.2.7. Rosenberg self-esteem scale (RSES; Rosenberg, 1965; German BDI-II 6.38 (4.68) 5.39 (5.36) 2.72 (3.30)
version: Ferring and Filipp, 1996) YMRS 1.30 (1.27) 1.20 (1.55) .87 (1.35)
The RSES is a self-report measure of global self-esteem. It consists HRSD 3.54 (2.17) 2.33 (2.02) 1.53 (1.75)

of 10 items answered on a 4-point Likert scale. Internal consistency


Notes. M=mean; SD=standard deviation; n=sample size; RSQ=Relationship Scales
was .86. It has been internationally used in a wide range of studies Questionnaire; DAS=Dysfunctional Attitude Scale; RSES=Rosenberg Self-Esteem Scale;
dealing with affective disorders (e.g. Blairy et al., 2004; Munford, 1994; BDI-II=Beck Depression Inventory II; YMRS=Young Mania Rating Scale;
Schmitz et al., 2003). HRSD=Hamilton Rating Scale for Depression.

2.3. Procedure
reduced (Analysis 4). Alpha levels of .05 were used for all statistical
Participants were individually tested. In the first session, partici- tests. All p-values will be reported exactly.
pants were interviewed using the SCID-I to confirm the diagnosis and
current symptom status in addition with the YMRS, the HRSD, and
BDI-II. The second session took place within the next 7 days, and levels 3. Results
of residual depressive symptoms were assessed using the self-rating
BDI-II. Afterwards, the participants were given the different self-report 3.1. Differences between unipolar, bipolar, and control group
instruments. It was ensured that the experimenter at the second
session was kept blind with regard to diagnostic status. Participants Means and standard deviations are displayed in Table 2. Not
received an allowance (40€) for their participation. surprisingly, significant differences between the three groups were
found in BDI-II scores, F(2, 179)=10.52, p < .001, partial η2=.11. The
2.4. Statistical analysis unipolar and the bipolar groups had higher BDI-II scores than the
control group (p < .001 and p=.004), but did not differ from each other
This cross-sectional study used a between-subjects design. One-way (p=.70).
ANOVAS were calculated to analyse differences between groups (i.e. Looking at the RSQ no differences were found between the groups
unipolar, bipolar, control group). Bonferroni post-hoc tests followed in with respect to the scale ‘FSeparation’ and, F(2, 178)=2.47, p=.09,
case a significant effect has been detected. There were four dependent η2=.03. The three groups differed in fear of ‘FCloseness’ F(2, 178)
variables: attachment scales (RSQ), dysfunctional attitudes (DAS), self- =6.23, p=.002, partial η2=.07, with unipolar patients showing higher
esteem (RSES), and depressive symptoms (BDI-II). fear of closeness compared to controls (unipolar vs. controls: p=.002;
For testing the regression model, Pearson's correlations and point- bipolar vs. controls: p=.22; unipolar vs. bipolar: p=.25). On the scale
biserial correlations of the dependent variables were calculated first. ‘FConfidence’ also a significant effect was observed, F(2, 178)=11.80, p
Those consisted of the four scales of the RSQ, the DAS, the RSES, and < .001, partial η2=.12. Post-hoc tests indicated that this was mainly due
the BDI-II. The method followed previous studies (compare Roberts to the unipolar and the bipolar groups reporting less confidence than
et al., 1996). Thus, four hierarchical multiple regression analyses were the control group (unipolar vs. controls: p < .001; bipolar vs. controls:
tested for each model. For an overview, see Table 1. Attachment p=.001; unipolar vs. bipolar: p=.96). For ‘Autonomy’ significant
insecurity was hypothesized to be associated with higher levels of differences between the three groups were observed, F(2, 178)=3.73,
depressive symptoms (Analysis 1). Specifically, this association was p=.03, partial η2=.04. Post hoc tests revealed that this was mainly due
expected to be mediated by cognitive factors; therefore dysfunctional to differences between unipolar and control group (p=.02; bipolar vs.
attitudes should predict lower self-esteem (Analysis 2) which, in turn, controls: p=.38, unipolar vs. bipolar: p=.70). As a first conclusion,
should predict more depressive symptoms (Analysis 3) so that the individuals with a remitted unipolar disorder showed a more insecure
direct path between attachment style and depressive symptoms is attachment style than healthy controls as indicated by higher scores in
FCloseness and higher scores in Autonomy. Patients with BD only
Table 1 differed from the latter in ‘FConfidence’.
Mediation model based on the following four hierarchical regression analyses.
We also found differences in cognitive vulnerability measured by
Analysis 1 Analysis 2 Analysis 3 Analysis 4
the DAS, F(2, 179)=5.43, p=.005, partial η2 =.06. The unipolar and the
bipolar groups showed higher scores compared to the control group
Step 1 Predictor “Mood “Mood “Mood “Mood (unipolar vs. controls: p=.009; bipolar vs. controls: p=.02), but did not
disorder disorder disorder disorder differ from each other (p > .999). Furthermore, a significant difference
lifetime” lifetime” lifetime” lifetime”
Criterion BDI-II DAS RSES BDI-II
was found between groups in the RSES, F(2, 179) =10.32, p < .001,
partial η2=.10. Post-hoc tests revealed that both patient groups differed
Step 2 Predictors RSQ RSQ RSQ RSQ from the control group by reporting lower self-esteem (unipolar vs.
DAS DAS controls: p < .001; bipolar vs. controls: p < .001; unipolar vs. bipolar: p
RSES
> .999).
Criterion BDI-II DAS RSES BDI-II
Since no significant differences between unipolar and bipolar
Notes. RSQ=Relationship Scales Questionnaire; DAS=Dysfunctional Attitude Scale; subjects were found in core measures, we were able to combine the
RSES=Rosenberg Self-Esteem Scale; BDI-II=Beck Depression Inventory II. groups for all further analyses.

112
K. Fuhr et al. Journal of Affective Disorders 212 (2017) 110–116

Table 3 t(178)=3.45, β=.24, p=.001 and t(178)=4.65, β=.32, p < .001, with this
Correlation matrix of variables. model explaining 19.4% of variance of the BDI-II scores, F(2, 178)
=21.43, p < .001, R=.44, R2 =.19, ΔR2=.10. Conducting the same
Variable (n=182) 1 2 3 4 5 6 7
analysis with the DAS as the dependent variable in the second
1. Mood disorder – regression analysis, a lifetime history of a mood disorder significantly
lifetime predicted the DAS scores, t(178)=2.28, β=.16, p=.02, and the RSQ
2. RSQ- .16*
scale ‘FCloseness’ also predicted the DAS scores, t(178)=5.58, β=.38, p
FSeparation
3. RSQ-FCloseness .22** −.02 < .001. This model explained 19.9% of variance of the DAS, F(2, 178)
4. RSQ- .34*** .43*** .54*** =22.13, p < .001, R=.45, R2=.20, ΔR2 =.14. In the third analysis, a
FConfidence lifetime history of a mood disorder contributed to the prediction of self-
5. RSQ-Autonomy .18* −.03 .26*** .16* esteem, t(177)=−2.60, β=−.15, p=.01, as well as the DAS scores t(177)
6. DAS .24** .44*** .42*** .59*** .09
=−8.16, β=−.50, p < .001, and the RSQ scale ‘FCloseness’, t(177)
7. RSES −.32*** −.33*** −.47*** −.51*** −.10 −.63***
8. BDI-II .31*** .19* .37*** .37*** .09 .32*** −.46*** =−3.82, β=−.23, p < .001. This model explained 47.2% of variance of
the RSES, F(3, 177)=52.73, p < .001, R =.69, R2=.47, ΔR2 =.37. The
Notes. n=sample size; RSQ=Relationship Scales Questionnaire; DAS=Dysfunctional fourth model consisting of the predictor variable ‘lifetime history of an
Attitude Scale; RSES=Rosenberg Self-Esteem Scale; BDI-II=Beck Depression Inventory mood disorder’ in the first step and the RSQ scale ‘FCloseness’, the
II.
* DAS, and the RSES as predictors in the second step with the BDI-II as
p < .05.
**
p < .01. dependent variable, was also significant, F(4, 176) =16.23, p < .001,
***
p < .001. R=.52, R2 =.27, ΔR2 =.17. A total of 27.0% of variance could be
explained. A lifetime history of a mood disorder, the RSQ scale
3.2. Testing the path model ‘FCloseness’, and the self-esteem predicted the BDI-II scores, t(176)
=2.42, β=.17, p=.02; t(176)=2.46, β=.18, p=.02; t(176)=−3.56,
Correlations in the overall sample were calculated using the β=−.32, p < .001. The DAS scores did not predict depressive symptoms,
relevant instruments for the path model (Table 3). The dichotomous t(176)=.13, β=.01, p=.90. The direct association between ‘FCloseness’
dummy variable ‘mood disorder lifetime’ was defined to control any and BDI-II was reduced, when entering the potential mediating factors.
potential confounding effect of the diagnosis of a mood disorder on the The mediators did not totally explain the association, given that the
path model. The variable then was included in the correlation and scale ‘FCloseness’ still had a direct influence on the severity of
regression analyses. To test the path model, the RSQ scales had to symptoms. The first path model consisting of the four regression
correlate with the BDI-II, the DAS and the RSES, and those had to analyses is displayed in Fig. 1.
correlate with each other. The RSQ scale ‘Autonomy’ which did
correlate neither with the DAS nor with the RSES, could therefore
3.4. Mediation model concerning ‘FConfidence’
not be used in the regression analyses (see Table 3).
Then, four hierarchical regression analyses were calculated to test
Likewise, four hierarchical regression analyses were calculated with
the path model with the three scales ‘FCloseness’, ‘FSeparation’, and
the RSQ scale ‘FConfidence’. In the first regression analysis, the
‘FConfidence’. The strong correlations of ‘FSeparation’ and ‘FCloseness’
variable ‘mood disorder lifetime’ and the RSQ scale ‘FConfidence’ both
with ‘FConfidence’ (r =.43, p < .001 and r=.54, p < .001, Table 3)
predicted depressive symptoms, t(178)=2.91, β=.21, p=.004, and
caused a problem of multicollinearity. Thus, the path models were
t(178)=4.11, β=.30, p < .001. A variance of 17.5% could be explained
tested for each scale separately. As a first step in every analysis, the
in this model, F(2, 178) =18.82, p < .001, R=.42, R2 =.18, ΔR2=.08.
influence of the dummy variable ‘mood disorder lifetime’ was con-
Conducting the analysis with the DAS as the dependent variable in the
sidered. As a second step, the particular predictors were included.
second regression analysis, a lifetime history of a mood disorder did
Analysis 1 tested the direct associations between the particular RSQ-
not predict the DAS scores, t(178)=.81, β=.05, p=.42, but the RSQ scale
scale and the subsyndromal depressive symptoms. Then, separate
‘FConfidence’ predicted the DAS scores, t(178)=8.86, β=.57, p < .001.
analyses were conducted for each predictor and criterion. Analysis 2
This model explained 34.7% of variance of the DAS, F(2, 178)=47.27,
involved again as a first step the dummy variable ‘mood disorder
p < .001, R=.59, R2 =.35, ΔR2=.29. In the third analysis, a lifetime
lifetime’ and as a second step the cognitive factor DAS. Analysis 3
history of a mood disorder a contributed to the prediction of self-
involved the RSES. The final path model consisting of analysis 2, 3, and
esteem, t(177)=−2.41, β=−.14, p=.02, as well as the DAS scores, t(177)
4 will be shown in separate figures for each of the three RSQ-scales. For
=−7.19, β=−.50, p < .001, and the RSQ scale ‘FConfidence’, t(177)
an overview of the four hierarchical regression analyses see Table 1.
=−2.37, β=−.17, p=.02. This model explained 44.6% of variance of the
RSES, F(3, 177) =47.49, p < .001, R=.67, R2=.45, ΔR2 =.34. In the
3.3. Mediation model concerning ‘FCloseness’ fourth and final regression analysis, the variables ‘lifetime history of a
mood disorder’ and the RSES predicted the BDI-II scores, t(176)
Both predictors, a lifetime history of a mood disorder as well as the =2.17, β=.15, p=.03, and t(176)=−4.00, β=−.35, p < .001. The RSQ
RSQ scale ‘FCloseness’ significantly contributed to the prediction of scale ‘FConfidence’ and the DAS scores did not predict depressive
subsyndromal depressive symptoms in the first regression analysis, symptoms, t(176) =1.78, β=.15, p=.08, and t(176)=−.19, β=−.02,

Fig. 1. Path model for the RSQ subscale FCloseness based on the four hierarchical regression analyses. Beta weights and their significances of the fourth regression analysis are
displayed. Beta weights in brackets represent the results from the first regression analysis. RSQ=Relationship Scales Questionnaire; DAS=Dysfunctional Attitude Scale;
RSES=Rosenberg Self-Esteem Scale; BDI-II=Beck Depression Inventory II. * p < .05; ** p < .01; *** p < .001.

113
K. Fuhr et al. Journal of Affective Disorders 212 (2017) 110–116

Fig. 2. Path model for the RSQ subscale FConfidence based on the four hierarchical regression analyses. Beta weights and their significances of the fourth regression analysis are
displayed. Beta weights in brackets represent the results from the first regression analysis. RSQ=Relationship Scales Questionnaire; DAS=Dysfunctional Attitude Scale;
RSES=Rosenberg Self-Esteem Scale; BDI-II=Beck Depression Inventory II. * p < .05; ** p < .01; *** p < .001.

p=.85. The model achieved significance, F(4, 176)=15.29, p < .001, significance level in the first regression analysis. The third path model
R=.51, R2 =.26, ΔR2=.16. A total of 25.8% of variance could be consisting of the four regression analyses is displayed in Fig. 3.
explained. The direct association between ‘FConfidence’ and BDI-II
was fully mediated through higher dysfunctional attitudes and lower 4. Discussion
self-esteem ‘FConfidence’ had no significant direct influence on the
BDI-II anymore. The second path model consisting of the four We found that remitted patients with a history of mood disorders
regression analyses is displayed in Fig. 2. were more likely to report higher depressive symptoms, higher scores
in dysfunctional attitudes, and lower self-esteem compared to healthy
3.5. Mediation model concerning ‘FSeparation’ controls, but did not differ from each other. The results are comparable
to other findings whereby similar cognitive vulnerability theories are
With the RSQ-scale ‘FSeparation’ we also calculated the four relevant not only for depression but also for bipolar disorders (Alloy
hierarchical regression analyses. In the first analysis, we found that et al., 2006b; Scott and Pope, 2003; Scott et al., 2000). Concerning
‘FSeparation’ did not reach the significance level in predicting the attachment style, we could also replicate former studies with a stronger
subsyndromal depressive symptoms in the first regression analysis, attachment insecurity in form of higher fear of closeness (FCloseness)
t(178) =1.96, β=.14, p=.05, but having had a history of a mood and wish of autonomy and independency (Autonomy), and lower
disorder lifetime significantly predicted the BDI-II scores, t(178) confidence in other people (FConfidence) as indicated by the RSQ
=4.03, β=.29, p < .001, with this model explaining 11.5% of variance subscales for unipolar patients compared to controls (Roelofs et al.,
of the BDI-II, F(2, 178)=11.59, p < .001, R=.34, R2 =.12, ΔR2 =.02. 2011; Whisman and McGarvey, 1995). Results are also in line with
Conducting the same analysis with the DAS as the dependent variable former studies reporting about a more anxious (Morriss et al., 2009) or
in the second regression analysis, a lifetime history of a mood disorder more avoidant attachment style in bipolar patients compared to
significantly predicted the DAS scores, t(178) =2.65, β=.18, p=.009, healthy people (Kökcü and Kesebir, 2010).
and the RSQ scale ‘FSeparation’ also predicted the DAS scores, t(178) As expected, we were able to confirm the path model in which
=6.11, β=.41, p < .001. This model explained 22.2% of variance of the attachment insecurity predicted higher levels of depressive symptoms,
DAS, F(2, 178) =25.37, p < .001, R =.47, R2=.22, ΔR2=.16. In the third and the association between attachment style and depressive symp-
analysis, a lifetime history of a mood disorder contributed to the toms was fully or partially mediated through higher dysfunctional
prediction of self-esteem, t(177)=−2.98, β=−.18, p=.003, as well as the attitudes and lower self-esteem. Therefore, we were able to replicate
DAS scores t(177) =−8.79, β=−.57, p < .001. The RSQ scale former studies (Hankin et al., 2005; Roberts et al., 1996) and also
‘FSeparation’ did not predict the RSES scores, t(177)=−.88, β=−.06, extent the existing evidence to clinical samples with not only unipolar
p=.38. This model explained 43.1% of variance of the RSES, F(3, 177) but also bipolar disorders. The significant influence of the history of a
=44.68, p < .001, R=.66, R2 =.43, ΔR2=.33. mood disorder on the path model reveals that the model was more
The fourth model with the predictor variables ‘lifetime history of a relevant for patients with former mood episodes. This is important,
mood disorder’ in the first step and the RSQ scale ‘FSeparation’, the since the more the negative cognitive style or the more depressive
DAS, and the RSES as predictors in the second step, and the BDI-II as symptoms the higher is their risk for further relapses (Alloy et al.,
dependent variable, was also significant, F(4, 176)=14.25, p < .001, 2006c; Judd et al., 1998, 2002). Negative interpersonal experiences
R=.50, R2 =.25, ΔR2 =.15. A total of 24.5% of variance could be early in life also significantly increase the risk of relapses and the
explained. A lifetime history of a mood disorder, and the self-esteem outcome of treatment (Nanni et al., 2012). To answer the question
predicted the BDI-II scores, t(176)=2.56, β=.18, p=.01, and t(176) when and how the risk for relapses increase researchers have focused
=−4.32, β=−.38, p < .001. The DAS scores and the RSQ scale on life events that are able to activate the negative experiences and lead
‘FSeparation’did not predict depressive symptoms, t(176)=.46, β=.04, to mood symptoms or relapses (Alloy et al., 1999; Hammen et al.,
p=.65, and t(176)=.22, β=.02, p=.83. The direct association between 1985; Ingram, 2001; Ingram and Luxton, 2005). Cognitive Behavioral
‘FSeparation’ and BDI-II was reduced, when entering the potential Therapy (CBT) is effective in preventing further relapses both in
mediating factors. The scale ‘FSeparation’ had no significant direct unipolar and bipolar disorders for example by providing strategies to
influence on the severity of symptoms, but also did not reach cope life events with and upcoming subsyndromal symptoms (Fava

Fig. 3. Path model for the RSQ subscale FSeparation based on the four hierarchical regression analyses. Beta weights and their significances of the fourth regression analysis are
displayed. Beta weights in brackets represent the results from the first regression analysis. RSQ=Relationship Scales Questionnaire; DAS=Dysfunctional Attitude Scale;
RSES=Rosenberg Self-Esteem Scale; BDI-II=Beck Depression Inventory II. * p < .05; ** p < .01; *** p < .001.

114
K. Fuhr et al. Journal of Affective Disorders 212 (2017) 110–116

et al., 1998; Miklowitz, 2008; Paykel et al., 2005; Scott et al., 2007). Alloy, L.B., Reilly-Harrington, N., Fresco, D.M., Whitehouse, W.G., Zechmeister, J.S.,
1999. Cognitive styles and life events in subsyndromal unipolar and bipolar
Moreover, the role of attachment style for mood disorders underlines disorders: stability and prospective prediction of depressive and hypomanic mood
the relevance of addressing early life experiences in psychotherapy and swings. J. Cogn. Psychother. 13, 21–40.
build a secure relationship between patient and therapist (Bowlby, Alloy, L.B., Abramson, L.Y., Smith, J.M., Gibb, B.E., Neeren, A.M., 2006a. Role of
parenting and maltreatment histories in unipolar and bipolar mood disorders:
1977). Interpersonal Psychotherapy (IPT) which focuses on the inter- mediation by cognitive vulnerability to depression. Clin. Child Fam. Psychol. Rev. 9,
personal aspects of depression should therefore be more efficacious 23–64.
compared to CBT in the treatment of depressive disorders in those Alloy, L.B., Abramson, L.Y., Whitehouse, W.G., Panzarella, C., Rose, D.T., Hogan, M.E.,
2006c. Prospective incidence of first onsets and recurrences of depression in
patients with more insecure attachment style (de Mello et al., 2005) individuals at high and low cognitive risk for depression. J. Abnorm. Psychol. 115,
even if other studies suggest that both treatments show similar effects 145–156.
(see for example Luty et al., 2007). Finally, patients suffering from Alloy, L.B., Abramson, L.Y., Hogan, M.E., Whitehouse, W.G., Rose, D.T., Robinson, M.S.,
Kim, R.S., Lapkin, J.B., 2000. The temple-wisconsin cognitive vulnerability to
chronic Major Depression with early trauma experiences benefit more
depression project: lifetime history of Axis I psychopathology in individuals at high
from a psychotherapy that focuses on interpersonal aspects (Nemeroff and low cognitive risk for depression. J. Abnorm. Psychol. 109, 403–418.
et al., 2005). American Psychiatric Association, 1994. Diagnostic and Statistical Manual of Mental
Disorders 4th ed. American Psychiatric Publishing, Inc., Arlington, VA, US.
Bartholomew, K., 1990. Avoidance of intimacy: an attachment perspective. J. Soc. Pers.
4.1. Limitations Relatsh. 7, 147–178.
Bartholomew, K., Horowitz, L.M., 1991. Attachment styles among young adults: a test of
The following limitations of the examination should be mentioned. a four-category model. J. Personal. Soc. Psychol. 61, 226–244.
Bartholomew, K., Moretti, M., 2002. The dynamics of measuring attachment. Attach.
Because of the cross-sectional nature of the present investigation, the Hum. Dev. 4, 162–165.
study does not allow drawing conclusions about the causality of the Bauer, M.S., Crits-Christoph, P., Ball, W.A., Dewees, E., 1991. Independent assessment
discovered relationships between attachment style, cognitive variables, of manic and depressive symptoms by self-rating: scale characteristics and
implications for the study of mania. Arch. Gen. Psychiatry 48, 807–812.
and the risk for depression. Long-term studies are needed to really Beck, A.T., 1967. Depression: Clinical, Experimental, and Theoretical Aspects. Harper
ascertain the association of experiences early in childhood to later and Row, New York.
depression. However, there are methodological difficulties when mea- Beck, A.T., 1976. Cognitive Therapy and the Emotional Disorders. International
Universities Press, Oxford England.
suring adult attachment. Most of the studies used measures that assess
Beck, A.T., Steer, R.A., Brown, G.K., 1996. Beck Depression Inventory II – Manual. The
adult attachment in samples of adults. Drawing conclusions about the Psychological Corporation, Dan Antonio.
attachment style that was developed in childhood based on the Blairy, S., Linotte, S., Souery, D., Papadimitriou, G.N., Dikeos, D., Lerer, B., Kaneva, R.,
Milanova, V., Serretti, A., Macciardi, F., Mendlewicz, J., 2004. Social adjustment and
interaction with an important caregiver is, thus, not applicable with
self-esteem of bipolar patients: a multicentric study. J. Affect. Disord. 79, 97–103.
adult attachment assessments. Information about the underlying Bowlby, J., 1977. The making and breaking of affectional bonds. II. Some principles of
mechanisms linking early childhood experiences to the cognitive psychotherapy. The fiftieth Maudsley lecture. Br. J. Psychiatry 130, 421–431.
processes in the presence cannot be obtained (see for example Bowlby, J., 1980. Attachment and Loss. Basic Books, New York, NY US.
Bowlby, J., 1988. A Secure Base: parent-child Attachment and Healthy Human
Morley and Moran, 2011). Many researchers have also questioned Development. Basic Books, New York, NY US.
the validity of self-reports and instead suggested to use the adult Brown, G.W., Harris, T., 1978. Social origins of depression: a reply. Psychol. Med. 8,
attachment interview as a gold standard (e.g. Bartholomew and 577–588.
Butzlaff, R.L., Hooley, J.M., 1998. Expressed emotion and psychiatric relapse. Arch. Gen.
Moretti, 2002). Another limitation is due to the sample size that was Psychiatry 55, 547–552.
still too small for mediation analyses even if most researchers were not Cummings, E.M., Cicchetti, D., Greenberg, M.T., 1990. Toward a transactional model of
able to provide sample sizes larger than 187 participants (see for relations between attachment and depression. In: Attachment in the Preschool
Years: Theory, Research, and Intervention. University of Chicago Press, Chicago, IL
example Fritz and MacKinnon, 2007). US, pp. 339–372
Nevertheless, our study underlines the importance of addressing Fava, G.A., Rafanelli, C., Grandi, S., Conti, S., Belluardo, P., 1998. Prevention of
cognitive biases and negative interpersonal experiences early in life in recurrent depression with cognitive behavioral therapy: preliminary findings. Arch.
Gen. Psychiatry 55, 816–820.
the treatment with unipolar and bipolar patients. It is essential to Ferring, D., Filipp, S.-H., 1996. Messung des Selbstwertgefühls: befunde zu Reliabilität,
provide more helpful emotion regulation strategies to deal with Validität und Stabilität der Rosenberg-Skala. Diagnostica 42, 284–292.
stressful life events to prevent further relapses of mood episodes. First, M.B., Spitzer, R.L., Gibbon, M., Williams, J.B., 1995. Structured Clinical Interview
for DSM-IV Axis I Disorders – Patient Edition (SCID- I/P, Version2.0). Biometrics
Research Department, New York State Psychiatric Institute, New York.
Declaration of interest Frank, E., Prien, R.F., Jarrett, R.B., Keller, M.B., Kupfer, D.J., Lavori, P.W., Rush, J.A.,
Weissman, M.M., 1991. Conceptualization and rationale for consensus definitions of
There were no conflicts of interest. terms in major depressive disorder: remission, recovery, relapse, and recurrence.
Arch. Gen. Psychiatry 48, 851–855.
Fritz, M.S., MacKinnon, D.P., 2007. Required sample size to detect the mediated effect.
Acknowledgments Psychol. Sci. 18, 233–239.
Gilbert, P., McEwan, K., Hay, J., Irons, C., Cheung, M., 2007. Social rank and attachment
in people with a bipolar disorder. Clin. Psychol. Psychother. 14, 48–53.
Funding: This research was supported in part by the Deutsche Gotlib, I.H., Hammen, C.L., 1992. Psychological Aspects of Depression: Toward a
Forschungsgemeinschaft (DFG Me 1681/13-1), awarded to Thomas D. Cognitive-Interpersonal Integration. John Wiley & Sons, Oxford England.
Meyer and Martin Hautzinger. Griffin, D.W., Bartholomew, K., 1994. The Metaphysics of Measurement: The Case of
Adult Attachment, Attachment Processes in Adulthood. Jessica Kingsley Publishers,
First results have been presented as a poster at the ‘6. London England, 17–52.
Workshopkongress für Klinische Psychologie und Psychotherapie Hamilton, M., 1960. A rating scale for depression. J. Neurol. Neurosurg. Psychiatry 23,
−27. Symposium der Fachgruppe Klinische Psychologie und 56–61.
Hammen, C., Marks, T., Mayol, A., DeMayo, R., 1985. Depressive self-schemas, life
Psychotherapie der Deutschen Gesellschaft für Psychologie’ in Zurich, stress, and vulnerability to depression. J. Abnorm. Psychol. 94, 308–319.
Switzerland, May, 21–23, 2009. Hankin, B.L., Kassel, J.D., Abela, J.R.Z., 2005. Adult attachment dimensions and
Ethical approval for the study was obtained from the German specificity of emotional distress symptoms: prospective investigations of cognitive
risk and interpersonal stress generation as mediating mechanisms. Personal. Soc.
Psychological Society (Deutsche Gesellschaft für Psychologie, DGPs).
Psychol. Bull. 31, 136–151.
Hautzinger, M., Joormann, J., Keller, F., 2005. Skala dysfunktionaler Einstellungen
References (DAS) – Testmanual. Hogrefe, Göttingen.
Hautzinger, M., Keller, F., Kühner, C., 2006. Das Beck Depressionsinventar II. Deutsche
Bearbeitung und Handbuch zum BDI II. Harcourt Test Services, Frankfurt a. M.
Abramson, L.Y., Metalsky, G.I., Alloy, L.B., 1989. Hopelessness depression: a theory- Ingram, R.E., 2001. Developing perspectives on the cognitive-developmental origins of
based subtype of depression. Psychol. Rev. 96, 358–372. depression: back is the future. Cogn. Ther. Res. 25, 497–504.
Alloy, L.B., Abramson, L.Y., Walshaw, P.D., Neeren, A.M., 2006b. Cognitive vulnerability Ingram, R.E., 2003. Origins of cognitive vulnerability to depression. Cogn. Ther. Res. 27,
to unipolar and bipolar mood disorders. J. Soc. Clin. Psychol. 25, 726–754. 77–88.

115
K. Fuhr et al. Journal of Affective Disorders 212 (2017) 110–116

Ingram, R.E., Luxton, D.D., 2005. Vulnerability-stress models. In: Hankin, B.L., Abela, to depression in Blacks. J. Black Psychol. 20, 157–174.
J.R.Z. (Eds.), Development of Psychopathology: A Vulnerability-stress Perspective.. Nanni, V., Uher, R., Danese, A., 2012. Childhood maltreatment predicts unfavorable
Sage Publications, Inc., Thousand Oaks, CA US, 32–46. course of illness and treatment outcome in depression: a meta-analysis. Am. J.
Joyce, P.R., 1984. Parental bonding in bipolar affective disorder. J. Affect. Disord. 7, Psychiatry 169, 141–151.
319–324. Nemeroff, C.B., Heim, C.M., Thase, M.E., Klein, D.N., Rush, A.J., Schatzberg, A.F.,
Judd, L.L., Schettler, P.J., Akiskal, H.S., 2002. The prevalence, clinical relevance, and Ninan, P.T., James P. McCullough, J., Weiss, P.M., Dunner, D.L., Rothbaum, B.O.,
public health significance of subthreshold depressions. Psychiatr. Clin. North Am. Kornstein, S., Keitner, G., Keller, M.B., 2005. Differential responses to
25, 685–698. psychotherapy versus pharmacotherapy in patients with chronic forms of major
Judd, L.L., Akiskal, H.S., Maser, J.D., Zeller, P.J., Endicott, J., Coryell, W., Paulus, M.P., depression and childhood trauma. FOCUS 3, 131–135.
Kunovac, J.L., Leon, A.C., Mueller, T.I., Rice, J.A., Keller, M.B., 1998. Major Paykel, E.S., Scott, J., Cornwall, P.L., Abbott, R., Crane, C., Pope, M., Johnson, A.L.,
depressive disorder: a prospective study of residual subthreshold depressive 2005. Duration of relapse prevention after cognitive therapy in residual depression:
symptoms as predictor of rapid relapse. J. Affect. Disord. 50, 97–108. follow-up of controlled trial. Psychol. Med. 35, 59–68.
Kamkar, K., Doyle, A.-B., Markiewicz, D., 2012. Insecure attachment to parents and Ramel, W., Goldin, P.R., Eyler, L.T., Brown, G.G., Gotlib, I.H., McQuaid, J.R., 2007.
depressive symptoms in early adolescence: mediating roles of attributions and self- Amygdala reactivity and mood-congruent memory in individuals at risk for
esteem. Int. J. Psychol. Stud. 4, 3. depressive relapse. Biol. Psychiatry 61, 231–239.
Kennedy, S.H., Fulton, K.A., Bagby, R.M., Greene, A.L., Cohen, N.L., Rafi-Tari, S., 2006. Reinecke, M.A., Rogers, G.M., 2001. Dysfunctional attitudes and attachment style among
Sexual function during bupropion or paroxetine treatment of major depressive clinically depressed adults. Behav. Cogn. Psychother. 29, 129–141.
disorder. Can. J. Psychiatry/ Rev. Can. Psychiatr. 51, 234–242. Roberts, J.E., Gotlib, I.H., Kassel, J.D., 1996. Adult attachment security and symptoms of
Kökcü, F., Kesebir, S., 2010. The relationship between attachment style, and depression: the mediating roles of dysfunctional attitudes and low self-esteem. J.
temperament, personality and bipolar symptoms: a controlled study on bipolar Personal. Soc. Psychol. 70, 310–320.
patients and their children. Turk. Psikiyatr. Derg. 21, 309. Roelofs, J., Lee, C., Ruijten, T., Lobbestael, J., 2011. The mediating role of early
Lam, D., Wright, K., Smith, N., 2004. Dysfunctional assumptions in bipolar disorder. J. maladaptive schemas in the relation between quality of attachment relationships and
Affect Disord. 79, 193–199. symptoms of depression in adolescents. Behav. Cogn. Psychother. 39, 471–479.
Lojko, D., Rybakowski, J.K., 2007. L-thyroxine augmentation of serotonergic Rosenberg, M., 1965. Society and the Adolescent Self-image. Princeton University Press,
antidepressants in female patients with refractory depression. J. Affect. Disord. 103, Princeton, New Jersey.
253–256. Schmitz, N., Kugler, J., Rollnik, J., 2003. On the relation between neuroticism, self-
Luty, S.E., Carter, J.D., McKenzie, J.M., Rae, A.M., Frampton, C.M.A., Mulder, R.T., esteem, and depression: results from the National Comorbidity Survey. Compr.
Joyce, P.R., 2007. Randomised controlled trial of interpersonal psychotherapy and Psychiat 44, 169–176.
cognitive–behavioural therapy for depression. Br. J. Psychiatry 190, 496–502. Scott, J., Pope, M., 2003. Cognitive styles in individuals with bipolar disorders. Psychol.
Masand, P.S., Eudicone, J., Pikalov, A., McQuade, R.D., Marcus, R.N., Vester-Blokland, Med. 33, 1081–1088.
E., Carlson, B.X., 2008. Criteria for defining symptomatic and sustained remission in Scott, J., Colom, F., Vieta, E., 2007. A meta-analysis of relapse rates with adjunctive
bipolar I disorder: a post-hoc analysis of a 26-week Aripiprazole study. psychological therapies compared to usual psychiatric treatment for bipolar
Psychopharmacol. Bull. 41, 12–17. disorders. Int. J. Neuropsychopharmacol. 10, 123–129.
de Mello, M.F., de Jesus Mari, J., Bacaltchuk, J., Verdeli, H., Neugebauer, R., 2005. A Scott, J., Stanton, B., Garland, A., Ferrier, I.N., 2000. Cognitive vulnerability in patients
systematic review of research findings on the efficacy of interpersonal therapy for with bipolar disorder. Psychol. Med. 30, 467–472.
depressive disorders. Eur. Arch. Psychiatry Clin. Neurosci. 255, 75–82. Steffanowski, A., Oppl, M., Meyerberg, J., Schmidt, J., Wittmann, W.W., Nübling, R.,
Mestel, R., 1994. Deutsche Übersetzung des RSQ. Klinik für Psychosomatische Medizin, 2001. Psychometrische Überprüfung einer deutschsprachigen Version des
Grönenbach, (Unveröffentlicht). Relationship Scales Questionnaire (RSQ). In: Bassler, B. (Ed.), Störungsspezifische
Miklowitz, D.J., 2008. Adjunctive psychotherapy for bipolar disorder: state of the Ansätze in der stationären Psychotherapie. Psychosozial Verlag, Giessen.
evidence. Am. J. Psychiatry 165, 1408–1419. Weissman, A.N., 1979. The dysfunctional attitude scale: a validation study, dissertation
Miklowitz, D.J., Goldstein, M.J., Nuechterlein, K.H., 1995. Verbal interactions in the abstracts international. ProQuest Inf. Learn., 1389–1390.
families of schizophrenic and bipolar affective patients. J. Abnorm. Psychol. 104, Whisman, M.A., McGarvey, A.L., 1995. Attachment, depressotypic cognitions, and
268–276. dysphoria. Cogn. Ther. Res. 19, 633–650.
Miklowitz, D.J., Goldstein, M.J., Nuechterlein, K.H., Snyder, K.S., 1988. Family factors Wittchen, H.-U., Zaudig, M., Fydrich, T., 1997. SKID. Strukturiertes Klinisches Interview
and the course of bipolar affective disorder. Arch. Gen. Psychiatry 45, 225–231. für DSM-IV. Achse I und II. Handanweisung. Hogrefe, Göttingen.
Morley, T.E., Moran, G., 2011. The origins of cognitive vulnerability in early childhood: Young, R.C., Biggs, J.T., Ziegler, V.E., Meyer, D.A., 1978. A rating scale for mania:
mechanisms linking early attachment to later depression. Clin. Psychol. Rev. 31, reliability, validity and sensitivity. Br. J. Psychiatry 133, 429–435.
1071–1082. Zimmerman, M., McGlinchey, J., Posternak, M., Friedman, M., Attiullah, M., Boerescu,
Morriss, R.K., van der Gucht, E., Lancaster, G., Bentall, R.P., 2009. Adult attachment in D., 2006. How should remission from depression be defined? The depressed patient's
bipolar 1 disorder. Psychol. Psychother.: Theory, Res. Pract. 82, 267–277. perspective. Am. J. Psychiatry 163, 148–150.
Munford, M.B., 1994. Relationship of gender, self-esteem, social class, and racial identity

116

Vous aimerez peut-être aussi