F C R E

Operator: LinPing

Dispatch: 12.14.06 PE: Alison McGonagle No. of Pages: 10 Copy-editor:

Journal Name Manuscript No. Proofreader: Liu Hongrong

FAMILY Erard/COMMENT O 2 45Association riginal 1531-2445Publishing Family USA of ON FCRE Court Review Incand Malden, Article FamilyERICKSON ET Courts, 2007 Blackwell COURT REVIEWConciliationAL.

PICKING CHERRIES WITH BLINDERS ON: A COMMENT ON ERICKSON ET AL. (2007) REGARDING THE USE OF TESTS IN FAMILY COURT*
Robert E. Erard, Ph.D.

Keywords:

custody; Rorschach; norms; reliability; validity; assessment; objective; projective

Erickson, Lilienfeld, and Vitaccos article (2007/this issue), A critical examination of the suitability and limitations of psychological tests in family court, invites attorneys, judges, and mental health professionals to rely on it as an objective guide for determining the selection and use of psychological tests as a basis for psychological expert testimony in child custody disputes and other family court matters. In describing the use and selection of tests, they argue strongly against a selective approach to evidence and associated conrmatory bias. On p. 167, they write: Moreover, the cherry-picking of invalid data in an impressionistic manner is wholly inconsistent with science and violates the tenets of Daubert. However, cherry-picking is precisely what they do in marshalling their arguments for their preferred instruments and against those that they do not favor. Their cherry-picking of invalid data in an impressionistic manner, and other troubling departures from a dispassionate, scientic approach to their analysis can be seen most plainly in their failure to accurately describe several of the instruments that they attempt to evaluate, their unbalanced focus on the relative strengths of multiscale objective personality inventories (particularly, the MMPI-2; Butcher, Graham, Ben-Porath, Tellegen, & Dahlstrom, 2001) versus the weaknesses of performance-based (or, as they prefer, projective) instruments, and their selective citation of the relevant literature on which they rely for their evidence. Such biases are also evident in the shift in tone and reliance on loaded language as they depart from objective analysis and become polemic in their discussion of tests that they do not favor (cf., Erard & Evans, 2006). A balanced discussion of the strengths and weaknesses of the MMPI-2 would not have limited itself to problems of interpreting defensive records and considering contextual
Correspondence: rerard2000@ameritech.net

FAMILY COURT REVIEW, Vol. 45 No. 2, April 2007 171180 2007 Association of Family and Conciliation Courts

EC

TE

PR O O

Erickson, Lilienfeld, and Vitaccos (2007) review of the suitability and limitations of psychological tests invites legal and mental health professionals to rely on it as an objecting guide for selecting, using, and admitting psychological tests in family court matters. Unfortunately, their discussion is marred by a pronounced bias in favor of multi-scale objective personality inventories and against performance based or projective instruments. This bias is evident not only in their unbalanced emphasis on the strengths of the former and weaknesses of the latter, but also in their use of selective citations and loaded language in launching what amounts to a polemical argument in support of tests that they favor. Their discussion of the Rorschach is particularly misleading. This comment refutes their unwarranted criticisms of the theoretical underpinnings of the Rorschach, its research base, its norms, its interscorer reliability, the validity of its scores, and its admissibility in the courtroom. The value of multi-method assessments that include the use of direct clinical observation of performance under standardized conditions in custody evaluations is highlighted.

172 FAMILY COURT REVIEW

factors associated with the purpose and setting of the testing. It might, for example, have also considered any of the following problems: 1) Unresolved questions about the compatibility of MMPI-2 highest two-point codes1 with the original MMPI code types (given that most of the empirically-based interpretive literature is based on the latter), particularly among non-psychiatric patients and in the common circumstance in which code types are not well-dened (Dalhstrom, 1992, but see Tellegen & Ben-Porath, 1993). 2) The fact that the majority of obtained code types fail to show adequate temporal stability,2 raising questions about whether detailed MMPI-2 based personality descriptions are reliable, particularly across changing life circumstances. 3) The problem that the internal consistency of most MMPI-2 clinical scales is fairly low, that there is considerable item overlap between scales, and that a great deal of the overall variance in the test is based on a general distress factor which suffuses most clinical, content, and supplementary scales (Tellegen et al., 2003). 4) The problem that there may not be just a paucity of MMPI-2 peer-reviewed studies establishing associations between particular MMPI-2 ndings and extra-test correlates in a child custody population, but actually an absence of any such studies (Graham, 2000; p. 357).3 5) The problem that MMPI-2 items all depend on test takers verbal self-understanding and self-presentation, such that apparent agreement between MMPI-2 results and interview and other self-report data may be the result of spurious mono-method variance rather than independent conrmation (Erard, 2005; Ganellen, 1996; Meyer, 1999; Meyer, Riethmiller, Brooks, Benoit, & Handler, 2000).

1) A major problem with criterion contamination in the most recent validation study as described in the current manual (Rogers, Salekin, & Sewell, 2000). 2) The fact that the Positive Predictive Power of some MCMI-III scores falls below .50, meaning they are wrong more often than they are correct (Retzlaff, 2000). 3) Evidence of gender bias in MCMI-III Personality Scale scores (Hyman, 2004; Lampel, 1999; McCann et al., 2001)a particular problem in family law, where the litigants naturally tend come in opposite-sexed pairs. 4) Complicated adjustments of Base Rate [BR] scores on Personality Scales based on validity scale scores that create considerable uncertainty about whether what is really being measured is response style or stable personality traits (Halon, 2001).

My point is by no means to discourage the responsible use of the MMPI-2, the MCMIIII, and similar instruments. Each of the criticisms just mentioned is answerable to some degree, but there is no getting around the fact that these objective instruments are no more valid for all purposes; no less dependent on the training, skill, and judgment of experienced clinicians; and no less problematic with respect to psychometric concerns than some of the projective techniques that Erickson et al. have singled out for scrutiny. Apparently reecting a determined bias against the Rorschach and any other personality assessment technique that does not consist of a self-report questionnaire, Erickson et al. pepper their section on projective tests with words that sound more prosecutorial than

EC

A balanced discussion of the MCMI-III (Millon, Millon, & Davis, 1994) might also have considered:

TE

PR O O

Erard/COMMENT ON ERICKSON ET AL. 173

scientice.g., vague, purport, enshrouded, suspect, outdated, indefensible, pseudoscientic. It is hardly surprising that Erickson et al. fail to give projective tests the kid-gloves treatment that characterizes their assessment of the MMPI-2. At least one of the authors (SOL) has vigorously campaigned against the Rorschach and other performancebased measures, averaging about 2 such published articles annually for the past 8 years.4 The basis for this single-minded antagonism is not entirely clear. Erickson et al. reveal a lack of serious interest in understanding performance-based techniques and how they work, as demonstrated by their failure to recognize how they are actually used in contemporary clinical practice. Contrary to these authors descriptions, the Rorschach Comprehensive System (Exner, 2002) is decidedly atheoretical and in no way dependent on what they pejoratively call psychoanalytical theoretical underpinnings (Erickson et al., 2007/this issue, p.160). In the Comprehensive System (CS), the Rorschach is understood primarily as a problem-solving task (Exner, 1993, p. 28), rather than as a screen for unconscious projections. Further, while the Rorschach is still sometimes called a projective test, no one trained in the CS assumes that all or even most of the responses given to the inkblots is based on the psychological defense known as projection (see Exner, 1993, p. 24 and pp. 5253; Weiner, 1998, pp. 6 8). The Rorschach Structural Summary does not require any more or less subjective interpretation than an MMPI-2 or MCMI-III prole (Medoff, 2003; Meyer, Mihura, & Smith, 2005). Erickson et al. are also mistaken about Rorschach CS scoringRorschach protocols cannot be scored by any existing computer software (albeit such software can be used to check inconsistencies in scoring). Turning to the Thematic Apperception Test [TAT] (Murray, 1943), the authors barely touch on systematic and validated approaches to its use, such as the Social Cognition and Object Relations Scale [SCORS] (Westen, 1991) and the Cramer Defense Scales (Cramer, 1991). They grudgingly acknowledge (p. 164) that the TAT may have some relevance to the perception of interpersonal relationships, but fail to recognize that this is a personality feature of critical importance in assessing parent-parent and parent-child relationships in family law cases. In evaluating the various Bricklin instruments, Erickson et al. recite several signicant criticisms previously made by others, but their descriptions of these tests are misleading: 1) in stating that social desirability, distortion, and parental inuence are presumed to be bypassed by the Bricklin Perceptual Scales (BPS, Bricklin, 1990) methodology,5 and 2) in implying (Erickson et al., 2007/this issue, Table 2) that any evidence exists, in the published literature that they cite, that the Bricklin instruments are in fact unreliable or invalid (Bricklin, 1993; Bricklin & Halbert, 2004a; Bricklin & Halbert, 2004b). Although the prejudice Erickson et al. manifest against any assessment methods other than verbal self-report measures has led, in my opinion, to distortions in their description and assessment of most of the instruments they discuss, I will conne the rest of this brief comment to their treatment of the Rorschach because of its particular utility in child custody evaluations (Calloway, 2005; Erard, 2005; Roseby, 1995; Weiner, 1999; Weiner, 2005a). Erickson et al. claim that whereas the MMPI-2 has hundreds of peer-reviewed studies behind it (actually, it is on the order of 10,000), the Rorschach has a questionable peer-review record (Erickson et al., 2007/this issue, Table 2). They write, Critics have been unopposed in their assertion that well over half of the studies cited in the manual were never formally peer reviewed (p. 161). Actually, Ritzler, Erard, and Pettigrew (2002a) did oppose this claim: by their count, over 90% (i.e., more than 400) of these studies were published in peer-reviewed journals.6 In fact, the Rorschach is the second most-researched

EC

TE

PR O O

174

FAMILY COURT REVIEW

personality instrument in history after the MMPI (Butcher & Rouse, 1996; Exner, 1997), with nearly 2000 articles published on it just between 1974 and 1994. In a survey of manuscripts concerning personality assessment methods appearing in peer-reviewed journals over a 10-year span (19851995), the Rorschach was a close second to the MMPI-A (Butcher et al., 1992) in the number of manuscripts published supporting its validity and reliability (Ritzler, 1996). Nor has there been any sign of a lag in published peer-reviewed research supporting the validity and utility of the Rorschach in the past 10 years. Erickson et al. make much of the supposed inadequacy of Rorschach norms for both adults and children and the Rorschachs concomitant tendency to pathologize normal behavior.7 In support of their position, they describe how a limited number of local samples, many of them using tests administered and scored by graduate students, sometimes in non-standard conditions, have differed from standard CS norms compiled from protocols administered and scored by experts in standardized conditions over a period of 20 years and based on a large, stratied national sample (Weiner, 2005b). Erickson et al. seem to be oddly unaware of the existence of new national reference data based on 450 participants (Exner & Erdberg, 2003, Chapter 12). These new CS norms clearly show that Exners original norms, while requiring some adjustments to keep pace with changes in the CS and with population shifts, were no more seriously out of kilter than those of the MMPI at the time it was re-standardized. As for childrens norms, Erickson et al. choose to dismiss normative data collected by Exners staff on nearly 1,600 children and rely on a single study (Hamel, Shaffer, & Erdberg, 2000) of Rorschach protocols of 100 children in unspecied conditions by a single examiner, using apparently nonstandard inquiry methods, and yielding many scores that make no clinical or psychometric sense to anyone experienced in administering the test to children (I. Weiner, personal communication, July 30, 2006). Erickson et al. go on to cite two studies as a basis for the conclusion that the scoring reliability of the Rorschach is poor. One study (Acklin, McDowell, Verschell, & Chan, 2000) involved 2 graduate students scoring 40 protocols; the other (Guarnaccia, Dill, Sabatino, & Southwick, 2001) used mostly graduate students scoring only 20 responses and did not include any actual protocols. Because of certain technical characteristics of the statistics used, such small samples of responses are likely to result in underestimates of reliability (Meyer et al., 2002). Acklin et al. (2000) would be surprised to learn that Erickson et al. cited their study as critical of Rorschach scoring reliability: Acklin et al. actually found that most CS coding decisions yielded acceptable to excellent levels of reliability. The Guarnaccia et al. (2001) study was later roundly criticized on statistical and methodological grounds (McGrath, 2003). Guarnaccia et al. themselves acknowledged, [i]n our view, the results of this study do not necessarily call into question the inherent consistency of the CS scoring rules (p. 472). By comparison, Meyers (1997) meta-analysis of 16 published studies strongly supported the overall reliability of the Rorschach. More recently, Meyer et al. (2002) conducted a much more sophisticated and comprehensive study, using a larger sample (219) of protocols scored in actual clinical practice at a variety of levels of experience and found that 95% of the ratings had excellent reliability and none of them were poor. Similar results were found in a study (Viglione & Taylor, 2003) examining 84 protocols with 1,732 responses for 68 major Rorschach variables and in a recent study of the eld reliability of Rorschach scoring with 84 adolescent psychiatric inpatients (McGrath et al., 2005). The scoring reliability of the Rorschach has been rmly established (Society for Personality Assessment, 2005). Erickson et al. proceed to attack the validity of the Rorschach, arguing that the majority of Rorschach scores have never been individually validated. First, once again, this is simply

EC

TE

PR O O

Erard/COMMENT ON ERICKSON ET AL. 175

not true. Validation of each Rorschach score has been described in Exner (2002), and most important variables are subject to continuing validation, either individually or in combination, in a host of ongoing studies. Scores for which the validity does not hold up over time are gradually modied or dropped from the System. Second, accurate interpretation of the Rorschach CS does not hang on the perfect validity of every component score, but rather on a complex and systematic comparison and testing of multiple scores and indices that have been empirically developed from them, all considered simultaneously and in relation to each other and to extra-test data (Exner, 2002; Weiner, 1998). This process of careful comparison and cross-checking among scores enhances the validity of the overall interpretation in each of several major areas of personality functioning. Erickson et al.s peculiar denition of meta-analysis (that is, the mathematical syntheses combining all Rorschach scores, p. 161) implies a profound misunderstanding of the scientic procedures surrounding this important technique. They dismiss as irrelevant a major meta-analysis by an independent blue-ribbon panel (Hiller, Rosenthal, Bornstein, Berry, & Brunell-Neuleib, 1999; also see Rosenthal, Hiller, Bornstein, Berry, and BrunellNeulieb, 2001) that conclusively demonstrated that the validity of the Rorschach is essentially equivalent to (not merely that its validity may even approach that of ; Erickson et al., 2007/this issue, p. 161) the MMPI-2. Erickson et al. contend that evidence for the global validity of the Rorschach does not tell us enough about particular valid uses. Nevertheless, they go on to make sweeping statements to the effect that nearly all uses of the Rorschach are invalid. They cannot logically have it both waysthey cannot condemn the meta-analyses for being too global and then go on themselves to make global criticisms regarding the invalidity of the Rorschach. Throughout their discussion of Rorschach reliability and validity, Erickson et al. ignore an extensive body of peer-reviewed literature that fails to support their conclusions and refutes their criticisms. A comprehensive selection of the literature both supportive of and critical of the Rorschach is available in the bibliography attached to a white paper on the Rorschach published by the Board of Trustees of the Society for Personality Assessment (Society for Personality Assessment, 2005, also available online at www.personality.org). In at least one instance, the degree of cherry-picking reected in Erickson et al.s disregard of relevant, peer-reviewed literature is simply astounding. Erickson et al.s references include an article (Grove, Barden, Garb, & Lilienfeld, 2002) which their second author (SOL) co-authored as a rejoinder to a previous article supporting the admissibility of Rorschach-based expert testimony under Daubert (Ritzler et al., 2002a8) and to which Ritzler, Erard, & Pettigrew (2002b) then repliedall three articles appearing side-by-side in the same issue of Psychology, Public Policy, and Law. Citing the Grove et al. (2002) article in support of their contention that Rorschach-based testimony should be held inadmissible, Erickson et al. studiously avoid any mention of either our article to which Grove et al. (2002) was written as a rejoinder or of the reply Ritzler et al. wrote to their piece.9 Although they strive to write authoritatively about what kinds of testimony meet applicable evidentiary standards, Erickson et al. do not appear to understand these standards. For example, they seem to believe that the Daubert criteria are minimal and that such criteria must all be applied in every casenotions which are contradicted by explicit statements found in Daubert and its progeny (Daubert at 579583; Kumho Tire at 226). Further, their assertion that there are some states which are Frye jurisdictions but still have adopted the Daubert principles through case law (p. 166) is unintelligible, since such labels as Frye jurisdiction and Daubert jurisdiction have no meaning apart from the analysis and conclusions of published case law.

EC

TE

PR O O

176 FAMILY COURT REVIEW

* I would like to thank Greg Meyer and my wife, Barbara Erard, for their helpful suggestions on previous drafts of this comment. 1. The 2 most elevated clinical scales, taken in combination (often called the code type), are usually the main focus for cookbook MMPI-2 interpretations. 2. Graham (2000, see Table 8.3) estimated that two point code types remained consistent on retesting only between 25 and 30 percent of the time. 3. In contrast, peer-reviewed studies of relevant extra-test correlates of the Rorschach in custody cases have been published, e.g., Johnston, Williams, & Oleson, 2005; Valente-Torre, Cavani, & Brusca, 1987, as cited in Johnston et al., 2005). 4. See Garb, Wood, Lilienfeld, & Nezworski, 2002; Garb, Wood, Lilienfeld, & Nezworski, 2005; Gill, 2006; Grove, Barden, Garb, & Lilienfeld, 2003; Lilienfeld, Lynn, & Lohr, 2004; Lilienfeld, Wood, & Garb, 2000; Wood & Lilienfeld, 1999; Wood, Lilienfeld, Garb, & Nezworski, 2000a; Wood, Lilienfeld, Garb, & Nezworski, 2000b; Wood, Lilienfeld, Nezworski, & Garb, 2001; Wood, Nezworski, Garb, & Lilienfeld, 2001a; Wood, Nezworski, Garb, & Lilienfeld, 2001b; Wood, Nezworski, Lilienfeld, & Garb, 2003; Wood, Nezworski, Garb, & Lilienfeld, 2006. 5. Although Erickson et al. actually cite to Bricklins Test Manual Supplement #9 (Bricklin, 1993), they completely ignore its lengthy and detailed discussion of how to detect the operation of parental inuences and other distortions in childrens ratings on the BPS, offered precisely because there is no presumption that the test bypasses these problems (see pp. 3 8). 6. Erickson et al. neglect, on the other hand, to mention that MCMI-III computerized narrative reports that are used in custody cases are notoriously overpathologizing, a fact that has led even MCMI-III defenders (McCann et al., 2001) to caution: [f]or this reason, the MCMI-III interpretive reports should be used with extreme caution in child custody settings (p. 41). (Also see Craig, 2005.) 7. Many of the unpublished studies are dissertations. It might also be noted in any event that in the course of developing a scoring system for virtually any test, most of the foundational studies are not published apart from their description in the manual or book introducing the method. 8. This article was itself a commentary on a prior article (Grove & Barden, 1999).

EC

TE
NOTES

The most important issue regarding testimony based on psychological tests is not simply, as Erickson et al. put it, that no psychological test is infallible, but rather that no personality test can or should be evaluated as a purely independent measure of any legally relevant construct. Tests provide lenses through which various types of legally relevant facts and observations can be studied and understood in light of applicable psychological theories, constructs, and norms. Erickson et al. are more than willing to grant that contextual considerations are important in interpreting so-called objective tests, but when interpreting performance-based measures, these authors apparently require that custody evaluators base their opinions on some simple set of zero-order correlational relationships between particular test variables and particular behavioral criteria. This is simply not how good clinicians and forensic psychologists do assessment. As in medicine (cf., Henen, Kipen, & Poulten, 2000) and in many other elds, tests are used in a process of incremental hypothesis-testing and theory-building that requires considerable skill in recognizing certain broad patterns and constructs and applying them to a particular case (Erard, 2005; Groth-Marnat, 1999; Hilsenroth & Stricker, 2004; Kleinmuntz, 1990; Ritzler et al., 2002a). Reliable and valid psychological assessment cannot be based exclusively on interviews, multiscale inventories, and other verbal self-report measures; to be comprehensive, assessment must also include direct clinical observation of performance under standardized conditions. Like true-false tests and essay exams in evaluating students, self-report tests like the MMPI-2 and performance-based measures like the Rorschach offer complementary contributions in developing rich, three-dimensional evaluations in family law matters.

PR O O

Erard/COMMENT ON ERICKSON ET AL. 177

9. Readers interested in more illuminating discussions concerning the admissibility and evidentiary weight of Rorschach-based expert testimony may wish to consider Calloway (2005); Erard (2005); Gacono, Evans, & Viglione (2002); Hamel, Gallagher, & Soares (2001); Hilsenroth & Stricker (2004); McCann (1998); Medoff (2003); Meloy, Hansen, and Weiner (1997); Ritzler et al. (2002a; 2002b); Society for Personality Assessment (2005); Weiner (2005a; 2005b); and Weiner, Exner, & Sciara (1996); but also see, in addition to those cited by Erickson et al. (2007/this issue), Garb, Wood, Nezworski, Grove, & Stejskal (2001); Grove & Barden (1999); and Wood, Nezworski, Stejskal & McKinzey (2001).

REFERENCES

Acklin, M. W., McDowell, C. J., Verschell, M. S., & Chan, D. (2000). Interobserver agreement, intraobserver reliability, and the Rorschach Comprehensive System. Journal of Personality Assessment, 74, 1547. Bricklin, B. (1990). Bricklin Perceptual Scales. Furlong, PA: Village Publishing. Bricklin, B. (1993). Test Manual Supplement #9. Furlong, PA: Village Publishing. Bricklin, B., & Halbert, M. H. (2004a). Can child custody data be generated scientically? Part I. American Journal of Family Therapy, 32, 119 138. Bricklin, B., & Halbert, M. H. (2004b). Perception-of-Relationships Test and Bricklin Perceptual Scales: Validity and Reliability Issues, Part II. American Journal of Family Therapy, 32, 189203. Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., & Dahlstrom, W. G. (2001). The Minnesota Multiphasic Personality Inventory-2 manual for administration. (Rev. ed.). Minneapolis, MN: University of Minnesota Press. Butcher, J. N., & Rouse, S. V. (1996). Personality: Individual differences and clinical assessment. Annual Review of Psychology, 47, 87111. Butcher, J. N., Williams, C. L., Graham, J. R., Archer, R. P., Tellegen, A., & Ben-Porath, Y. S. (1992). The Minnesota Multiphasic Personality Inventory for Adolescents (MMPI-A) manual for administration. Minneapolis, MN: University of Minnesota Press. Calloway, G. C. The Rorschach: Its use in child custody evaluations. Journal of Child Custody, 2(12), 143158. Craig, R. J. (Ed.). (2005). New directions in interpreting the Millon Clinical Multiaxial Inventory-III (MCMI-III). New York: Wiley. Cramer, P. (1991). The development of defense mechanisms: Theory, research, and assessment. New York: Springer-Verlag. Dahlstrom, W. G. (1992). Comparability of two-point high-point code patterns from original MMPI norms to MMPI-2 norms for the restandardization sample. Journal of Personality Assessment, 59, 153164. Daubert v. Merrell Dow Pharmaceuticals, Inc., 113 U.S. 579 (1993). Erard, R. E. (2005). What the Rorschach can contribute to custody and parenting time evaluations. Journal of Child Custody, 2(12), 119 142. Erard, R. E., & Evans, F. B. (2006, July/August). Rorschach not what critics and potshots say it is. National Psychologist, 15(4), 13. Erickson, S. K., Lilienfeld, S. O., & Vitacco, M. J. (2007, April). A critical examination of the suitability and limitations of psychological tests in family court. Family Court Review, 45(2), 153170. Exner, J. E. (1993). The Rorschach: A comprehensive system (3rd ed., Vol. I). New York: Wiley. Exner, J. E. (1997). The future of Rorschach in personality assessment. Journal of Personality Assessment, 68, 3746. Exner, J. E. (2002). The Rorschach Comprehensive System: Basic foundations and principles of interpretation, vol. 1. New York: Wiley. Exner, J. E., & Erdberg, P. (2003). The Rorschach: Advanced interpretation. New York: Wiley. Frye v. United States, 293 F. 1013 (D.C. Cir. 1923). Gacono, C. B., Evans, F. B., & Viglione, D. J. The Rorschach in forensic practice. Journal of Forensic Psychology Practice, 2(3), 33 54. Ganellen, R. J. (1996). Integrating Rorschach and the MMPI-2 in personality assessment. Mahwah, NJ: Erlbaum. Garb, H. N., Wood, J. M., Lilienfeld, S. O., & Nezworski, M. T. (2002). Effective use of projective techniques in clinical practice: Let the data help with selection and interpretation. Professional Psychology: Research and Practice, 33(5), 454 463. Garb, H. N., Wood, J. M., Lilienfeld, S. O., & Nezworski, M. T. (2005). Roots of the Rorschach controversy. Clinical Psychology Review, 25, 97118. Garb, H. N., Wood, J. M., Nezworski, M. T., Grove, W. M., & Stejskal, W. J. (2001). Toward a resolution of the Rorschach controversy. Psychological Assessment, 13, 433448.

EC

TE

PR O O

178

FAMILY COURT REVIEW

Gill, R. E. (2006, May/June). Rorschach not what its cracked up to be? National Psychologist, 15(3), p. 21. Graham, J. R. (2000). The MMPI-2: Assessing personality and psychopathology (3rd ed.). New York: Oxford University Press. Graham, J. R., Timbrook, R. E., & Ben-Porath, Y. S. (1991). Code-type congruence between MMPI and MMPI-2: Separating fact from artifact. Journal of Personality Assessment, 57(2), 205215. Groth-Marnat, G. (1999). Handbook of psychological assessment (3rd ed.). New York: Wiley. Grove, W. R., & Barden, H. N. (1999). Protecting the integrity of the legal system: The admissibility of testimony from mental health experts under Daubert/Kumho analyses. Psychology, Public Policy, and Law, 5, 224242. Grove, W. R., Barden, C., Garb, H. N., & Lilienfeld, S. O. (2002). Failure of Rorschach-ComprehensiveSystem-based testimony to be admissible under the Daubert/Kumho Standard. Psychology, Public Policy, and Law, 8(2), 216234. Guarnaccia, V Dill, C., Sabatino, S., & Southwick, S. (2001). Scoring accuracy using the Comprehensive System ., for the Rorschach. Journal of Personality Assessment, 77, 464474. Halon, R. L. (2001). The Millon Clinical Multiaxial Inventory-III: The normal quartet in child custody cases. American Journal of Forensic Psychology, 19, 5775. Hamel, M., Gallagher, S., & Soares, C. (2001). The Rorschach: Here we go again. Journal of Forensic Psychology Practice, 1(3), 7987. Hamel, M., Shaffer, T. W., & Erdberg, P. (2000). A study of nonpatient preadolescent Rorschach protocols. Journal of Personality Assessment, 75, 280 294. Henen, M. S., Kipen, H. M., & Poulten, S. R. (2000). Reference guide on medical testimony. Reference manual on scientic evidence, pp. 439 484. Federal Judicial Center, Washington, D.C. Hiller, J. B., Rosenthal, R., Bornstein, R. F., Berry, D. T. R., & Brunell-Neuleib, S. (1999). A comparative meta-analysis of Rorschach and MMPI validity. Psychological Assessment, 11, 278296. Hilsenroth, M. J., & Stricker, G. (2004). A consideration of challenges to psychological assessment instruments used in forensic practice: Rorschach as exemplar. Journal of Personality Assessment, 83(2), 141152. Hyman, D. J. (2004). Unsupported gender differences on some personality disorder scales of the Millon Clinical Multiaxial Inventory-III. Professional Psychology: Research and Practice, 35(1), 105110. Johnston, J. R., Walters, M. G., & Oleson, N. W. (2005). Clinical ratings of parenting capacity and Rorschach protocols of custody-disputing parents: An exploratory study. Journal of Child Custody, 2(12), 159178. Kleinmuntz, B. (1990). Why we still use our heads instead of formulas: Toward an integrative approach. Psychological Bulletin, 107, 296 310. Kumho Tire Co., v. Carmichael, 526 U.S. 137 (1999). Lampel, A. K. (1999). Use of the Millon Clinical Multiaxial Inventory-III in evaluating child custody litigants. American Journal of Forensic Psychology, 17, 1931. Lilienfeld, S. O., Lynn, S. J., & Lohr, J. M. (Eds.). (2004). Science and pseudoscience in clinical psychology. New York: Guilford. Lilienfeld, S. O., Wood, J. M., & Garb, H. N. (2000). The scientic status of projective techniques. Psychological Science in the Public Interest 1, 27 66. McCann, J. T. (1998). Defending the Rorschach in court: An analysis of admissibility using legal and professional standards. Journal of Personality Assessment, 70, 125 144. McCann, J. T., Flens, J. R., Campagna, V Collman, P., Lazzaro, T., & Connor, E. (2001). The MCMI-III in child ., custody evaluations: A normative study. Journal of Forensic Psychology Practice, 1(2), 2744. McGrath, R. E. (2003). Enhancing accuracy in observational test scoring: The comprehensive system as an example. Journal of Personality Assessment, 81(2), 104110. McGrath, R. E., Pogge, D. L., Stokes, J. M., Cragnolino, A., Zaccario, M., Hayman, J., Piacentini, T., & WaylandSmith, D. (2005). Field reliability of Comprehensive System scoring in an adolescent inpatient sample. Assessment, 12(2), 199209. Medoff, D. (2003). The scientic basis of psychological testing: Considerations following Daubert, Kumho, and Joiner. Family Court Review, 41(2), 199 213. Meloy, J. R. (1991). Rorschach testimony. The Journal of Psychiatry and Law, 19, 221235. Meloy, J. R., Hansen, T. L., & Weiner, I. B. (1997). Authority of the Rorschach: Legal citations during the past 50 years. Journal of Personality Assessment, 69, 5362. Meyer, G. J. (1997). Assessing reliability: Critical corrections for a critical examination of the Rorschach Comprehensive System. Psychological Assessment, 9, 480 489. Meyer, G. J. (1999). The convergent validity of MMPI and Rorschach scales: An extension using prole scores to dene response and character styles on both methods and a reexamination of simple Rorschach response frequency. Journal of Personality Assessment, 72, 1 35. Meyer, G. J., Hilenroth, M. J., Baxter, D., Exner, J. E., Fowler, J. C., Piers, C. C., & Resnick, J. (2002). An

EC

TE

PR O O

Erard/COMMENT ON ERICKSON ET AL. 179

examination of interrater reliability for scoring the Rorschach Comprehensive System in eight data sets. Journal of Personality Assessment, 78(2), 219 274. Meyer, G. J., Mihura, J. L., & Smith, B. L. (2005). The interclinician reliability of Rorschach interpretation in four data sets. Journal of Personality Assessment, 84, 296314. Meyer, G. J., Riethmiller, R. J., Brooks, R. D., Benoit, W. A., & Handler, L. (2000). A replication of Rorschach and MMPI convergent validity. Journal of Personality Assessment, 74(2), 175 215. Millon, T., Millon, C., & Davis, R. (1997). Manual for the Millon Clinical Multiaxial Inventory-III (MCMI-III). (2nd ed.). Minneapolis, MN: National Computer Systems, Inc. Murray, H. (1943). Thematic Apperception Test. Cambridge, MA: Harvard University Press. Retzlaff, P. D. (2000). Comment on the validity of the MCMI-III. Law and Human Behavior, 24, 499500. Ritzler, B. (1996). The status of personality assessment. Presidential address at a meeting of the Society for Personality Assessment, Denver, CO. Ritzler, B., Erard, R., & Pettigrew, G. (2002a). Protecting the integrity of Rorschach expert witnesses: A reply to Grove and Barden (1999) re: the admissibility of testimony under Daubert/Kumho analyses. Psychology, Public Policy, and Law, 8(2), 201215. Ritzler, B., Erard, R., & Pettigrew, G. (2002b). A nal reply to Grove and Barden: The relevance of the Rorschach Comprehensive System for expert testimony. Psychology, Public Policy, and Law, 8(2), 235246. Rogers, R., Salekin, R. T., & Sewell, K. W. (1999). Validation of the Millon Clinical Multiaxial Inventory for Axis II disorders: Does it meet the Daubert standard? Law and Human Behavior, 23, 425443. Rosenthal, R., Hiller, J. B., Bornstein, R. F., Berry, D. T., & Brunell-Neuleib, S. (2001). Meta-analytic methods, the Rorschach, and the MMPI. Psychological Assessment, 13, 449451. Roseby, V (1995). Uses of psychological testing in a child-focused approach to child custody evaluations. Family . Law Quarterly, 28, 97110. Shaffer, T. W., Erdberg, P., & Haroian, J. (1999). Current nonpatient data for the Rorschach, WAIS-R, and MMPI2. Journal of Personality Assessment, 73, 305 316. Society for Personality Assessment. (2005). The status of the Rorschach in clinical and forensic practice: An ofcial statement by the Board of Trustees of the Society for Personality Assessment. Journal of Personality Assessment, 85, 219237 (available at http://www.personality.org). Tellegen, A., & Ben-Porath, Y. S. (1993). Code-type comparability of the MMPI and the MMPI-2: Analysis of recent ndings and criticism. Journal of Personality Assessment, 61, 489500. Tellegen, A., Ben-Porath, Y. S., McNulty, J. L., Arbisi, P. A., Graham, J. R., & Kaemmer, B. (2003). MMPI-2 Restructured Clinical (RC) Scales: Development, validation, and interpretation. Minneapolis, MN: University of Minnesota Press. Viglione, D. J., & Taylor, N. (2003). Empirical support for interrater reliability of Rorschach Comprehensive System coding. Journal of Clinical Psychology, 59, 111121. Weiner, I. B. (1998). Principles of Rorschach interpretation. Mahway, NJ: Lawrence Erlbaum Associates. Weiner, I. B. (1999). What the Rorschach can do for you: Incremental validity in clinical applications. Assessment, 6, 327339. Weiner, I. B. (2005a). Rorschach assessment in child custody cases. Journal of Child Custody, 2(3), 99119. Weiner, I.B. (2005b). The utility of Rorschach assessment in clinical and forensic practice. Independent Practitioner, 25, 7683. Weiner, I. B., Exner, J. E., & Sciara, A. (1996). Is the Rorschach welcome in the courtroom? Journal of Personality Assessment, 67, 422 424. Westen, D. (1991). Social cognition and object relations. Psychological Bulletin, 109(3), 429455. Wood, J. M., & Lilienfeld, S. O. (1999). The Rorschach Inkblot Test: A case of overstatement? Assessment, 6(4), 341351. Wood, J. M., Lilienfeld, S. O., Garb, H. N., & Nezworski, M. T. (2000a). The Rorschach test in clinical diagnosis: A critical review, with a backward look at Gareld (1947). Journal of Clinical Psychology, 56(3), 395430. Wood, J. M., Lilienfeld, S. O., Garb, H. N., & Nezworski, M. T. (2000b). Limitations of the Rorschach as a diagnostic tool: A reply to Gareld (2000), Lerner (2000), and Weiner (2000). Journal of Clinical Psychology, 56(3), 441448. Wood, J. M., Lilienfeld, S. O., Nezworski, M. T., & Garb, H. N. (2001). Coming to grips with negative evidence for the comprehensive system for the Rorschach: A comment on Gacono, Loving, and Bodholdt; Ganellen; & Bornstein. Journal of Personality Assessment, 77, 4870. Wood, J. M., Nezworski, M. T., Garb, H. N., & Lilienfeld, S. O. (2001a). The misperception of psychopathology: Problems with norms of the Comprehensive System for the Rorschach. Clinical Psychology: Science and Practice, 8(3), 350 373.

EC

TE

PR O O

180

FAMILY COURT REVIEW

Wood, J. M., Nezworski, M. T., Garb, H. N., & Lilienfeld, S. O. (2001b). Problems with the norms of the Comprehensive System for the Rorschach: Methodological and conceptual considerations. Clinical Psychology: Science and Practice, 8(3), 397 402. Wood, J. M., Nezworski, M. T., Garb, H. N., & Lilienfeld, S. O. (2006). The controversy over Exners Comprehensive System for the Rorschach: The critics speak. Independent Practitioner, 26, 7382. Wood, J. M., Nezworski, M. T., Lilienfeld, S. O., & Garb, H. N. (2003). Whats wrong with the Rorschach?: Science confronts the controversial inkblot test. New York: Jossey-Bass. Wood, J. M., Nezworski, M. T., Stejskal, W. J., & McKinzey, R. K. (2001). Problems of the Comprehensive System for the Rorschach in forensic settings: Recent developments. Journal of Forensic Psychology Practice, 1, 89103.

EC

TE

Robert E. Erard, Ph.D. is clinical director of Psychological Institutes of Michigan, P where he practices .C., clinical and forensic psychology. He has served as a court-appointed expert in family law matters on a regular basis for over 20 years. He is co-editor of the Clinical Case Applications Section of the Journal of Personality Assessment and an editorial board member of the Journal of Child Custody. He is a member of the Board of Trustees of the Society for Personality Assessment; a past president of the Michigan Psychological Association; a past president of the Michigan Inter-Professional Association on Marriage, Divorce, and the Family; and a former member of the APA Council of Representatives. Dr. Erards publications include articles on the forensic use of the Rorschach, child custody evaluation and mediation, expert testimony, and the release of raw test data. He has presented national workshops on the ethics of clinical and forensic assessment and on child custody evaluations and has made over 50 professional presentations in the areas of family law, psychological assessment, and expert testimony and the Rules of Evidence.

PR O O