Defining Issues Test2: Spring 2009

Defining Issues Test-2: Spring 2009

November 2009

prepared by Office of Planning and Assessment

Defining Issues Test2: Spring 2009

Table of Contents
Executive Summary....................................................................................................................................... 3 Introduction .................................................................................................................................................. 4 Sample........................................................................................................................................................... 5 Results ........................................................................................................................................................... 6 Conclusion ................................................................................................................................................... 16 References .................................................................................................................................................. 18 Attachments................................................................................................................................................ 19

Defining Issues Test2: Spring 2009

Executive Summary To help measure the effectiveness of taking classes at TTU on teaching ethical reasoning skills, the Defining Issues Test, Version 2 (DIT-2) was administered to TTU students. According to the Center for the Study of Ethical Development (2003), the DIT-2 is a measure of moral judgment derived from Kohlbergs model of moral development (Kohlberg, 1984). The DIT-2 includes five hypothetical moral dilemmas, each followed by 12 issues that could be involved in making a decision about the dilemma. Participants are asked what decision they would make in each dilemma and which issues they consider most important in making the decision. These responses are scored to find which moral schema students follow in making moral decision: Personal interests schema: considering what will benefit me and help others to like me Maintaining norms schema: considering what will maintain the law and social order Postconventional schema: considering human rights and other moral principles During the spring 2009 semester, some TTU students were invited to participate in two administrations of the DIT-2 online, a pretest at the beginning of the semester and a posttest at the end of the semester. The tests were scored by the Center for the Study of Ethical Development with a final sample size of 78 students from the pretest and 18 students from the posttest. Nine of these students participated in both administrations of the test. Looking at the combination of both samples there are some interesting potential patterns in scores by demographic variables. Women appear to focus more on postconventional issues than men in the sample. Conservative students appear to focus more on maintaining norms issues. Older students seem to have higher U scores (U scores reflect the degree of match between the moral decision students make and the issues they report are most important in making the decision). Future research with a larger sample could help clarify if these patterns are typical for TTU students and if there are any other patterns of interest. Comparing the pretest and the posttest as two independent samples, there is a significant increase in postconventional issues endorsed by students. Comparing the pretest and the posttest as a matched-pairs sample of the nine students that took the test during both administrations, there is a significant increase in the U scores. These are both hopeful changes in the ethical understanding; future research could help clarify if these changes are typical for TTU students. In line with the QEP Student Learning Outcomes, the DIT-2 helps test students ability to identify, articulate, and reflect critically on ethical issues from multiple perspectives. The results of this report show some patterns that would suggest that taking classes at TTU is helping students to meet these outcomes. To help clarify these patterns, future administrations of the DIT-2 should come at the beginning and end of different types of TTU classes with larger samples of students participating in both administrations of the test.

Defining Issues Test2: Spring 2009

Introduction According to the Center for the Study of Ethical Development (2003), the Defining Issues Test, Version 2 (DIT-2), is a measure of moral judgment derived from Kohlbergs model of moral development (Kohlberg, 1984). The DIT-2 includes five hypothetical moral dilemmas, each followed by 12 issues that could be involved in making a decision about the dilemma. Participants rate each issue and choose the first, second, third, and fourth most important issues for each of the five dilemmas. Participants are also asked what decision they would make in each moral dilemma (see Attachment A for the online version of the DIT-2 used for this report). The results of the test provide the following scores (Center for the Study of Ethical Development, 2003):
Stage 23 Score Personal Interest Schema Score: this score represents the proportion of items selected that represent considerations from Stage 2 (focus on the personal interest of the actor making the moral decisions) and Stage 3 (focus on maintaining friendships, good relationships, and approval). Maintaining Norms Schema Score: this score represents the proportion of items selected that represent consideration from Stage 4 (focus on maintaining the existing legal system, roles, and formal organizational structure). Postconventional Schema Score: this score represents the proportion of items selected that represent considerations from Stage 5 (focus on appealing to majority while maintaining minority rights) and Stage 6 (focus on appealing to intuitive moral principles or ideals). New Index Score: this score represents the degree to which Postconventional items are prioritized plus the degree to which Personal interest items receive lower ratings than the Postconventional items. This score is adjusted to have the same mean and standard deviation as the P score to allow for comparisons. Utilizer Score: This score represents the degree of match between which items the participants rated as most important and what decision participants say they would make in the moral dilemma. Humanitarian/Liberalism Score: this score represents the number of reported decisions for the moral dilemmas that match those chosen by a group of experts (professionals in the field of political science and philosophy). Scores range from 0 to 5 out of the possible 5 moral dilemma decisions that can match. Religious Orthodoxy Score: this score represents the sum of the rated importance and rank for one specific item from the Cancer moral dilemma that evokes the notion that only God can determine whether or not someone should live or die. Antisocial Score: this score represents the degree to which items are selected that represent considerations that reflect an anti-establishment attitude. These considerations presuppose Stage 4, but fault the establishment for being inconsistent with their purpose.

Stage 4P Score

P Score

N2 Score

U Score

Hum/Lib Score

Cancer10 Score

A Score

Defining Issues Test2: Spring 2009

Sample To help measure the effectiveness of taking classes at TTU, the DIT-2 was administered to participating groups at two different times during the Spring 2009 Semester. The DIT-2 was administered online. The first administration of the test was at the beginning to middle of the course (i.e., the pretest). In many cases, the pretest administration was too late in the course of the class to be considered a true pretest. The Ethics in the Curriculum Task Force recruited the two participating courses: a management course and a philosophy course. Participants also included student who worked as part-time employees within Student Affairs. These participants were also enrolled in courses across campus. Invitations were sent to a random sample of students at TTU as we. 86 students participated in this first administration of the test. The test results were sent to the Center for the Study of Ethical Development to be scored. The returned scores suggested that eight of these respondents be filtered from the results for a final sample size of 78 students. The second administration of the test was at the end of the course to the same groups from the pretest. 19 students participated in this second administration of the test. These test results were also sent to the Center for the Study of Ethical Development to be scored. The returned scores suggested that one of these respondents be filtered from the results for a final sample size of 18 students. Of these 18 students, nine participated in both the first administration and second administration of the test to provide for a matched comparison. The results of both administrations were also combined to check for overall patterns, including only the second administration for these nine students that participated in both administrations. This combined sample includes the results of all 87 students that participated in at least one administration of the test without their results being filtered from the analysis. The following charts show the representation of this combined sample according to sex, age group, class, and political ties: Sample by Sex
60 50 40 30 20 10 0 Male Female 35 30 25 20 15 10 5 0 18 -19 20 - 21 22 - 24 over 24

Sample by Age Group

Sample by Class
35 30 25 20 15 10 5 0

Defining Issues Test2: Spring 2009

Sample by Political Ties
35 30 25 20 15 10 5 0 Very Liberal Some Neither Liberal Some Cons. Very Cons.

The sampling methods did not produce the desired sample sizes to generalize the findings, particularly when considering that only nine participants completed both administrations, but the sample does appear to have some diversity and may provide some meaningful implications for future research. Results Moral Schema Scores by Demographics We will first look at patterns in the three moral schema scores (Stage 23, Stage 4P, and P scores) for the combined sample according to the demographic variables of sex, age group, class, and political ties. Remember that the Stage 23, Stage 4P, and P scores reflect the proportion of items selected that represent considerations from a personal interest schema, a maintain norms schema, and a postconventional schema, respectively. Moral Schema Scores by Sex
40 35 30 25 20 15 10 5 0 STAGE23 STAGE4P PSCORE

Male (N=49) Female (N=38)

In this sample, it appears that the female participants have slightly higher P scores and slightly lower Stage 23 and Stage 4P scores, suggesting a slightly higher level of moral understanding. This would coincide with the research that has shown that females generally score better on the DIT than males (Center for the Study of Ethical Development, 2003). This difference does not appear to be large for this sample, but further research may help to clarify if the pattern of women doing better than men holds true with TTU students.

Moral Schema Scores by Age Group

45 40 35 30 25 20 15 10 5 0 STAGE23 STAGE4P PSCORE

Defining Issues Test2: Spring 2009

18 -19 (N=21) 20 - 21 (N=32) 22 - 24 (N=25) over 24 (N=9)

There does not appear to be any particular pattern in schema scores for the different age groups. Moral Schema Scores by Class
50 40 30 20 10 0 STAGE23 STAGE4P PSCORE Freshman (N=5) Sophomore (N=32) Junior (N=14) Senior (N=25) Master's (N=10) Doctoral (N=1)

Considering that the single Doctoral students schema scores does not represent a large enough sample for comparison, there does not appear to be any particular pattern in schema scores for the different student classifications. Moral Schema Scores by Political Ties
45 40 35 30 25 20 15 10 5 0 STAGE23 STAGE4P PSCORE Very liberal (N=7) Somewhat liberal (N=13) Neither liberal nor conservative (N=16) Somewhat conservative (N=33) Very conservative (N=18)

It appears that the students that consider themselves somewhat conservative or very conservative have higher Stage 4P scores. This would coincide with the research on DIT scores according to political ties (Center for the study of Ethical Development, 2003). There do not appear to be other noteworthy patterns in schema scores by political ties. Primary Moral Schema Types by Demographic Based on the moral schema scores, students can be given a primary moral schema type according to the following types:
Type 1: predominant in personal interests Type 2: predominant in personal interests, transitioning to maintaining norms Type 3: predominant in maintaining norms, transitioning from personal interests Type 4: predominant in maintaining norms Type 5: predominant in maintaining norms, transitioning to postconventional Type 6: predominant in postconventional, transitioning from maintaining norms Type 7: predominant in postconventional

Defining Issues Test2: Spring 2009

The following graphs show the primary moral schema types of participants by sex, age group, class, and political ties.
Primary Schema Type: Males (N=49) 0.4 0.3 0.2 0.1 0 Type Type Type Type Type Type Type 1 2 3 4 5 6 7 0.4 0.3 0.2 0.1 0 Type Type Type Type Type Type Type 1 2 3 4 5 6 7 Primary Schema Type: Females (N=38)

Just over half of the males in the sample appear to be transitioning between personal interests and maintaining norms schemas. The females seem to have more students transitioning between maintaining norms and postconventional schemas. Future research with a larger sample and more participants taking pre and posttests could help clarify if these patterns are typical of TTU students and if students primary moral schema type changes as they take ethics classes as TTU.

Primary Schema Type: Ages 18 -19 (N=21) 0.4 0.3 0.2 0.1 0

Defining Issues Test2: Spring 2009

Primary Schema Type: Ages 20 - 21 (N=32) 0.4 0.3 0.2 0.1 0

Type Type Type Type Type Type Type 1 2 3 4 5 6 7 Primary Schema Type: Ages 22 - 24 (N=25) 0.4 0.3 0.2 0.1 0 Type Type Type Type Type Type Type 1 2 3 4 5 6 7 0.4 0.3 0.2 0.1 0

Type Type Type Type Type Type Type 1 2 3 4 5 6 7 Primary Schema Type: Ages over 24 (N=9)

Type Type Type Type Type Type Type 1 2 3 4 5 6 7

There seem to be many transitioning between personal interests and maintaining norms schemas in each age group, with more transitioning toward a postconventional schema in the older age groups. Future research could help clarify if these patterns are typical of TTU students.

Primary Schema Type: Freshman (N=5) 0.5 0.4 0.3 0.2 0.1 0

Defining Issues Test2: Spring 2009

Primary Schema Type: Sophomore (N=32) 0.4 0.3 0.2 0.1 0

Type Type Type Type Type Type Type 1 2 3 4 5 6 7 Primary Schema Type: Junior (N=14) 0.5 0.4 0.3 0.2 0.1 0 Type Type Type Type Type Type Type 1 2 3 4 5 6 7 Primary Schema Type: Master's (N=10) 0.4 0.3 0.2 0.1 0 Type Type Type Type Type Type Type 1 2 3 4 5 6 7 1 0.8 0.6 0.4 0.2 0 0.4 0.3 0.2 0.1 0

Type Type Type Type Type Type Type 1 2 3 4 5 6 7 Primary Schema Type: Senior (N=25)

Type Type Type Type Type Type Type 1 2 3 4 5 6 7 Primary Schema Type: Doctoral (N=1)

Type Type Type Type Type Type Type 1 2 3 4 5 6 7

Some of these subgroup samples are small, making it difficult to interpret patterns, but it seems that the older classes (seniors and Masters students) have moved away from the personal interests and more towards the postconventional schema types. Future research could help clarify if these patterns are typical of TTU students.
Primary Schema Type: Very Lib. (N=7) 0.5 0.4 0.3 0.2 0.1 0 Type Type Type Type Type Type Type 1 2 3 4 5 6 7 0.5 0.4 0.3 0.2 0.1 0 Type Type Type Type Type Type Type 1 2 3 4 5 6 7 Primary Schema Type: Some Lib. (N=13)

10

Primary Schema Type: Neither (N=16) 0.5 0.4 0.3 0.2 0.1 0

Defining Issues Test2: Spring 2009

Primary Schema Type: Some Cons. (N=33) 0.5 0.4 0.3 0.2 0.1 0

Type Type Type Type Type Type Type 1 2 3 4 5 6 7

Type Type Type Type Type Type Type 1 2 3 4 5 6 7

Primary Schema Type: Very Cons. (N=18) 0.5 0.4 0.3 0.2 0.1 0 Type Type Type Type Type Type Type 1 2 3 4 5 6 7

With the smaller sample sizes, there do not appear to be any clear patterns according to political ties. Future research with a larger sample may find clearer patterns for primary moral schema type by political ties among TTU students. N2 and U Scores by Demographics We will now look at patterns in the N2 scores and U scores for the combined sample according to the demographic variables of sex, age group, class, and political ties. Remember that the N2 score reflects the degree to which postconventional items were prioritized and prioritized higher than personal interest items. Remember that the U Score reflects the degree of match between the items prioritized and the moral decision. N2 Score by Sex
50 40 30 20 10 0 Male (N=49) Female (N=38) 0.2 0.16 0.12 0.08 0.04 0 Male (N=49) Female (N=38)

U Score by Sex

The female N2 scores were slightly higher and the male U scores were slightly higher, but there does not appear to be any noteworthy pattern according to sex for the N2 scores or U scores.

11

N2 Score by Age Group

50 40 30 20 10 0 18 -19 (N=21) 20 - 21 (N=32) 22 - 24 (N=25)

Defining Issues Test2: Spring 2009

U Score by Age Group
0.2 0.15 0.1 0.05 0 over 24 (N=9) 18 -19 (N=21) 20 - 21 (N=32) 22 - 24 (N=25) over 24 (N=9)

There does not appear to be any pattern according to age group for N2 scores, but the U scores appear to improve with older participants. This may reflect a more established moral decision making process in older participants. N2 Score by Class
50 40 30 20 10 0 0.2 0.15 0.1 0.05 0

U Score by Class

There appear to be lower N2 scores for the Juniors and the Doctoral student and a lower U score for the Doctoral student. These groups may be too small, though, for any meaningful comparisons. N2 Score by Political Ties
50 40 30 20 10 0 Very Lib. Some Neither Some Very (N=7) Lib. (N=16) Cons. Cons. (N=13) (N=33) (N=18) 0.18 0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 Very Some Neither Some Very Lib. Lib. (N=16) Cons. Cons. (N=7) (N=13) (N=33) (N=18)

U Score by Political Ties

12

There appear to be slightly lower N2 scores for those students on the conservative end, but the difference does not appear to be very large. The students that described themselves as very liberal have lower utility scores. This group may be too small for a meaningful comparison. Cancer10, Hum/Lib, and A Scores by Demographics We will now look at patterns in the Cancer10, Hum/Lib, and A scores for the combined sample according to the demographic variables of sex, age group, class, and political ties. Remember that the Cancer 10 score reflects the level of religious orthodoxy, the Hum/Lib score reflects the number of the students moral decisions that match a group of experts, and the A score reflects the level of anti-establishment sentiments. Cancer10, Hum/Lib, and A Scores by Sex
5 4 3 2 1 0 CANCER10 HUMLIB ASCORE Male (N=49) Female (N=38)

Defining Issues Test2: Spring 2009

The female students seem to have slightly lower Cancer10 scores and slightly higher A scores. Further research may help clarify whether this pattern is characteristic of students at TTU. Cancer10, Hum/Lib, and A Scores by Age Group
6 5 4 3 2 1 0 CANCER10 HUMLIB ASCORE 18 -19 (N=21) 20 - 21 (N=32) 22 - 24 (N=25) over 24 (N=9)

The Cancer10 scores seem to be higher for the older students. There does not appear to be any pattern according to age group for Hum/Lib or A scores.

13

Cancer10, Hum/Lib, and A Scores by Age Group

12 10 8 6 4 2 0 CANCER10 HUMLIB ASCORE

Defining Issues Test2: Spring 2009

Freshman (N=5) Sophomore (N=32) Junior (N=14) Senior (N=25) Master's (N=10) Doctoral (N=1)

The Masters students seem to have higher Cancer10 scores than the other classes of students. The Doctoral student has a high A score. These groups may be too small for any meaningful comparison. Cancer10, Hum/Lib, and A Scores by Political Ties
6 5 4 3 2 1 0 CANCER10 HUMLIB ASCORE Very Lib. (N=7) Some Lib. (N=13) Neither (N=16) Some Cons. (N=33) Very Cons. (N=18)

Cancer10 scores are higher for the somewhat conservative and very conservative students, Hum/Lib scores are lower for the very liberal students, and A scores are lower for the very conservative students. These groups may be too small for any meaningful comparison.

14

Comparison of Pretest and Posttest of DIT We will first compare the first and second administration of the DIT as two independent samples. We will then compare the nine students that participated in both administrations of the test.
Comparison of First and Second Administration of DIT as Two Independent Samples First Administration (N=78) Mean Stage23 Stage4P P score N2 score U score Cancer10 Hum/Lib A score 28.4103 35.5128 29.7692 28.7463 0.2669 4.3462 1.8442 1.3077 Std. Dev. 11.4885 13.7442 14.4401 14.0773 1.1220 2.8046 1.0890 1.9759 Second Administration (N=18) Mean 25.5556 30.5556 37.8889 36.1972 0.1656 3.7778 2.5000 1.5556 Std. Dev. 12.0890 15.2594 17.0463 17.4172 0.1495 2.5795 1.2005 1.9166 T-statistic -0.9330 -1.2953 1.9174 1.7335 -0.7749 -0.8475 2.1757 0.5031 P-value 0.1766 0.0992 0.0291 0.0431 0.2201 0.1994 0.0160 0.3080

Defining Issues Test2: Spring 2009

There are three significant differences between the two administrations of the test: an increase in P scores, an increase in N2 scores, and an increase in Hum/Lib scores. Given the small second administration sample size and the short duration between administrations, further research could help clarify whether these patterns are descriptive of the results of TTU ethics classes or if other patterns might arise. Future research should seek out a larger sample to take the DIT during both administrations to give a better explanation for the differences. We will now look at the changes in scores from the nine students that did participate in both of these administrations of the DIT.
Change in DIT Scores for Students that Participated in both the First and Second Administration Change in Scores (N=9) Mean Stage23 Stage4P P score N2 score U score Cancer10 Hum/Lib A score 2.6667 0.0000 -4.0000 -4.8589 0.0989 -0.3333 -0.4444 -0.2222 Std. Dev. 15.9687 7.6158 19.2614 16.3057 0.1222 2.5981 1.1304 1.9221 T-statistic 0.5010 0.0000 -0.6230 -0.8940 2.4274 -0.3849 -1.1795 -0.3688 P-value 0.3149 0.5000 0.2753 0.1987 0.0207 0.3552 0.1360 0.3688

15

There is only one significant change: an increase in the U scores. This may suggest that taking classes at TTU help students to make decisions more in line with their moral reasoning. Future research should use a larger sample size participating in both administrations to explore if this pattern holds for classes at TTU and to test for other significant patterns. The results from this small sample size seem to have been skewed by some more extreme observations in the data.

Defining Issues Test2: Spring 2009

Summary There are some interesting potential patterns in moral schema scores by demographic variables. Women appear to have slightly higher P scores and slightly lower Stage23 and Stage4P scores. It also appears that students with conservative ties have higher Stage4P scores. Future research with a larger sample could help clarify if these patterns are typical for TTU students and if there are any other patterns of interest. There also appear to be some potential patterns in primary moral schema type that could be explored by future research. There are also some interesting patterns in the other scores from the DIT. It appears that the U score increases with age. It makes sense that as students get older they solidify their moral decision making with their moral schema. The very liberal group had a much lower U score, but this information comes from a small sample size. It will be interesting to see if this pattern holds with a larger sample of TTU students. Cancer10 scores also seem to increase with age and are higher for conservative students. Comparing the first and second administration of the tests as two independent samples, there is a significant increase in P scores, N2 scores, and Hum/Lib scores. Comparing the first and second administration of the tests as a matched-pairs sample of the nine students that took the test during both administrations, there is a significant increase in the U scores. These changes are all hopeful; future research with a larger sample taking the test at the beginning and end of an ethics class could help find if these patterns are typical for students in TTU classes. Conclusion The DIT-2 helps test for the moral schemas that students use to consider moral dilemmas and to make moral decisions. There are three QEP Student Learning Outcomes that the DIT-2 can help test for: 7. Identify ethical issues in personal, professional, and civic life from their own perspective as well as that of others 8. Articulate ethical issues in personal, professional, and civic life from their own perspective as well as that of others 9. Reflect critically on ethical issues in personal, professional, and civic life from their own perspective as well as that of others The results of this report show some patterns that would suggest that taking classes at TTU is helping students to meet these outcomes. The shift in scores, with higher P scores, N2 scores, and Hum/Lib scores suggests that students are learning to look at moral or ethical issues from different perspectives. An increase in U scores for the nine students that took the DIT-2 both

16

administrations suggests that students are becoming more aware of their own moral position. Learning to see moral dilemmas from other perspectives and being able to follow ones own moral schema in the decisions they make helps students to identify, articulate, and reflect critically on ethical issues from their own perspective as well as that of others.

Defining Issues Test2: Spring 2009

The administrations of the DIT-2 in this report came too close together and drew too few participants to make solid conclusions about how ethics classes at TTU have affected these three student learning outcomes for students. Future administrations of the DIT-2 should come at the beginning and end of the semester with larger samples of students in various courses participating in both administrations of the test.

17

Defining Issues Test2: Spring 2009

References
Center for the study of Ethical Development. (2003). Guide for DIT-2. Minneapolis, MN: Bebeau, M. J., & Thoma, S. J. Kohlberg, L. (1984). Essays on moral development: The nature and validity of moral stages (Vol. 2). San Francisco: Harper & Row.

18

Defining Issues Test2: Spring 2009

Attachments Attachment A: Screenshots Online Version of the DIT-2 used for this Report

19

Defining Issues Test2: Spring 2009

20

Defining Issues Test2: Spring 2009

21

Defining Issues Test2: Spring 2009

22

Defining Issues Test2: Spring 2009

23

Defining Issues Test2: Spring 2009

24

Defining Issues Test2: Spring 2009

25

Defining Issues Test2: Spring 2009

26

Defining Issues Test2: Spring 2009

27

Defining Issues Test2: Spring 2009

28

Defining Issues Test2: Spring 2009

29