NONCOGNITIVE TEST

ASSESSING SEVENTH GRADE STUDENTS OF JUNIOR HIGH

SCHOOLS MOTIVATION THROUGH QUESTIONNAIRE




BY
PUTU TIKA VIRGINIYA
NIM: 1229081036
B 1






POSTGRADUATE
ENGLISH EDUCATION STUDY PROGRAM
UNDIKSHA UNIVERSITY
2013
ASSESSING SEVENTH GRADE STUDENTS OF JUNIOR HIGH
SCHOOLS MOTIVATION THROUGH QUESTIONNAIRE

1. Motivation - Questionnaire
In this paper, questionnaire is made to assess motivation of seventh grade students
of junior high school as the non-cognitive skill.

2. Grand Theory
Non-cognitive attributes are those academically and occupationally
relevant skills and traits that are not specifically intellectual or analytical in nature
(Rosen et al., 2010). They include a range of personality and motivational habits
and attitudes that facilitate functioning well in school. Non-cognitive traits, skills,
and characteristics include perseverance, motivation, self-control, and other
aspects of conscientiousness (Borghans et al. in Rosen et al., 2010). Although they
develop throughout childhood, non-cognitive attributes developed during
adolescence have been shown to have a significant and lasting impact on success
in life (e.g., Rauber in Rosen et al., 2010).
Poor non-cognitive attribute and skill development may accumulate over
time; poor skills and habits developed and internalized early on may lead to less
desirable educational and economic outcomes in adulthood, compounding
cognitive or academic problems. Non-cognitive attributes and skills may play an
important role in reversing or limiting delays or deficiencies in cognitive
development and academic achievement, and they may complement direct efforts
to improve academic learning. Non-cognitive skills research is a relative
newcomer to the study of factors affecting student achievement and attainment.
Non-cognitive traits and behaviors, however, might be as important asor even
more important thancognitive skills in determining academic and employment
outcomes (Heckman et al. in Rosen et al., 2010). For example, Jencks and his
colleagues in Rosen et al. (2010) found that noncognitive skills such as study
habits, industriousness, and perseverance show almost identical effect sizes as
cognitive skills on occupational attainment.
Elements of self-discipline measured through commonly used teacher,
parent, and child self-report instruments have outperformed IQ test scores in
predicting grades and attainment, in addition to accounting for achievement
differences between boys and girls with similar IQ levels (Duckworth & Seligman
in Rosen et al., 2010). Given the growing evidence that non-cognitive skills can
affect academic outcomes, as well as these continuing debates, education
researchers, practitioners, and policy makers would benefit from knowing which
non-cognitive skills matter for various educational outcomes and how to go about
defining and measuring those skills. This paper attempts to discuss one of the non
cognitive skills and develop an instrument item to measure the non cognitive
skills of seventh grade of junior high school students.
Collaborating with the Spencer Foundation, Rosen et al. (2010) identified
seven non-cognitive skills and traits to investigate:
1. motivation
2. effort
3. self-regulated learning
4. self-efficacy
5. academic self-concept
6. antisocial and prosocial behavior
7. coping and resilience
Motivation, effort, and self-regulated learning all pertain to developing problem-
solving strategies in ways that are oriented toward academic success. Self-efficacy
and academic self-concept studies focus on confidence and belief in ones ability
to succeed academically. Antisocial behavior and coping are actions and attitudes
that are not directly focused on academic achievement, yet they still may
influence academic success. Among the non cognitive skills above, motivation is
chosen to be discussed further in this paper.
Motivation is found substantial debate about the components of motivation
but general concordance in the effects of motivation on academic achievement.
Broadly defined, motivation in educational settings (i.e., achievement motivation)
is a desire to accomplish academic activities successfully. Evidence provided in
the reviewed articles supported the conclusions that intrinsic motivation, high
expectations of success and high task value, and mastery goal orientations all were
related to higher tested achievement, educational attainment, and other
academically favored outcomes like effort and engagement. At the same time, the
diversity of perspectives and the measures they rely on make evaluating the size
of motivations influence and its role vis--vis other theories and other non-
cognitive skills difficult to ascertain.
Motivation is a core psychological concept that has a long research
history. Besides an extensive literature on general motivation in the psychological
research literature, the educational research literature has produced a substantial
body of work on achievement motivation (i.e., motivational processes involved in
academic outcomes). Motivation and its attendant concepts involve other non-
cognitive factors such as engagement, effort, and self-efficacy in a complex
process that interrelates background factors, immediate social contexts, and
individual behavior.
In order to understand achievement motivation and its connection to
academic outcomes is distinguishing the several theoretical traditions that diverge
on the question of which aspects of motivation to study (Rosen et al., 2010).
These traditions share some common elements, but they place special emphasis on
their own conceptualizations of motivation. Achievement motivation can be
described simply as the desire to obtain academic success. In this framework,
achievement motivation is a more specific definition of motivation generally, and
both are understood as desires to accomplish or do well on a task. This desire is
understood as both a cognitive and an affective orientation; that is, it includes
either instrumental judgments or beliefs about the personal or social value of a
task or goal and emotional attitudes and reactions to the task. This desire to
achieve is also recognized to vary according to social situation and individual
experiences; it may be prompted or induced by circumstances, or it may be a
relatively stable orientation to a domain of activity (Hidi & Harackiewicz, 2000;
Weiner, 1990 in Rosen et al., 2010).
Within this broad framework, different traditions focus on particular
aspects of achievement motivation and connect motivation to certain other
attitudes and perceptions. These traditions include intrinsic/extrinsic theories,
expectancy-value theory, and achievement goal theory. Across these theoretical
traditions, researchers describe motivation as deriving from two basic sources:
interest in or enjoyment of a task or goal itself, and the value of external rewards
attached to the task or goal.
The first theoretical tradition focuses on these two elemental factors
themselves, identifying them as intrinsic and extrinsic motivation, and serves as a
foundation for subsequent theories. The second tradition, expectancy-value
theory, describes both intrinsic and extrinsic motivation as task values and
incorporates them into a model that includes self-efficacy. The third tradition,
achievement goal theory, parallels the intrinsic/extrinsic distinction by
investigating mastery and performance goalsmore specifically, academic and
cognitive conceptions of motivation.
In terms of specific measurement tools, most of the studies reviewed in
Rosen et al. (2010) (40 articles) used student reports of motivation from
questionnaires. A directly administered questionnaire is given to a group of people
assembled at a certain place for a specific purpose which in this case the
questionnaire is administered to the seventh grade students of junior high school.
According to Ary et al. (2010), the main advantage of directly administering
questionnaires is the high response rate, which typically is close to 100 percent.
Other advantages are the low cost and the fact that the researcher is present to
provide assistance or answer questions. The disadvantage is that the researcher is
usually restricted in terms of where and when the questionnaire can be
administered. Also, when a population is limited (e.g., parents of freshmen in a
specifi c university), the results of the survey will be equally limited in terms of
generalizability. However, a multimode approach in which researchers use
combinations of these methods in the same study is quite common. In fact, Fowler
in Ary et al. (2010) states that mixing modes is one of the best ways to minimize
survey nonresponse, because it enables researchers to reach people who are
inaccessible via a single mode.
This paper is making available the method of questionnaire to measure
students motivation which is using attitude scales. Attitude scales use multiple
responsesusually responses to statementsand combine the responses into a
single scale score (Ary et al., 2010). Attitudes of individuals or groups are of
interest to educational researchers. An attitude may be defi ned as a positive or
negative affect toward a particular group, institution, concept, or social object.
The measurement of attitudes presumes the ability to place individuals along a
continuum of favorablenessunfavorableness toward the object. If researchers
cannot locate an existing attitude scale on their topic of interest, they must
develop their own scales for measuring attitudes. There are two types of attitude
scales mentioned in Ary et al. (2010): (1) summated or Likert (pronounced
Likert) scales and (2) bipolar adjective scales. The first type of attitude scales,
Likert scales, is used to administer the questionnaire in measuring the students
motivation in this paper.
The Likert scale (1932), named for Rensis Likert who developed it, is one
of the most widely used techniques to measure attitudes. A Likert scale (a
summated rating scale) assesses attitudes toward a topic by presenting a set of
statements about the topic and asking respondents to indicate for each whether
they strongly agree, agree, are undecided, disagree, or strongly disagree. The
various agree disagree responses are assigned a numeric value, and the total scale
score is found by summing the numeric responses given to each item. This total
score assesses the individuals attitude toward the topic.
A Likert scale is constructed by assembling a large number of statements
about an object, approximately half of which express a clearly favorable attitude
and half of which are clearly unfavorable. Neutral items are not used in a Likert
scale. It is important that these statements constitute a representative sample of all
the possible opinions or attitudes about the object. It may be helpful to think of all
the subtopics relating to the attitude object and then write items on each subtopic.
To generate this diverse collection of items, the researcher may find it helpful to
ask people who are commonly accepted as having knowledge about and definite
attitudes toward the particular object to write a number of positive and negative
statements. Editorial writings about the object are also good sources of potential
statements for an attitude scale.
For pilot testing, the statements, along with five response categories
arranged on an agreementdisagreement continuum, are presented to a group of
subjects. This group should be drawn from a population that is similar to the one
in which the scale will be used. The statements should be arranged in random
order so as to avoid any response set on the part of the subjects. The subjects are
directed to select the response category that best represents their reaction to each
statement: strongly agree (SA), agree (A), undecided (U), disagree (D), or
strongly disagree (SD). There has been some question regarding whether the
undecided option should be included in a Likert scale. Most experts in the field
recommend that the researcher include a neutral or undecided choice because
some respondents actually feel that way and do not want to be forced into
agreeing or disagreeing.
To score the scale, the response categories must be weighted. For
favorable or positively stated items, strongly agree is scored 5, agree is scored 4,
undecided is scored 3, disagree is scored 2, and strongly disagree is scored 1. For
unfavorable or negatively stated items, the weighting is reversed because
disagreement with an unfavorable statement is psychologically equivalent to
agreement with a favorable statement. Thus, for unfavorable statements, strongly
agree would receive a weight or score of 1 and strongly disagree a weight of 5.

3. Conceptual Definition and Operational Definition
3.1 Conceptual Definition
Non-cognitive attributes are those academically and occupationally
relevant skills and traits that are not specifically intellectual or analytical in nature
(Rosen et al., 2010). Collaborating with the Spencer Foundation, Rosen et al.
(2010) identified seven non-cognitive skills and traits: (1) motivation, (2) effort,
(3) self-regulated learning, (4) self-efficacy, (5) academic self-concept, (6)
antisocial and prosocial behavior, and (7) coping and resilience. Motivation,
effort, and self-regulated learning all pertain to developing problem-solving
strategies in ways that are oriented toward academic success. Self-efficacy and
academic self-concept studies focus on confidence and belief in ones ability to
succeed academically. Antisocial behavior and coping are actions and attitudes
that are not directly focused on academic achievement, yet they still may
influence academic success. Among the non cognitive skills above, motivation is
chosen to be discussed further in this paper.


3.2. Operational Definition
Non-cognitive attributes are those academically and occupationally
relevant skills and traits that are not specifically intellectual or analytical in nature
(Rosen et al., 2010). Collaborating with the Spencer Foundation, Rosen et al.
(2010) identified seven non-cognitive skills and traits: (1) motivation, (2) effort,
(3) self-regulated learning, (4) self-efficacy, (5) academic self-concept, (6)
antisocial and prosocial behavior, and (7) coping and resilience. Among the non
cognitive skills above, motivation is chosen to be discussed further in this paper.
Motivation is found substantial debate about the components of motivation but
general concordance in the effects of motivation on academic achievement.
Broadly defined, motivation in educational settings (i.e., achievement motivation)
is a desire to accomplish academic activities successfully. It is measured through
questionnaire using one type of attitude scales called Likert scales.

4. Items/Instrument
Student Survey of Motivation (Adapted from Motivation and Interest
Research Project - Student Survey, the University of Sydney)

Please indicate your responses to this survey on the answer sheet provided using
the following key:
Strongly
Disagree
Disagree Neutral Agree
Strongly
Agree
A B C D E
1 2 3 4 5

1. Considering the difficulty of this unit of study, the teacher, and my skills, I
think I will do well in this unit of study.
2. The most important thing for me right now is improving my overall grade
point average, so my main concern in this unit of study is getting a good
grade.
3. I want to do well in this unit of study because it is important to show my
ability to my family, friends, employer, or others.
4. I am confident I can learn the basic concepts taught in this unit of study.
5. The most satisfying thing for me in this unit of study is trying to
understand the content as thoroughly as possible.
6. I believe I will receive an excellent grade in this unit of study.
7. My most important goal in my studies is to prepare myself for entering a
career.
8. Im confident I can do an excellent job on the assignments and tests in this
unit of study.
9. Im confident I can understand the most complex material presented by the
instructor in this unit of study.
10. It is important for me to learn the course material in this unit of study.
11. Understanding the subject matter of this unit of study is very important to
me.
12. Im certain I can master the skills being taught in this unit of study.
13. Getting a good grade in this unit of study is useful for me to learn.
14. I expect to do well in this unit of study.
15. I have to be in higher education in order to do the sort of work I want to
do.

5. Index of Discrimination (ID)
After administering the attitude scale to a preliminary group of
respondents, the researcher does an item analysis to identify the best functioning
items. The item analysis typically yields three statistics for each item: (1) an item
discrimination index, (2) the percentage of respondents marking each choice to
each item, and (3) the item mean and standard deviation.
The item discrimination index shows the extent to which each item
discriminates among the respondents in the same way as the total score
discriminates. The item discrimination index is calculated by correlating item
scores with total scale scores, a procedure usually done by computer. If high
scorers on an individual item have high total scores and if low scorers on this item
have low total scores, then the item is discriminating in the same way as the total
score. To be useful, an item should correlate at least .25 with the total score. Items
that have very low correlation or negative correlation with the total score should
be eliminated because they are not measuring the same thing as the total scale and
hence are not contributing to the measurement of the attitude.
The researcher will want to examine those items that are found to be
nondiscriminating. The items may be ambiguous or double barreled (containing
two beliefs or opinions in one statement), or they may be factual statements not
really expressing feelings about the object. Revising these items may make them
usable. The item analysis also shows the percentage of respondents choosing each
of the five options and the mean and standard deviation for each item. Items on
which respondents are spread out among the options are preferred. Thus, if most
respondents choose only one or two of the options, the item should be rewritten or
eliminated. After selecting the most useful items as indicated by the item analysis,
the researcher should then try out the revised scale with a different group of
subjects and again check the items for discrimination and variability.
Below is the calculation of index of discrimination (D).
Index of Discrimination is a measure of how well an item is able to
distinguish between students who are knowledgeable and those who are not. The
possible range of the discrimination index is -1.0 to 1.0. The item can be
considered acceptable if the discrimination index is above or equal to 0.40.
However, there are some considerations regarding to the discrimination index belo
0.40.
Discrimination Index Decision
0.40 to 1.00 Accepted
0.30 to 0.39 Accepted with revision
0.20 to -1.0 Dropped
There are some steps as the procedure to measure discrimination index.
1) Rank the students test score from the highest to the lowest.
2) Separate 27% of the students at the top and the 27% at the bottom.
3) Total the number of correct answer of the upper group and the lower
group.
4) Calculate the discrimination index using the following formula.

L

Note: D : Discrimination Index
UG : total number of students who answered correctly in the upper
group
LG : total number of students who answered correctly in the lower
group
N : total number of students

Upper Group of the students are 10 students, can be calculated as follows:
Upper group = 36 27% = 9.72 = 10 students
Lower Group of the students are 10 students, can be calculated as follows:
Lower group = 36 27% = 9.72 = 10 students
No UG LG UG-LG D
1 50 30 20 2.22
2 38 14 24 2.67
3 40 15 25 2.78
4 41 14 27 3
5 40 26 14 1.56
6 38 27 11 1.22
7 40 27 13 1.44
8 38 25 13 1.44
9 40 21 19 2.11
10 43 23 20 2.22
11 40 23 17 1.89
12 36 25 11 1.22
13 33 25 8 0.89
14 34 16 18 2
15 32 14 18 2

6. Content Validity
Validity concerns the extent to which the scale really measures the attitude
construct of interest. It is often difficult to locate criteria to be used in obtaining
evidence for the validity of attitude scales. Some researchers have used
observations of actual behavior as the criterion for the attitude being measured.
This procedure is not often used because it is often difficult to determine what
behavior would be the best criterion for the attitude and also because it is
expensive.
One of the easiest ways to gather validity evidence is to determine the
extent to which the scale is capable of discriminating between two groups whose
members are known to have different attitudes. To validate a scale that measures
attitudes toward organized religion, a researcher would determine if the scale
discriminated between active church members and people who do not attend
church or have no church affiliation. A scale measuring attitudes toward abortion
should discriminate between members of pro-life groups and members of pro-
choice groups. By discriminate, we mean that the two groups would be
expected to have significantly different mean scores on the scale. Another method
of assessing validity is to correlate scores on the attitude scale with those obtained
on another attitude scale measuring the same construct and whose validity is well
established.
The content validity of the items are first analyzed using expert judgments.
In this case, there are two judges to judge the validity by using the following
table.
Criteria for content validity are as follows.
0.8 - 1 Very high
0.6 - 0.79 High
0.4 - 0.59 Moderate
Below 0.4 Low
The expert judgments of relevant and irrelevant test items made to assess
reading comprehension of seventh grade students of junior high school can be
seen in the following table.
Judge I
Judge II
Irrelevant Relevant
Irrelevant
(A) (B)
- -
Relevant
(C) (D)
- 1, 2, 3, 4, 5, 6, 7,
8, 9, 10, 11, 12,
13, 14, 15

Then, the result of content validity testing is analyzed by the formula of Gregory
and below is the calculation.
ontent alidity

.00
As the index of content validity is 1.00 and considered as very high, the multiple
choice test items made in this paper to assess reading comprehension of seventh
grade students of junior high school can be said as valid.

7. Empirical Validity
The instrument with non-dichotomy-scaled data is the test which the data
are in the gradation scale (e.g. a questionnaire with the scale of 1 5). Pearsons
Product Moment Correlation is used to measure the empirical validity of an
instrument with non-dichotomy-scaled data. r critical value (r
cv
) is used to draw
conclusion with (df = n-2). If r
obs
exceeds r
cv
(r
obs
> r
cv
), the item is valid.
The statements used in the present study are then analyzed empirically in
terms of the validity of the test using the formula below.
r
n(xy) (x) (y)
[n x

(x)

] [n y

(y)

]

Note:
x : score of respondent on the discovered item
y : total score of respondent
n : number of respondent

X r hitung
r tabel
(db= 34)
Decision
1 0.890657 0.3291 V
2 0.873144 0.3291 V
3 0.858828 0.3291 V
4 0.904195 0.3291 V
5 0.751701 0.3291 V
6 0.713502 0.3291 V
7 0.787648 0.3291 V
8 0.722137 0.3291 V
9 0.765035 0.3291 V
10 0.795436 0.3291 V
11 0.754931 0.3291 V
12 0.656499 0.3291 V
13 0.556737 0.3291 V
14 0.81815 0.3291 V
15 0.781806 0.3291 V
From the calculation above, it can concluded that the 15 statements used in
questionnaire to measure students motivation are considered valid.

8. Reliability
Reliability is a consistency of an instrument. An instrument with high
reliability tends to give a relatively same result at any given time the instrument is
used. Some statistical formula of reliability are Spearman-Brown Formula,
Kuder-Richadson Formula (KR), Hoyt Formula. Those formulas are specific to
the characteristic of the data. The more general formula was developed by
Cronbach called Alpha Cronbach Formula which can be used to both dichotomy
and politomy (non-dichotomy) type data.
The reliability of the test items used in this paper is analyzed using Alpha
Cronbach formula as the following because the data gathered is classified into
politomy (non-dichotomy) type data.

)
Note:
r : Reliability coefficient of instrument (cronbach alpha)
k : The number of test items

: Total variance of items

: The variance of the set of students total scores

Using the above formula, the reliability of the test items in this paper can be
calculated as follows.

(
.

.
)
. ( .)
. (.)
.
The instrument is said having high reliability if the coefficient is > 0.60 (Ghozali,
2002). Hence, the structure test items were considered to be reliable enough to
gather the data for the present study.

References:
Rosen, Jeffrey A.; Glennie, Elizabeth J.; Dalton, Ben W.; Lennon, Jean M.;
Bozick, Robert N. 2010. Noncognitive Skills in the Classroom: New
Perspectives on Educational Research. USA: RTI International.


Ary, Donald; Jacobs, Lucy Cheser; Sorensen, Chris; Razavieh, Asghar. 2010.
Introduction to Research in Education. Eighth Edition. Wadsworth:
Cengage Learning.

Motivation and Interest Research Project - Student Survey, the University of
Sydney.