Factors that Affect Validity of Classroom Assessments (Nitko, 1996):

Content Representativeness and Relevance Does my assessment procedure emphasize what I have taught? Do my assessment tasks accurately represent the outcomes specified in my school's or state's curriculum framework? Are my assessment tasks in line with the current thinking about what should be taught and how it should be assessed? Is the content in my assessment procedure important and worth learning?

Thinking Processes and Skills Represented

Do the tasks on my assessment instrument require students to use important thinking skills and processes? Does my assessment instrument represent the kinds of thinking skills that my school's or state's curriculum framework states are important? Do students actually use the types of thinking I expect them use on the assessment to complete the assessment? Did I allow enough time for students to demonstrate the type of thinking I was trying to assess?

Consistency with other Classroom Assessments

Is the pattern of results in the class consistent with what I expected based on my other assessments of them? Did I make the assessment tasks to difficulty or too easy for my students?

Reliability and Objectivity

Do I use a systematic procedure for obtaining quality ratings or scores from students' performance on the assessment? Does my assessment instrument contain enough tasks relative to the types of learning outcomes I am assessing?

Fairness to Different Types of Students

Do you word the problems or tasks on your assessment so those students with different ethnic and socioeconomic backgrounds will interpret them in appropriate ways? Did you modify the working or the administrative conditions of the assessment tasks to accommodate students with disabilities or special learning problems? Do the pictures, stories, verbal statement, or other aspects of my assessment procedure perpetuate racial, ethnic, or gender stereotypes?

Economy, Efficiency, Practicality, Instructional Features

Is the assessment relatively easy for me to construct and not too cumbersome to use to evaluate students? Is the time needed to use this assessment procedure better spent on teaching students instead? Does your assessment procedure represent the best use of your time?

Multiple Assessment Usage

Are the assessment results used in conjunction with other assessment results?

Factors which can influence reliability, validity and practicality

Dr. R. Green, Aug 2006

EUROPOS SJUNGA

Test [1]
quality of items number of items difficulty level of items level of item discrimination type of test methods number of test methods

10

Test [2]
time allowed clarity of instructions use of the test selection of content sampling of content invalid constructs

Dr. R. Green, Aug 2006

11

Test taker
familiarity with test method attitude towards the test i.e. interest, motivation, emotional/mental state degree of guessing employed level of ability

Dr. R. Green, Aug 2006

12

Test administration
consistency of administration procedure degree of interaction between invigilators and test takers time of day the test is administered clarity of instructions test environment light / heat / noise / space / layout of room quality of equipment used e.g. for listening tests
Dr. R. Green, Aug 2006 13

Scoring
accuracy of the key e.g. does it include all possible alternatives? inter-rater reliability e.g. in writing, speaking intra-rater reliability e.g. in writing, speaking machine vs. human

Dr. R. Green, Aug 2006

14

How can we measure validity?

According to Henning [1987] non-empirically, involving inspection, intuition and common sense empirically, involving the collection and analysis of qualitative and quantitative data

Dr. R. Green, Aug 2006

15

Construct validity evidence is usually obtained through such statistical analyses as factor analysis [looks for items which group together], discrimination; also through retrospection procedures Content validity this type of validity cannot be measured statistically; need to involve experts in an analysis of the test; detailed specifications should be drawn up to ensure the content is both representative and comprehensive

Dr. R. Green, Aug 2006

16

Predictive validity can be determined by investigating the relationship between a test taker's score e.g. on IELTS/TOEFL and his/her success in the academic program chosen problem - other factors may influence success e.g. life abroad, ability in chosen field, peers, tutors, personal issues, etc.; also time factor element
Dr. R. Green, Aug 2006 17