Ch.

6: Reliability and
Validity in Measurement
and Research

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

Validity and Reliability

Validity: How well does the

measure or design do what it
purports to do?
Reliability: How consistent or
stable is the instrument?

Is the instrument dependable?

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

Random and Systematic

Error

Random Error: Chance fluctuations

Tend to cancel out over repeated

measurements.

Systematic Error: Fluctuations that

are slanted in a particular
direction.

Also known as bias

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

Types of Reliability

Test-Retest Reliability: Degree of

temporal stability of the instrument.

Assessed by having instrument completed by

same people during two different time
periods.

Alternate-Forms Reliability: Degree of

relatedness of different forms of test.

Used to minimize inflated reliability

correlations due to familiarity with test items.

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

Types of Reliability (cont.)

Internal-Consistency Reliability:
Overall degree of relatedness of all
test items or raters.

Also called reliability of components.

Item-to-Item Reliability: The reliability

of any single item on average.

Judge-to-Judge Reliability: The reliability

of any single judge on average.

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

Spearman-Brown Formula

Used to estimate the internal consistency

of a test or judges overall.
nr
SB
ii
The formula is: R

1 n 1 rii

Where:
RSB = the overall internal consistency reliability
n = the total number of items in the test
r = the average intercorrelation among the
ii
items or among the judges.
Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

Reliability, Replication, and

External Validity

External Validity: The generalizability

of an inferred causal relationship.
The dependability of causal
generalizations is based on replicable
findings.

But how know if have replicated a

finding?
Role of the effect size.

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

Validity in Test and

Instrument Construction

Content Validity: Is the relevant

material adequately sampled?
Criterion Validity: How well does
the test correlated with outcome
criteria?

Concurrent Validity
Predictive Validity

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

Construct Validity in Test

Development

Construct Validity: What does the

test really assess?

Does the test have the ability to

discriminate?

Establishing construct validity

Convergent validity
Discriminant validity

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

Validity and Causal

Inference in Experimental
Design

External Validity
Construct Validity: The validity of the
hypothetical idea linking the IV and
DV.
Statistical Conclusion Validity: Are the
statistical conclusions well-grounded?
Internal Validity: Ability to rule out
plausible rival hypotheses.

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall