Types of Validity

Criterion Validity Predictive Validity Concurrent Validity

Content Validity

Construct Validity

Convergent Validity

Discriminant Validity

Adapted from Sekaran, 2004

Types of Validity

Construct Validity
Extent to which hypotheses about construct are supported by data 1. Define construct, generate hypotheses about
constructs relation to other constructs 2. Develop comprehensive measure of construct & assess its reliability 3. Examine relationship of measure of construct to other, similar and dissimilar constructs

E.g. Height & Weight; Networking & career outcomes study; Learning Style Orientation measure

Establishing Construct Validity

Multi-trait multi-method matrix

Convergent validity coefficient
Absolute size of correlation between different measures of the same construct should be large & significantly different from zero

Discriminant validity coefficient Correlations between the same construct measured by

different methods should be significantly different from correlations of
Different constructs measured by different methods (across methods & across constructs) Different constructs measured by same method (method bias)

Convergent & Discriminant Validity

Corr b/w Objective (O) & Self-Reports (SR) of Height & Weight

O-H
O-H 1.00

SR-H

O-W

SR-W

SR-H
O-W SR-W

.98
.55 .68

1.00
.56 .69 1.00 .92 1.00

Convergent & Discriminant Validity of Objective & Self-Reports of Height & Weight

Convergent validity (across methods) Objective and subjective measures of height are
correlated .98 Objective and subjective measures of weight are correlated .92

Discriminant Validity of Objective & Self-Reports of Height & Weight Discriminant validity (across constructs) Objective measures of height & weight are corr .55 Subjective measures of height & weight are corr.69
STRONG CASE: Are the correlations b/w the same construct measured by different methods significantly different from corr b/w different constructs measured by same methods

i.e., Are .92 & .98 significantly different from .55 & .69? Convert rs to z scores and compare

Discriminant Validity of Objective & Self-Reports of Height & Weight Discriminant validity (across constructs) Objective height & subjective weight are corr .68 Subjective height & objective weight are corr .56
WEAK CASE: Are the correlations b/w the same construct measured by different methods significantly different from corr b/w different constructs measured by different methods

i.e., Are .92 & .98 significantly different from .56 & .68

Establishing Construct Validity in Networking Study For convergent validity, different measures of the same construct should be highly correlated
Note: In networking study, diff measures= diff subscales Corr b/w diff measures of career success (promotion, salary, perceived career success) range from .20 to .36 (moderate support) .20 & .36 should not be sig diff from each other Corr b/w diff measures of networking (increasing internal visibility, socializing etc.) range from .03 to .43 (weak support) .03 & .43 should not be sig diff from each other

Establishing Construct Validity in Networking Study

For discriminant validity, different measures

of the same construct should be more highly correlated than different measures of different constructs
Correlations b/w career success & networking (.01 to .35) should be sig different from Corr b/w diff measures of career success (.20 to .36) Corr b/w diff measures of networking (.03 to .43)

Learning Style Study

Developed items by generating critical

incidents (Study 1)
N=67 Yes/no responses to statements Recall of learning events Two types of learning: theoretical, practical (2) Two types of outcomes=success, failure (2) 2 x 2 events per participant 112 items constructed in total

Learning Style Study

Study 1 Part 2: Created 112 items from critical incidents

Administered to 154 participants
5-point likert scale (agree/disagree) Extracted 5 factor solution w/factor analyses 54 items loaded highly on the 5 factors Content validity: item sorting by 8 grad students Also administered personality scale

Learning Style Study Item Development Study (study 1)

Convergent Validity High reliabilities of subscales of Learning Style (.81-.91) Corr b/w different measures (subscales) of learning style
=.01 to.32 but 1 only corr is significant
Weak support for convergent validity of new learning style measure

Discriminant validity Corr b/w different measures of different constructs

(Learning Style & personality) .42 to .01 should be lower than and sig diff from corr b/w different measures of same construct (subscales of learning style) .01 to .32

Learning Style Orientation Measure

Validation Study (study 2)

N=350 -193 New learning style, Personality, old Learning style, preferences for instructional & assessment methods Construct validity Confirmatory factor analysis confirms 5
dimensions Reliability of new learning style subscales=.74 to .87 compared to
Reliability of old learning style subscales=.83 to .86 Reliability of personality subscales=.86 to .95

Learning Style Orientation Measure

Validation Study (study 2)

Convergent validity Corr b/w similar measures of key construct
=Corr b/w diff subscales of new learning style 01 to .23 should be comparable to
Corr b/w similar measures of other constructs in the study Diff subscales of old learning style .23 to .40 Diff subscales of personality .01 to .27

Learning Style Orientation Measure

Validation Study (study 2)

Discriminant validity Corr b/w measures of similar constructs= Corr
b/w new learning style subscales & old learning style = .01 to .31 Corr b/w measures of different constructs
Corr b/w new learning style & personality subscales is .01 to .55 Corr b/w old learning style & personality subscales= .02 to .38

Learning Style Orientation Measure

Validation Study
Incremental validity (aka construct validity) Additional variance explained by old vs. new learning style
measures in preferences for assessment & instruction

DV
Subjective assessment Interactional instruction

LSOM LSI (new) (old) .15 .01 .21 .04

Informational instruction

.06

.00

Types of Validity

Criterion Validity Predictive Validity Concurrent Validity

Content Validity

Construct Validity

Convergent Validity

Discriminant Validity

Adapted from Sekaran, 2004