Académique Documents
Professionnel Documents
Culture Documents
Evaluation
Prof Keithia Wilson
Academic Leader Student
Success & Retention
Importance of
Evaluation
Kirkpatrick (1994)
Model of Evaluation Overview
4 levels of outcome evaluation
1.
2.
3.
4.
Level 1 Reaction
Level 2 Learning
Level 3 Behaviour
Level 4 Results
Level 1 - Reaction
Measures participants reactions to the training program,
including:
Level 1 - Reaction
Level 1 - Reaction
Some FYE examples
Starting@Griffith Survey questions on
Orientation (retrospective student
survey)
School specific Orientation Program
evaluation at the end of O-Day & the
Common Time (ongoing Orientation
Program) student surveys, focus groups
Peer Mentoring Program final evaluation
by mentors & mentees
Level 2 - Learning
Measures what participants have learned from
involvement in the program
What is measured needs to relate to what
was covered in the program viz. learning
objectives
Typically covers knowledge, skills, or
attitudes
Needs to include both rating scales & openended questions
Can include self-report & tests of actual
knowledge
Level 2 - Learning
Some examples
1.
Student self-report
* I feel confident that I know what to do to be successful
academically
this semester
not at all confident
very confident
1
2
3
4
5
6
7
* What have you learned about the transitions you will need to
make as
a commencing student in order to be successful?
2. Student knowledge
* I know the name of my First Year Advisor
yes
no (circle one)
* I know the difference between a lecture and a tutorial (circle one)
Strongly Disagree | Disagree | Neutral | Agree | Strongly Agree
Level 2 - Learning
Some FYE examples
Starting@Griffith Survey data evaluates
learning outcomes in terms of knowledge &
attitudes (e.g., knowing where to go to get
help; perceptions of good teaching)
Outcome evaluation for Peer Mentees (What
are your 3 most important learnings from your
participation in the Peer Mentoring Program?)
First-Assessment-First-Feedback amber alert
intervention (What have you learned as a
result of your participation?)
Level 3 - Behaviour
Measures the transfer of knowledge, skills &
attitudes from the training context to in vivo
or real-life contexts (the classroom & study).
Evaluate both before & after the program if
practical
Use survey, focus groups, interviews with
students, mentors, staff (convenors,
sessionals)
Starting@Griffith Survey (Did you attend
Orientation? How much time are you
spending on study?)
Level 3 - Behaviour
Some FYE examples
Starting@Griffith survey data
Attendance at E-Day, O-Day &
Common Time/ASP Program
Attendance at lectures & tutes
Assessment submission rates
in courses
Level 4 - Results
Measures return-on-investment, or the
extent to which the program has produced
results.
Some FYE examples include
Hard student outcomes such as * pass rates in courses for assessment items
* GPA in semester 1, 2 & FY overall
* student retention into year 2
Soft student outcomes such as
* student satisfaction
Evaluation
Key question What difference do our FYO&E
interventions (e.g., Orientation, Peer
Mentoring, Attendance, First Assessment
Intervention etc.) make overall?
Program/Intervention1 Reactions2
Learning 3 Behaviour 4 Overall
Productivity/Result
Program/InterventionFeelKnowDowhat
is the result or difference overall?