PROGRAM EVALUATION

GNRS 5311 MODULE 4

Meredith French, BSN

Debra Morris, BSN
Anne Nance, BSN
Chlo Williams, BSN

OBJECTIVES
Define program evaluation in nursing education.
Discuss theory-driven and method-driven evaluation
methods.
Understand how the complexity theory drives a program
evaluation.
Describe the elements necessary to evaluate each
component of the evaluation plan.
Analyze the specific criteria determined by the Commission
on Collegiate Nursing Education (CCNE).

PROGRAM EVALUATION
Required for accreditation and approval by State
Board of Nursing (Pross, 2010)
Driven historically by the accreditation process

(Sauter

et al., 2012)

Link to and integration of accreditation programs are

still in use (Kalb, 2009; Escallier & Fullerton, 2012)
Key relationships of elements can be overlooked

PROGRAM EVALUATION
THEORIES
Underlying theoretical foundation
Method-oriented versus theory-driven
Method-oriented approaches have been popular
et al., 2012)

Method-oriented ignore views and concerns of

stakeholders (Chen, 2012)
Theory-driven models should be considered

(Sauter

THEORY-DRIVEN PROGRAM
EVALUATION
Tests correct program theory and successful
implementation
More scientifically sound, with causal explanations
and involved mechanisms (Coryn et al., 2011)
Information generated improves existing or future
programs (Chen, 2012)
Chens theory-driven model used for nursing
programs (Sauter et al, 2012)

THEORY-DRIVEN PROGRAM
EVALUATION
Evaluation-Nursing Education
Chens Evaluation
Programs

Normative outcome evaluation

Develop program theory
Define activities to achieve goals
Explain linkages between programs
Identify/prioritize goals & outcomes

Normative treatment evaluation

Evaluate congruency between
expected and implemented treatment

Mission & goal evaluation

Develop mission, philosophy &
framework
Identify program goals & outcomes
Ensure program goals & outcomes
meet professional standards

Program evaluation
Evaluation of teaching
effectiveness

Adapted from Sauter, M.K., Gillespie, N.N., & Knepp, A. (2012). Educational program evaluation.
In D.M. Billings & J.A. Halstead (Eds.), Teaching in nursing (p 507-508).

THEORY-DRIVEN PROGRAM
Evaluation-Nursing Education
EVALUATION
Chens Evaluation
Programs

Implementation Environment
evaluation
Participant evaluation
Implementer dimension
Delivery mode dimension
Implementing organization
dimension

Environment Evaluation
Student dimension
Faculty dimension
Delivery mode dimension including
courses
Organization dimension including
leadership

Adapted from Sauter, M.K., Gillespie, N.N., & Knepp, A. (2012). Educational program evaluation.
In D.M. Billings & J.A. Halstead (Eds.), Teaching in nursing (p 507-508).

THEORY-DRIVEN PROGRAM
EVALUATION
Evaluation-Nursing Education
Chens Evaluation
Programs

Implementation Environment
evaluation
Inter-organizational relationships
Impact evaluation
Intervening mechanisms
Generalization evaluation

Environment Evaluation
Inter-organizational relationship
Outcomes assessment
Intervening mechanisms
Generalization evaluation

Adapted from Sauter, M.K., Gillespie, N.N., & Knepp, A. (2012). Educational program evaluation.
In D.M. Billings & J.A. Halstead (Eds.), Teaching in nursing (p 507-508).

COMPLEXITY THEORY
Theory-driven evaluation
method
Less rigid approach
Looks at the program in its
entirety
(Frye & Hemmer, 2012)

COMPLEXITY
THEORY

Ever-changing knowledge and

practice patterns, therefore nursing
programs are complex systems
A program as a whole is greater
than the sums of its parts
Allows educators to
structure useful program evaluation that
accommodate a programs true
complexity
avoid an overly narrow approach

Provides a useful perspective for

choosing an evaluation theory that

COMMISSION ON COLLEGIATE NURSING

EDUCATION (CCNE)
Specific criteria for evaluation:
Baccalaureate
Masters
Doctor of Nursing Practice (DNP)
Post-graduate Advanced Practice Nurse
Self-study
Standards for accreditation

EVALUATION COMPONENTS
Purpose of program evaluation
Mission
Goals
Outcomes
Implementation
Commitment
Resources
Curriculum/Teaching-Learning

UNIVERSITY OF TEXAS
MEDICAL BRANCH

CCNE
Required documentation
Standards and guidelines
Official documentation
Appointment, promotion, and tenure
Unit reports
Official correspondence
Handbooks, manuals, calendars, etc.
Advertising

CCNE: RESOURCES
Budget/Resources

Degree Requirements

Nursing personnel

Full-time faculty

Location space

Student-ratio

Equipment/Supplies

Supporting documentation

Budget
Degree/certification/experience
CV
Bargaining/policies/documents

CCNE: TEACHING-LEARNING
Curriculum
Objectives
Outcomes
Nursing
standards/guidelines
Teaching-learning
Environment
Competencies

Required documentation

Syllabi
Didactic/clinical learning
Student performance
Evaluations
Affiliation agreements
Documents
Assignments
Formal data and licensure

CCNE: ASSESSMENT
Assessment and
achievement

Required documentation

Quantitative/qualitative

Completion rate (each degree)

Completion rates

Faculty outcomes

Licensure/certification rates

Formal complaints

Employment rates

Documents

Formal complaints

Data

CONCLUSION

REFERENCES
Chen, H. T. (2012). Theory-driven evaluation: conceptual framework, application and advancement.
In R. Strobl, O. Lobermeier, & W. Heitmeyer (Eds.), Evaluation von programmen und projekten fur
eine demokratische kultur (pp. 17-40). Weisbaden: Springer. doi:10.1007/978-3-531-19009-9_2
Commission on Collegiate Nursing Education (2013). Retrieved from
http://www.aacn.nche.edu/ccne-accreditation/Standards-Amended-2013.pdf
Coryn, C.L.S., Noakes, L.A., Westine, C.D., & Schroter, D.C. (2011). A systematic review of theorydriven evaluation practice from 1990 to 2009. American Journal of Evaluation, 32(2), 199-226. doi:
10.1177/1098214010389321
Doll Jr, W.E. & Trueit, D. (2010). Complexity and the health care professions. J Eval Clin Pract. 16(4),
841-848.
Escalier, L.A., & Fullerton, J.T. (2012). An innovation in design of a school of nursing evaluation
protocol. Nurse Educator, 37(5), 187-191. doi: 10.1097/NNE.0b013e318262eb15
Frye, A.W., & Hemmer, P.A. (2012) Program evaluation models and related theories: AMEE guide no.
67. Medical Teacher, 34, 288-299. Doi:10.3109/0142159X.2012.668637
Kalb, K.A. (2009). The three Cs model: the context, content, and conduct of nursing education.
Nursing Education Perspectives, 3(30), 176-180.
Sauter, M.K., Gillespie, N.N., & Knepp, A. (2012). Educational program evaluation. In D.M. Billings &
J.A. Halstead (Eds.), Teaching in nursing (pp. 503-549).