Evaluation Plan

Stephanie Fong, Sakinah Hunafa, Jenna Jenkins, Spencer Landis

FRIT 7237: Evaluation of Educational Needs and Programs

Fall 2017
 Implementation Evaluation:

Implementation Data Sources

Evaluation
Questions

P1: Were the Initial Experience: Data regarding the initial experience could be
initial gathered through professional learning workshop observations. These
experience and observations would determine time accuracy with activities as well as
follow up accuracy of participants. They would ensure proposed objectives were
activities covered during the professional learning activities. Gathering
implemented as attendance information of teachers and staff would help to determine
planned? actual participants. A further survey could be done to determine this
with teachers confirming times and dates of workshop attendance
along with open-ended comments on regarding if changes occurred.
The “Summary of Project activities” table in the proposal details
activities and the dates chosen for completion. This can help
determine activity implementation as stated in regards to time and
content covered. Follow up Activities: Data for follow up activities
could be gathered through observation of the professional learning
workshops to determine that activities are going as scheduled. Data
for the electronic communication activities could be gathered through
survey to see if they were implemented as well as the meetings and
the Project Debrief meeting.

P2: What is the Initial program activities: This data can be collected through a focus
quality of the group of the participating teachers discussing the useful activities for
initial program construction and implementation of the module. Teachers would also
activities? discuss whether or not the activities provided enough practice using
the NSSM to incorporate them into their PBL modules correctly and
efficiently. Finally, they would discuss technology introduction for the
modules. A survey with specific closed and open-ended questions
regarding teacher response to the quality of the activities including
teacher opinion on future improvements would be useful too. Teachers
could rate communication between the staff and teachers during the
continued module creation. The survey could also determine whether
teachers felt the small group activities with other teachers had
sufficient participation to be helpful. Another question posed could be
to ask if teachers noticed the UBD model in use during the workshops
and presentations given. The October 2011 survey completed by the
Braxton County math teachers provides more data for a starting point
of teacher NSSM knowledge. This allows for a comparison of pre-
program knowledge and post professional learning knowledge.

P3: Who are The participants are math teachers in Braxton County, Georgia.
the program Braxton County was identified as an eligible system for the grant
participants program that focuses on the application of New State Standards for
and how were Mathematics in the creation and use of mathematics problem-based
they recruited? learning modules. Teachers will be recruited from a pool of educators
who will be teaching math at the elementary, middle, or high school
levels in Braxton county during the 2015-2016 school year. The
project will focus initially on educators teaching middle school grades,
but any mathematics teacher in Braxton County is welcome to
participate up to full capacity of 16 participants. Braxton County’s
School’s Director of Professional Learning, Ms. Carla Lawton, will
assist the Project Director in recruiting participants. Email and paper
mailing will be used to advertise the project. This data can be
collected through a teacher survey asking participants if they were
recruited or made aware of the opportunity to participate through
email, paper mail, or personal contact.

P4: What is the The follow up and support activities provide many opportunities for
quality of participants and faculty members to share ideas, ask questions, discuss
follow up and results, and provide feedback. Once PBL modules are finalized and
support implemented teacher participants will conduct self-evaluations of the
activities? implementation of modules and share the results with project staff
members. Great Southern University faculty members will conduct a
mid-implementation meeting with teacher participants to discuss
questions and concerns related to the implementation of the PBL
modules. There will also be a final meeting in which teacher-created
self-evaluations, implementation, and best practices will be discussed.
Following the workshop, participants and project staff will
communicate through electronic tools such as email and wikis. Wikis
allow for discussion of the project components and sharing of
information during the Fall semester when the modules are
implemented. When the project is finished, public areas of the wiki
will be used to share the PBL modules that were created with the
public. Interviews and/or surveys can collect data to determine if
participants felt well supported, if questions were answered in a timely
manner, and if the support activities were meaningful.
 Summative Evaluation:

Outcome Evaluation Outcome Evaluation Data Collection Activities

Questions

O1: To what extent Data to determine the usefulness and NSSM alignment of
were teachers able to PBL modules could be collected in the following ways:
develop PBL modules 1.The proposal stated that the PBL modules would be
that were connected to available for viewing on Wiki after completion. The modules
local business and can be viewed with a checklist of important aspects that must
industries, aligned with be included.
NSSM, and 2.Faculty communication with teachers to complete the
incorporated modules to ensure that they meet requirements along with
appropriate uses of rubrics.
technology? 3.Teacher interviews by the faculty members to ascertain how
(Objectives 1, 2, and 3) well teachers were able to develop their PBL modules.
Specific questions regarding NSSM alignment and technology
use will further assist in gathering data. This data will help
decide if changes should be made. Further questions regarding
teacher understanding of PBL will determine any possible
workshop modifications to improve teacher knowledge..
Finally, a survey can be conducted to ask questions regarding
Wiki use for the PBL modules. These questions will help to
determine whether teachers came upon some issues with the
site such as ease of use or the technology including Internet
connectivity issues. This data will help to see if Wiki should
be used in future as well as the technology incorporated.

O2: To what extent Teacher participants will implement their finalized PBL
were teachers able to modules with their students and conduct a self- evaluation of
implement and the implementation. Self-evaluation results and student work
evaluate those will be used to evaluate the success of the PBL module and
modules? (Objective 4) identify areas for improvement.

Objective Indicator Data Source

1. Create PBL Modules draw on content and -Post-Project Survey

Modules based on processes from local -Observation/Participation in
local businesses businesses and industries Professional Learning Workshops
and industries

2. Create PBL Module content, tasks, and -Observation/Participation in

Modules assessments are aligned with Professional Learning Workshops
addressing NSSM appropriate NSSM -Wiki Checklist, Self-Evaluation
3. Integrate PBL modules contain -Post-Project Survey
technology into activities that effectively use -Wiki Checklist, Self-Evaluation
PBL experiences technology -Observation/Participation in
Professional Learning Workshops

4. Implement and Record of implementation -Post-Project Self-Evaluation

evaluate PBL Record of self-evaluation -Wiki Checklist, Self-Evaluation
Modules

Data Collection Schedule/Summary:

Both qualitative and quantitative data collection instruments were used resulting in a
mixed-methods design. Quantitative surveys were administered to teacher participants
before the project started and also at the conclusion of the project in late 2015 or early
2016. Surveys were used to collect data in the areas of how participants were selected,
level of support available to participants, self-evaluations, and ease of technology use in
creating PBL modules. Qualitatively, mid-implementation meetings were held with
teacher participants as well as a final meeting. These meetings discussed areas including
implementation questions/concerns, self-evaluation, level of implementation support, and
best practices. Additionally, faculty observed and participated in the professional
learning workshop where the design of modules occurred in order to ensure local
businesses were consulted and well-established instructional design models were used.

Data set Date of Instruments Data collected by:

collection already
developed?

Self-Evaluations of Fall 2015 or Yes Great Southern Faculty

Implementation Early 2016

Mid- Implementation Meeting Fall 2015 Yes Great Southern Faculty

Final Meeting Fall 2015 or Yes Great Southern Faculty

Early 2016

Teacher Perceived Needs and October 2011 Yes Great Southern Faculty
Interest Survey

Pre-Project Survey Mon, June 25 No Great Southern Faculty

Post-Project Survey Early 2016 No Great Southern Faculty

Observations and Participation Mon, June 25- No Great Southern Faculty

in Professional Learning Friday, June
Workshops 29