Running head: STAR E.L.

PROGRAM EVALUATION 1

Evaluating the Effectiveness and Utilization of the STAR Early Literacy Initiative

Bianca M. Cheatham

University of West Georgia

STAR E.L. PROGRAM EVALUATION 2

Table of Contents

Description of Program…………………………………………………………………………3

Stakeholders.……………………………………………………………………………………4

Logic Model.……………………………………………………………………………………5

Purpose of Evaluation.………………………………………………………………………….5

Evaluation Questions and Hypotheses.…………………………………………………………6

Variables and Constructs of Interest.…………………………………………………………...7

Proposed Methodology.………………………………………………………………………...7

Participants.……………………………………………………………………………………..7

Design and Methodology.………………………………………………………………………9

Procedure.………………………………………………………………………………………9

Timeline for Evaluation.………………………………………………………………………..11

Proposed Analyses.……………………………………………………………………………..12

Evaluation Budget.……………………………………………………………………………...13

Evaluator’s Qualifications.……………………………………………………………………...14

Dissemination Plan.……………………………………………………………………………..15

Appendix A: STAR Early Literacy Initiative of Lorraine Elementary Logic Model…………...17

Appendix B: Letter to Participants and Consent………………………………………………..18

Appendix C: STAR Early Literacy Assessment Program Evaluation Initial Survey………….20

Appendix D: STAR Early Literacy Assessment Program Evaluation Final Survey…………..22

Appendix E: Teacher Interview Questions…………………………………………………….25

Appendix F: Administrator Interview Questions………………………………………………27

STAR E.L. PROGRAM EVALUATION 3

Description of the STAR Early Literacy assessment

The STAR Early Literacy assessment is a component of the Renaissance 360 testing program

(Renaissance Learning, Inc., 2018). It is utilized with learners in kindergarten, first, and second

grades at Lorraine Elementary School and other elementary schools within the Rockdale County

Public Schools district. The STAR Early Literacy screener tracks student progress within the

following specific skill-based categories alphabetic principle, concept of word, visual

discrimination, phonemic awareness, phonics, structural analysis, vocabulary, sentence level,

paragraph level comprehension, and early numeracy (Renaissance Learning, Inc., 2018). A grade

equivalency, reading level, and student growth percentile are also scored within the assessment.

It has been in use for the past six years. As the state’s evaluation system has shifted from

achievement-based to growth-based, Lorraine Elementary School has developed a plan to

administer the Renaissance 360 assessments, every six weeks to the students within our building.

This is a significant increase from the typical, district-required, quarterly screeners the learners

have participated in within the past. Along with the assessments, teachers are being required to

track the student shifts in grade equivalency, reading level, and student growth percentile scores.

All students will be monitored to ensure growth is made within the program. Data briefing

meetings are conducted with teachers, along with the RTI coordinator, and the administration

team.

The objective of this proposal is to evaluate how well the program is working on measuring

students' knowledge and understanding. Additionally, the STAR Early Literacy assessment was

selected due to a problem that the school is currently experiencing. Many teachers have shared

that they felt their young learners’ score did not match their present levels of performance. As
STAR E.L. PROGRAM EVALUATION 4

many kindergarteners and even first-grade students lack maturity, many do not fully comprehend

the importance of these online screeners. If so much weight and time are being invested in these

assessments to track learning growth, evaluate how often the STAR Early Literacy assessment

matches what the teachers see in their classrooms becomes a necessary measure to provide

valuable information to both the administration team and teaching staff within the building.

Stakeholders

The stakeholders for this program include all kindergarten, first, and second-grade

students and teachers at Lorraine Elementary, along with the school administrators and Response

to Intervention (RTI) Coordinator. Every one of these learners participates in these assessments

multiple times throughout the school year. The STAR Early Literacy screener is used to progress

monitor their individual growth and development in grade-level standards and skills throughout

the academic year.

Each grade level teacher is held responsible for implementing the STAR Early Literacy

screener in their classroom every six weeks within the assessment window. Classroom teachers

are provided with ten iPads to use for instructional purposes. These devices are expected to be

used for small-group or individualized implementation of this assessment. Then, they are

supposed to analyze the data and results collected looking for areas of student growth, in addition

to strengths and weaknesses. The school administration team, including the assistant principal,

principal, and testing coordinator aid the educators in evaluating the data culminated from the

STAR Early Literacy screeners. They host meetings to discuss student progress and growth.

Additionally, the RTI coordinator supports teachers in implementing new strategies to use to

meet the needs of all learners better. This could be through acceleration or remediation teaching

techniques and accommodations.

STAR E.L. PROGRAM EVALUATION 5

Logic Model

See Appendix A.

Purpose of the Evaluation

The purpose of this evaluation is to assess the effectiveness level of the STAR Early

Literacy assessment at Lorraine Elementary School. Many teachers have voiced concerns about

the validity of the testing program. As teachers are required to administer this screener every six

weeks to students, in addition to tracking data trends and growth percentiles, it is vital to know

how the data collected from this assessment compares to classroom-based performance. Student

performance in running records, fluency checks, comprehension activities, classroom

assessments, and performance-based activities can all relate to a teacher’s overall measure of a

child’s learning. This evaluation is necessary to provide teachers and administrators within the

school building indication of wither or not the program is accurately measuring student learning.

This will allow them opportunities to compare the culmination of classroom-based observations

to the evidence provided through STAR Early Literacy reports. All stakeholders, including

students, teachers, administrators, and the RTI coordinator will benefit from this evaluation.

After speaking with the RTI coordinator, it was decided that this evaluation will rely

heavily on focus groups and surveys. As each grade-level will already be meeting in data-

briefing meetings to review student growth, these groups of individuals could form focus groups

that would be geared towards sharing the overall trends and effectiveness of the STAR Early

Literacy assessment as it pertains to individual classrooms. This type of evaluation would also fit

the focus of this proposal because it provides opportunities for adjustment for teaching strategies

and testing implementation. It is also a source of excellent feedback. Adding a survey

component to the study will provide a second layer of feedback. The survey questions will ask
STAR E.L. PROGRAM EVALUATION 6

educators to indicate which areas the STAR Early Literacy screener most accurately matches the

students’ classroom-assessment would be an excellent source of data collection to see how it

directly correlated to student learning.

Evaluation Questions

The goal of this evaluation would be to determine the overall level of effectiveness of the

utilization of the STAR Early Literacy assessment within the lower-grades at Lorraine

Elementary. This evaluation hypothesizes that the data collected through the use of the online

screener known as the STAR Early Literacy assessment within kindergarten, first, and second

grades will not directly align with in-class student performance. While some evidence collected

within the data will provide indicators of if the students are learning grade-level content, all areas

will not match teacher-created assessments and hands-on learning opportunities. Additionally,

this evaluation will focus on the investigation of the following questions.

1. How do students’ age and grade affect their performance on the STAR Early Literacy

assessment?
2. Currently, what percentage of teachers indicate aligned assessment results between

student performance and the STAR Early Literacy assessment?

3. Will data-driven focus groups, comprising of grade-level teachers, administration

leaders, and the RTI coordinator result in positive educational changes for

instructional needs in classrooms?

4. How will data collected from each STAR Early Literacy assessment (given every six

weeks) affect individual teacher instruction?

Constructs
STAR E.L. PROGRAM EVALUATION 7

The primary constructs of interest identified within this evaluation relate to student

performance, the STAR Early Literacy screener, focus groups, differentiation, and classroom

assessments. Student performance is defined as the individual progression of learning or lack

thereof. The STAR Early Literacy screener is an online assessment, produced by Renaissance

360, that all kindergarten, first, and second-grade learners take every six weeks at Lorraine

Elementary School (Renaissance Learning, Inc., 2018). Focus groups concentrate on the

meetings, consisting of grade-level teachers, administration, and RTI coordinator to disseminate

data collected from each implementation of the STAR Early Literacy screener. The variety of

teaching strategies used to support a modality of learners within of classroom is known as

differentiation. Classroom assessments are defined as any formal or informal method teachers

use to evaluate student learning. Additional variables that will affect this evaluation are student

age and grade level.

Proposed Methodology

Participants

Participants in this evaluation include all kindergarten, first, and second-grade students at

Lorraine Elementary in addition to their homeroom teachers. Members from the administration

staff will also participate in the study. They are the principal, assistant principal, testing

coordinator, and RTI coordinator. The student scores on the six-week STAR Early Literacy

screeners will be used to determine the level of impact and overall effectiveness levels these

assessments are having on classroom instruction. Additionally, homeroom teachers will be able

to share how their classroom evaluations of students about unit tests, running records, and

reading levels compare to those estimated by the STAR Early Literacy assessments.
STAR E.L. PROGRAM EVALUATION 8

Recruitment for this evaluation process will take place through it promotion with the

school’s RTI coordinator and assistant principal. As the testing protocol for the use of all

Renaissance 360 products used at Lorraine Elementary has recently shifted from once a quarter

to every six weeks, many teachers are concerned about the over-testing of learners. They are

also voicing concerns that the products are not correlating to students’ classroom levels of

performance. The results of this evaluation could provide teachers with an increased sense of

security in using the assessments or help administration see that they are not as reliable as they

were once predicted to be. Since this change has significantly impacted teachers and students

within the building frequently, I feel as though many will be active participants. The data is

relevant to all kindergarten, first, and second-grade teachers, which is a vast population of staff

members within the building. As the evaluator is a first-grade teacher, she possesses a personal

relationship with the teachers at Lorraine Elementary, which will increase their motivation to

participate in the evaluation. All of these factors combined make external incentives unnecessary.

There are ethical considerations that must be considered within this study. The primary

one focuses on confidentiality. All student data collected from the STAR Early Literacy program

and teacher-provided lessons, must be kept confidential. Teacher and student names must not be

released. In the case that any particular classes need to be further examined pseudonyms must be

given. A second ethical consideration relates to teacher-provided data. As many of these

assessments are created and graded by the same individual, the scoring process could be

somewhat subjective when comparing scores across grade-level where multiple assessments are

utilized.

Design and Methodology

STAR E.L. PROGRAM EVALUATION 9

The research methods included in this evaluation combines the use of quantitative and

qualitative data and is referred to as a mixed-methods approach. The data within this evaluation

will be collected through the use of test scores, surveys, and focus groups. Since all of these

measures blend the use of quantitative and qualitative data, a mixed-methods approach has been

selected. The test scores received from the STAR Early Literacy screeners themselves will serve

as quantitative data for the study. Additionally, all kindergarten, first, and second-grade teachers

at Lorraine Elementary school will be asked to complete two online surveys as qualitative data.

The results collected from the first survey will help form three categories of individuals. They

include individuals who see major, some, or minor correlation between classroom assessments

and the STAR Early Literacy screener, before the formation of focus groups. Once complying

the data, I will be able to identify the educators, on each grade-level, who support the online

screeners and those who strongly oppose it. Being aware of these teachers’ strong emotions, like

allow me to keep a close eye on their sharing and data results.

Further discussions and findings will take place within each grade-specific focus group.

The conversations that take place in these focus-groups will serve as additional forms of

qualitative data. The second survey will also be used. This will identify how or if the focus

groups or data briefing meetings affected teacher instructional strategies thus resulting in positive

shifts in student performance. They will allow teachers the opportunity to share the number of

students whose STAR Early Literacy scores correlated within class performance.

Procedure

Before beginning the data collection process, the evaluator will discuss this proposal

with the administration team at Lorraine Elementary School. Data collection will start in October

of 2018, immediately following the school’s fall break. Once approval has been given, a formal
STAR E.L. PROGRAM EVALUATION 10

meeting will be conducted with all kindergarten, first, and second-grade teachers, explaining the

purpose of this evaluation and an introduction to the survey. Following the meeting, a letter

explaining the program evaluation (Appendix B) will be emailed out to the teachers along with a

link to the initial survey. Participation in the survey will be monitored for optimum results. A

follow-up email will also be sent thanking the first wave of teachers for their immediate response

and reminding others that there is still time to complete the survey.

The evaluator will meet with the RTI coordinator to STAR Early Literacy results from

July and September screenings. She will begin to analyze this data looking for trends and hot-

spots within classes and grade-levels. This analyzation process will continue as the STAR Early

Literacy screeners will be given to students on a six-week basis. The data will indicate positive

or negative changes in student growth towards grade-level literacy standards through the

calculation of grade equivalency, reading level, and student growth percentile.

Teachers will also participate in their first focus-group meeting, where they too will look

at their screener results, compare them to classroom performance, and plan instructional

interventions to meet student needs better. Focus-groups meetings will be held monthly. Minutes

are taken at each meeting, depicting the exact action plans that the grade-level discussed and the

students involved. These minutes will serve as excellent sources of data, as they will indicate the

changes made to instructional content in-between screeners.

Interviews with the RTI coordinator, Assistant Principal, and Principal will be held in

November of 2018, while teacher interviews will begin in January of 2019 and be completed

within February of 2019. The data obtained from these interviews will help identify teacher and

administrator perception of the program, in addition to how effective its utilization has been

within the classroom and school on an individual level.

STAR E.L. PROGRAM EVALUATION 11

Timeline for Evaluation

Table 2

Evaluation Timeline

Initial meeting with administration and kindergarten, first, and second

grade teachers to discussion evaluation process
October 2018
Approval of evaluation plan and ethics

Collect STAR Early Literacy assessment results from July and September
screenings

Have stakeholders participate in initial survey

Conduct first grade-level focus meetings

Analyze STAR Early Literacy assessment results from July and September
screenings

Analyze initial survey result

November 2018
Collect November STAR Early Literacy assessment results

Conduct second grade-level focus meetings

Analyze STAR Early Literacy assessment results from November

December 2018 Interview RTI coordinator, Assistant Principal, and Principal

Collect January STAR Early Literacy assessment results

January 2019 Conduct third grade-level focus meetings

Begin interviewing kindergarten, first, and second grade teachers

Analyze January STAR Early Literacy assessment results

February 2019 Conclude interviewing kindergarten, first, and second grade teachers
March 2019 Collect March STAR Early Literacy assessment results

Begin writing written report and preparing presentation

April 2019 Analyze teacher and administrative staff interview in each of trends. (

Continue writing written report and preparing presentation

May 2019 Collect and analyze May STAR Early Literacy assessment results

Have stakeholders participate in final survey

STAR E.L. PROGRAM EVALUATION 12

Calculate overall growth trends in STAR Early Literacy to student

performance in classroom standards

Edit written report and presentation

Dissemination: Presentation of final evaluation and findings to

administration team and then to faculty members
Proposed Analysis

The proposed analyses for this evaluation includes the use of a mixed-methods approach.

Quantitative measures will be used to analyze STAR Early Literacy, and closed-response survey

question results through descriptive statistics, resulting in standard deviation scores and

percentages. Qualitative means will be utilized to examine interview and closed-response survey

results through content analysis. The evaluator will review the data searching for frequency

counts on repetitive phrases and words. The evaluator will also support the utilization of clean

data by recording all interviews, preparing transcriptions, using Google Forms to create and

collect survey results, and ensuring that all STAR Early Literacy results are accurately monitored

and reported.

The evaluation of participants will be described in a variety of ways. As the surveys will

be created through the use of Google Forms, percentages, bar graphs, and pie charts will easily

be representative of the data collected. The additional data obtained from the frequency of

repetitive phrases and words will also be displayed within an informational table, as will the

correlational data between instructional strategies and impacts of STAR Early Literacy results.

When more explicitly addressing the evaluation questions, the focus led toward the use of either

quantitative or qualitative methods. The first and second evaluation questions relate to

quantitative research. The first is how do students’ age and grade affect their performance on the

STAR Early Literacy assessment? The data results for this question can be identified through the
STAR E.L. PROGRAM EVALUATION 13

descriptive statistical analysis of the student growth percentiles on the STAR Early Literacy over

time compared to other students. For example, one can evaluate if second-grade learners

achieved higher growth percentages, compared to others the same age in their grade level and

also compared to first and kindergarten students. The second evaluation question states currently,

what percentage of teachers indicate aligned assessment results between student performance

and the STAR Early Literacy assessment? Evidence to support this answer will be collected from

both the initial and final teacher survey results.

The next two questions rely on data composed of qualitative research methods. The third

question is will data-driven focus groups, comprising of grade-level teachers, administration

leaders, and the RTI coordinator result in positive educational changes for instructional needs in

classrooms? The minutes from the monthly focus group meetings and the response from all

personnel interviews will help identify how effective these groups were at enhancing

instructional opportunities for all learners. The fourth and final question is how will data

collected from each STAR Early Literacy assessment (given every six weeks) affect individual

teacher instruction? Like the third, the findings to this question heavily relies on qualitative

approaches and responses from staff members in surveys and interviews.

Evaluation Budget

This evaluation will be conducted free of charge, as it is the result of a school assignment.

It will be completed as a component of the evaluator’s graduate-level coursework pertaining to

her specialist’s degree. The full budget details can be located below in Table 3.

Table 3

Evaluation Budget

Budget Category Amount Description

Evaluation Fee $2,030 10 hours of interviews (0.5
STAR E.L. PROGRAM EVALUATION 14

hour x 16 teachers) at $35 per

Fees were calculated based on hour = $280
a 5-hour working week within
a 24-week period (totaling to 4 hours of interviews (1 hour x
be 2 semesters) 4 administrators) at $35 per
hour = $140

6 hours meetings with final

focus groups at $35 per hour =
$210

40 hours of document and

survey data collection at $35
per hour = $700

40 hours of data analysis &

report writing at $35 per hour
= $700

Office Supplies $150 Printed STAR Early Literacy

reports, copies of teacher
assessments, printed survey
results, additional office
supplies

Total $2,180
Evaluator’s Qualifications

The evaluator obtains a Dual Bachelor of Science degree in Early Childhood and Special

Education from the North Georgia College and State University, from which she also received a

reading endorsement. Additionally, she holds a Master of Education in Elementary Reading and

Mathematics from Walden University. Currently, the evaluator is in her final semester of

graduate school, focused on receiving a Specialist in Education Degree in Media with a

concentration in Instructional Technology. She has seven years of teaching experience and has

worked with a variety of learners. Previous experiences include special education, kindergarten

inclusion, and third grade. The evaluator is currently working as an inclusion first-grade teacher

in a 2:1 technological device environment at Lorraine Elementary School. Ten student iPads are
STAR E.L. PROGRAM EVALUATION 15

used on a daily basis in addition to four classroom desktops, an interactive whiteboard, and an

Apple TV.

The evaluator will be an inclusive member of the data collection process within this

evaluation, as her students also participate in the STAR Early Literacy assessments every six

weeks. She will join in the first-grade focus group, but will not be an active member within the

others. Her role as an educator within the building will allow for an excellent vantage point, as it

will make many co-workers feel more comfortable expressing their experiences openly with her.

Dissemination Plan

After the final STAR Early Literacy assessment has been given and analyzed along with

the final survey results, a report and presentation of the findings will be created and provided to

all stakeholders at a faculty meeting. A written report will also be included as a final step of the

results. This presentation and discussion of this evaluation will also take place with the

administration staff before the faculty meeting. All of these events will occur prior to the end of

the school year.

STAR E.L. PROGRAM EVALUATION 16

References

Renaissance Learning, Inc. (2018). STAR Early Literacy – Assessment product for skills

mastery. Retrieved September 1, 2018, from

https://www.renaissance.com/products/assessment/star-360/star-early-literacy-skills/
STAR E.L. PROGRAM EVALUATION 17

Appendix A

STAR Early Literacy Initiative of Lorraine Elementary Logic Model

The STAR Early Literacy Initiative is a program at Lorraine Elementary that aims to help every student grow in their learning in an
individualized manner. All kindergarten, first, and second grade students will take the STAR Early Literacy screener once every six
weeks. These results will then be evaluated by teachers, administrators, and the RTI coordinator. These individuals will devise plans
to ensure successful “growing” opportunists for all students through the use of accelerated and remedial teaching strategies that are
catered to the represented learning needs within each classroom.

INPUTS ACTIVITIES OUTPUTS OUTCOMES*

Initial Intermediate Longer-Term
6 Kindergarten Teachers STAR Early Literacy Data tracking student Positive Positive Positive
screener given every six progress of Teachers will receive Students will Data from the
5 First-Grade Teachers weeks alphabetic principle, direct training on STAR receive more screeners led to
concept of word, Early Literacy individualized growth in every
5 Second-Grade Teachers Grade-Level visual instruction student as a result of
Data/Growth Meetings discrimination, Increased techniques. “master teacher”
1 Principal conducted with phonemic awareness, understanding of STAR differentiation.
administration team to phonics, structural Early Literacy screener Trends in student
1 Assistant Principal communicate results to analysis, vocabulary, for all stakeholders. performance will be
through bi-weekly sentence level, monitored by all Promoting the use of
1 Testing Coordinator meetings. paragraph level stakeholders. this STAR Early
comprehension, and Negative Literacy model within
1 RTI Coordinator Monthly RTI meetings to early numeracy. Time commitment- other schools.
strategize individualized increase of testing Negative
16 Classrooms acceleration and Calculation of grade during instructional Inability to properly Negative
remediation strategies equivalency, reading time administer Consistently skewed
160 iPads (10 per level, and student screeners within results. Complete
classroom) Use the same grade-level growth percentile Technological issues testing window discrepancy between
classroom-based with the STAR Early student performance
18 Weeks assessments to track Literacy program or Inaccurate testing in class and online
performance-based student iPads. results. Discrepancy assessment. Lack of
progress of student use of STAR Early
performance on Literacy
assessment versus
class assignments.
STAR E.L. PROGRAM EVALUATION 18

Appendix B

Letter to Participants and Consent

Dear Lorraine Elementary Staff Members,

Thank you so much for your time and consideration in participating in this program

evaluation. As you know our school has increased our participation in the STAR Early Literacy

assessment screeners from quarterly to a continuous six-week process to monitor student growth.

While our main priority it to ensure that we are providing every student at Lorraine Elementary

School with a world-class education, the utilization of this program and its correlation to student

performance in the classroom still needs to equate within our setting. Your responses within a

survey, interviews, and even focus groups will help do exactly that. We would like to learn about

your experiences comparing the data from STAR Early Literacy to in-class participation and test

scores. These participation will significantly impact the nature of this evaluation process and

might even effect our utilization of this program, as a school. All responses will remain

anonymous and all data relating to personal information or class details will remain confidential.

An initial and final survey will be included as part of the program. Each should take less than 15

minutes to complete. You can choose not to respond to questions and even end the survey at your

leisure. If you are willing to become a stakeholder in this program evaluation please indicate

your consent below.

Thank you so much for your time and consideration in participating in this evaluation process. If

you have questions or concerns regarding this program evaluation please contact:

Bianca Cheatham

770-483-0657 ext. 35152.

bcheath1@my.westga.edu
STAR E.L. PROGRAM EVALUATION 19

CONSENT
I have read the information above and am willing to participate in this study. I also give my
permission for all survey responses and interview comments to be used within the findings
report.

Participant's name ______________________________ Date __________

Participant's signature ______________________________ Date __________

Evaluator's signature _____________________________ Date __________

STAR E.L. PROGRAM EVALUATION 20

Appendix C

STAR Early Literacy Assessment Program Evaluation Initial Survey

Please indicate your level of agreement with the following statements in relation to the use of

STAR Early Literacy Assessment?

Strongly Disagree Neutral Agree Strongly

Disagree Agree
The STAR Early Literacy
provides an accurate measure
of overall student performance
    
(Scaled Score).

Monitoring student growth

through six-week assessment     
period has been beneficial

The STAR Early Literacy is

designed in a developmentally
appropriate manner for young
    
learners.

My students’ performance on
classroom assessments matches
their STAR Early Literacy
    
scores.

My students carefully select

their answer choices and
perform their best when taking     
the STAR Early Literacy
screeners.

The data collected from the

STAR Early Literacy
assessments has helped me     
identified areas of strengths and
weaknesses within my students.

The STAR Early Literacy

provided an accurate measure
of my students’ reading fluency,     
when compared to teacher-
driven running records.

The STAR Early Literacy

provided an accurate measure
of my students’ reading
levelers, when compared to
    
teacher-facilitated reading
groups.
STAR E.L. PROGRAM EVALUATION 21

I feel as though my students are

growing as a result of the
utilization of the STAR Early
    
Literacy screeners.

I understand the data generated

by the STAR Early Literacy     
program.

I feel comfortable sharing the

STAR Early Literacy data with
others (administrators, grade-
    
level team, parents).
1. Please provide any specific comments or feedback that you would like to share regarding

your school’s implementation of the STAR Early Literacy program:

2. Please provide the following demographic information:

Faculty Member Position:

 kindergarten teacher

 first-grade teacher

 second-grade teacher

 administrator

 other

 Prefer not to answer

Number of years within the educational field:

 1-5

 6-10

 11-15

 20 or more
STAR E.L. PROGRAM EVALUATION 22

Appendix D

STAR Early Literacy Assessment Program Evaluation Final Survey

Please indicate your level of agreement with the following statements in relation to the use of

STAR Early Literacy Assessment?

Strongly Disagree Neutral Agree Strongly

Disagree Agree
The STAR Early Literacy
provides an accurate measure
of overall student performance
    
(Scaled Score).

Monitoring student growth

through six-week assessment     
period has been beneficial

The STAR Early Literacy is

designed in a developmentally
appropriate manner for young
    
learners.

My students’ performance on
classroom assessments matches
their STAR Early Literacy
    
scores.

My students carefully select

their answer choices and
perform their best when taking     
the STAR Early Literacy
screeners.

The data collected from the

STAR Early Literacy
assessments has helped me     
identified areas of strengths and
weaknesses within my students.

The STAR Early Literacy

provided an accurate measure
of my students’ reading fluency,     
when compared to teacher-
driven running records.

The STAR Early Literacy

provided an accurate measure
of my students’ reading
levelers, when compared to
    
teacher-facilitated reading
groups.
STAR E.L. PROGRAM EVALUATION 23

I feel as though my students are

growing as a result of the
utilization of the STAR Early
    
Literacy screeners.

I feel understand the data

generated by the STAR Early     
Literacy program.

I feel comfortable sharing the

STAR Early Literacy data with
others (administrators, grade-
    
level team, parents).

The grade-level focus groups

helped to identify instructional
strategies and data-relevant     
information to increase student
performance in my classroom.

Additional training in the STAR

Early Literacy program helped
me feel more confident     
understanding my students’
performance.

I feel as though my opinion has

changed regarding the STAR
Early Literacy program due to
    
the evaluation of the program.

1. Please provide any specific comments or feedback that you would like to share regarding

your school’s implementation of the STAR Early Literacy program:

2. Please provide the following demographic information:

Faculty Member Position:

 kindergarten teacher

 first-grade teacher

 second-grade teacher

 administrator

 other

 Prefer not to answer

STAR E.L. PROGRAM EVALUATION 24

Number of years within the educational field:

 1-5

 6-10

 11-15

 20 or more
STAR E.L. PROGRAM EVALUATION 25

Appendix E

Teacher Interview Questions

1. What is your name?

2. What school district do you currently work for?
3. What is your current position and how long have you been held this position?
4. What are your primary responsibilities and tasks within this position?
5. What are your previous work experiences?
6. What is your educational background?
7. What are the demographics of your current class related to student populations (ethnic

group, special education populations percentages)?

8. How many certified and classified staff members does your school have working with

students in your grade level?

9. Are you aware of if your school or school district has a plan of action regarding student

growth? If so, please elaborate.

10. How does your school use the STAR Early Literacy assessment? More specifically, how

effective have you found this program’s use within your building?
11. How and when are these assessments administered?
12. How do teachers within your building perceive this program?
13. How developmentally appropriate is the STAR Early Literacy assessment for young

learners?
14. Are there any assessments have been found inaccurate in correlation to the student

performance within your class?

15. What information is provided within the data collected from these programs?
16. Are there any thoughts that you would like to share with me regarding the use of

Renaissance 360 testing programs, the school’s growth plan, or the correlation between

student achievement and assessment?

STAR E.L. PROGRAM EVALUATION 26

Appendix F

Adminstrator Interview Questions

1. What is your name?

2. What school district do you currently work for?
3. What is your current position and how long have you been held this position?
4. What are your primary responsibilities and tasks within this position?
5. What are your previous work experiences?
6. What is your educational background?
7. What are the demographics of your current school related to student populations (ethnic

group, special education populations percentages)?

8. How many certified and classified staff members does your school have working with

students in kindergarten, first, and second grades?

9. Are you aware of if your school or school district has a plan of action regarding student

growth? If so, please elaborate.

10. How does your school use the Renaissance 360 testing program? More specifically, how

effective have you found this program’s use within your building?
11. How and when are these assessments administered?
12. How do teachers within your building perceive this program?
13. Which assessments are taken by various grade-levels?
14. How developmentally appropriate is the STAR Early Literacy assessment for young

learners?
15. Are there any assessments have been found inaccurate in correlation to student

performance?
16. What information is provided within the data collected from these programs?
17. Are there any thoughts that you would like to share with me regarding the use of

Renaissance 360 testing programs, the school’s growth plan, or the correlation between

student achievement and assessment?