The Accelerated Reader Program at Diamond Lakes Elementary School

Patricia Krezinski Tony Combs Katherine Hathcock

Program Evaluation
FRIT 8435 Fall 2011 Georgia Southern University November 28, 2011

Table of Contents

I. II.

Executive summary Introduction to the report A. Purpose of the evaluation B. Audiences for the evaluation report C. Limitations of the evaluation D. Overview of report contents Focus of the evaluation A. Description of the evaluation object B. Evaluative questions used to focus the study C. Information needed to complete the evaluation Brief overview of evaluation plan and procedures Presentation of evaluation results A. Summary of evaluation findings B. Interpretations of evaluation findings Conclusions and recommendations A. Criteria and standards used to judge evaluation object B. Judgments about evaluation object (strengths and weaknesses) C. Recommendations Appendix A. Instrument used for data analysis

I.

I. II.

I.

I.

Executive Summary

The purpose/objective of our evaluation is to determine whether or not the Accelerated Reading program featured at Diamond Lakes Elementary School improves the reading comprehension of participating second grade students based on the standards used by the Accelerated Reader Program. Elementary schools throughout the nation implement this program in order to increase the reading motivation of its students. By using three evaluators, one internal and two external, we will use both qualitative and quantitative data in order to analyze the effect of this program. As a result of the evaluation, this study should show the value and merit of the program, which will determine what improvements (if any) should be implemented along with whether or not the program should continue. The following evaluation questions were created in order to serve as a guide to provide this information: Does the program improve student reading comprehension? Does the program increase the reading level (as defined by the Advantage-TASA Open Standard used by the Accelerated Reader Program) students read? What is the opinion of stake-holders (administrators, teachers, parents, and students) of the impact the AR program has had on participating students?

The Renaissance Learning database was accessed in order to collect data for the reading comprehension and reading level of all participating second graders. A survey was distributed via email in order to collect stakeholder opinions about the program. Evaluation findings show that the Accelerated Reader Program at Diamond Lakes Elementary School:
3

 Provides an opportunity for participating students to improve their reading comprehension scores and reading levels.
Provides an additional resource for teachers to use in order to address the reading

comprehension needs of his/her students. Provides an additional differentiation technique that focuses on the students in a more individualized manner.

Recommendations
Although faced with obstacles that lead to somewhat limited information, the data does reveal notable considerations about The Accelerated Reader Program at Diamond Lakes Elementary School:
The teacher questionnaires (although quite limited in response) show a positive response

to the program and that some teachers view the program as beneficial to student reading comprehension and motivation.
Student records indicate that when the program is implemented and used properly, there

is an increase in reading comprehension and reading level. The data presented in the evaluation report shows the benefits of the program. With the consideration that the students are still just beginning the program for the participating year, the data justifies the potential the program has in order to continue its implementation (see Presentation of Evaluation Results section for more detailed information). As a result, the evaluators make the following recommendations:
It is recommended that the Accelerated Reader Program is continued so that students are

provided an opportunity to utilize this resource in order to truly benefit from the program.

 It is recommended that in order to further improve the program, teachers in the school

system that use AR should meet so that the teachers with the best results in the program could share their strategies. It is recommended that all teachers provide an opportunity for their students to participate in the program in order to provide for them a chance to increase their reading comprehension.
It is recommended that all schools in the Richmond County School System implement

the Accelerated Reader Program so that all students in the school system are provided an opportunity to benefit from the potential of the program the data reveals.

Introduction

The purpose of this evaluation is to determine whether or not the Accelerated Reader Program from Renaissance Learning is responsible for raising the reading and comprehension levels of students in the second grade at Diamond Lakes Elementary School. We chose to evaluate second graders because they are the most voracious readers in the school. We will look at diagnostic reports of their reading practice quizzes to determine if the Accelerated Reader program truly accelerates comprehension. The audience for this evaluation is the parents, faculty, staff, and students at Diamond Lakes Elementary School. The school has students from pre-kindergarten to fifth grade. The total population is around six hundred students, which is why we chose to evaluate just the second graders in order to represent a sampling of the student population. The school is in the suburbs of Augusta, Georgia and most of our students moved out of the inner city into this lower middle class neighborhood. Incomes of parents are moderate and most students qualify for free or reduced lunch. The faculty is composed of teachers with degrees from Bachelors to Specialist and one with a Doctorate, and at least 15 to 18 faculty members have a Masters degree. There are teaching assistants in grades PreK, K, and 1st grade, all of which have at least a high school diploma or an Associates degree.

Limitations
Limitations to our evaluation were mainly time constraints. There is just not enough time to study the reading comprehension rates for all 600 students. Additionally, the AR system itself was not up and running at the beginning of the year due to technical difficulties affiliated with the system. Therefore student scores and levels could were not collected for the month of September as originally anticipated (in order to fully cover the first nine week period the year). Furthermore, the teacher survey was not well received and less than 5 responded. In retrospect, all three evaluators should have been more involved in the collection of the stakeholder perceptions. Reminder emails should have been sent to remind teachers to complete the survey. Additionally, communicating more with the administration in order to agree upon some type of incentive could have been arranged. Due to time constraints and busy schedules, surveys were not distributed to parents of the participating students and the students themselves. Letters home would have been the most beneficial approach to gaining feedback from the parents with possibly the informal interview depending on the willingness of the parent. Creating student surveys and having the classroom teacher distribute them during class in order to maintain a comfortable and less-intimidating environment would have been the best approach to gather this data. The failure to interview these stakeholders is considered a major flaw of the evaluation. Teachers may be biased in their feelings towards the AR Program, and parents could contribute an outside looking in perspective of it while the students could provide for us the most genuine perspective since they are the ones actually participating. An overview of the reports content includes reading diagnostic reports from the Accelerated Reader web site. The program is web hosted and all the information is there and can be
7

customized to give the kind of information that we need for evaluating the second graders at the school. As a result of this availability, there was no need for the evaluators to contact the classroom teachers for information pertaining to student reading comprehension and reading levels. In the diagnostic reports we can find the students ZPD or zone of proximal development that will give us the range each student is reading in. We will look at where the students were in September and then where they are in November. Our evaluation will look at growth and advancement of the students reading level or lack thereof over the two month period in order to postulate the justification of the continued use of the Accelerated Reader Program. Please refer to the entire report for clarification and additional information.

Focus of the Evaluation

The evaluation object is the Accelerated Reader program. The program is web hosted and works with a student from where they are and attempts to raise their comprehension level as they progress through the reading levels. Students begin by taking a STAR reading quiz and from this information the students English/Language Arts teacher will set their zone of proximal reading development and set a goal for the student to work towards by the end of the nine week grading period. The students are given a range so that there will always be a good selection of books to choose from when they come to the media center. The books with Accelerated Reader quizzes are labeled and color coded to make it easier to see the grade level. Books range from level 0.1 all the way to 12.9. A ZPD for a second grader might range from 1.4 2.8. Students never lose the lower end of their zone they only add to the upper level of their zone. Their zone continually expands to include more and more books. The second graders we are evaluating are just moving from mostly picture books to short chapter books like Junie B. Jones and The Magic Tree House Series. When the program works as it should, teachers are in control and track their students progress. They make sure a student takes at least one or two quizzes per week. Some teachers even use the Accelerated Reader quiz for a weekly grade. That is really not the intent of the program, but schools implement the program in different ways.

Overview of Evaluation Plan and Procedures

In order to evaluate the Accelerated Reader Program at Diamond Lakes Elementary School, a collection of second grade student scores on AR Tests and reading levels was accessed through the Renaissance Learning Database. Additionally, stakeholder perceptions and opinions of the program were collected through electronic surveys along with informal interviews. As mentioned in the introduction, challenges faced by the evaluators included lack of stakeholder participation, time constraints with collecting data, and technical problems at the beginning of the school year. If these challenges had not occurred, we perhaps would have been able to provide a more adequate representation of the stakeholder opinion of the program, a broader representation of the program as it relates to the students, and a longer span from which to show student progress due to the use of the program. Qualitative data was collected using a descriptive, cross sectional design by sending a survey via email in order to address convenience and the internal evaluator informally interviewing participating students. Since less than 5 stakeholders responded, coding was not needed and responses were simply categorized into negative or positive. Quantitative data was collected using a causal design with case study elements by accessing the Renaissance Learning Database to analyze second grade student reading comprehension scores and reading levels between September and November. All staff members of Diamond Lakes Elementary School were sent a survey to document stakeholder perception of the AR Program (Appendix A). All AR data of the ninety second grade students of the participating school year at Diamond Lakes Elementary School were analyzed to provide information pertaining to the aforementioned reading comprehension scores and reading levels. Collectively, quantitative and qualitative data gathered from approximately 94 individuals linked to the Accelerated Reader Program was analyzed in order to complete this evaluation.
10

Presentation of Evaluation Results

11

The tables below display information taken from charts on the Accelerated Reader (AR) administrative site. The information pertains to the students who participated in AR from the entire second grade class at Diamond Lakes Elementary school. The tables display two criteria: reading comprehension and reading level. Two criteria were chosen because reading comprehension is determined by quizzes and we felt that quizzes alone were not efficient enough to determine the effectiveness of AR. Again, due to time constraints and technical issues with the system at the beginning of the year, the information only pertains to the months of October and November. However, even in this limited time period small gains are already evident. There are also charts that display the entire second grade class by teacher to help show a better picture of the gains of the AR program. Teacher A Amount of participating students Number of students whose reading comprehension scores increased 5% Number of students who increased one reading level 12 in class 1 participant 1 0

As shown with Teacher A, although only one student participated in the program, there is an increase of 5% even though there is no increase in reading level.

Teacher B Amount of participating students Number of students whose reading 22 in class 19 participants 8
12

comprehension scores increased 5% Number of students who increased one reading level 5

As shown with Teacher B, out of the 19 participants, 8 students have shown a 5%

increase in reading comprehension and 5 have increased their reading level. Teacher C Amount of participating students Number of students whose reading comprehension scores increased 5% Number of students who increased one reading level 22 in class 16 participants 7 3

As shown with Teacher C, out of the 10 participants, 7 have shown a 5% increase in reading comprehension and 3 have increased their reading level.

Teacher D Amount of participating students Number of students whose reading comprehension scores increased 5% 21 in class 17 participants 9

13

Number of students who increased one reading level

As shown with Teacher D, out of the 17 participants, 9 have shown a 5% percent increase and 2 have increased their reading level . Teacher E Amount of participating students 20 in class (students did not participate until November. No comparison data available.) 0 0

Number of students whose reading comprehension scores increased 5% Number of students who increased one reading level

As shown with Teacher E, students did not participate until November therefore no comparison data is available.

As shown in the Reading Comprehension chart, there were percentage increases in all participating classes (keeping in mind Teacher Es class did not begin until November).

As shown in the Reading Levels chart, there was an increase in percentage of reading levels for the majority (3 out of 5) of the participating classes.
14

Stakeholder Perceptions

Due to the low return-rate of the surveys, we had to make assumptions about the opinions of the stakeholders in that the responses we did get represent the overall perceptions. With the thought that if a stakeholder had a major problem with the AR Program, he/she would have taken the opportunity we provided (an anonymous, electronic survey) to express those concerns, the surveys returned showed overall positive feelings towards the program. As mentioned under the executive summary, the responders view the program as beneficial to student reading comprehension and motivation.

15

Conclusions and Recommendations

Conclusively, the evaluative data displaying student percentage gains stakeholder perceptions of the Accelerated Reader Program, this evaluation study concludes:

Program Strengths:
Provides an opportunity for participating students to improve their reading comprehension scores and reading levels. Provides an additional resource for teachers to use in order to address the reading comprehension needs of his/her students. Provides an additional differentiation technique that focuses on the students in a more individualized manner.

Program Weaknesses:
Not all teachers are utilizing the program in order to provide students with another resource to improve their reading comprehension.
Reading comprehension is gauged by quizzes; if a student does not test well their scores

may not relay their progress.

16

Recommendations:
It is recommended that the Accelerated Reader Program is continued so that students are provided an opportunity to utilize this resource in order to truly benefit from the program.
It is recommended that in order to further improve the program, teachers in the school

system that use AR should meet so that the teachers with the best results in the program could share their strategies. It is recommended that all teachers provide an opportunity for their students to participate in the program in order to provide for them a chance to increase their reading comprehension. It is recommended that all schools in the Richmond County School System implement the Accelerated Reader Program so that all students in the school system are provided an opportunity to benefit from the potential of the program the data reveals.

As a result of the evaluation study, it is our recommendation that the Accelerated Reader Program be continued at Diamond Lakes Elementary School. However, participating teachers should provide the opportunity for their students to participate in the program since increases in reading comprehension and reading levels have been displayed even at the beginning of the program for this year. Since busy schedules and time constraints seem to be ever-present factors, teachers with the best results in the program could share their strategies.

17

References

(2011). Renaissance learning. Retrieved from http://www.renlearn.com/.

(2011). SurveyMonkey. Retrieved from http://try.surveymonkey.com/?gclid=CP7ws9eo2qwCFYpY7Aod9GWCpA. (actual survey sent can be found at http://www.surveymonkey.com/s/DJRGKJ9)

18

Appendix

Appendix A Diamond Lakes Elementary School Staff Survey

1. Are you using the Accelerated Reader Program with your students? Why or why not? 2. What do you think of the Accelerated Reader Program in terms of usefulness to you and your students? Why? 3. In your perception, what are the important goals and objectives of the Accelerated Reader program when used properly? 4. Which components are most critical to success of the program? 5. What if any concerns do you have about the program? About it's outcomes? It's operations? Other issues? 6. What do you hope to learn from an evaluation of this program? Why is this important to you? 7. How could you use the information provided by the results of this evaluation? (Would you use it to make decisions, to enhance your understanding?)

19