Running Heading: Parents Speak Up About High-Stakes Heading

Survey Project: Parents Speak Up about New York High-Stakes Testing

Joe Goldberg

Mary Jo Hoeft

Karen Humphrey

Theresa Neddo

Katherine Reichel

Syracuse University

April 21, 2016

Parents Speak Up about New York State High-Stakes Testing

Table of Contents
General Study Description…………..…………………………….…………………..3
Sampling Procedures………………..…………………………….…………………...6
Instrumentation……….……………….………………………….……………………9
Data Collection Procedures……………..……………………….…………………...11
Data Analysis Procedures………..……………………………….…………………..14
Reporting and Ethics…………….………………………………….….……….…….18
Pretest/Pilot Test………….…………………………………….……….………….....20
References……….…………………………………………….……….……………...24
Appendix……….………………………………………………….…….…………….25

2
Parents Speak Up about New York State High-Stakes Testing

General Study Description

Problem
Over the last five to eight years, New York State has undergone numerous educational
changes. One educational change has been in standardized statewide tests. These tests, which
are given to third through eighth grade students in New York State, have become a source of
concern in recent years. Teachers, administrators, and parents have growing apprehensions over
the use of these tests as a measure of student growth and teacher accountability. Students in third
through eighth grades in New York State began being tested under the No Child Left Behind
(NCLB) legislation and have continued into the era of Race to the Top. During the Race to the
Top era, 48 states including New York State adopted the Common Core Standards. Statewide
standardized testing continued in third through eighth grades and now encompasses assessing
students on their proficiency knowledge of the Common Core Standards. Students are tested
once a year, for duration of two to three days, in the subject areas of English Language Arts and
mathematics. Protests of state testing from teachers, administrators, and parents across New
York State and the nation have occurred since the beginning of NCLB but what is very different
today is the breadth and depth of the protests.

Many parents across the state have banded together through local and national
organizations to address their worries for statewide standardized testing. One organization called
NYS Allies for Public Education are “…parents and concerned community members who firmly
believe in the power of public education and its fundamental link to the success of a thriving
community and democratic government” (NYS Allies for Public Education). These public
groups, as John Dewey expressed, are participating in a democratic society for the well-being of
public education. Stitzlein adds to Dewey’s thoughts by stating, “Participatory democracy
emphasizes citizens as actively working together to shape public institutions and policies, rather
than merely being voting members of society who remain largely passive” (p. 58).

Within the past two testing years, a growing number of parents have chosen to “opt-out”
or refuse to allow their children to take part in these standardized tests. Data provided from the
New York State Education Department website state that in 2015 about 240,000 students have
refused these tests. On average about 26% of students eligible to take the English Language Arts
test refused with some districts having as high as 89% refusal rate. On average about 31% of
eligible students refused to take the mathematics test with some districts having as high as 91%
refusal rate, (New York State Education Department). Therefore, this survey attempts to
understand the reasons why many parents/legal guardians made the choice for their children to
either opt out or to participate in the third through eighth grade state tests.

Question
Our primary question for this study is as follows:
 What are the reasons and/or the influences that caused many parents/legal guardians
to make the choice for their children to either opt out or to participate in the third
through eighth grade state tests?

3
Parents Speak Up about New York State High-Stakes Testing

Purpose
The purpose of this study includes description, explanation, and exploration. The
descriptive goal is to summarize the distribution of perspectives regarding participation in the
New York State third through eighth grade tests. Within this sample population, discovering the
distribution of certain participant in or opt-out characteristics will help us to learn more about the
sample population. The distributions of these characteristics can be created from this sample
include level of education, household income, age of the parents. The explanation goals are 1)
examine the distribution of characteristics as variables and 2) explain the relationship between
the variables and the parents’ decision to allow their child to take or not take the state tests. The
exploration goal is to examine our current methodologies used for this study to determine if other
methodologies may be more useful in future studies on this topic.
In this study, the independent variable is the beliefs of the parents/legal guardians. These
beliefs may include educational policy ideas, the welfare of the child, and community attitudes.
The dependent variable is the decision parents/legal guardians made about their child taking or
not taking the third through eighth grade New York State tests.

Diagram of Variables

4
Parents Speak Up about New York State High-Stakes Testing

Design
The survey design will consist of a cross-sectional study. We believe that data from a
single encounter with a sample group of parents/legal guardians will provide us with adequate
information to understand the statistics regarding the school district selected which is Cicero-
North Syracuse’s whole population of New York State parents/legal guardians who opt their
child out or in taking part in the New York State tests.

Population
The population will consist of parents/legal guardians whose children attend public
schools in Cicero-North Syracuse (CNS) Central School District and who have a child(ren)
eligible to participate in the third through eighth grade New York State tests for ELA and
mathematic. Our units of analysis will consist of the parents/legal guardians who are listed on
the district’s student management system as the primary household on record for the 2015-2016
school year.

This population was selected because of their location, demographics, and enrollment.
As researchers at Syracuse University, the location of this study is within a reasonable distance
from campus to work with district officials. The district has a diverse demographic population of
students consisting of ethnicities that include American Indian, Asian, black, Hispanic, and
white. The district is a large suburban district with 8,804 students in grades K-12. Of those
students, 14% are students with disabilities and 34% are economically disadvantaged (New York
State Education at a Glance). The last two years of data reported by the New York State
Education Department demonstrated that CNS had a high percentage of students who were opted
out of both the New York State English Language Arts and mathematics tests.

Data Collection
Data will be collected via a self-administered questionnaire. Researchers will mail a
prior notice post card to the parents/legal guardians, who are living in the primary postal address
of record the district has on file for the child. One week later, a cover/introductory letter with
explanation of the study and the questionnaire will be mailed to the residence. The questionnaire
will include specific directions on how to complete the survey. Follow up correspondence that
includes a reminder letter with an additional questionnaire will be mailed four weeks after the
initial questionnaire was sent. Parents/legal guardians will be instructed to complete the paper
questionnaire and return it in a self-addressed stamped envelope.

5
Parents Speak Up about New York State High-Stakes Testing

Sampling Procedures

Universe
The universe in the study consists of parents/legal guardians of children in grades three
through eight.

Population of Interest
The population of interest in the study consists of parents/legal guardians of children in
grades three through eight during the 2015-2016 school year.

Unit of Analysis
The units of analysis are parents/legal guardians in primary households of record (as
identified by the district) in the Cicero North Syracuse School District who has children in
grades three through eight in the 2015-2016 school year.

Survey Population
The survey population will consist of parents/legal guardians in primary households of
record with children enrolled in grades three through eight in the Cicero-North Syracuse School
District who are eligible to participate in the New York State tests in ELA and mathematics
during the 2015-2016 school year. This population was chosen because the last two years of
data reported by the New York State Education Department demonstrated that CNS had a high
percentage of students who were opted out of both the New York State ELA and mathematics
tests.

Sampling Design
For the purpose of sampling, a stratification of parent/legal guardians in households of
record in the Cicero-North Syracuse School District with children in grade levels three through
eight will take place prior to sample selection. A stratified sample with a random start will select
every sixth address to yield 500 households of record and therefore, parents/legal guardians in
our sample. To eliminate duplicate mailings, if a household is selected through this process that
has already been selected for a child in a different grade level (due to having multiple children in
the grade levels being tested), the next selection will be chosen and the process will continue.

A questionnaire will be mailed just after the New York State ELA and math tests are
administered in April of 2016. The sampling frame will consist of all households within the
Cicero-North Syracuse School District, in which, one or more third through eighth grade
students enrolled in the district reside. We will send an initial postcard on May 2, 2016 to
introduce the project and ask households to be on the lookout for our survey. The survey will be
mailed on May 9, 2016. We will provide a cover letter with the questionnaire that describes the
researchers and the study. A reminder letter with an additional questionnaire will be mailed to
6
Parents Speak Up about New York State High-Stakes Testing

non-respondents four weeks after the initial letter and questionnaire are sent out to households.
For the purpose of mailing reminder letters and questionnaires to non-respondents, we will code
the questionnaires by parent/legal guardian in household of record and create a key to avoid
sending a second letter and questionnaire to a household that has already responded. We will
explain in our letter of introduction that the key will be destroyed and respondent confidentiality
will be maintained. A pilot study will also be conducted on a smaller sample of a comparable
district.

A possible survey sampling frame may look like the following and would include all
students in grades three through eight:

Student Grade Parent/Guardian Address

EX. Jones, Mary 3 Sharon Jones 341 South Street

Cicero, NY 13111

EX. Kilcoyne, Kevin 3 Mark and Sheila Kilcoyne 65 Ramble Road

North Syracuse, NY 13111

EX. Zlaus, Peter 3 Klaus Zlaus 991 E. Waltham Rd.

Cicero, NY 13111

EX. Ackerman, Victoria 4 John and Wendy Ackerman 4181 Lake Road Way
North Syracuse NY 13111

EX. Anyella, Margaret 4 Jean and Fritz Anyella 8776 Harwich Rd.
Cicero NY 13111

Limitations of Sampling Frame

As with all sampling procedures, this sampling has limitations. At this time the
feasibility of obtaining names and addresses of parents/legal guardians in primary households of
record in the CNS School District has not been fully explored. We feel this information would be
valuable to the district and as a courtesy; we will notify the superintendent and board of
education when the results are posted on the website we have set up to report our findings. We
feel this would likely increase the chances that they would be willing to accommodate our
research needs.

Some parents/legal guardians may feel the questionnaire is an invasion of their privacy.
We will do our best to assure them that responses will remain confidential. Any tracking using
code keys will be destroyed once the collection of questionnaires is complete.

7
Parents Speak Up about New York State High-Stakes Testing

Since districts in New York State are required to keep accurate student enrollment and
demographic records, CNS School District lists of parents/guardians and addresses will be
complete and it is unlikely that there will be omissions. It is possible that some of the
questionnaires will not be deliverable to the participate household for various reasons, such as a
parent not updating a new address after a move.

8
Parents Speak Up about New York State High-Stakes Testing

Instrumentation

Format
Once the sample is selected, the following procedures will occur:

1. Households with children in grades three through eight who reside in the CNS School
District will be sent a prior notice post card introducing the survey study and notifying
them that a questionnaire will be sent within one week. This postcard will be mailed
May 2, 2016.

2. A cover letter and questionnaire will be sent to households with children in grades three
through eight who reside in the CNS School District. The cover letter and questionnaire
will arrive soon after the completion of the 2015-2016 New York State ELA and
mathematics testing in April 2016. The cover letter and questionnaire will be mailed on
May 9, 2016.

3. A reminder letter with an additional questionnaire will be mailed on June 6, 2016 to non-
respondents.

Materials
All primary households of record included in the survey sample will receive a prior
notice postcard to inform the them that a questionnaire regarding New York State 3-8 grade
testing will be coming in the mail shortly afterwards. The survey will be a self-administered
questionnaire that will be delivered to the participants using the United States Postal Service.
Included with the questionnaire will be a cover letter that explains the study and a pre-stamped
return envelope. Copies of these materials are provided in the appendix of this project.

The research findings will be submitted for journal publications in the field of education
and will be shared with local and larger media outlets via email. The research team will also
direct participants (as specified in the consent letter), and the CNS school district superintendent
and board of education and the New York State Education Department as a professional courtesy
(via email), to the website created by the research team with all of the study’s findings.

Once the sample is selected, the following procedures will occur:

1. Households will be sent a prior notice post card introducing the survey study and
notifying them that a questionnaire will be sent within one week. This postcard will be
mailed May 2, 2016.

2. A cover letter and questionnaire will be sent to households with children in grades three
through eight who reside in the CNS School District. The cover letter and questionnaire
will arrive soon after the completion of the 2015-2016 New York State ELA and
mathematics testing in April 2016. The cover letter and questionnaire will be mailed on
May 9, 2016.

9
Parents Speak Up about New York State High-Stakes Testing

3. A reminder letter with an additional questionnaire will be mailed on June 6, 2016 to non-
respondents.

10
Parents Speak Up about New York State High-Stakes Testing

Data Collection Procedures

Instrumentation
Fed Ex/Kinko’s Copy Service will be used. The prior notice post card will be printed on both
sides of heavy weight paper. The cover letter that will accompany the questionnaire will be
printed on white paper. The four page questionnaire will be printed on green paper on two sheets
of paper printed on both sides.

According to the Fed Ex/Kinko’s price list accessed on April 1, 2016, the prices for copying
are $.07 for a single side copy, $.14 for a double-sided copy and $.19 for specialty paper. On the
top of the questionnaire there will be a unique identification number. These numbers, with the
corresponding household names will be recorded on the Survey Key in order to track who
returns the survey for follow up purposes.

Timeline for Data Collection Procedures

ELA Test Administration: April 5-7 Math Test Administration: April 13-15
Activity Date
Initiate request to Cicero North Syracuse School District for their February 10, 2016
cooperation in providing grades 3-8 family addresses
Prepare survey return mail log March 10, 2016
Use sampling procedure to select sample. April 1, 2016
Print initial run of postcards and questionnaires April 4, 2016
Produce 3 sets of mailing labels April 4, 2016
Purchase envelopes and stamps for mailing April 4, 2016
Prepare mailing, including hand-numbering of questionnaire April 5-6, 2010
Mail “Expect This” pre-questionnaire postcard May 2, 2016
Mail cover letter and questionnaire May 9, 2016
Conduct preliminary data run May 23, 2016
Conduct mid-survey assessment, including calculation of non-response May 30, 2016
Mail reminder letter and questionnaire to non-respondents June 6, 2016
Close survey June 20, 2016
Complete data entry June 24, 2016
Conduct computations on data and response June 27, 2016
Conduct data cleaning June 27, 2016
Destroy key July 1, 2016
Set up website to allow findings to be shared with CNS, NYS August 1, 2016
Department of Education and our participants

Questionnaire Procedures

Mailing and Postage

We will be using first-class postage for the mailing of the postcard and both mailings of
the questionnaire. In addition, we will be putting postage on the return envelopes included with
the questionnaire.

11
Parents Speak Up about New York State High-Stakes Testing

Monitoring/Logging of Returns
As questionnaires are received, they will be logged in, based on their unique
identification number, on the Survey Return Log. The identification numbers will be cross-
checked with the survey key to inform the team of which households have yet to return their
questionnaires. This list will be utilized for follow-up mailings.

Sample Survey Key

Name Address Mailing List (ML) #

1
2
3

Sample Survey Log

ML # Date Date Follow- Coding Coding Checked

Sent Return up Complete
Mailing
1 4/14/16
2 4/14/16
3 4/14/16

Supervision
The team leader will be responsible for supervision of all staff and procedures. Team
members will work in pairs in order to cross-check each other’s work to ensure that records are
accurate. The team leader will randomly cross check 10% of each team’s work to ensure
consistency.

Total Cost Estimate

Primary costs are for printing of instrumentation materials, postage and envelopes. We
are estimating a need for 300 follow-up letters (60% or 6 out of 10) as we hope for an initial 40%
return rate. The cost of this survey project will be funded independently by this team of
researchers.

Printing Costs
Item # sides x # of copies x $ copy Total
Post card copies 1 x 500 x $.19 $95.00
Letter copies 1 x 500 x $.07 $35.00
Questionnaire copies 3 (double) x 500 x $.19 $285.00
Estimated number of letters for second mailing 1 x 300 x $.07 $21.00
Estimated number of questionnaires for second 3(double) x 300 x $.19 $171.00
mailing
Total Cost of Printing $607

12
Parents Speak Up about New York State High-Stakes Testing

Postage Costs
Postage for postcard mailing 500 x $.35 $175.00
Postage for 1st Questionnaire with stamped 2 x 500 x $.49 $490.00
return envelope
Postage for estimated 2nd Questionnaire mailing 2 x 300 x $.49 $294.00
with stamp on for return
Total Cost for Postage $959
Other Materials
Envelopes 1600 x $.07 $112.00
Mailing labels 1 x 500 x $.05 (post card) $25.00
2 x 500 x $.05 (initial) $50.00
2 x 300 x $.05 (follow up) $30.00
Total Cost for other materials $217

Report of Findings will be posted on a website that the research team will create
An email will be sent to CNS No cost
board members,
superintendent, and NYS
Department of Education to
let them know the website
address
Cover/Introduction letter No additional cost
included with the
questionnaire to households
will include the website
location and the date the
results will be available.

Total Costs
Printing $607
Postage $959
Other Materials $217
Total Cost for Project $1783

13
Parents Speak Up about New York State High-Stakes Testing

Data Analysis Procedures

Data Storage and Retrieval
The data for this study will be hand entered by the survey team into Excel for record
keeping and into SPSS (Statistical Package for the Social Sciences) for later analysis. All
computer files will be password protected with the password known only to the survey team.
These files will be saved on a personal computer, backed up on the Syracuse University server
with folders created for this project. Each folder will also be password protected. Passwords will
be changed weekly to increase security. When the data analysis has been completed, the folders
will be transferred to a portable flash drive and stored in a locked cabinet. The key that contains
the ID numbers and participants’ names will be destroyed once the data collection and data
cleaning have been completed.

Data Coding
All data possible will be coded numerically.
 Question 1 and 2 will be coded 1= Yes, 2=No
 Questions 3 and 4 will be coded 4=Highly Likely, 3 = Likely, 2 = Not Likely, 1 =
Not Applicable
 Questions 5-6 will be coded 1=No, 2= Yes with a line for the explanation which
will be coded following qualitative protocol
The demographic questions 7- 11 will be coded as follows and for later ease of access, be
placed on the top of our data entry sheets:
 Age of Survey Respondent: 1= Less than 25, 2 = 26-35, 3=36-45, 4=46-55,
5=56+
 Relationship to child (children): 1=Mother, 2=Father, 3=Grandparents, 4=Legal
Guardian, 5=Other with a line for explanation.
 Grade Level of your Children: use actual grade number (s) (3rd grade=3)
 Highest level of Education of Survey respondent: 1=High school Diploma,
2=Associate Degree, 3=Undergraduate Degree, 4=Graduate Degree, 5= Other
with a line for explanation
 Do you or someone you know work in public education: 1=No, 2=Yes with a line
for explanation
 Household Income Range: 1=$0-$24,999, 2=$25,000-$49,999, 3=$50,000-
$74,999, 4=$75,000-$99,999, 5=$100,000-$124,999, 6=$125,000-$149,999, 7=
$150,000- $174,999, 8=$175,000-$199,999, 9=$200,000 and up.
Each question will be edge coded by one member of the survey team and then double
checked by another team member for accuracy. After this cross checking to reduce possible
errors, the data will be entered directly into SPSS from the surveys. This also helps to reduce
potential error by limiting the number of times data is transferred from one format to another.
The responses required writing will be entered word for word and coded later using qualitative
data analysis protocol.

14
Parents Speak Up about New York State High-Stakes Testing

Data Cleaning Procedures

As described above, edge coding will be double checked on each survey by a different
team member to reduce coding error. Also, SPSS will be set up with code limits that will not
allow a number to be put in that is not assigned to that question. For example, in the opt-out
questions, only numbers 1-4 will be accepted. A 0 or number above 5 that is entered will receive
an error message response. Contingency cleaning does not apply to these survey questions as
each is independent of any other response. As a final measure, data will be entered by two
people. One will read the data and the second will type the codes into SPSS. The project
manager or a designee will then check the data entered with the survey to both increase accuracy
and to reduce potential errors.
Sample Codebook
VAR. 1 Age
What is the household age range of the parents/caregivers participating in this survey?
1=Less than 25
2=25-35
3=36-46
4= 47-57
5=58+
VAR. 2 Relationship
What is the relationship to the child(ren)?
1=Mother
2=Father
3=Grandparent
4=Legal Guardian
5=Other
Explain___________________
VAR. 3 Grade
Grade Level of your child in April 2016
3= 3rd grade
4=4th grade
5=5th grade
6=6th grade
7=7th grade
8=8th grade
VAR. 4 Education level
What is the highest level of education completed in your household?
1=High School
2=Associate Degree
3=Undergraduate Degree
4=Graduate Degree
5=Other
Explain_________________

15
Parents Speak Up about New York State High-Stakes Testing

VAR. 5 Work in Pub Ed

Do you or someone in your family work in public education? If yes, in what capacity?
1= No
2= Yes
Capacity______________
VAR. 6 Income
What is your approximate average household income?
1=$0-$24,999
2=$25,000-$49,999
3=$50,000 to $74,999
4=$75,000-$99,999
5= $100,000-124,999
6=$125,000-149,999
7=$150,000-$174,999
8=$175,000-$199,999
9=$200,000 and up
VAR. 7 ELA Test Participation
Did your child in grades 3-8 participate in the New York State English Language Arts test?
1=Yes
2= No
VAR. 8 Math Test Participation
Did your children in grades 3-8 participate in the New York State mathematics test?
1=Yes
2= No

For VAR. 9-13 Determine the degree to which each statement was a factor in your participation
decision

VAR. 9 Testing Preparation

New York State Testing Preparation in the Classroom
4=Highly Likely
3=Likely
2=Not Likely
1=Not Applicable

VAR. 10 Measurement of Child’s Achievement

New York State testing used as a measurement of my child’s academic achievement.
4=Highly Likely
3=Likely
2=Not Likely
1=Not Applicable

16
Parents Speak Up about New York State High-Stakes Testing

Data Analysis
The data for this survey will be analyzed with the SPSS program using quantitative measures
for all of our data has been translated into numbers and qualitative coding for those answers that
have been written in.
 Descriptive Analysis will be used to summarize the distribution of attitudes regarding
participation in the NYS 3rd-8th grade tests. Distribution frequency tables, standard
deviation and means of central tendency (mode, mean, median, and range) will be used to
reduce the data and generalize the findings. Association measures between variables
utilizing a data matrix will be determined using a joint distribution table. Tests for
significance will be performed using a chi square.
 Explanation Analysis will focus on examining the distribution of trait as variables and
explain the relationship between the variable and the families who made the choice to opt
their children out of NYS testing. A path analysis will be performed to help determine
predominant patterns among our many variables. Once those patterns have been
determined, a discriminant analysis will be performed so we can see how families who
opt out differ from those who opt in.
 Exploration Analysis will examine our current methodologies used for this study to
determine if other methodologies may be more useful in future studies on this topic.

Sample Summary Table

Highly Likely Likely Not Likely Not Applicable
Testing Preparation

Accurate measure of achievement

Assessing Common Core Standards

Teacher evaluation connected to

state testing outcomes

Federal Government Involvement

Personal schooling experience

17
Parents Speak Up about New York State High-Stakes Testing

Reporting and Ethics

Reporting
Following the timeframe as outlined in the timeline procedures of the design of this
research study, the findings of this study will be released on Monday, August 1st—one month
after all data has been collected, entered, cleaned and analyzed, and after the key has been
destroyed. This lag time allows for any potential technical or other problem that arises to be
identified and resolved, as well as for oversight of and reflection on the process, research
findings, and its significance. The research team will have ample time to meet and review the
execution of the steps in the design of the study in order to evaluate its efficacy prior to the
release of the findings.

The research findings will be submitted for journal publications in the field of education
and will be shared with local and larger media outlets via email. The research team will also
direct participants (as specified in the consent letter), and the CNS school district superintendent
and board of education and the New York State Education Department as a professional courtesy
(via email), to the website created by the research team with all of the study’s findings.

General findings (e.g. what percentage of respondents opted in or out of testing, their
attitudes regarding relevant educational matters, influences respondents cited for their testing
decisions for their children, and demographic trends, etc.) and any correlative relationships
identified by the researchers will be specified. The report will feature a basic summary overview
of the major findings, accessible for laypeople, followed by a detailed, comprehensive account of
the methodologies, procedures, raw data and analysis, as well as identification of the research
team for examination of the integrity of the study by any interested parties. The former should
suffice for many participants or the general public who might access the report, and the latter
should suffice for more interested members of the public and experts in the field.

Ethics

Institutional Review Board Clearance

All members of the research team have completed the appropriate Collaborative
Instructional Training Initiative (CITI) required by the Syracuse University for research with
human subjects. The research team shall also obtain Institutional Review Board (IRB) approval
prior to conducting the survey research procedures, thus having third party oversight with the
safety of subjects in the survey as primary concern while obviating potential conflict of interest.
This avenue enables the minimization of risk to the respondents through the vetting of research
materials, informed consent procedures and paperwork, research instrumentation, and the
collecting, processing, storage, and reporting of data. A copy of the IRB is included in the
appendix of this study.

Informed Consent
A written consent form will be included with the questionnaire mailing to the households.
The participants will be asked to read the informed consent form prior to completing the
questionnaire and to contact the researchers for any questions or concerns. If participants choose

18
Parents Speak Up about New York State High-Stakes Testing

to complete the questionnaire, they are asked to sign the consent form and return it with the
completed questionnaire in the self-addressed stamped envelope.
The informed consent form addresses the nature of the study, the expectations and rights
for those willing to participate, and the risks and benefits of involvement in the study. The
consent form notifies the participants of the confidential nature of their participation and enables
them the option to withdrawal from the study at any point while completing the questionnaire.
There is a space provided at the end of the consent form for the participants to make their
approval of their participation known by providing their signature. A copy of the informed
consent form is included in the appendix of this study.

Confidentiality
Data will be collected via a self-administered questionnaire and confidential information
will transport via the federally protected U.S. Postal system directly back to the research
participants. The researchers will include a cover/introductory letter of explanation for the
purpose of the study and instructions for completing the optional questionnaire. The
superintendent will provide the mailing list to the research team. The researchers will populate
the names and mailing addresses of the parents/guardians who were selected for the sample using
the already-procured mailing labels, materials and postage. The introductory letter explicates the
confidential nature of the study, using a coding system to minimize access of the identifying
information, defining precisely the expectations they may fairly have for privacy and care of
information they may provide. The relevance of the study is enumerated in the introductory letter
to enable parents with children eligible for state testing a voice in the pedagogical, political
process, as it faces challenge and possible future alteration.

Data Storage
Data storage procedures include a process of hand-entry by the research team, utilizing
responses that are coded and absent any other identifying information. The coding will be done
by the research team using the relevant parent and student names and addresses identified and
provided by the superintendent of the CNS school district. The computer files are password
protected, known to only the research team, and those passwords are changed weekly to increase
security and obviate being compromised. The key, which matches the coded numbers to the
actual survey respondent names and addresses, will remain under lock and key with the research
team, and is scheduled for destruction on July 1, 2016, after data has been collected, computed,
and cleaned, as indicated in the timeline.

The aforementioned procedures work toward minimizing unnecessary links between

respondent answers to the survey questions and any identifiers, as no-one other than the research
team will have access to the survey response answers, nor to the key that links the names,
addresses, or any other inherently personalized information that connect responses to
respondents. Non-project members will have no access to completed surveys and will not be
involved in the process of assigning coded numbers to the survey materials. Small categories
will be avoided in the presentation of data to avoid identification through implicit means, and the
destruction of the key will be ensured by the research team to ensure confidentiality.

19
Parents Speak Up about New York State High-Stakes Testing

Pre-test/Pilot Test

Pretest Rationale
In a graduate survey research class, the research team supplied classmates with a pre-test
of the instrument for the purpose of detecting areas where misconceptions, questions, or lack of
clarity occurred. The goal was to become aware of potential problems with the instrument that
may not be anticipated. This input was a crucial piece in assisting to create the best instrument
possible.

Pretest Activities
We provided copies of the prior notice postcard, introductory letter, and questionnaire to
each classmate, as well as to Dr. Smith. They completed the questionnaire and read the letters to
provide feedback pertaining to clarity or where additions or deletions might improve the quality
of the survey. Suggestions were made for possible improvements on the postcard, introductory
letter as well as the questionnaire. The research team compiled the comments and reviewed each
one individually. While all the feedback was helpful, special attention was paid to portions
where multiple comments were made and where several similar comments were made on a
particular question.

As we were working through the suggestions from classmates, we realized some minor
changes could add to the clarity of the instrument and ultimately increase our response rate. For
example, several classmates suggested that the postcard did not state what the team plans to do
with the results of the survey, and therefore may make respondents reluctant to participate. We
agreed that adding a line explaining our plans to share the results with those households we
surveyed, as well as, the district’s superintendent, the board of education, and the New York
State Education Department would be wise and satisfy any respondents who were concerned
about how we were planning to use the data.

The feedback also pointed out some areas where the cover/introductory letter could be
improved. Some minor adjustments were made to the language in the letter and some additional
contact information for questions and concerns were added. Again, these changes will add to the
clarity of our purpose and will likely lead to a better response rate.

Major revisions were made in the questionnaire as a result of the feedback received from
the classmates. A particular area of concern was the format of the questions. Several classmates
stated that the questionnaire layout was confusing as some portions had respondents circle
responses and other areas required to check boxes. Several classmates suggested that we add a
category for “other” so responses that were not listed could be added. Another area of the
questionnaire that had repeated comments and suggestions was the question about teacher
evaluation. Classmates helped us realize that those parents who do not work in the education
field may not be aware that teacher performance is based, in part, on test results. The
respondents may not have an opinion on whether or not the tests scores of students provide a fair
assessment of teacher performance. This is a major concern for educators whose own children
were eligible to take the test and we feel those parents wanted to make a political statement by
holding their children out of the testing. However, given the number of households that will
receive the questionnaire, it is likely that the majority of respondents will not be educators. This
20
Parents Speak Up about New York State High-Stakes Testing

made us think about the order in which we listed the possible reasons parents may have allowed
or disallowed their children to take part in the tests. We decided to change the order of the
choices we presented. We want to be sure that we provide choices that most parents can relate to
first in our list. We do not want to risk parents thinking that the list does not include choices that
influenced their decisions and ultimately skip the question or give up on the questionnaire
altogether.

Another comment that appeared multiple times in our feedback was that we did not
provide an “other” category where parents could provide a response that influenced their
decision but was not listed. We felt that this was a very important suggestion if we were to
capture as much information as possible. We added the “other” response where appropriate
along with a line where other influences could be noted.

During this process, the research team finally decided that the format of our instrument
may not be been the best way to get to the information we want to retrieve from parents. Our
team decided to change the format of the questionnaire to the use a Likert scale with the
responses, “highly likely”, “likely,” “not likely,” “not applicable,” as options respondents will
have. We feel this will give us not only those factors that influenced parental decision regarding
the testing but we will also learn to what extent the factors influenced the decisions. We can
calculate what factors had major impacts on parental decisions, what factors had minor impacts
on their decisions and of course, what factors did not impact their decisions at all. Because we
have now changed the format of our survey, our data analysis procedures have also required
revision. Ultimately, a Likert scale will make coding of the information much more
straightforward and therefore be less likely to lead to errors.

Receiving feedback from our classmates provided us with information that we found
helpful in creating an instrument and accompanying communications that will better serve our
purpose and will significantly improve both the number of responses we receive and the quality
of the information contained in those responses.

Pretest Parameter and Sampling Error Estimates

While we have been initially focused on the feedback of classmates regarding the clarity
and precision of the instrument, the next steps will be to scrutinize the responses themselves.
From this information, we can calculate parameter estimates and sampling error estimates. We
will then be able to predict with an acceptable level of accuracy what our expectations will be for
responses in our final larger study.

Pilot Test

The feedback from the pre-test was very useful but the intended audience and method of
administration of our study differs significantly from the class experience. Therefore, a pilot test
is required. The intent is to run the pilot test for the purpose of revealing misinterpretations and
omissions in the instrument, as well as errors in data collection, processing, and analysis
21
Parents Speak Up about New York State High-Stakes Testing

procedures. Further, the pilot test will improve our understanding of the study’s topic. The pilot
test will be carried out on households in the East Syracuse Minoa School District, a district
comparable to the Cicero-North Syracuse School District. This will allow us to test our
procedures without running the risk of low response rate from our study district due to repetition.

The pilot test will consist of a self-administered questionnaire, each with a unique
identification number for the purpose of tracking responses and identifying non responders. We
will send a prior notice post card to 20% of households in the district in which a third through
eighth grader resides during the 2015-2016 school year. This 20% will be chosen as from a
stratified sample by grade levels (third through eighth) with a randomized start. A week later, a
letter of explanation and the questionnaire will be mailed to these households. Respondents will
be asked to complete the questionnaire and return it to the research team in the self-addressed
stamped envelope provided. Follow up correspondence and an additional questionnaire will be
sent four weeks later to remind those who have not responded to please do so. If we do not
receive a 40% to 60% response rate, the research team will begin to call the households from
which a response was not received. Of importance, will be the reasons behind the lack of
response to the pilot survey. We want to be sure to correct any possible flaws in our survey
population, instrument, and accompanying processes that could result in non-response in our
final survey.

Responses will be recorded on our Survey Return Log based on their unique
identification number. The identification number will be cross-checked with the survey key to
inform the team of which households have yet to return their questionnaires. This list will be
used for follow-up mailings.

Data for the pilot test will be hand entered by the survey team in Excel for record keeping
and in SPSS for later analysis. All work will be password protected (password changed weekly)
and stored on a personal computer, the SU server, and iCloud. Once the study is completed the
stored folders will be transferred to a flash drive and stored in a locked cabinet. The key
containing names of participants and ID numbers will be destroyed.

All data will be coded numerically in the exact form stated in the study. Each question
will be edge coded by one member of the survey team and them double checked by another team
member for accuracy. The data will then be entered directly into SPSS from the surveys,
reducing the possibility of error by limiting the number of times the information is transferred.
SPSS will be set up with code limits that will not allow a number to be entered for a specific
question that is not assigned as a possible response to that question. A second measure for
accuracy, the data will be entered by two people, one reading the codes and a second typing them
into SPSS.

Data in the pilot test will be analyzed using quantitative measures as all of our data will
be translated into numbers. Descriptive Analysis will be used to summarize the distribution of
attitudes regarding participation in the NYS third through eighth grade tests. Distribution
frequency tables, standard deviation and means of central tendency will be used to reduce the
data and generalize the findings. Association measures between variables utilizing a data matrix

22
Parents Speak Up about New York State High-Stakes Testing

will be determined using a joint distribution table. Tests for significance will be performed using
a chi square.

Explanation Analysis will focus on examining the distribution of traits as variables and
explain the relationship between the variable and the families who made the choice to opt their
child out of NYS testing. A path analysis will be performed to help determine predominant
patterns among our many variables. Once those patterns have been determined, a discriminant
analysis will be performed so we can see how families who opt out differ from those who opt in.
Exploration Analysis will examine our current methodologies used for this study to determine if
other methodologies may be more useful in future studies on the topic.

23
Parents Speak Up about New York State High-Stakes Testing

References
Dewey, J. (1916). Democracy and Education. New York: Macmillan Company.

Information and Reporting Service. (n.d.). Retrieved from New York State Education Department:
http://www.nysed.gov

New York State Education at a Glance. (n.d.). Retrieved April 19, 2016, from www.data.nysed.gov

New York State Education Department. (n.d.). Retrieved from New York State Education Department:
http://www.nysed.gov

NYS Allies for Public Education. (n.d.). Retrieved April 15, 2016, from NYS Allies for Public Education:
http://www.nysape.org

Schaeffer, B. (2012). Resistance to High-Stakes Testing Spreads. District Administration, 34-42.

Stitzlein, S. M. (2015). Improving Public Schools Through the Dissent of Parents: Opting Out of Tests,
Demanding Alternative Curricula, Invoking Parent Trigger Laws, and Withdrawing Entirely.
Educational Studies, 57-71.

24
Parents Speak Up about New York State High-Stakes Testing

Survey Project: Parents Speak Up About New York State High-Stakes Testing
Appendix

25
Parents Speak Up about New York State High-Stakes Testing

Appendix A: Prior Notice Post Card

Your Thoughts Count!

Parents Speak Up about New York State High-Stakes Testing

Dear Parent or Guardian,

We are a Syracuse University doctoral
student survey research team who will be sending
out a questionnaire to parents/legal guardians of
children in grades 3-8 residing in the Cicero-North
Syracuse Central School District. We are seeking
your feedback as to what prompted your decision
for your child to take or not take the New York
State standardized tests during the 2015-2016
school year.
It is important as a parent/legal guardian
to be able to share your decisions in regard to
standardized state testing. Findings will be shared
on a website created by the research team.
Our research team will be sending you a
brief questionnaire within the next week.

Thank you in advance for your input.

Syracuse University Doctoral Student Survey

Research Team
School of Education
Joe Goldberg
Mary Jo Hoeft
Karen Humphrey
Theresa Neddo
Katherine Reichel

26
Parents Speak Up about New York State High-Stakes Testing

Appendix B: Questionnaire Cover Letter/Introductory Letter

School of Education

330 Huntington Hall

Syracuse, NY 13244

May 9, 2016

Dear Parent/Guardian,

We are a Syracuse University doctoral student research team from the School of Education. We are
contacting parents/legal guardians in the Cicero-North Syracuse (CNS) School District for feedback as to
what prompted your decision to opt out or allow your child in (this will be filled in based on the random
3-8 grade student selected) grade to participate in the third through eighth grade New York State testing
during April 2016.

We want your voice to be heard. It is imperative for parents/legal guardians, the school district, and the
New York State Education Department to know the reasoning behind parental decisions to opt out their
children or to allow their children to participate in the New York State English Language Arts and
mathematics tests for students in grades 3-8.

Your responses may assist other parents or guardians in gaining a deeper understanding of community
perspectives of standardized testing. Your feedback will also be shared with the CNS district and the
New York State Education Department to aid in future decisions regarding Common Core State Testing.

Participation consists of completing a 15 minute questionnaire which is included. All responses are
confidential and will only be referenced with a coding system. Your name will not be associated with any
part of the questionnaire. A consent letter is included along with the questionnaire. The consent letter
will inform you of your rights as a participant along with the risks and benefits for participating in this
study. Upon your completion of the questionnaire, please return the consent form and questionnaire in the
self-addressed stamped envelope. If you have any questions, concerns, or complaints about the
research, please contact Dr. Nick Smith at 315-443-5259 and nlsmith@syr.edu or Joe Goldberg at 315-
420-4857 and jagold05@syr.edu.

You may access the findings of the study at www.statetesting.com on August 1st. Thank you for your
assistance. Your time and feedback are appreciated.

Sincerely,

Syracuse University Doctoral Student Survey Research Team

School of Education
Joe Goldberg
Mary Jo Hoeft
Karen Humphrey
Theresa Neddo

27
Parents Speak Up about New York State High-Stakes Testing

Katherine Reichel

Appendix C: Reminder Letter

School of Education

330 Huntington Hall

Syracuse, NY 13244

June 6, 2016

Dear Parent/Guardian,

On May 9, 2016 our research team mailed you a questionnaire related to the New York State
English Language Arts and Mathematics tests administered to children in grades 3 through 8.
We are contacting parents/legal guardians in the Cicero-North Syracuse (CNS) School District
for feedback as to what prompted your decision to opt out or allow your child in (this will be
filled in based on the random 3-8 grade student selected) grade to participate in the third through
eighth grade New York State testing during April 2016. As previously noted, your participation
involves completing a 15 minute questionnaire. All questionnaire responses are kept
confidential and identified only with a unique coding system.

As of this follow up mailing, we have yet to receive your questionnaire. Our research team has
enclosed an additional questionnaire in the event the original questionnaire was misplaced. At
your earliest convenience, please return the enclosed completed questionnaire along with the
consent form in the self-addressed stamped envelope provided. If you have any questions,
concerns, or complaints about the research, please contact Dr. Nick Smith at 315-443-5259 and
nlsmith@syr.edu or Joe Goldberg at 315-420-4857 and jagold05@syr.edu.

You may access the findings of the study at www.statetesting.com on August 1st. Thank you for
your assistance. We look forward to receiving your responses with the week.

Sincerely,

Syracuse University Doctoral Student Survey Research Team

School of Education
Mary Jo Hoeft
Karen Humphrey
Joe Goldberg,
Theresa Neddo
Katherine Reichel

28
Parents Speak Up about New York State High-Stakes Testing

Appendix D: Informed Consent Letter

SYRACUSE UNIVERSITY, SCHOOL OF EDUCATION

Survey Project: Parents Speak Up about New York High-Stakes Testing

Allow us to introduce ourselves. Our names are Mary Jo Hoeft, Karen Humphrey, Joe Goldberg,
Theresa Neddo, and Katherine Reichel. We are a group of doctoral students in the School of
Education at Syracuse University inviting you to take part in a research study examining the
factors that lead parents and guardians to opt their child(ren) in or out of New York State testing
in grades three through eight. We are asking you to take part because you have a child(ren) who
were recently eligible to take the New York State tests in English Language Arts and
mathematics during the 2015-2016 school year.

Please read this form carefully and contact us using the email addresses and phone numbers
below if you have any questions or concerns before agreeing to take part in the study. If you
agree to be included in this study, you will need to consent to participating by signing at the
bottom of this document (please include this consent form when you mail back the questionnaire
in the self-addressed, stamped envelope) and complete the accompanying short self-administered
questionnaire.

The questionnaire will include a few questions about your demographics, your views on a small
variety of education-related topics, and your opinion on what degree various factors might have
influenced your decision to have your child(re) take the New York State English Language Arts
and mathematics tests or not. The questionnaire should only take approximately 15 minutes to
complete. All information will be kept confidential.

There is the risk that you may find some of the questions about your household income, age,
decision to have your child(ren) take the New York State English Language Arts and
mathematics tests or not, as well as the factors that led you to your decision, to be sensitive.
Taking part in this study is completely voluntary. You may skip any questions that you do not
want to answer. If you decide not to take part or to skip some of the questions, it will not be held
against you in any way. If you decide to take part, you are free to withdraw at any time.

There are possible benefits to you. This questionnaire may enable you to voice your thoughts and
possible concerns regarding state tests. The findings of this study regarding reasons parents
chose to have their children take state tests or not will be made available to the public—without
any identifying information for individuals who participated—and might possibly inform
decisions by policy makers in the field of education. You may access the findings of the study at
www.statetesting.com on August 1st.

29
Parents Speak Up about New York State High-Stakes Testing

The records of this study will be kept private. In any sort of report we make public, we will not
include any information that will make it possible to identify you. Research records will be kept
in a locked file; only the researchers will have access to the records.

If you have any questions, concerns, or complaints about the research, please contact Dr. Nick
Smith at 315-443-5259 and nlsmith@syr.edu or Joe Goldberg at 315-420-4857 and
jagold05@syr.edu. If you have any questions about your rights as a research participant, have
questions, concerns, or complaints that you wish to address to someone other than the
investigator, or if you cannot reach the investigator, contact the Syracuse University Institutional
Review Board at 315-443-3013.

Statement of Consent: I have read the above information, and have received answers to any
questions I asked. I consent to take part in the study.

Your Signature ___________________________________ Date ________________________

Your Name (printed) ____________________________________________________________

PLEASE BE CERTAIN TO INCLUDE THIS CONSENT FORM IN THE SELF-ADDRESSED

STAMPED ENVELOPE WHEN RETURNING THE COMPLETED QUESTIONNAIRE.

30