Confidenti al

New York City Department of Education

18 December 2013
Findings and Recommendations Report
EY
5 Times Square
New York, NY
10036
Tel: +1 212 773 3000
ey.com
Mr. Dennis Walcot t
Chancellor
The New York City Depart ment of Educat ion
52 Chambers St reet
New York, NY 10007
Dear Mr. Walcott :
We have completed our test ing procedures rel ated to t he implementat ion of the recommendat ions set fort h in the
New York City Depart ment of Educat ion ( Depart ment ) Office of Auditor General ( OAG ) Hi gh School Academi c
Data Audit report dat ed February 23, 2012 specific to st udent discharges, academic requirements, and Regent s
scoring.
The at t ached report summarizing t he result s of t he three phases of t he audi t s was prepared in accordance with Work
Order #14 dated June 22, 2012. The Execut ive Summary, including background and a summary of result s, i s included
on page one of t he at t ached report . Furt her explanat ion of t he findi ngs and recommendat ions f rom our t esti ng
procedures are included in t he att ached report beginning on page t wo.
We value t he opport unity to work wit h you and Depart ment management and sincerely appreciate t he cooperat ion
and assistance provided t o us during the project . We would be pleased t o f urt her discuss any aspect of our
procedures or t his report wit h you or ot her members of t he Depart ment s management at your convenience. If you
have any quest ions, please call David Milkosky at +1 (732) 516-4225.
This report is intended solely for t he informat ion and use of t he Department .
December 18, 2013
Copies:
Shael Polakow-Suransky, Chief Academic Officer
Court enaye Jackson-Chase, General Counsel

TABLE OF CONTENTS
I. EXECUTIVE SUMMARY ........................................... .1
A. INTRODUCT ION AND COMPLIANCE AUDIT OBJECTIVES . . . . . . . . . . . . . . . . . . . . 1
B. SUMMARY OF RESULTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
II. STUDENT DISCHARGES .............................................................. 2
A. SCOPE A ND MET HODOLOGY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
B. RESULT S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
C. RECOMMENDATI ONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
III. REGENTS HIGH SCHOOL EXAMINATION SCORING ..................... 4
A. SCOPE A ND MET HODOLOGY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
B. RESULT S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
C. RECOMMENDATI ONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
IV. REGENTS EXAMINATION AND ACADEMIC COURSE CREDIT ....... 7
A. SCOPE A ND MET HODOLOGY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
B. RESULT S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
C. RECOMMENDATI ONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1
I. EXECUTIVE SUMMARY
A. INTRODUCTION AND COMPLIANCE AUDIT OBJ ECTIVES
A risk-based audit of School Year 2010-2011 high school st udent dat a conduct ed by t he New York City Depart ment of
Educat ion ( Depart ment ) Office of Audit or General ( OAG ) ident ified dat a pat terns indicat ing a need for policy
clarificat ion and syst ems improvement . The audit focused on school dat a pat terns in t he following areas: credit and
exam accumulat ion, st udent discharges and Regent s examinat ion scoring. The result s of t he audit and related
recommendat ions were document ed in t he OAGs High School Academic Dat a Audit Report ( Report ) dated February
23, 2012.
To t est t he effect iveness of t he implement at ion of t he recommendat ions set fort h in t he Report and assist t he
Depart ment as it cont inues t o review st udent s academic and performance dat a and assess t he value of dist ributed
scoring of Regent s examinat ions, t he Depart ment engaged Ernst & Young LLP ( EY ) to conduct compliance audit s in
2011-12 in each of t he following t hree areas:
a. Student Discharges. Assess whet her st udent discharge det erminat ions and t he associated discharge
codes recorded in t he Depart ment s Aut omate The Schools ( ATS ) syst em were support ed wit h t he
appropriate support ing records;
b. Regents Scoring. Assess whet her t he scoring result s from t he Depart ment s dist ribut ed scoring model
pilot were wit hin t he Depart ment s accept able range of scorer variability
1
.
c. Academic Requirements. Det ermine whet her st udent s met t he minimum course credit s and Regent s
examinat ion requirement s for earning a Regent s diploma as defined by t he New York St at e Educat ion
Depart ment ( NYSED ).
The audit s will assi st t he Depart ment as it cont inues t o review policies and pract ices pert aining t o st udent discharges,
Regent s exam scoring, and t he implement at ion of academic requirement s.
B. SUMMARY OF RESULTS
Upon examinat ion of high school academic dat a in each of t he t hree areas of focus ident ified above, t he following key
observat ions emerged.
20 out of 305 st udent s were discharged without adequat e document at ion t o support t he discharge code.
Of t he 1,536 Regent s exams rescored by independent scorers:
938 exams originally had been scored as passing. Upon rescoring, 30 (3.2%) received a failing score and
exceeded t he Depart ment s accept able range of scorer variabilit y.
598 exams originally had been scored as failing. Upon rescoring, 13 (2.2%) received a passing score and
exceeded t he Depart ment s accept able range of scorer variabilit y
Four out of 310 st udent s (1.3%) who graduat ed wit h a Regent s diploma in June 2012 did not meet the
minimum physical educat ion credit requirement s. No ot her credit requirement errors were ident ified.
The audit findings for discharges and Regent s Examinat ion and Course Credit did not yield significant errors. As such,
t heir applicat ion t o t he test ed populat ions would not impact t heir overall original result s. As t o t he Regent s
Examinat ion Rescoring, t he audit included a subject ive test t hat precluded EY from ext rapolat ing any result s t o t he
universe t est ed.
1
Duri ng the audi t peri od that is the subj ect of the February 23, 2012 Report, R egents examinati ons were scored by teachers assigned to
the students schools. T he Report flagged that practi ce as an area of concern and recommended, among other opti ons, that the
Department consi der phasi ng i n a system-wi de shi ft to non-vested-i nterest scori ng, i . e. , use of raters who are not affi li ated wi th the
school whose students exams are bei ng scored. T hi s recommendati on for a di stributed scori ng model was i mplemented i n School Y ear
2011/2012 as a pilot proj ect which had non-vested-interest scorers score the non-multiple choice porti on of 127, 531 June 2012
Regents exami nati ons i n selected subj ect areas. Of that number, a sample of 1, 536 Regents exami nati ons was re-scored by an
independent second ti er of non-vested-i nterest scorers. T he di fferenti al between the ori gi nal non-vested-interest score and the re-
score performed by i ndependent scorers was calculated by EY and compared to the acceptable range of scorer vari abi li ty provi ded by
Department management.
2
II. STUDENT DISCHARGES
When a st udent leaves t he New York Cit y ( NYC ) public school syst em t o enroll in a school wit hin t he Unit ed St at es
or a foreign count ry, t he st udent is said t o have been discharged from t he public school; a corresponding discharge
code is ent ered in Aut omat e t he Schools ( ATS ), t he Depart ment s data syst em for t racking st udent informat ion.
Each discharge is required t o be support ed wit h records as defined by the Depart ment s 2011-2012 Transfer,
Discharge, and Graduat ion Code Guidelines ( Guidelines ).
St udent s discharged from NYC public schools wit h t he discharge codes t hat are wit hin t he scope of t his audit are also
removed from the schools graduat ing cohort . As such they are not counted for purposes of calculat ing t he schools
graduat ion rat e and ot her performance measurement s report ed in t he Depart ment s High School Progress Report s.
A. SCOPE AND METHODOLOGY
EY t est ed discharge t ransact ions in ATS effect uated during t he 2011-2012 school year for t he following codes:
Discharge
Code
2
Description
06 (E/ P/ W) Dischar ged t o a NYC or NY St at e Parochial School
08 (E/ P/ W) Dischar ged t o a NYC or NY St at e Privat e School
11 (E/ P/ W/ Z) Dischar ged t o a school out side NYC
20 Early Admission t o a f our-year Universit y
25 Already received a High School Diploma out side t he NYC public school syst em at t ime of enrollment
Using common st at ist ical formulae for at t ribut e t est ing of Simple Random Samples (SRS), EY determined t hat a
randomly selected sample of 305 st udent s at 18 different schools discharged during t he period Sept ember 8, 2011
t hrough April 30, 2012, would achieve t he required 95%confidence level and 5%precision rate under t he assumpt ion
of a 25%error rat e.
3
For each discharge within t he select ed sample, EY requested t hat t he school submit support ing document at ion. EY
t hen performed a desk review of t he document at ion t o assess whet her it support ed the st udent s ATS discharge
st at us in accordance with t he Guidelines.
B. RESULTS
Using the Guidelines as t he st andard, EY det ermined that , in t he case of 20 out of 305 st udent s t he document at ion
provided by the school did not support t he ATS discharge st at us as follows:
Discharge Code Number in Sample Number of Exceptions
06 (E/ P/ W) 45 1
08 E/ P/ W) 19 6
11 (E/ P/ W/ Z) 240 13
20 0 0
25 1 0
Tot al 305 20
2
T he discharge letter codes are used to defi ne the documentati on required to support the discharge status as follows:
E: Confi rmed enrollment i n an academi c exchange program
P: Confirmed enrollment i n a pri vate school or setti ng as mandated by the Committee on Speci al Educati on
W: Confi rmed enrollment at a new school
Z: Confi rmed move outsi de the USA
3
T he desi red confidence and preci si on levels along wi th assumed error rates were provided by Department management.
3
The audit determined an except ion rat e for t he sample of 6.56%. However, given t he potent ial for sampling error, t he
audit result s reflect , wit h 95%confidence, a 93.4%non-except ion rat e wit h +/ -2.93%margin of error (or 90.51%t o
96.37%).
For t he 20 st udent s discharged wit hout adequate support ing document at ion for t he code used in ATS, it was
det ermined t hat :
11 st udent s who were coded 11W, 8W or 11P in ATS indicat ing t hat t hey had enrolled at a new school had
act ually dropped out of high school and had enrolled in a non-high school diploma grant ing programs, enrolled
in GED programs, or obt ained employment .
5 st udent s were discharged in ATS using code 11W, 6W or 11P indicat ing t hat t hey had enrolled at a new
school, but no evidence was submit t ed t hat t he st udent s had re-enrolled in a high school wit hin t he United
St at es.
3 st udent s were discharged using code 11Z, but no acceptable evidence was submit ted confirming t he
count ry and/ or cit y t o which t he st udent had moved.
1 st udent had been discharged in error; t he st udent was st ill enrolled in t he school.
C. RECOMMENDATIONS
i. Discharge codes for t he affected st udent s should be updated in ATS to reflect t he appropriate code based on
t he Depart ment s guidelines and records submit t ed by t he schools.
ii. School personnel performing discharges should t horoughly review all support ing document at ion t o verify t hat
it meet s t he st andards set fort h in t he Depart ment s current guidelines before recording t he discharge st at us
in ATS.
iii. The development and implement at ion of a checklist wit h t he minimum required document at ion t o support
each discharge code may be beneficial in reducing t he risk of using t he incorrect code or missing
document at ion. It is furt her recommended that t he Depart ment consider creat ing st andardized t emplates on
which t he discharging schools must record t he subst ance of oral communicat ions wit h parent s and/ or ot her
schools about t he discharge. Such forms should be signed and dat ed by t he school st aff ent ering t he
informat ion and maint ained along wit h ot her discharge records.
4
III. REGENTS HIGH SCHOOL EXAMINATION SCORING
To earn a Regent s diploma, st udent s must pass Regent s examinat ions in at least five subject areas. St udent s pass
rat es on Regent s examinat ions in all subject s cont ribut e t o a school' s Progress Report grade. Pass rat es on t he
English and mat h Regent s examinat ions also cont ribute t o a school' s New York St at e different iat ed account ability
st at us.
Generally, Regent s examinat ions are st ruct ured as follows:
A mult iple-choice sect ion (Part I), which is usually bet ween 25 t o 50 quest ions; and,
A long-answer/ essay ( open response port ion ) sect ion (Part II), which consist s of eit her a select ion of
det ailed quest ions for which t he work must be shown, or a set of essay t opics requiring writ t en responses.
The open response port ion of a Regent s examinat ion is scored by Depart ment t eachers using a rubric developed by
NYSED for t he purpose of det ermining t he raw score for each open response. For example, if an essay can be
awarded a maximum of 3 point s, and t he scorer det ermines t hat t he essay merit s 1 point based on t he rubric, t he raw
score for t hat essay is 1. The raw scores for t he mult iple choice and open response port ions of t he exams are
convert ed t o a scale score using a conversion t able provided by NYSED t o calculate t he final Regent s examinat ion
score.
Based on requirement s set fort h by NYSED and t he 2010-2011 audit Report s findings and recommendat ions, t he
Depart ment moved t o a dist ributed scoring model for scoring Regent s examinat ions whereby t he rat ers are not
affiliat ed wit h the school whose st udent s exams are being scored and t herefore have no vested int erest in score
result s. In school year 2011-12, Depart ment management undert ook a pilot of t he Report recommendat ion by which
t he open response port ion of t he Regent s examinat ions was scored by non-vest ed-interest scorers. Because open
response scoring requires scorers t o exercise professional judgment in using t he rubric, t here is an expect at ion t hat
t here will be some degree of variabilit y in t he point s different rat ers would assign for t he same open response answer.
A. SCOPE AND METHODOLOGY
In order t o assess t he variabilit y of scoring result s under t he dist ributed scoring model, st udent s responses on the
open response port ions of t he June 2012 Regent s examinat ions in select ed subject areas were ident ified by
Depart ment management for re-scoring by a second independent group of teachers cert ified in t he sampled subject
areas.
4
Depart ment management ident ified all June 2012 Regent s examinat ions t hat were scored under t he dist ribut ed
scoring model. In order t o calculat e a st at ist ically valid sample size of examinat ions from t hat universe t hat would be
subject to re-scoring, EY used common st at ist ical formulae for at t ribut e test ing of Simple Random Samples and
crit eria of 95%confidence level, 5%precision rat e and 25%error rat e.
5
EY applied random sampling met hodology
wit hin t he following subject areas t o select t he Regent s examinat ions for re-scoring.
4
The audit did not i nclude re-scori ng of multi ple choice questions because that secti on is graded by electronic scanner, thereby reduci ng
the risk of answers bei ng scored and/or totaled i ncorrectly.
5
The desi red confidence and precisi on levels along with the assumed error rates were provi ded by Department management.
# Regents Exam
Number of Exams for
Re-scoring
Total Population
1. Int egrat ed Algebra 309 34,798
2. Comprehensive English 309 23,380
3. Living Envir onment 309 30,353
4. US Hist ory and Gover nment 309 31,254
5. Global Hist or y and Geography 300 7,746
Tot al
1,536
127,531
5
Depart ment management t hen engaged teachers cert ified in t he examinat ion subject areas t o re-score t he sampled
Regent s examinat ions. At t he conclusion of t he re-scoring, Depart ment management collect ed t he Regent s
examinat ions and applied a process
6
similar t o t hat employed for t he original set of Regent s examinat ions t o
calculat e t he final scores. Using t he original score and re-score document at ion provided by Depart ment management ,
EY calculated whet her t he difference bet ween the original exam score and t he re-score fell wit hin t he accept able
range of scorer variabilit y
7
as defined by Depart ment management upon t heir review of t he dist ribut ion of re-scoring
differences for each subject .
B. RESULTS
In analyzing t he result s of t he dist ributed scoring model and subsequent rescoring effort s, EY concent rat ed on exams
t hat originally received a passing score (65 or above) but were rescored as failing and exams t hat originally received a
failing score but were rescored as passing. Wit hin each subject , t he difference bet ween t he original score less t he
rescore was calculat ed for each exam. These differences were normalized about t he mean difference f or t he
part icular exam. The examinat ion subject s for which t he normalized scores t hat were above 1 or below -1 st andard
deviat ion (i.e., t he scored difference is one st andard deviat ion away from t he average) are ident ified in the t ables
below.
Of t he 1,536 exams rescored, 938 exams were originally scored as passing and 598 were originally scored as failing.
Of t he 938 exams originally scored as passing:
62 exams were rescored as failing
Of t hese, 30 exams (or 3.2%of t he t ot al exams originally scored as passing) reflect ed a normalized score t hat
was above 1 or below -1 st andard deviat ion.
6
EY performed a detai led walkthrough of the Department s procedures for scanni ng the exami nati ons using the Scantron Scan T ools
software and prepari ng the ori gi nal score and re-score reports, i ncludi ng a vali dati on that the parameters used to calculate the fi nal
score were i n accordance wi th the NY SEDs Chart for Converti ng T otal R aw Scores to Scale Scores for each exami nati on.
7
Department management defi ned the acceptable range of scorer vari ability as + / - 1 standard deviation. Standard devi ati on shows how
much variati on or di spersi on there is from the mean.
# Regents Exam
Originally Scored as
Passing
Originally Scored as
Failing
1. Int egrat ed Algebra 151 158
2. Compr ehensive English 173 136
3. Living Envir onment 218 91
4. US Hist ory and Government 204 105
5. Global Hist ory and Geography 192 108
Tot al
938
598
6
Of t he 598 exams originally scored as failing:
44 exams were rescored passing
Of t hese, 13 exams (or 2.2%of t he t ot al exams originally scored as failing) reflect ed a normalized score t hat
was above 1 or below -1 st andard deviat ion.
C. RECOMMENDATIONS
i. Depart ment management should cont inue it s effort s t o implement dist ribut ed scoring across all t he schools
and t o provide t raining t hat reinforces t he proper applicat ion of rubrics and scoring prot ocols.
# Regents Exam
Originally
Scored as
Passing
From Pass
Rescored to Fail
Exceeded
Acceptable
Range
% Exceeded
Acceptable
Range
1. Int egrat ed Algebra 151 3 3 2.0%
2. Compr ehensive English 173 16 6 3.5%
3. Living Envir onment 218 8 5 2.3%
4. US Hist ory and Gover nment 204 16 9 4.4%
5. Global Hist ory and Geography 192 19 7 3.6%
Tot al 938
62 30
3.2%
# Regents Exam
Originally
Scored as
Failing
From Fail
Rescored to
Pass
Exceeded
Acceptable
Range
% Exceeded
Acceptable
Range
1. Int egrat ed Algebra 158 5 0 0.0%
2. Compr ehensive English 136 25 8 5.9%
3. Living Envir onment 91 3 1 1.1%
4. US Hist ory and Gover nment 105 8 3 2.9%
5. Global Hist ory and Geography 108 3 1 0.9%
Tot al 598
44 13
2.2%
7
IV. REGENTS EXAMINATION AND ACADEMIC COURSE CREDIT
A. SCOPE AND METHODOLOGY
Depart ment management provided EY wit h a dat a file of t ranscript s for all st udent s graduat ing from NYC public
schools in June 2012 wit h a Regent s Diploma (45,495 in t ot al).
8
Using common st at ist ical formulae for at t ribut e
t est ing of Simple Random Samples, EY det ermined t hat a randomly selected sample of 310 st udent s would be
sufficient t o achieve 95%confidence and 5%precision under t he assumpt ion of a 25%error rat e.
9
The samples select ed were test ed t o verify t hat each st udent met the following crit eria:
a. Academic Course Credit :
The minimum course credit requirement s for earning a Regent s diploma are defined by t he Part 100 Regulat ions of
t he New York St at e Commissioner of Educat ion.
10
St udent s must obt ain cert ain minimum credit requirement s in t he
following subject areas:
English 8 credit s
Mat hemat ics 6 credit s
Social St udies 8 credit s, including t he following courses:
Global Hist ory 4 credit s
U.S. Hist ory 2 credit s
Economics 1 credit
Part icipat ion in Government 1 credit
Science 6 credit s, including t he following courses:
Life Science at least 2 credit s
Physical Science at least 2 credit s
Language Ot her Than English 2 credit s
Visual Art s, Music, Dance and/ or Theat er 2 credit s
Elect ives 7 credit s
11
b. Regent s Examinat ions:
Except for t hose at t ending New York St at e approved Consort ium schools
12
, st udent s must obt ain
passing scores in five Regent s examinat ions t o meet t he requirement for a Regent s diploma
(Comprehensive English, Global Hist ory and Geography, U.S. Hist ory and Government , one life or
physical science, and one math Regent s.)
13
8
Charter school graduates ( i . e. , students assigned to Distri ct 84 in the data fi le) were excluded because these schools are not required
to use the Department s transcript recordi ng system.
9
T he desi red confidence and preci si on levels, along with the assumed error rates, were provided by Department management.
10
Generally, students are requi red to have earned 44 course credits and passed fi ve Regents exami nati ons as a conditi on of receivi ng a
Hi gh School Regents Diploma. However, there are several exempti ons that must be considered to determi ne if these requirements have
been met, such as certai n Regents exami nati on waivers for students who enter a New Y ork State School in grade 11 or 12; di fferent
Regents exami nati on requi rements for students who attend a Consortium school; and exempti ons to the 44 credi ts requi rement for
students who graduate early.
11
Si nce elective courses do not have thei r own designati on i n ST ARS, compli ance was measured by checki ng for all other subj ect areas
and confirming that students had at least 44 total credi ts.
12
Consortium schools have a waiver from NY SED allowing students to graduate usi ng portfoli o assessments i n place of Regents exams i n
math, sci ence, and soci al studies.
13
Regents Exami nations are i ncluded as meeti ng the corresponding requirement i f thei r mark i s a number from 65 to 100 ( i nclusi ve) , or
if the mark examinati on score i s PR. However, students may meet exami nati on requirements for a Regents di ploma by attai ning
certai n scores on other types of exami nati ons, such as A dvanced Placement exami nati ons, SA T Subj ect tests, and others i denti fi ed by
NY SED.
8
St udent s who at t ended Consort ium schools must have a passing score on the English Regent s
examinat ion and four Performance Based Assessment Tasks (PBAT) examinat ions.
14
To t est t he above crit eria, a t wo-st ep approach was applied as follows:
Step 1. Electronic audit tool
EY developed an elect ronic t ool t o det ermine whet her, based on t he dat a contained within t he Depart ment s St udent
Transcript and Academic Recording Syst em ( STARS ), t he sampled st udent graduat es had sat isfied t he course
credit s and Regent s examinat ion requirement s. For any sample st udent for whom sat isfact ion of t he requirement s
could not be validated t hrough t he elect ronic audit tool, t he t ransact ion was forwarded to Independent Reviewers
( Reviewers )
15
for a desk review of available records.
Step 2. Manual review of transcripts and other student academic records
Reviewers confirmed t he course credit and/ or Regent s examinat ion deficiency ident ified by t he EY elect ronic audit
t ool. For any st udent who did not meet t he requirement s, t he Reviewers were assigned t o review t he records
submit t ed by t he school. EY collected t he addit ional document at ion from t he schools and performed a quality review
16
of t he result s compiled by t he Reviewers.
B. RESULTS
Of t he 310 st udent s sampled, no except ions for Regent s examinat ions and academic course credit s were not ed.
17
111 st udent s passed all t hree crit eria in St ep 1.
199 st udent s passed after t heir t ranscript s were manually reviewed (Step 2).
a) Other Part 100 Compliance Issues Credits in Physical Education and Health
St udent s are required t o earn four credit s in physical educat ion ( PE ) and one credit in health t o earn a Regent s
diploma.
18
For any st udent in the sample for whom sat isfact ion of t he PE or healt h course credit requirement
could not be validated t hrough t he elect ronic audit tool, t he t ransact ion was reviewed t hrough a manual review of
t he st udent s t ranscript .
14
If the school awardi ng the di ploma is a Consorti um school, the exami nati on requi rements for a Regents di ploma are as follows: one
Comprehensive Engli sh Regents; one soci al studi es PBAT ( score of competent ) ; one sci ence PBAT ( score of competent ) ; one math
PBA T ( score of competent ) .
15
The I ndependent Revi ewers, who were former Guidance Counselors and Supervi sors of Guidance, had parti ci pated in the pri or
academi c data audi t conducted by OAG and had been trai ned i n the focus areas covered by the audi t.
16
EY vali dated the completeness and accuracy of the results provi ded by the Independent R evi ewers and veri fi ed that the results aligned
wi th the supporti ng documentati on.
17
Prior to 2012-13, the Department di d not have a standardized set of course codes for schools to use when enteri ng course data i nto
ST A RS. A s a result, schools reported course data in ST ARS in di fferent ways and, in some cases, used di fferent course codes for the
same course, thereby necessitating a manual transcri pt review for the 199 students who di d not initi ally meet one or more cri teri a i n
step 1. For the 199 students for whom sati sfacti on of the requi rements could not be establi shed through STA RS data, student transcri pt
data was forwarded to R evi ewers for a desk revi ew of avai lable records. T he Revi ewers revi ewed the selected student transcripts to
assess whether the students received the requi red mi nimum credi ts, met the Regents examinati ons requirements and met the requi red
credi t di stri buti on by subject i n compli ance with the mi ni mum number of credit requi rements.
18
Students must take PE every year that they are i n hi gh school. NY SED regulati ons provide an excepti on to the PE requi rement when a
student has met all other requi rements in fewer than four years, i n which case the student may graduate without having met the
remai ni ng P E credits they would otherwise need to earn.
9
For t he 310 randomly selected st udent s, t he Reviewers manually reviewed 199 t ranscript s and noted t hat 4
st udent s (or 1.29%of t he sample) did not meet t he minimum PE or healt h course credit requirement s. Given t he
pot ent ial for sampling error, t he audit result s reflect with 95%confidence t hat t he percent age of except ions in t he
populat ion lies bet ween 0%and 2.71%.
b) Other Course and Crediting Issues Documentation of Course Content
In analyzing t ranscript s, t he Reviewers ident ified courses whose t it les did not sufficient ly clarify t he nat ure of t he
course or did not appear on t heir face t o align wit h subject area cont ent or sequencing st andards. In cert ain
cases, t he schools were unable t o provide the relevant syllabus or ot her evidence t o indicat e t hat a course aligned
clearly wit h learning st andards in t he related subject . That informat ion was shared wit h t he school and net work
t eam leaders t o support school curriculum development and follow up where warranted.
C. RECOMMENDATIONS
i. Depart ment Management should cont inue to st andardize the course coding in STARS to facilit at e alignment
of courses t aken and credit s earned wit h academic requirement s and credit ing policies.
ii. The retent ion requirement for document at ion of course content (syllabi) should be clarified and reinforced.
EY
Assurance | Tax | Transactions | Advisory
About EY
EY i s a global leader in assurance, tax, transaction and
advisory services. Worldwide, our 167, 000 people are
united by our shared values and an unwavering
commi tment to quali ty. We make a di fference by helping
our people, our clients and our wi der communi ties achi eve
thei r potenti al.
Ernst & Young refers to the global organization of member
firms of Ernst & Y oung Global Limited, each of which is a
separate legal enti ty. Ernst & Young Global Limited, a UK
company limited by guarantee, does not provide services to
cli ents. For more informati on about our organi zati on,
please vi si t www. ey.com.
Ernst & Young LLP is a cli ent-serving member firm of Ernst
& Young Global and of Ernst & Y oung Americas operating
i n the US.
2013 Ernst & Young LLP
All Ri ghtsReserved.