Académique Documents
Professionnel Documents
Culture Documents
University Students
Author(s): Jeremy P. Smith and Robin A. Naylor
Source: Journal of the Royal Statistical Society. Series A (Statistics in Society), Vol. 164, No. 2
(2001), pp. 389-405
Published by: Blackwell Publishing for the Royal Statistical Society
Stable URL: http://www.jstor.org/stable/2680559 .
Accessed: 06/01/2011 06:13
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at .
http://www.jstor.org/action/showPublisher?publisherCode=black. .
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.
Blackwell Publishing and Royal Statistical Society are collaborating with JSTOR to digitize, preserve and
extend access to Journal of the Royal Statistical Society. Series A (Statistics in Society).
http://www.jstor.org
J. R. Statist.Soc. A (2001)
164, Part 2, pp. 389-405
1. Introduction
The question of why studentdrop-outrates vary across highereducationalinstitutions is
both importantand topical and has been the focus of considerableresearch activity,
especiallyin theUSA. For theUK, unliketheUSA, mostof theanalysishas been based on
university level data (see, forexample,Johnesand Taylor (1989, 1990)) or, in the case of
studentlevel data, has been conductedon relativelysmall samples of students(see Johnes
(1990) and Johnesand Taylor(1991)). Large scale microdataon UK university studentshave
not been available. This has now changedwithresearchaccess to the fullset of individual
studentlevel informationstoredin the UniversitiesStatisticalRecords (USR). (The USR
preceded the Higher Education StatisticsAgency as the depositoryfor all the statistical
returnsfromwhat are typicallyreferredto as the 'pre-1992' universities,predatingthe
abolitionofthebinarydivide.)These data, togetherwithmergeddata on schools,providethe
information thatwe exploitin the currentpaper.
A primarymotivationfor studyingthe non-completionrates of UK university students
stemsfromthe fact that the Governmentis currently developinga range of performance
indicatorsfor highereducation institutions.A firstwave of indicatorswas publishedin
December1999(see HigherEducationFundingCouncilforEngland(1999a,b)) and included
a performance measurebased on thenon-completion rate of studentsby institution.
One of
theprimaryobjectivesofthepublishedperformance indicatorsis to improveon rawrankings
Addressfor correspondence:
Robin A. Naylor,Departmentof Economics,University
of Warwick,Coventry,CV4
7AL, UK.
E-mail:robin.naylor@warwick.ac.uk
25
20
15
ia110 ;
0
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 48 40 52
University bymak wvthcrawl
(ordered rate)
Fig. 1. Distribution
ofwithdrawing *, females;*, males
studentsbyuniversity:
394 J. P. Smithand R. A. Naylor
16
14
c12
10
4
IL6'|
2
0 +
B C D FGA GB H J K L M N P Q R T V W X Y
Course
type
Fig. 2. Distribution
ofwithdrawing
studentsbycourse:*, females;*, males
Table 1. Drop-out
ratesbyspecificcharacteristics
A-levelscores
> 8.5 3.73 4.25
7.5-8.5 4.68 6.59
5.5-7.5 5.77 9.61
0-5.5 8.09 15.31
Schooltype
LEA 7.27 10.82
Independent 5.96 9.08
Furthereducation 5.65 9.35
Other 5.22 8.94
Parentaloccupationt
SC I 5.57 8.22
SC II 6.37 9.53
SC IIINM 6.23 9.77
SC IIIM 7.38 10.42
SC IV 7.81 12.68
SC V 6.45 12.24
Unemployed 14.21 18.82
SC II, intermediate;
tSC I, professional; SC IIINM, skilled
non-manual;SC IIIM, skilled
manual;SC IV,partly
skilled;
SC V, unskilled.
3.2. Statistical
modelling
We conducta binomialprobitregression analysisof the probability
thatan individual
withdrawsfromuniversity.
(Our resultsare notsensitive
to thechoicebetweenprobitand
logit estimation.Similarly,Dey and Astin (1993) found that resultsfromlogit and probit
Droppingout of University 395
A~~~~~~0 M
00000000000000000 00 mIDC CAcq00
00 0000 0
~~~~C)
O n cl E-
> .t t00m C) CD Co -O 00
mmD N ON 0C0
bk )
moN C) O) C) ) E- C CD
rq CD CD CDC)C)C)???nrnC)C)?? N CD C D ONC
????? 0c? - - ??
?l 0??
X~~~~~~~~~~~~~~~~~~~~~
00
8~~~~~~~~.
Q 0000#204Xm 000e0 00
~~~~~~~~~~Cs cs ;3 .o.oo.o .
.o o.o
,ooot ro
;.3
I I~~~~~~~~~-
In I 1 1
w~~~~~~~di .- q) r. r. oo - 0O0
'~ ~~~
a-cd ~
ad'= ~~~~
o -c:s 0 to ~ ~ ~ ~ 0
t I11
4-1
s snN I
00
-1 II cd 00> 0 t I0 - 2 -
-+- 0 0 0 d
to
S CO OOO0 t O O 00 0
cjO
0 ~~ 0 0 ~~
0 t4 OO
A~~~~~~~~~~~~-
000 M00CD 0000 00)00 000 00000 000-
E-,, 00)
cs~~~~~~~.
00 E- oo,tn Nt --- -- r- kn oN O N ) kn 01 E- :t o: o: o o CA ON ON
a)
o -~~~~~~~~~~~~~~~~~~~~~~~~~~~~C
~~~~~~~~~~~~~~~~1~0.- DC-C
ONC DC0)C
C . - 0 n11
_b~~~~~~~4 U o $o
yo:ooo
y om
50~~~~~~~ bb
t~~~~~~+,
6<Ove
E g,?U cd cn E _ 0 0 +. V ;
00 00 .~~~~~~~~~~~
0 ~~IQC 0 C
Droppingout of University 397
4. Results
Probitestimatesof theprobabilityof droppingout are presentedin Table 2, separatelyfor
male and female students.The equation includes controlsfor educational background,
degreesubjectand relatedattributes,
personalcharacteristics, of theuniversity
characteristics
department,unemployment in the countyof prior residenceand the universityattended.
Summarystatisticson themeansof each of theexplanatoryvariablesare presentedin Table
3. We discussthe estimatedmarginaleffectsforindividualuniversities
in Section5.
4.1. Educationalbackground
Withrespectto educationalbackground,Table 2 showsthatthestudent'sA-levelpointshave
Educationalbackground Personalcharacteristics
(continued)
Qualifications Nationality
BritishTechnicalEducationCounciletc. 0.046 0.024 UK 0.911 0.919
Other 0.032 0.043 European Union 0.031 0.035
No formalqualifications 0.031 0.036 Overseas 0.057 0.046
A-levels 0.821 0.821 Self-fundedoverseas 0.038 0.035
Highers 0.071 0.075 Universityresidence
A-levelpoints 7.050 6.934 Livingat home 0.096 0.112
Higherspoints 7.065 7.056 Livingoffcampus 0.110 0.113
Subjectpreparedness
Science 0.111 0.135 Degreesubjectand relatedattributes
Otherscience 0.020 0.011 Degree subject
Social science 0.076 0.141 B, medicalrelated 0.017 0.047
Languages 0.016 0.030 C, biologicalscience 0.063 0.112
Literature 0.042 0.042 D, agriculture 0.011 0.012
School type F, physicalscience 0.132 0.068
LEA school 0.418 0.451 GA, mathematics 0.064 0.043
Furthereducationcollege 0.100 0.106 GB, computing 0.054 0.011
Independentschool 0.285 0.259 H, engineering 0.184 0.033
Otherschool 0.198 0.185 J,technology 0.015 0.008
Single-sexschool 0.244 0.282 K, architecture 0.013 0.006
School points L, social science 0.109 0.130
Englandand Wales 17.180 16.936 M, law and politics 0.080 0.085
Ireland 37.326 34.784 N, businessadministration 0.060 0.062
Scotland 48.802 49.403 P, communications 0.002 0.006
Total pupils 835.679 803.415 Q, literature and classics 0.044 0.107
R, modernEuropeanlanguages 0.021 0.076
Personalcharacteristics T, otherlanguages 0.006 0.014
Age < 24 years 0.760 0.779 V, humanities 0.071 0.084
Age 24-27 years 0.177 0.141 W, creativearts 0.012 0.026
Age 28-33 years 0.040 0.035 X, education 0.006 0.026
Age > 34 years 0.023 0.044 Y, othersubjects 0.037 0.043
Married 0.020 0.040 Otherdegreeinformation
Parentaloccupation Departmentsize 5.053 4.952
SC I 0.183 0.189 Jointdegree 0.136 0.179
SC II 0.410 0.429 4-yeardegree 0.217 0.272
SC IIINM 0.114 0.108 Sandwichdegree 0.073 0.057
SC IIIM 0.104 0.093
SC IV 0.063 0.054 Economicactivity
SC V 0.011 0.008 Unemployment 5.887 6.011
Unemployed 0.069 0.079
398 J. P. Smithand R. A. Naylor
statisticallysignificant
effectson thedrop-outprobabilitiesforboth males and females.For
males,a pointon theA-levelaverage(equivalentto one extraA-levelgradefora studentwith
twoA-levels)reducesthedrop-outprobability by about 1.4 percentagepoints,ceterisparibus.
For females,the marginaleffect,althoughstatisticallysignificant, is smaller.For males,
significanteffectsare also associated with the average points score in Scottishor Irish
Highers.The analysisalso examinesthe effectof the students'A-level (or Highers)score
relativeto the average scoredby all A-level(or Highers)candidatesin the previousschool
attended.The resultssuggestthat,forboth males and females,the drop-outprobabilityis
lowerforstudentswho performed well relativeto the averageof theirschool peer group.
The analysisalso attemptsto take accountof the effectsof academicpreparednessin the
sense of the closenessof the matchbetweenthe subjectstudiedat university and the prior
subjectsstudiedat A-level.For specificsubjectareas, we definestudentsas poorlyprepared
fortheirdegreesubjectiftheyfailedto have at least two relevantsubjectsamongtheirprior
qualifications.For the purposes of interpretation, the default,for each subject dummy
variable,is a studentwho is well matchedto the course that theyare taking.The results
suggestthatthesubjectmatchis importantonlyforscienceand social-science-based subjects.
(Science includesbiology,physicalscience,mathematics,engineering, medical relatedand
technology.Other science is definedas agriculture,computingand architecture.)For
example,studentswho are poorlymatchedto social sciencesubjectsat university are around
1.3 percentagepointsmorelikelyto drop out in thecase offemalesand 2 percentagepointsin
thecase of males. The lack of significance in thecase of languagesis likelyto stemfromthe
factthatsignificant familiarityis a prerequisite formostuniversity languagecourses:in other
words,thereis relativelylittlevariationin thisvariable.Of languagestudents,only 1.6% of
males and 3% of femalesare poorlymatchedto theirdegreesubject.The resultsgivesupport
to the view that studentswho are well preparedforuniversity are less likelyto drop out,
especiallyforstudentstakingmore science-baseddegreesubjects.
Table 2 also showstheestimatedeffects ofpreviousschoolcharacteristics on theindividual
student'sdrop-outprobability.Of theschool typevariables,theindependentschoolvariable
is significantin themodelsforbothmales and females.The coefficient impliesthata student
whohad previouslyattendedan independent schoolis about4 percentagepointsmorelikelyto
drop out thana studentwho had previouslyattendedan LEA school. Againstthis,students
who had attendedsingle-sexschools are 1 percentagepointless likelyto drop out. Control
variablesare includedfortheaverageperformance of thelast school attendedagainstofficial
DepartmentforEducationand Employment criteria.Thereis no clearpatternof effects. The
size of school does not appear to have a significant effecton the drop-outprobability.
Dummy variableswere also includedfor othertypesof universityentryqualifications,
such as BritishTechnicalEducation Council qualifications, otherqualificationsand no for-
mal qualifications.There are some significanteffectsacross the different types of entry
qualification,thoughtheseare not reported.
Compared witha studentfroma social class II (SC II) background i.e. whereparental
occupationwas definedas technicalor intermediate professional therewas no significant
differencein thedrop-outprobabilityforstudentsfromothersocial class backgroundswith
theexceptionof male studentsfromSC I (professionaloccupations).These studentsare less
likelyto drop out. (Dummy variablesfor social class backgroundare also interactedwith
unemployment, as is discussedbelow.) Parentsof studentsfromSC I backgroundsare most
likelyto have studiedat universityand thismayincreasethepreparednessand motivationof
such studentsto completetheirdegreestudies.It willbe interesting to compareour findings
withdata for cohortsenteringuniversity afterthe introductionin 1998 of 'home student'
tuitionfees.
Table 2 shows thatnon-UK European studentsare significantly more likelyto drop out
than are UK students,as are overseasstudents(witha verylarge marginaleffect) unless
theyare self-funding, in whichcase theyare less likelyto drop out evencomparedwithUK
students.Self-funding studentsmay have a greatercommitment to completetheirstudies.
Studentswho liveat theparentaladdressare around2-2.5 percentagepointsmorelikelyto
drop out, and studentswho live offcampus are around 5 percentagepointsmore likelyto
drop out than are studentswho live on campus. These findingsare consistentwith the
hypothesesof Tinto and otherson the importanceof social integrationat university.
4.4. Unemployment
From Table 2, it appears that, for both male and female students,the county level
unemploymentrate has a well-determined positive effecton the individual drop-out
probability.The estimatedmarginaleffectimpliesthat if the local unemployment rate is
higherby 5 percentagepointsthenthedrop-outratewillbe 1 percentagepointhigher.Given
the possibilitythat the effectof local unemployment on the withdrawalrate mightdiffer
across differentgroupsof students,we interactthelocal unemployment ratewitha dummy
variableforstudentsfroma low social class background(i.e. classes SC IIIM, SC IV, SC V
(see Table 1 for definitions)and unemployed).For males, this interactiontermis both
positiveand significant, indicatingthat studentsfromlower social class backgroundsare
more sensitiveto local labour marketconditions,as hypothesizedin Section2. That social
class influencedthedrop-outprobabilitiesof studentsstartingcoursesin 1989underlinesthe
importanceof ensuringfinancialsupportforstudentsfrompoorerbackgrounds.A second
corollaryis thatthereis a potentialclash of incentivesforuniversities
attempting to widen
access whilebeingconstrainedby performance indicatorsbased on withdrawalrates.
Finally,we have examinedthe Oaxaca (1973) decompositionto investigatethe extentto
whichmale-femaledifferences in
in the drop-outprobabilityare attributableto differences
characteristicsratherthanin theparametersof theestimateddrop-outfunction.The results
implythatonlyarounda sixthof themale-femaledifference can be explainedby differences
in observedcharacteristics.This confirms thehypothesisthatstudentwithdrawalbehaviour
is verydifferentby gender.
The next section of the paper is concernedwith derivingrankingsof universityper-
formancefromthe universitymarginaleffectsestimatedfromthe probit models for the
probabilityof droppingout. As well as usingthesemarginaleffectsto generateperformance
measuresby institution, we have also regressedthe estimateduniversitymarginaleffectson
the teachingqualityassessmentscoresof each university, separatelyformales and females.
We found a highlysignificantnegative relationship,indicatingthat the probabilityof
withdrawaldiminishesas measuredteachingqualityrises.This givessupportto theargument
Droppingout of University 401
ratesand university
5. Drop-out indicators
performance
In announcingthepublicationof a performance rates,the
indicatorbased on non-completion
UK Government'sFunding Council has statedthat
'Ifall students within
qualified theexpected wouldbe 100percentefficient'
time,thentheinstitution
(HigherEducationFundingCouncilforEngland,1999c).This viewof university withdrawal
is not uncontroversial. Althoughclearlycostsare associatedwithuniversity withdrawalboth
forinstitutions and forindividuals(see, forexample,the discussionof costs in DesJardins
(1999)) it is unlikelythatthe optimalwithdrawalrateis 0. Johnesand Taylor (1989) argued
thatnon-completion does not mean thata studenthas receivedno benefitfromtheirstudies.
Indeed,somewithdrawing studentsmayreturnto theirstudiesat a laterdate. (Unfortunately,
we cannot obtain an estimateof theproportionof withdrawing studentswho subsequently
return.Studentswho maintaintheirregistrationwhile delayingtheir studies keep their
personalidentification code. In contrast,therecordsofstudentswhowithdrawareterminated.
Any returning studentis thenallocated a new personalidentification code.) Furthermore, a
successful matching betweendegreecoursesand theabilitiesand preferences ofstudentsis likely
to requireadaptabilitysuchthatsomewithdrawalis desirableon efficiency grounds.Similarly,
indicatorsof non-completion can potentiallyconflictbothwithpoliciesof wideningaccess to
highereducationand withthemaintenanceof academicquality(see Cave et al. (1997)). These
factorsshould be bornein mindwheninterpreting data on non-completion.
The university performanceindicatorspublishedby the UK Governmentin December
1999are intendedto be an improvement on thepublicationof rawdata whichhave tendedto
generatepotentially misleadinguniversity leaguetablesin thepressand media.The published
performance indicatorsprovidecontextvariablesby whichthe performanceof universities
can be interpreted in the lightof relevantcircumstances, such as the school or social class
backgroundsofthestudentintake.In thecase ofuniversity drop-outrates,thedata are at the
aggregateuniversity level and the methodof adjustmentinvolvesonly a small numberof
variables.Johnesand Taylor (1990) provideda methodfor the constructionof university
performance againstthecriterionof thenon-completion rateusinguniversity leveldata: see
also Johnes(1992). MacPherson and Paterson (1990) argued for universityperformance
measuresto be based on microdatabut, at that time,large scale individuallevel data for
whole cohortswere not available to academic researchers.Astin (1997), in an analysisof
retentionratesamongUS universities, arguedthatit is vitalto controlforthecharacteristics
of studentswhen assessing institutionalperformance.In the currentpaper, we have
developeda model of the individualstudentdrop-outprobabilitybased on administrative
studentlevel data. This can be used as the basis for the constructionof a university
performance indicator.In thissectionofthepaper,we outlinea methodforconstructing such
a performance indicatorforthenon-completion rate.As importantly, we discusssome of the
limitationsof performance indicatorsin thisarea.
From the estimateduniversitymarginaleffects,derivedfromthe models reportedin
Section4, we can deriverankingsof university performance. Fig. 3 comparestheseadjusted
universityrankingswith the raw or unadjustedrankings,as depictedin Fig. 1 for male
students.Fig. 3 shows the sensitivity of the rankingsto the adjustmentfor the control
variablesincludedin theunderlying estimationprocedure.Each pointin Fig. 3 represents an
402 J. P. Smithand R. A. Naylor
institutionand the co-ordinatesrepresentthe adjusted and unadjustedrankings.Ranked
number 1 on the horizontalaxis is the universitywith the lowest unadjusted drop-out
probabilityand, on the verticalaxis, is the universitywith the lowest marginal effect
estimatedfromtheprobitmodelpresentedin Section4. If a university is observedon the450
line,thisindicatesthat its rankpositiondoes not change aftercontrollingforstudentand
coursecharacteristics.
It is clearfromFig. 3 thattherankingof universities on thebasis of theadjustedeffectsis
verydifferent fromthatbased on theunadjusteddata. The correlationbetweentheadjusted
and unadjustedrankingsof universities is 0.66 for male studentsand the mean absolute
movementis 9.4 rank places, with a substantialvariationaround this mean. The largest
moveis 39 places. Onlyfiveofthetop-rankeduniversities on theunadjustedbasis retaina top
10 rankingafteradjustment.Similarly,only threeof the bottom 10 maintaina bottom 10
rankingafteradjustment.
An importantquestion concernshow confidentwe can be in the exact orderingof
universities in performance-based rankings.In theliteratureon theperformance of schools,
Goldstein and Spiegelhalter(1996) concluded that the best statisticalanalysis of school
performancegeneratesrankingssuch that most schools cannot be separated with any
significantdegree of confidence.This result,theyargued,underminesthe validityof the
league table exercisefor schools. To examinethis issue in the contextof our rankingof
universities againstthecoursenon-completion we calculatetheconfidenceintervals
criterion,
aroundthepointestimatesofeach university's marginaleffecton thewithdrawalprobability,
relativeto the medianuniversity.
Fig. 4 presentsthe95% confidenceintervalsfortheadjusteduniversity marginaleffectson
thedrop-outprobabilitiesformales,orderedbytheirmarginaleffect. A lineis drawnthrough
the point estimateforthe median university. This line cuts the majorityof the confidence
intervals,indicatingthatwe can have littleconfidencein the rank positionof most of the
universities relativeto themedian.For males,onlythetop six and thebottom12 universities
performsignificantly fromthe median.
differently
50
0) 40
.C
30
0 10 20 30 40 50
Unadjusted universityranking
Fig. 3. rankingsformales
Comparisonof adjusted and unadjusteduniversity
Droppingout of University 403
15
10
5)
0)
-10
University
(ordered effect
bymarginal on withdrawal
rate)
The resultsfor femalesare very similarto those for males. For female students,the
correlationbetweentheadjustedand unadjustedrankingsofuniversities is 0.77 and themean
absolutemovementis 8.2 rankplaces,witha maximummove of 38 places. Sevenof thetop-
rankeduniversities on theunadjustedbasis retaina top 10 rankingafteradjustment,and six
of thebottom10 maintaina bottom10 rankingafteradjustment.As formales,we can have
littleconfidencein the rank positionforfemalesof most of the universitiesrelativeto the
median: only threeof the top fouruniversities togetherwiththe bottomfouruniversities
performsignificantly fromthe median.
differently
6. Discussion
We have estimateda binomial probit model of the individual student'sprobabilityof
withdrawalfromuniversity. The data set is based on the individualstudentlevel USR for
1990-1993forthe fullpopulationof undergraduatestudents,embarkingon a 3- or 4-year
degreeprogrammein the autumnof 1989 at a pre-1992university. We have matchedthese
data both to school level informationusing data obtained from the Departmentfor
Education and Employmentand fromthe Scottishand NorthernIreland Officesand to
countyleveldata on unemployment.
Among other results,we find that the probabilityof dropping out of universityis
influencedsignificantlyby pre-university education,personal attributes,the degreesubject
and characteristicsof the departmentand university. The countylevel unemployment rate
also has a well-determinedpositiveeffecton theindividualdrop-outprobability.On average,
an increase in the local unemploymentrate of 5 percentagepoints raises the drop-out
probabilityby around 1 percentagepoint. For male studentsfrom lower social class
backgrounds,theeffectof local unemployment is morethantwiceas large,implyingthatthe
on-goingparticipationof such studentsat university is much more sensitiveto local labour
marketconditions.That social class influencedthedrop-outprobabilitiesof studentsstarting
courses in 1989 suggeststhe desirabilityof devisingfundingpolicies in such a way as to
ensurefinancialsupportforpoorerstudents.
404 J. P. Smithand R. A. Naylor
The data that we have analysedprovidea richsource of informationon UK university
students.None-the-less, thereare limitationsof thedata and thesehave imposedconstraints
on the modellingprocedure.First,the data set refersexclusivelyto studentswho entered
universityand, consequently,we have not been able to address the issue of selectioninto
university.Second,thedata do not capturestudentswho substantially delaythecompletion
of theirstudiesand, ultimately, thesestudentsmay be disproportionately more likelyto fail
to complete.However,we have suggestedthat such studentsare anywayrelativelyfewin
number.Third,some of thestudentswho are observedto drop out may subsequentlyreturn
to studylater,perhapselsewhere,but thereare no administrative data withinformation on
this.Fourth,althoughthereare likelyto be diversereasonsfordroppingout ofuniversity, we
have modelledthedrop-outprobabilityas a dichotomousvariableratherthanby adoptinga
competingrisksframework, forexample.This is because the data do not providereliable
information on thereason forwithdrawal,eitherdirectlyor indirectly throughresponsesto
thefirst-destination
survey.
Finally,we have shown how the analysisof the individualstudentdrop-outprobability
mightbe used to constructa university performance indicator.We have demonstrated thatan
adjustedrankingofuniversities, based on estimateduniversityceterisparibusmarginaleffects
on thedrop-outprobability, is ratherdifferentfroman unadjustedranking.We have argued
thata further limitationof drawingup a league table of universitiesbased on the estimated
marginaleffectson the drop-outprobabilityis that the confidenceintervalsare verywide
around the individualuniversity point estimates:we can have littlestatisticalconfidencein
therankings.This is so fortherelatively homogeneousgroupof pre-1992universities. It will
be interestingto compare our resultswith an analysis of more recentcohorts for the
expandeduniversity sector.An issue whichwe do not pursuein the currentpaper concerns
the question of what is the appropriateset of controlvariablesto include for generating
universityperformance indicators.For a discussionof thisin thecontextof graduatelabour
marketoutcomes,see Smithet al. (2000).
Acknowledgements
We are gratefulto Wiji Arulampalam,Keith Cowling,GeraintJohnes,Abigail McKnight,
Mark Stewartand JimTaylor and to therefereesforhelpfulcomments.Many people have
givenus invaluablehelp in generatingthe data set: in particular,we thankJohnMcClure
and PeggyPaull at the UniversitiesCouncil on Admissionsof Students,JohnMcNeill at
Warwick and staff at the Department for Education and Employment,the Higher
Education FundingCouncil for England,the HigherEducation StatisticsAgencyand the
Schools Register.We acknowledgeboth the USR, as the originaldepositors,and the UK
Data Archivefor the use of the data set SN:3456 USR. None of these individualsor
organizationsbears any responsibility presentedin
forany of the analysisor interpretation
thispaper.
References
Astin,A. (1979) Four CriticalYears. San Francisco:Jossey-Bass.
(1997) How 'good' is yourinstitution's retentionrate?Res. HigherEduc., 38, 647-658.
Booth,A. L. and Satchell,S. E. (1995) The hazardsof doinga PhD: an analysisof completionand withdrawalrates
of BritishPhD studentsin the 1980s.J. R. Statist.Soc. A, 158, 297-318.
Cave, M., Hanney,S., Henkel,M. and Kogan, M. (1997) The Use of Performance Indicatorsin HigherEducation.
London: Kingsley.
Droppingout of University 405
DesJardins,S., Ahlburg,D. and McCall, B. (1999) An eventhistorymodel of studentdeparture.Econ. Educ. Rev.,
18, 375-390.
Dey, E. and Astin,A. (1993) Statisticalalternativesforstudyingcollegestudentretention: a comparativeanalysisof
logit,probitand linearregression.Res. HigherEduc., 34, 569-581.
Fielding,A., Belfield,C. and Thomas, H. (1998) The consequencesof drop-outson the cost-effectiveness of 16-19
colleges.Oxf Rev. Educ., 24, 487-511.
Fitz-Gibbon,C. (1996) Monitoring Education.Trowbridge:Redwood.
Goldstein,H. and Spiegelhalter, D. J. (1996) League tablesand theirlimitations:statisticalissuesin comparisonsof
institutional performance (withdiscussion).J. R. Statist.Soc. A, 159, 385-443.
HigherEducation FundingCouncil forEngland(1999a) Performance indicatorsin highereducation.Report99/11.
HigherEducation FundingCouncil forEngland,Bristol.
(1999b) Performanceindicatorsin highereducationin the UK. Report99/66.HigherEducation Funding
Council forEngland,Bristol.
(1999c) Press Release, Dec. 3rd. HigherEducation FundingCouncil forEngland,Bristol.
Johnes,G. (1992) Performance indicatorsin highereducation:a surveyofrecentwork.Oxf.Rev.Econ. Poly,8, 19-34.
Johnes,J. (1990) Determinantsof studentwastagein highereducation.Stud. HigherEduc., 15, 87-99.
Johnes,J. and Taylor,J. (1989) Undergraduatenon-completion rates:differences betweenUK universities. Higher
Educ., 18, 209-225.
(1990) Performance Indicatorsin HigherEducation.Oxford:OxfordUniversity Press.
(1991) Non-completionof a degreecourseand its effecton the subsequentexperienceof non-completers in
the labour market.Stud. HigherEduc., 16, 73-81.
Kalsner,L. (1992) Issues in College studentretention.HigherEduc. ExtensnServ. Rev.,3.
Krueger,A. (1999) Experimentalestimatesof educationproductionfunctions.Q. J. Econ., 114, 497-532.
Layard,R. and Psacharopoulos,G. (1974) The screeninghypothesisand thereturnsto education.J. Polit.Econ.,82,
985-998.
MacPherson,A. and Paterson,L. (1990) Undergraduatenon-completion rates:a comment.HigherEduc., 19, 385-
390.
Noel, L. and Levitz,R. (eds) (1985) IncreasingStudentRetention.San Francisco:Jossey-Bass.
Oaxaca, R. (1973) Male-femalewage differentials in urban labour markets.Int. Econ. Rev., 14, 693-709.
Porter,0. (1990) Undergraduate Completion and PersistenceinFour-yearCollegesand Universities. WashingtonDC:
National Instituteof IndependentColleges and Universities.
Riley,J. (1979) Testingthe educationalscreeninghypothesis.J. Polit. Econ., 87, 227-252.
Smith,J.,McKnight,A. and Naylor,R. (2000) Graduateemployability: policyand performance in highereducation
in the UK. Econ. J., 110, 382-411.
Spence,M. (1973) Job marketsignalling.Q. J. Econ., 87, 354-374.
Tinto,V. (1975) Dropout fromhighereducation:a theoreticalsynthesis of recentresearch.Rev. Educ. Res., 45, 89-
125.
(1987) LeavingCollege:Rethinking theCauses and CuresofStudentAttrition. Chicago: University of Chicago
Press.
(1988) Stages of studentdeparture:reflections on the longitudinalcharacterof studentleaving.J. Higher
Educ., 59, 438-455.
(1997) Classrooms as communities:exploringthe educationalcharacterof studentpersistence.J. Higher
Educ., 68, 599-623.