WATER RESOURCES RESEARCH,

VOL. 19, NO. 1, PAGES 251-259, FEBRUARY

1983

Evaluation of Maximum Likelihood Parameter Estimation Techniquesfor

Conceptual Rainfall-Runoff Models' Influence of Calibration Data Variability
and Length on Model Credibility
SOROOSHSOROOSHIAN
1
Departmentsof Systemsand Civil Engineering,CaseInstitute of Technology,Case WesternReserveUniversity
Cleveland, Ohio 44106

VIJAI KUMAR GUPTA AND JAMES LLOYD FULTON

Departmentof SystemsEngineering,CaseInstituteof Technology,Case WesternReserveUniversity

Cleveland,Ohio 44106

The success
of an automaticcalibrationprocedureis highly dependenton the choiceof the objective
functionand the nature (quantity and quality) of the data used.The objectivefunctionshouldbe selected
on the basisof the stochasticpropertiesof the errorspresentin the data and in the model.Also, the data
shouldbe chosenso as to contain as much valuableinformationabout the processas possible.In this
paper we comparethe performanceof two maximumlikelihoodestimators,the AMLE, which assumesthe
presenceof first lag autocorrelatedhomogeneous
varianceerrors, and the HMLE, which assumesthe
presenceof uncorrelatedinhomogeneousvarianceerrors, to the commonlyused simple least squares
criterion,SLS.The modelcalibratedwasthe soilmoistureaccountingmodelof the U.S. National Weather
Service'sriver forecastsystem(SMA-NWSRFS). The resultsindicate that a properly chosenobjective
functioncan enhancethe possibilityof obtaininguniqueand conceptuallyrealisticparameterestimates.
Furthermore,the sensitivityof the estimationresultsto variouscharacteristics
of the calibrationdata, such
ashydrologicvariabilityand length,are substantiallyreduced.

tested in a simulation environment using a simple twoparameter

model and were shown to provide more reliable
It is generally acceptedthat the reliability of operational
parameter
estimates
than the SLS criterion (which is also a ML
conceptualrainfall-runoff(R-R) modelsusedin forecastingis
estimator
under
the
assumption
of uncorrelatedhomogeneous
highlydependenton the adequacyof the calibrationprocedure
employed.An approachto calibrationthat hasbeengainingin errors). Additional support for thesenew procedureswill be
popularity is the 'automatic' techniquewhich relies on the presentedaspart of thispaper.
The choiceof an optimizationalgorithmhas beendiscussed
capabilitiesof a digital computer. This approach has been
discussed
extensivelyin the literature [e.g., Dawdy and O'Don- by many researchers[lbbitt, 1970; Johnstonand Pilgrim, 1976;
nell, 1965; lbbitt and O'Donnell, 1971; Jacksonand Aron, 1971; Sorooshian,1981b;among others]. Iterative searchtechniques
Clarke, 1973; Monro and Anderson,1974; Johnstonand Pilgrim, (e.g., Rosenbrock's[1960] technique or Hooke and Jeeves
1976; Diskin and Simon, 1977; Pickup, 1977; Sorooshianand [1961] pattern search method, etc.) have been found to be
Dracup, 1980; Jamesand Burges, 1982; among others.]. Al- superior to gradient techniquesdue to the fact that explicit
though desirabledue to its speedof implementation,it hasbeen partial derivativesof the functionwith respectto the parameters are very difficult to obtain. The inefficienciesof search
criticizedwith respectto its accuracyand reliability.
The automatic estimation technique consistsof three ele- techniquesin the presenceof a large number of interacting
ments: (1) objectivefunction,(2) optimization algorithm, and parameters are well known. The effectivenessof these tech(3) calibration data. The problemsassociatedwith the choiceof niquesis highlydependenton the natureof the responsesurface
a suitableobjectivefunctionwere discussedin great length by generatedby the objective function being used. Sorooshian
proper
Sorooshianand Dracup [1980]. The most commonlyusedob- [ 198lb] demonstratedthat the choiceof a stochastically
jective function has been the simple least squares(SLS) cri- objectivefunction can result in a better responsesurface(i.e.,
terion. As discussedby Sorooshianand Dracup, the SLS cri- reducedincidenceof local optima, reducedparameterinteracterion might not be the bestchoicebecauseit doesnot take into tion), thusreducingthe possibilityof prematureterminationof
account the stochasticproperties of the measurementuncer- the algorithm.
It is evidentthat the success
of any calibrationprocedureis
tainties in a realisticmanner. They presentedobjectivefunchighly
dependent
on
the
nature
(quantity and quality) of the
tions basedon maximumlikelihood (ML) theory for two types
data used.It has often beensuggestedthat the calibrationdata
of error structures that are known to exist in rainfall and runoff
of the variousphenomenaexperimeasurements,namely, autocorrelated errors and heterosce- shouldbe as 'representative'
enced
by
the
watershed
as
possible.Many researchershave
dastic (inhomogeneousvariance)errors. These methods were
attemptedto satisfythis requirementby usingas long a length
Now at theDepartmentof HydrologyandWater Resources,
Uniof calibrationdata as waspossible.This approachhoweverhas
versityof Arizona, Tucson,Arizona 85721.
not provideddemonstratablysuperiorresults.The authorsbeCopyright 1983by the AmericanGeophysicalUnion.
lieve that it is not the length of data usedbut the information
containedin it and the efficiencywith whichthat informationis
Paper number 2W 1890.
0043-1397/83/002W-1890505.00
extracted that is important. The latter point relates to the
INTRODUCTION

251

252

SOROOSHIAN ET AL.: CONCEPTUAL RAINFALL-RUNOFF

UPPER

MODELS 1

ZONE

EE wTE[ I

L:R [ZE
FREE
TENSlOt4 WATER
.ZTw

WATER

czs

tZrP

LZrS

RSERV

PRUV,
ARY
A,
FLOW
Fig. 1. SMA-NWSRFScatchment
model[fromPeck,1976]

choiceof a stochastically
proper objectivefunction.Such a
choiceshouldreducethe sensitivityof the estimationresultsto
the type and quality of the calibrationdata. In addition,it is

that thereare certaininherentproblemsrelatedto modelstructure

that

must

be resolved

before

the full

benefit

of these

estimationtechniquescan be realized.Theseissuesare pursued

likely that the effort and cost entailedin obtaininglonger further in the other two papers [Sorooshianand Gupta, this
thisissue].
periodsof datamaynot bejustifiedby theimprovements
in the issue;GuptaandSorooshian,

parameterestimates.
In otherwords,oncea certainpoint has
MODEL USED
beenreached,the gainin informationacquiredby a stochasticallyproperestimationcriterioncan be marginaland the cost
The model usedwas the soil moistureaccountingmodel of
of its acquisitionand useunjustified.
the U.S. National Weather Serviceriver forecastsystem(SMAClearly, all of the componentsof the automaticcalibration NWSRFS) [seePeck, 1976]. This model is essentiallythe Sactechniquediscussed
abovemustbe studiedtogether.Thereis ramento watershedmodel of Burnashet al. [1973]. We will
no evidence in the literature that this has been done so far. An

integratedapproachisessential
to helpestablishto whatextent
the reliability of conceptualrainfall-runoffmodelsis relatedto
the effectiveness
of the calibrationprocedure.Only then can
onebeginto seriouslyexaminethe adequacyof themodelitself.
With thisviewin minda twoyearstudy,supported
by theU.S.
Officeof Water Researchand Technology(OWRT) and the
HydrologicResearchLab of the U.S. National WeatherService
(HRL-NWS) wasconducted.
The resultsof thisstudywill be
reportedin three papers.In this paper we will examinethe
followingquestions
relatedto the choiceof objectivefunction
and the calibration data.

avoid a discussionof this model becauseit has been fully

describedin the literature [e.g., Brazil and Hudlow, 1981]. A

schematic
representation
of the modeland the list of its parametersappearin Figure1 andTable 1,respectively.
DATA USED

A continuous18-yearrecord(water years(WY): 1952-1969)

of data for the Leaf River Basin(1949 km') near Collins,

Mississippiwas obtainedfrom the HRL-NWS. The data consistedof 6-hourly lumped precipitationvalues(millimetersof
depth),daily potentialevapotranspiration
estimates(millimetersof depth),and meandaily streamflowdischargerates(cubic
metersper day).

1. How sensitive
is the performance
of the estimationprocedureto calibrationdata variability(dry, average,and wet
datarecords),
andcanan appropriately
chosenobjective
funcOBJECTIVE FUNCTIONS USED
tion reducethissensitivity?
The relative performanceof three objectivefunctionswere
2. How sensitive
is the performance
of the estimationprocedureto calibrationdata length,and can an appropriately evaluatedin thisstudy.Theseobjectivefunctionsare asfollows.
chosenobjectivefunctionreducethissensitivity?
In the contextof this paper,'performanceof an estimation
procedure'
will bejudgedon thebasisof thefollowingcriteria:
1. Are the resultingparameterestimates
conceptually
real-

1. The SimpleLeastSquaresCriterion(SLS)

MinSLS
= ,2
O

istic ?

2. How reliableare the forecasts

obtainedusingtheseparameter estimates ?

3. How muchdo parameterestimates

varyfromdatasetto
data set ?

We will show that quite substantialimprovementsin the

calibrationresultscan be achievedby the useof a ML estimationprocedure.
However,the resultsof thisstudyindicate

/=1

(1)

wheregt-- qt,obs
-- qt,sim
= residualat timet andqt,obs,
qt,sim
are
the measuredand the simulatedflowsat time t, respectively,
9is
the set of parameters to be estimated,and n is the number of

data points. Note that the estimatesobtained using SLS are

equivalentto thoseobtainedusingML theory if the errorsare
assumedto be Gaussian,independent,and havehomogeneous
variance.

SOROOSHIANET AL.: CONCEPTUALRAINFALL-RUNOFF MODELS 1

TABLE

Notation

1.

Parameters

and State Variables

Included

NWS Acronym

in the SMA-NWSRFS

Catchment

253

Model
NWSP

Description

Parameter

Set

Parameters

T *
T2
T3'
T,
Ts
K*
K2
K3
Ax
A2
A3

UZTWM
UZFWM
LZTWM
LZFPM
LZFSM
UZK
LZPK
LZSK
PCTIM
ADIMP
RIVA

Z*
X*

ZPERC
REXP

P,,
Pe

PXADJ
PEADJ

Ps

PFREE

RSERV

decimal fraction to lower zone free water not transferable

to lower zone water tension

SIDE

ratio of deep rechargeto channelbaseflow

C
C2
C3
C,
Cs
C6

UZTWC
UZFWC
LZTWC
LZFPC
LZFSC
ADIMC

upper zone tension water contents,mm

upper zone free water contents,mm
lower zone tension water contents, mm
lower zone free primary contents,mm
lower zone free supplementalcontents,mm
tension water contents of the ADIMP area, mm

20.00
25.00
200.00
140.00
45.00
0.35
0.004
0.15
0.025
0.15
0.00
200.00
3.30
1.00
1.00
0.10

upper zone tensionwater capacity,mm

upper zone free water capacity,mm
lower zone tensionwater capacity,mm
lower zone primary free water capacity,mm
lower zone supplementalfree water capacity,mm
fractionaldaily upperzone free water withdrawalrate
fractionaldaily primary withdrawal rate
fractionaldaily supplementalwithdrawalrate
minimum imperviousarea (decimalfraction)
additionalimperviousarea (decimalfraction)
riparian vegetationarea (decimalfraction)
maximum percolationrate coefficient
percolationequationexponent
precipitationadjustmentfactor
ET demandadjustmentfactor
decimalfraction of percolatedwater going directly to
lower zone free water storage

0.30

State Variables

*Parameterschosento be optimized.

The Maximum LikelihoodEstimatorfor the Autocorrelated wherewtis the weightat time t, and it is computedby
Error Case(AMLE)
.

Wt =ft 2(x-1)

0.v
2n

212
MinAMLE=ln(2;0+lnl_p2
2020.v

wheref, is theexpectation
of q,.true
(eitherqt,ob,
or qt,sim)
and2 is

O,p,v 2

q-2'-
12,2(,= PEt
- 1)
2

(2a)

the unknown transformation parameter which stabilizesthe

variance.The implict expressionto estimate2 is

[,=lln
(f)l
I,=w,,21
- n[,iwt
In(f)t2]
=0 (3c)

where

Briefly, the HMLE

02
--H
1I--P2g2
+t=2
',(gt
Pgt-)21
(2b)

estimatoris derived basedon the assump-

tion that the errors are Gaussian

and p is estimatedfrom the implicitequation

(3b)

2 m

g1

gt-1 p3 +
t=2

ancematrixV,whereV,,,= 0't2andV,,,+s= 0 fors 4:0.

Stabilizationof the varianceis attemptedthroughthe useof
the Box-Cox [Box and Cox, 1964] power transformationwhich
relatesthe varianceof eacherror to its associatedoutput value.
[see $orooshianand Dracup, 1980; $orooshian,1981a]. In this

gtgt-1 P2
t=

q- 0'02-- '12q-

gt-12 P _

with mean zero and covari-

gtgt
- 1 "- 0

(2c)

study,f,= q,.ob,
wasusedin thecomputation
of theweights(the
originalprocedurereportedin the aforementionedpapersused

[1982] hasshownthat thisresultsin a more

where0'v2 is a constantvarianceterm and p is the first lag ft = qt.,im)-Fulton

stable estimationscheme.It is interestingto note that if the

variancesof the additiveerrorsare homogeneous(independent
of time or magnitude of the associatedflows), then the procedure will automatically selectthe value of 2 1.0. This resultsin w, = 1 for all t and the objectivefunctionreducesto the
SLS. If however,the varianceof the errors is proportional to a
3. The MaximumLikelihoodEstimatorfor the Heteroscedastic power function of the magnitude of the flows then the procedurewill selecta valueof 2 4: 1.0.Pertinent to our problemis
Error Case(HMLE)

autocorrelationcoefficient.The above estimator is developed

based on the assumptionthat the output errors are normally
distributedwith a constant variance and correlatedaccording
to a first lag autoregressive
scheme(for details,see$orooshian
andDracup[ 1980]).

the case where the error variance increases as the flow values

Min HMLE =
o,

w,et
2 n
t= 1

w,

(3a)

get larger ($orooshianand Dracup [1980] have in great length

discussedthe underlyingreasons).In this situation the maxi-

254

SOROOSHIAN
ET AL.: CONCEPTUALRAINFALL-RUNOFFMODELS 1

600f
5OO

18 year mean

4OO
300
200
100

time

(year)

Fig. 2. Streamflowyearly volumesummary(millimeters),Leaf River, Collins,Mississippi.

mum likelihood estimateof 2 will be lessthan unity, and this

ensuresthat in the objectivefunctionthe errorsassociatedwith
lower flows (which contain more reliable information) are
weightedmore heavily.

year data usingSLS, AMLE and HMLE as objectivefunctions

(runs3, 6, 9). The choiceof only 1 year for eachcalibrationrun
isjustifiedby a studyof the monthlyaccumulatedflow volumes
for eachof the respectiveyears(Figure 3). The figuresindicated
an annual pattern of seasonalhydrologicbehaviorwith 6 wet
OPTIMIZATION ALGORITHM AND CONVERGENCE CRITERIA USED
monthsand 6 dry months.Becausethe data reflectsa distinct
The optimization routine utilized was the pattern search annualcyclein the watershedresponse,it wasdecidedthat the
method of Hooke and Jeeves[1961] which was already im- minimum period of data requiredfor adequatecalibration of
plementedon the SMA-NWSRFS [see Monro, 1971]. Because the model (assumingtime stationaryvaluesfor the model pathe valuesof the three differentobjectivesfunctionsare essen- rameters) was one year. In stage 2, calibration with longer
tially noncomparable,it is obviousthat no function conver- periodsof data will be considered.
gencecriterioncould be usedthat would allow fair comparison
of the results.Furthermore,a functionconvergence
criterionis Stage2. Effectsof Data Length

essentiallyquite arbitrary, and it can causetermination of the

The aim of this stage was to study the sensitivityof the
searchprocedurewell beforethe parametersthat optimizethe estimationresultsto the length of data usedfor calibration.In
functionare obtained.In this study a parameterstepsizecon- addition to the one year studies,the following multiyear calivergencecriterion was employed.The searchwas terminated brationsruns were performedusingeachof the objectivefuncwhen three resolutions(halving)of the stepsizesof the search tions (SLS, AMLE, HMLE): (1) two years of calibration data
procedurewere unable to improve on the best functionvalue (WY 1962-1963)the wettestand driestyearson record,and (2)
obtained.Thus the precisionobtainedfor eachparameterwas three yearsof calibration data (WY 1961-1963) two wet years
within 0.125% of its nominal value.
and a dry year. This constitutessix additional calibration runs.
No constraintswere imposedon the parametervalues.This
reflectsthe belief that a proper estimation procedureshould
PARAMETERS ESTIMATED
arrive at parameter valuesthat are conceptuallyrealistic(i.e.,
The studiesreportedin thispaperinvolvedthe estimationof
physicallyacceptable).In other words,the optimizedparameter
five model parameters(seeTable 2). Thesewere selectedso as
valuesmust lie within the parametersubspaceconsideredfeasto representthe most importantaspectsof the behaviorof the
ible. If the criterion were to result in unrealisticvalues(e.g.,
model,that is,the thresholdbehaviorof the upperzonetension
negative storages,etc.) of the parameters, then one would
water storage(T), the mechanicsof the percolationprocess(Z
questionthe adequacyof that criterion.
and X), the thresholdbehaviorof the lower zone tensionwater
storage(T3), and the quick responseof the upper zone free
DESCRIPTION OF THE STUDIES
water storage (K). It is important to note that all of these
The studywasconductedin two stages.
parametersare difficult to computefrom a study of the hydrographs[see Peck, 1976]. All other parametervalueswere
Stage 1. Effectsof Data Variability
held constant.

The aim of this stagewas to study the sensitivityof the

estimationresultsto variabilityin the calibrationdata. It was
decidedto choosethree 1-yearrecordsfor this purpose.The
18-yeardata record (seeFigure 2 for the yearly streamflow
volumesummary)wasexaminedanda relativelydry year(WY
1952),an averageyear (WY 1960),and a wet year (WY 1962)
were selected.The followingnine calibrationruns were performed:dry yeardata usingSLS,AMLE, and HMLE asobjective functions(runs 1, 4, 7), averageyear data using SLS,
AMLE, and HMLE as objectivefunctions(runs2, 5, 8), wet

A set of parametervaluesfor the Leaf River watershedwere

obtained from the HRL-NWS

to be used as a basis for com-

parison(seeTable 2). These values,hereinafterreferredto as

'NWSP' parameter set,were the resultsof calibrationsconducted by HRL-NWS staff usinga combinationof manual and
automatictechniquesand 18 yearsof data. Thesevalueswere
used as the starting points for all the calibration runs. The
HRL-NWS also provided us with estimatesof the feasible
rangesof valuesthat theseparametersmight conceivablybe
expectedto take (seeTable 2).

SOROOSHIAN
ET AL.' CONCEPTUALRAINFALL-RUNOFFMODELS 1

VALIDATION

PROCEDURE

255

TABLE 2. Boundson the ParameterValuesSuggested

by NWS

In order to establishthe reliability of the resultsof an estimation procedure,it is importantto ensurethat the parameter
valuessatisfythe followingtwo conditions'(1)the valuesof the
parametersare conceptuallyrealisticand (2) the confidencein
the abilityto forecastusingthesevaluesis high.With respectto
1, the followingtestswere performed.First, the estimatedparametervalueswerecheckedagainstthe reasonablerangesuggestedby HRL-NWS. Second,the percentbias'(PBIAS)statistic of the residualsby flow group was examined to detect
defficiencies
in the reproductionof variousaspectsof the hydrograph. The percent bias statisticis defined as follows' for

Parameter

Starting Value

Suggested
Lower Bound

Suggested
Upper Bound

T1
K1

20.00
0.35

10.00
0.20

150.00
0.40

Z
X

200.00
3.30

10.00
1.00

220.00
3.50

T3

200.00

50.00

220.00

of the differencebetweenobservedand simulatedhydrograph

values.The secondmeasurewas the PBIAS of the residuals.(It
eachflowgroupi, PBIAS(i)= [Q-sim(i)-C)obs(i))/C)obs(i)],
where should be pointed out here that the SLS and the DRMS are
Qsim(i)
and Qobs(i)
are the simulatedand observedmeanflow for essentiallythe same;DRMS = SQRT (SLS/n).)A lower DRMS
group i. The flow groupswere chosenas the ranges(0-3, 3-6, value over the forecastperiod would indicatea closerrepro6-14, 14-40,40-85, 85-250,greaterthan 250m3/day).A good duction of the observedflows. A study of the percent bias
in the forecastperformance.In
estimationprocedureshouldresult in valuesof PBIAS consis- would reveal inconsistencies
addition
to
these
tests
the
following
two quantitieswere examtently closeto zero in all flow groups.
ined.

With respectto 2, the followingprocedurewas employed.A

The First Lag AutocorrelationCoefficient(p). This testindi&year-longcontinuousvalidation(forecast)period(WY 1964cates
the degreeof correlation present in the residualsand
1969) which did not includeany of the calibration years was
selectedfrom the eighteenyear record.As Figure 2 shows,these thereforeindicatesdegreeof systematicerror present.
The Variance Stabilizing Transformation Parameter
are essentiallyaverageyears.
(2).
This test indicatesthe presenceof inhomogeneityin the
Each parameter set obtained using the different estimation
variance of the residuals.
procedureswasthenusedto generatea streamflowhydrograph
for the forecastperiod and the model output comparedto the
RESULTS
measuredhydrograph. Since visual comparison is a highly
subjectivetool, two statisticalmeasureswereemployedfor the
StageI Results:Effectsof Data Variability
comparison.The first wasthe daily root mean square(DRMS)
The resultsof stage 1 appear in Table 3 (dry, average,and
wet 1-year calibrations).The parametersthat terminated outside the suggestedbounds are marked by an asterisk. The
important
resultsare asfollows.

5o
1. The SLS and AMLE proceduresresultedin at least two
or three of the five parametersoutside the bounds in all data
OC1 NOV I)FC IAN FEB MAR APR 'k'' IUN JUL AUG SEP
cases.In particular the SLS and AMLE estimatesfor T are
a)
Water Year 1952
unreasonablysmall. In contrast the HMLE estimatesare all
either within or reasonablycloseto the bounds,and in each
caseT hasa realisticvaluecloseto 40 mm.
2. SLS parameter estimatesobtained using dry, average,

100
and wet years show little similarity. The same is true for the
AMLE estimator.Note that the estimatefor T getsprogress
0
ively worsefrom dry to wet yearsfor both of theseestimators.
The HMLE has obtained similar estimatesfor T (of approxiOC 1' NOV DEC
J,'N FEB ,MAR APR
MAY IUN
IUL
AUG
";El'
mately40 mm) with all calibrationdata years.There is however
not much similarity among the other parametervaluesexcept
b)

Water

Year

that most of them have remained within the bounds.

1960

250

200

150

oo

50

O('I

NOV I)EC

I,\N

Ft.B '"b\R

c)

Water

APR

MAY

Year

1962

IUN

IUI

AUC; .SEP

Fig. 3. Monthly streamflow volume summary, Leaf River, Collins, Mississippi.

3. In each calibration data year case the lowest value of

calibrationDRMS statisticwas obtainedusingthe SLS, while
the highestvaluewasobtainedusingthe HMLE. This indicates
that the SLS parameterestimatesfit the calibrationdata closer
(in a leastsquaressense)than the HMLE parameterestimates.
During the forecastperiod, however, the HMLE parameter
estimatesprovided better DRMS values (in all cases),indicating closerreproductionof the hydrographover the forecast
period.The importanceof thisresultwill be discussed
later.
4. The HMLE parameterestimateshave consistentlyprovided lower forecastDRMS than the NWSP parameter set.
However, the SLS and AMLE showedimprovedforecastperformanceonly in the dry calibrationyearcase.
5. The HMLE forecastDRMS is consistentlygood irrespective of the type of data (i.e., dry, average,or wet) used for
calibration. In contrast, the forecast DRMS for the SLS and

256

SOROOSHIAN
ET AL.' CONCEP2-UAL
RAINFALL-RUNOFF
MODELS1

TABLE 3. Resultsof the One-Year Calibration Runs and Their ForecastsUsing NWSP ParameterSet as Starting Values
Final Parameter

Values for 1-Year Calibration

SLS

Runs

AMLE

HMLE

NWSP

Starting
Values

Parameters

20.00
0.35

K1
z

200.00

3.30
200.00

Wet

Dry

Wet

Dry

Year

Dry

Average
Year

Year

Year

Year

Year

Year

4.28*
0.43*
95.47
2.09
290.10'

2.56*
0.70*
51.88
1.17
258.50*

0.46*
0.64*
91.52
2.09
181.00

4.73*
0.57*
98.48
1.75
268.30*

1.19'
0.05*
61.38
1.89
300.30*

0.60*
0.46*
50.96
1.65
145.70

11859
5.70
-4.99
0.62

65833
13.43
2.83
0.51

163273
21.15
-4.78
0.53

1062
5.89
-6.76
0.66

1424
15.07
1.12
0.62

1564
21.48
-3.97
0.57

327

237

328

233

741

458

Calibration
Criterion
DRMS
PBIAS
P

Iterations

Average

Wet

Average

Year

Year

39.21
0.47*
34.68
1.22
146.20

37.22
0.30
156.60
3.52
186.10

42.15
0.65*
83.88
1.76
189.50

Period Statistics

2.99
6.96
- 17.38
0.70

4.66
16.53
-6.56
0.62

5.22
27.41
- 13.80
0.64

0.10
612

0.03
271

- 0.04

13.42
6.15

13.93
0.92

370

Six- Year F orecastPeriodStatistics( W Y 1964-1969)

DRMS
PBIAS

14.60
4.82

13.78
8.18

14.63
15.17

19.22
28.79

13.45
9.21

16.24
11.68

20.21
32.85

12.81

-!.71

0.61

0.59

0.63

0.70

0.60

0.66

0.72

0.61

0.61

0.59

),

0.10

0.30

0.38

0.30

0.32

0.34

0.31

0.08

0.06

0.08

*Values outsidethe suggestedbounds.

AMLE casesgot progressivelyworseas wetter year calibration

data was used.

6. With respectto the percent bias statistic (PBIAS) the

SLS and AMLE

have in all cases resulted in calibration

PBIAS

quite close to zero (within about 5%). However, the PBIAS

value for each of the HMLE calibrations is quite large and
negative,indicating underestimationof the flows. Once again
the forecast

results are different

from

the calibrations.

Stage2: Effects of Data Length

The resultsof stage2 appear in Table 4 (SLS, AMLE, and

HMLE multiyear calibrations).Becausethe performanceof
SLS and AMLE calibrations were similar, their results are
discussed
together.
Dry Year

The

HMLE has obtained consistentlysuperior forecast PBIAS

values. The SLS and AMLE forecastPBIAS values get progressivelyworsefrom dry to wet year case,indicatingmore and

more
'model
divergence'
during
theforecast
period.
7. The PBIAS(i)is depictedin Figure4. The SLS and
AMLE appear to be fitting the high-flowrangescloselywhile
leaving large percent errors in the low-flow ranges. This is
particularly apparent in the forecastperiod. The HMLE however, has tended to fit all flow groups in a fairly uniform
manner, and this property is consistentlyobservedin the calibration yearsas well as the forecastyears.Note that constant
percentbias by flow group indicatesa direct relationshipbe-

Wet

Average Year

Year

,..

SLS

80

AMLE
60

40

HMLE

NWSP

2O
0

-20
,

Flow Range

Flow Range

a)

Calibration

Flow Range

Period

PBIAS(i)

tweenabsoluteerror and flow magnitude,as theorized,so that

largererrorsare associatedwith higherflows.
8. The first lag autocorrelation coefficientis in all cases
around 0.5 to 0.6. This indicates the presenceof a strong
systematiccomponent in the residuals.This phenomenon is
observedduring the calibration and forecast years with not

muchvariation.
Notethatthepresence
of significant
correlation in the residuals seem to suggestthat conditions are
suitable for the use of the AMLE criterion. This will be dis-

Dry Year Sets

Average

Year Sets

Wet Year Sets

/\

8
t
20

cussed further in the conclusions.

9. The values of the variance stabilizing transformation

parameter2, whichrelatesto the HMLE objectivefunction,are
givenin Table 3. The resultsof the HMLE calibrationsindicate
that a great deal of heteroscedasticityis indeed present, as
indicatedby the 2 values(significantlylessthan one).The same
result is observedduring the forecastyearsthereby supporting
the choiceof the HMLE as an appropriate criterion to usefor
these data sets.

-20

Flow RanRe

Flow Rane
b) Forecast

'

'

'

'

Flow Ram:e

Year, I'BIAS(i)

Fig. 4. Percentbiasstatisticby flow range.

SOROOSHIAN ET AL.: CONCEPTUAL RAINFALL-RUNOFF

MODELS 1

257

TABLE 4. Resultsof Multi-yearCalibrationRunsandTheir Forecasts

UsingNWSP ParameterSetasStartingValues
Final ParameterValuesfor Multiyear Calibration
SLS

NWSP
Parameters

Kx
Z
x

T3

Starting
Values
20.00
0.35
200.00
3.30
200.00

AMLE

WY 1962-1963

WY 1961-1963

WY 1962-1963

WY 1962-1963

WY 1962-1963

WY 1961-1963

2 Years,
Wet and Dry

3 Years,
Wet and Dry

2 Years,
Wet and Dry

3 Years,
Wet and Dry

2 Years,
Wet and Dry

3 Years,
Wet and Dry

3.67*
0.63'
75.96
1.95
187.30

4.50*
0.32
173.40
2.82
195.80

2.86*
0.45'
51.61
1.60
171.10

3.45*
0.15'
104.50
2.19
172.60

42.11
0.59*
166.20
2.21
187.30

43.37
0.49*
242.00'
2.52
188.60

102405
16.75
-0.48
0.60

138649
19.49
1.22
0.57

2920
17.16
1.42
0.64

4583
20.40
3.54
0.61

3.72
22.29
- 15.21
0.70

399

251

535

427

694

Calibration
Criterion
DRMS
PBIAS
P

HMLE

Period Statistics

0.08
Iterations

4.02
22.29
- 9.47
0.63
-0.01

246

6-Year Forecast Period Statistics (WY 964-1969)

DRMS
PBIAS

14.60
4.82

0.61

0.10

17.59
22.96
0.67
0.25

16.10
17.28
0.65
0.25

17.76
25.76
0.69
0.30

16.38
20.58
0.67
0.28

13.50
- 1.64
0.63
0.08

13.93
- 2.24
0.64
0.07

*Values outsidethe suggested

bounds.
SLS and AMLE

calibrations.
shorter interval data were to be used, the situation might
1. As longercalibrationperiodsare used,the parameters change.
tend to attain more realistic values. The violations of the feasA very important result of this study, which we wish to
ible rangeare not as largein the 3-yearcaseas in the 1-year discusshere,is the factthat thoughthe HMLE obtainedpoorer
calibrations.The valuesof the parametersdo not seemto tend calibration DRMS values than the SLS, its forecast performancewasconsistentlysuperior.The SLS criterionhasbeen
to anyparticularvaluesaslongercalibration.periods
are used.
2. The forecastDRMS and PBIAS of the 3-year cases commonlyusedin the literaturebasedon the argumentthat the
improvedoverthe worstcase(wet 1-yearcalibrations)but not 'best'parameterset is one that most closelyfits the simulated
overthe dry 1-yearresults.
flowsto the observedflowsduringthe calibrationperiod.This
HMLE
calibrations.
approach,however,ignoresthe stochasticnature of the errors
in the data/model. As discussedby Sorooshianand Dracup
1. Most of the parametershaveremainedwithin or closeto
the suggestedboundswith no unrealisticvalues.The estimates [1980], this may resultin parametervaluesthat are biasedand
for Ta and T3 appearto be relativelyunaffectedby calibration unrealistic,resultingin poor forecasts.These studiesclearly
data length (or variability).The remainingparameters,how- supportthis assertion.In particular,the SLS (and AMLE) tend
ever,showlittle similarityin their values.
to adjustthe parametersto try and reproducethe peak flowsof
2. In all casesthe forecastDRMS is superiorto that ob- the hydrographcloselyat the expenseof the reproductionof
tained by the NWSP. The use of longer data sets has not the low flows.Sincethe model operatesusingthe massbalance
producedany statisticallysignificantdifferencesin the results. principle,the parametersresult in inconsistentforecastreproAs in the stage 1 studies,the PBIAS statistic indicatesan
ductionof both high and low flows(highsare still better reproencouraging
tendencyto slightlyunderestimate
thehydrograph ducedthan lows)with larger errorsin hydrographtiming. The
peaks.
HMLE parameter sets,however, consistentlyreproducedall
aspectsof the hydrograph.In particular,they tendedto underestimatethe peaks slightly. Note that this is a reasonable
DISCUSSION OF THE RESULTS AND CONCLUSIONS
property consideringthat rainfall gaugestend to underestimate
The resultsof this studyseemto indicatethat the HMLE is precipitationdue to effectsof high velocitywind during larger
able to provide more realisticparameterestimatesand more storms.
reliableforecastsand that it is lessaffectedby the nature and

length of the data usedfor calibration. The overall resultsalso

supportthe validity of the assumptions
underlyingthe HMLE.
The performanceof the SLS and AMLE is, however,quite
sensitiveto variabilityand lengthof the data. It is interestingto
note that although a significantdegree of first lag autocorrelation in the residuals was observed in all cases, the
AMLE did not perform noticeablybetter than the SLS. Becauseboth autocorrelationand heteroscedasticity
were detected in the residuals,we conjecturethat the effectsof heteroscedasticityare more severefor this data set. It is possiblethat if

TABLE

5.

Variation

in the Final Parameter Values

Coefficient of Variation, %
Parameter

SLS

AMLE

HMLE

53.0
29.0

66.0
65.0

6.0
26.0

47.0
29.0

35.0
13.0

58.0
38.0

22.0

32.0

10.0

258

SOROOSHIANET AL.' CONCEPTUALRAINFALL-RUNOFF MODELS 1

the latter issuecanbe addressed

it is necessary
to first solvethe
problemsassociated
with the structureof the modelthat give
riseto parameterinteractionand nonuniqueness.
Theseissues
are furtherdiscussed
by Sorooshian
and Gupta[this issue]and

1.00

rion

Value

0.95

GuptaandSorooshian
[this issue].
Finally, we make a brief comment about the choiceof convergencecriterion.Figure 5 containsa plot of the (normalized)
parametervaluesobtainedastheoptimizationproceeded.
Also
includedis a plot of the criterionvalue.This exampleis for the
caseof the wet 2-year calibration with SLS criterion. Notice
that thoughthe criterionvaluehas stabilizedaround 16 trials,
mostof the parameters(exceptfor T) are stillchangingsignificantly. In particular,Z changesby about 40% of its initial

0.90

0.85

10
No.

Parameter

1.4

20
of
trials

Value

1.0

T3

K1

0.6

Z
0.2

T1

valuebetweenthe 16thtrial and terminationof search(based

on parameterconvergence).
This indicatesthe presenceof significant parameterinteractionsand demonstratesclearly the
dangerinvolved in relying on convergence
basedsolelyon
changesin the criterion value.
In thispaperwe havepresented
the resultsof a studyinvolving the estimationof five parametersof the SMA-NWSRFS
model.A similarstudyinvolvingeightparametersand 1-year
calibrations(as discussed
in stage1) is documentedby Sorooshianet al. [1981]. The resultsof that study lend further
supportto the conclusions
reportedabove.

10
No.

of

20
trials

30

Fig. 5. Plot of normalizedcriterion and parameter values as the

optimizationproceededfor SLS wet 1-yearcalibration.

The fact that the estimation procedure using the HMLE

criterion obtainedrealisticparameterestimatesand uniformly
good forecastswith only 1-year calibration data sets is encouraging.It indicatesthat the method is able to extractmuch
of the available information from the data in a fairly efficient
manner.Thus efficiencyof this method may permit the useof a
shorter calibration data period without lossesin estimation
ability. This is desirablefrom a computationalpoint of view, as
the costsof calibration can be substantiallyreduced.
It is not clear at this stageas to how small a length of data
can be effectivelyused,but it is reasonableto assumethat not
lessthan one water year of data should be used.This would
help in trying to ensurereasonablyadequaterepresentationof
the variousphenomenaobservedduring a complete,seasonal
hydrologic cycle.For the same reason it is probably better to
choosea wet year to ensure adequate activation of all the
parametersof the model during calibration.In the view of the
authors, these issuesare critical and deserve further investigation.
The resultsalso raise some interestingquestionsrelated to
the uniquenessof the parameterestimates.In Table 5 we present a grossmeasureof the variation (coefficientof variation) in
eachestimate,basedon the parametervaluesfrom Tables3 and
4. The only parametersshowinga relatively small amount of

variabilityare T and T3, obtainedusingthe HMLE estimation

procedure.The large variation in the values of most of the
estimatesis disturbingin view of the fact that the parametersof
the model are assumedto be time-invariant lumped values,
representativeof the entirewatershed.There is no doubt that a
great deal of interaction existsbetween the parameters of the
model and that this is certainly one of the factorscontributing
to this result. But it also conceivablethat the parametersbest
representingthe watershed responsevary seasonallyor are
trended with respect to time due to physical changesin the
watershed(e.g.,deforestation,urbanization,etc).Clearly, before

Acknowledgments.
This work was supportedby fundsfrom the
Office of Water Researchand Technology,U.S. Departmentof Interior,grant 14-34-0001-9442
and by the HydrologicResearchLab of
the U.S. National WeatherService,grantNA 81 AAA 01746.

REFERENCES

Box, G. E. P., and O. R. Cox, The analysisof transformation,J. R. $tat.

Soc.Set. B, 26(2),211-252, 1964.
Brazil, L. E., and M.D. Hudlow, Calibration proceduresusedwith the
National Weather Serviceforecastsystem,in Water and Related
Land ResourcesSystems,editedby Y. Y. Haimes and J. Kindler, pp.
457-466, Pergamon,New York, 1981.
Burnash, R. J. C., K. L. Ferral, and R. A. McGuire, A generalized
streamflow system: Conceptual modeling for digital computers,
report, Joint Fed. State River ForecastCenter, U.S. Nat. Weather
Serv.and Calif. Dep. of Water Resour.,Sacramento,March 1973.
Clarke, R. T., A review of somemathematical models used in hydrology, with observationson their calibration and use, J. Hydrol.,
19(1),1-20, 1973.
Dawdy, D. R., and T. O'Donnell, Mathematical modelsof catchment
behaviour, J. Hydraul. Div. Am. Soc. Civ. Eng., 91(HY4), 123-137,
1965.

Diskin, M. H., and E. Simon,A Procedurefor the selectionof objective

functionsfor hydrologicsimulationmodels,J. Hydrol., 34(1/2), 129149, 1977.
Fulton, J. L., Discussion,modification, and extension of Some maximum likelihood techniquesfor modelcalibrationwith applicationto
rainfall-runoff models, report, SystemsEng. Dep., Case Western
ReserveUniv., Cleveland,Ohio, August1982.
Gupta, V. K., and S. Sorooshian,Uniquenessand observabilityof
conceptualrainfall-runoffmodel parameters:The percolationprocessexamined, Water Resour.Res.,this issue.
Hooke, R., and T. A. Jeeves,Direct solution of numerical and statistical
problems,J. Assoc.Cornput.Mack, 8,(2),,212-229, 1961.
Ibbitt, R. P., Systematicparameter fitting for conceptualmodels of
catchmenthydrology,Ph.D. thesis,Univ. of London, 1970.
Ibbitt, R. P., and T. O'Donnell, Fitting methodsfor conceptualcatchment models,J. Hydraul. Div. Am. Soc. Civ. Eng., 97(HY9), 13311342, 1971.
Jackson, D. R., and G. Aron, Parameter estimation in hydrology,
Water Resour.Bull., 7(3),457-472, 1971.
James,L. D., and S. J. Burges,Selection,calibration, and testing of
hydrologic models, in HydrologicModeling of Small Watersheds,
Monogr. 5, pp. 437-472, edited by C. T. Haan et al., American
Societyof AgriculturalEngineers,St. Joseph,Michigan, 1982.

SOROOSHIAN ET AL.: CONCEPTUAL RAINFALL-RUNOFF

Johnston,P. R., and D. Pilgrim, Parameter optimization for watershed

models,Water Resour.Res.,12(3),477-486, 1976.
Monro, J. C., Direct searchoptimizationin mathematicalmodeling
and a watershed model application, NOAA Tech. Memo NWS
Hydro-12,U.S. Dep. of Commer.,SilverSpring,Md., 1971.
Monro, J. C., and E. A. Anderson, National Weather Service river
forecastingsystem,J. Hydraul. Div. Am. Soc. Civ. Eng., 100(HY5),
621-630, 1974.

Peck, E. L., Catchmentmodelingand initial parameterestimationfor

the National Weather Serviceriver forecastsystem,NOAA Tech.
Memo NWS Hydro-31,U.S. Dep. of Commer.,Silver Spring,Md,
1976.

Pickup, G., Testing the efficienciesof algorithms and strategiesfor

automatic calibration of rainfall-runoff models, Hydrol. Sci. Bull.,
22(2), 257-274, 1977.
Rosenbrock,H. H., An automatic method for finding the greatestor
leastvalue of a function,Cornput.J., 3(*), 175-184, 1960.
Sorooshian, S., Parameter estimation of rainfall-runoff models with
heteroscedasticstreamflow errors: The noninformative data case, J.

Hydrol.,52, 127-138, 1981a.

MODELS 1

259

Sorooshian,S., Comparisonof two direct searchalgorithmsusedin

calibration of rainfall-runoff models, Water Related Land Resource

Systems,
editedby Y. Y. Haimesand J. Kindler,pp. 477-485,Pergamon, New York, 1981b.

$orooshian,$., and J. A. Dracup, Stochasticparameter estimation

proceduresfor hydrologicrainfall-runoffmodels:Correlatedand
heteroscedastic
error cases,Water Resour.Res.,16(2),430-442, 1980.
Sorooshian,
S.,and V. K. Gupta,Automaticcalibrationof conceptual
rainfall-runoffmodels:The questions
of parameterobservabilityand
uniqueness,
Water Resour.Res.,thisissue.
Sorooshian,S., V. K. Gupta, J. L. Fulton, and F. Arfi, Advanced
calibration methodologyfor hydrologicrainfall-runoff simulation
models,Tech. Rep. LSSPA-81-6, SystemsEng. Dep., Case Western
ReserveUniv., Cleveland, Ohio, 1981.

(ReceivedJune 1, 1982;
revisedNovember 15, 1982;
acceptedDecember1, 1982).