Advances in Geosciences, 7, 85–90, 2006
SRef-ID: 1680-7359/adgeo/2006-7-85
European Geosciences Union
© 2006 Author(s). This work is licensed
under a Creative Commons License.
Radar adjusted data versus modelled precipitation: a case study
over Cyprus
M. Casaioli1 , S. Mariani1,2 , C. Accadia1,* , M. Gabella3 , S. Michaelides4 , A. Speranza5 , and N. Tartaglione6
1 Agenzia per la Protezione dell’Ambiente e per i Servizi Tecnici (APAT), Rome, Italy
2 Department of Mathematics, University of Ferrara, Ferrara, Italy
3 Department of Electronics, Politecnico di Torino, Turin, Italy
4 Meteorological Service, Nicosia, Cyprus
5 Department of Mathematics and Informatics, University of Camerino, Camerino, Italy
6 Department of Physics, University of Camerino, Camerino, Italy
* present organization: EUMETSAT, Darmstadt, Germany
Received: 30 October 2005 – Revised: 28 December 2005 – Accepted: 29 December 2005 – Published: 30 January 2006
Abstract. In the framework of the European VOLTAIRE
project (Fifth Framework Programme), simulations of rela-
tively heavy precipitation events, which occurred over the is-
land of Cyprus, by means of numerical atmospheric models
were performed. One of the aims of the project was indeed
the comparison of modelled rainfall fields with multi-sensor
observations. Thus, for the 5 March 2003 event, the 24-h ac-
cumulated precipitation BOlogna Limited Area Model (BO-
LAM) forecast was compared with the available observations
reconstructed from ground-based radar data and estimated by
rain gauge data.
Since radar data may be affected by errors depending
on the distance from the radar, these data could be range-
adjusted by using other sensors. In this case, the Precipita-
tion Radar aboard the Tropical Rainfall Measuring Mission
(TRMM) satellite was used to adjust the ground-based radar
data with a two-parameter scheme. Thus, in this work, two
observational fields were employed: the rain gauge gridded
analysis and the observational analysis obtained by merging
the range-adjusted radar and rain gauge fields.
In order to verify the modelled precipitation, both non-
parametric skill scores and the contiguous rain area (CRA)
analysis were applied. Skill score results show some dif-
ferences when using the two observational fields. CRA re-
sults are instead quite in agreement, showing that in general
a 0.27◦ eastward shift optimizes the forecast with respect to
the two observational analyses. This result is also supported
by a subjective inspection of the shifted forecast field, whose
gross features agree with the analysis pattern more than the
non-shifted forecast one.
However, some open questions, especially regarding the
effect of other range adjustment techniques, remain open and
need to be addressed in future works.
Correspondence to: M. Casaioli
(marco.casaioli@apat.it)
Advances in
Geosciences
1 Introduction
Meteorological observations, derived from different instru-
ments, can be compared with model outputs in order to verify
the models themselves and quantify their skill since, focus-
ing on the relationship between observations and forecasts,
it is possible to underline the model’s ability to forecast cor-
rectly a meteorological event. Forecast verification can also
provide insight into the way atmospheric processes are mod-
elled.
Precipitation is often the object of verification studies:
point measurements over land are compared with modelled
precipitation while over the sea it is not possible, usually, to
perform the same kind of comparison. Nevertheless, satellite
or ground-based remote sensing can help evaluating rainfall
over the sea.
Performing this kind of operation is one of the purposes
of the European VOLTAIRE project (Fifth Framework Pro-
gramme; http://www.voltaireproject.org). After selecting
several relevant weather events over the island of Cyprus,
precipitation measured at rain gauges (hereafter RG) and re-
trieved by both the ground-based radar (hereafter GR) and
the Precipitation Radar on board the Tropical Rainfall Mea-
suring Mission (TRMM) was collected. These case studies
were also modelled using the hydrostatic BOlogna Limited
Area Model (BOLAM).
In this work, we focus on the relatively heavy precipitation
event occurred on 5 March 2003. This was due to a cyclonic
circulation slowly moving from the western to the eastern
Mediterranean Sea (Tartaglione et al., 2005).
Since GR data can be prone to distance-dependent errors,
some range adjustment may be helpful. In this study, TRMM
precipitation radar data were used to adjust the GR data, em-
ploying a (physically easy to be interpreted) two-coefficient
scheme. The forecast field was compared with both a RG
In order to verify precipitation forecast by BOLAM over
Cyprus, both the RG and GR data, which are managed by
the Meteorological Service of Cyprus, were used. The RG
86 and the GR location are shown in Fig. 1. Fig.Radar
M. Casaioli et al.: 2. The 24-h data
adjusted observational precipitation
versus modelled contours (mm)
precipitation
network
2 Casaioli et al.: Radar adjusted data versus modelled precipitation
0600 UTC 5 March to 0600 UTC 6 March 2003, obtained app
the Barnes analysis scheme.
the contiguous rain area analysis (CRA; Ebert and McBride,
2000) were applied for verifying the modelled precipitation.
The paper is organized as follows: in section 2, the ob-
servational data set and the radar range adjustment scheme island, we decided to merge GR data with the RG gri
are discussed; the atmospheric model BOLAM is presented analysis.
in section 3; a brief discussion of the applied verification The GR raw data can be corrected and/or adjusted, s
methodologies is proposed in section 4; results and conclu- some factors can alter the beam response. GR measures
sions are, finally, reported in section 5.
from a lateral direction. In the present case, the distanc
tween the echo and the radar varies from 10 to 120 km.
2 Ground-radar and rain gauge data cause of this large ratio of distances, the scattering vo
changes by a factor of over 100, since the volume incre
In order to verify precipitation forecast by BOLAM over with the square of the distance. The scattering volum
1. Cyprus,
Fig. Fig. Geographicalboth thedistribution
1. Geographical RG and GRover
distribution data,
over the which
theisland
islandareCyprus
of managed
of of the
Cyprus by
of the TRMM has a similar size in all the locations. This advan
147therain gauge stations,Service
Meteorological marked of as white
Cyprus, circles,
wereand the location
used. The RG
147 rain
of the gauge stations,
ground-based marked ◦ as white ◦ circles, and the location Fig.
Fig. 2.
2. ofThe
The 24-h observational
TRMM
24-h observational precipitation
its use forcontours
suggestsprecipitation estimating
contours (mm)
(mm) fromthe
frominfluence o
network and the radar (34.98 ◦ N,
GR location are32.73
shownE,in1310 Fig.m1.above mean
of theseaground-based
level), marked as radar
a black circle. N,
(34.98 32.73◦ isE,indicated
Orography 1310 minabove 0600 UTC
colour mean06:00 UTC 55 March
March to
GRanalysis
sampling
to 06:00
0600 UTCUTC 66 March
March 2003,
2003, obtained
obtained applying
volume (Gabella et al., 2005). For both ra applying
sea level), marked is indicated in colourthe theBarnes
Barnes scheme.
analysis scheme.
scale; visible in as a black
figure is thecircle.
TroodosOrography
mountain range.
the average reflectivity Z in the same 10-km circular ri
scale; visible in figure is the Troodos mountain range.
computed. Let us consider < GR >2π and < TRMM
island, we decided to merge GR data with the RG gridded
gridded analysis and an observed precipitation field obtained The the
analysis.Cyprusvalues GR canof the average
provide reflectivity,
information also onaveraged
precipita- in azimuth
For the selected event, precipitation recorded at the both
by merging the RG data with rainfall retrieved from the tion over GR raw data can be corrected and/or adjusted, size,
the GR
waters and TRMM.
surrounding Because
the island ofof their
Cyprus. since these two
How-
RG range-adjusted
network during GR data.24 hours, from 0600 UTC 5 Marchever,The the proximity
ables show of the
similar Troodos high
behaviour, massif
except in the south-
for rain
the decreased
some factors can alter the beam response. GR measures
to 0600 BothUTC 6 Marchskill2003
non-parametric scoreswas (Hanssenconsidered.
and Kuipers,Sinceeastern part
from asitivity
lateralanddirection.
of the
of GRTripylos
with peak in case,
In thedistance.
present the Averaging
northwestern
the distanceover part
be- the large
RG 1965;
givesSchaefer,
only point measurements,
1990; Wilks, 1995; Stephenson, a two-pass 2000) and Barnestween of
(see Fig. 1)
thethe produced
echoringsand the two obscured
radar varies (no-data)
from 10 to radar
120
reduces deviations caused by rain cells of sectors.
km. Be-
schemethe contiguous
(Barnes, rain 1964, area1973;
analysisKoch (CRA;etEbert al., and1983) was usedThus,
McBride, cause inofan attempt
this
intensity.
to obtain
large ratio a better observational
of distances, the scattering analysis volume
2000) were applied for verifying ◦ modelled precipitation.
the over Cyprus
changes by aand its
factor surrounding
of over 100, sea,
since even
the considering
volume increasesthat
in order to obtain a gridded 0.09 observational analysis: the Thus, the following factor:
The paper is organized astofollows: in Sect. 2,with the obser- much
with the square of the distance. The scattering volume the
precipitation occurred in the southeastern part of of
first observational
Fig. 1. Geographical analysis
distribution be
over compared
the island of Cyprus BOLAM
of the island,
TRMMwe hasdecided
a similartosize merge
in allGRthedata with the
locations. ThisRG gridded
advantage
vational data set and the radar range adjustment scheme are
was discussed;
obtained
147 rain gaugeby usingstations,only
markedRGasdata white (see
circles, Fig.
and 2). However,analysis.
the location < GR > for estimating the influence of the
suggests its use2π
the atmospheric model ◦ BOLAM ◦ is presented in of TRMM
m above meanmayGR sampling<
F =
be>corrected
in this of the ground-based
case, ofradar (34.98 N,
for32.73 E, 1310
Sect.
sea 3; a the
level),brief use
markeddiscussiononly RG
as a blackofcircle.
the applied forecast
Orography
verification
verification
is indicated method-
in colour
The GR rawvolume TRMM
data (Gabella
can 2π et al., 2005).
and/orFor both radars,
adjusted, since
be insufficient
ologies is for
proposed at least
in two
Sect. 4;reasons.
results and conclusions are, the
some average
factors reflectivity
can alter theZ in the response.
beam same 10-km GR circular
measures ringrainis
scale; visible in figure is the Troodos mountain range. is statistically described by using a weighted regression
computed. Let us consider < GR > and <
lateral direction. In the present2πcase, the distance be- TRMM > 2π
finally,
First, RG reported
are only in Sect. 5.
available on Cyprus’ western part andfrom a tween log(F ) and log(D), where D is the distance betw
the values of the average reflectivity, averaged in azimuth, for
coastal grid For the points (where
selected event,observed
precipitationprecipitation
recorded at spreadtween
is the both
the echo and the radar varies from 10 to 120 km. Be-
thethisecho
GR and
cause of largeand
TRMM. ratio GR.
Because The best
of
of distances, theirthe relationship,
size, these two
scattering volume vari- among the
by the RG Barnes network during 24may
procedure) hours,
suffer froman0600 UTC 5 Marchwithables show similar behaviour, except for the decreased sen-
underestimation changes found
by a by
factor Gabella
of over et
100,al. (2005),
since the seems
volume to be
increases the followin
2 toGround-based
respect to0600 UTC 6radar
the internal March
gridand 2003
rain was
points. gauge
Second, considered. Since
datathe precipitation sitivity ofsquare
GR with distance. Averaging over the volume
large area
RG gives only point measurements, a two-pass Barnes with the of the distance. The scattering
of high  of
comparison is(Barnes,
limited1964, by the presence ofal.,
the1983)
surrounding of the rings
sea.TRMM has areduces
similar sizedeviations caused
in all the by rain
locations. Thiscells D
advantage
scheme 1973; Koch
In order to verify precipitation forecast by BOLAM over et was used FdBsuggests
ofintensity. ≡ 10 ·itslog(F )= a0 + athe 1 · log =
The Cyprus
in
Cyprus, order toGR
both thecan
obtain RG a provide
gridded
and GR0.09 data,
◦
information
which arealso
observational on precipita-
analysis:
managed the
by
TRMM use for estimating influence Dgof the
tion the
over the waters surrounding thecompared
island ofwith
Cyprus.
Thus, the following factor:
GR sampling volume (Gabella et al., 2005). For both radars,

first observational
Meteorological analysisoftoCyprus,
Service be were used. BOLAM
The RGHow-the average reflectivity Z in the same 10-km circular ringDis
wasproximity
obtained < GR > = −4.1 − 10.1 · log ,
ever,network
the and theby of
GR using
theonly
location RGshown
Troodos
are data
high(see Fig.1.2). in
in massif
Fig. However,
the south-computed.
F =
2π
in this case, the use of only RG for forecast verification may < TRMM Let us>consider
2π
<GR>2π and <TRMM>2π (1) 40
the
easternbe part
For theand of theevent,
selected Tripylos peak inrecorded
precipitation
insufficient for at least two reasons.
the northwestern
at the RG partvalues of the average reflectivity, averaged in azimuth, for
network
(see Fig.First, during
1) produced 24
RG are only
h, from
two 06:00
obscured
available
UTC 5 March
(no-data)
on Cyprus’
to 06:00
westernradar UTC
part sectors.
and
is statistically
both GRwhere
and TRMM. adescribed
0 andBecauseaby using
1 are of theatheir
weighted
regression regression
size, these be-
coefficients
two vari- and Dg
6 March 2003, was considered. Since RG gives only point tween log(F ) and log(D), where D is the distance between
Thus, coastal
in an attempt to obtain a better observational
grid points (where observed precipitation is spread analysis ables coefficient
show similar for the
behaviour, normalization
except for the distance.
decreased sen-
measurements, a two-pass Barnes scheme (Barnes, 1964, the echo and withGR. distance.
The best relationship, among the ones
over1973;
Cyprus
by the Barnesand its surrounding
procedure) may suffer sea,an even considering
underestimation with thatsitivity
found
ofItGR
by is possible,
Gabella et al. thisAveraging
(2005), way, totobeover
seems use
the
the
two large area kinds of
different
following:
Koch et al., 1983) was used in order to obtain a gridded of the rings reduces deviations caused by rain cells of high
much0.09 respect to
precipitation the internal
occurred grid points. Second, the precipitation
◦ observational analysis: in the the
first southeastern
observational analysis part of the data: the original and the
range-adjusted  ones. The (ori
comparison is limited by the presence of the surrounding sea. intensity. D
to be compared with BOLAM was obtained by using only FdB ≡ 10 · log(F ) = a0 + a1 · log =
The Cyprus GR can provide information also on precipita- Thus, the following factor: Dg
RG data (see Fig. 2). However, in this case, the use of only
tion over the waters surrounding the island of Cyprus. How-

RG for forecast verification may be insufficient for at least D
ever, the proximity of the Troodos high massif in the south- = −4.1 − 10.1 · log , (2)
two reasons. < GR >2π 40
eastern part and of the Tripylos peak in the northwestern part F = (1)
First, RG1)areproduced
only available on Cyprus’ western
radarpart and <aTRMM >2π
(see Fig. two obscured (no-data) sectors. where 0 and a1 are the regression coefficients and Dg is a
coastal grid points (where observed precipitation is spread
Thus, in an attempt to obtain a better observational analysis coefficient for the normalization distance.
byover
the Barnes
Cyprusprocedure) may suffersea,
and its surrounding an underestimation
even consideringwiththat is statistically described
It is possible, bytousing
this way, a weighted
use two differentregression be-
kinds of GR
respect to the internal grid points. Second, the precipitation
much precipitation occurred in the southeastern part of the tween log(F ) and log(D), where D is the distance between
data: the original and the range-adjusted ones. The (original
comparison is limited by the presence of the surrounding sea. the echo and GR. The best relationship, among the ones
 Casaioli
M. Casaioli et al.: Radar adjusted data versus modelled precipitation et al.: Radar adjusted data versus modelled precipitation
87

found by Gabella et al. (2005), seems to be the following:

 
D
FdB ≡ 10 · log(F ) = a0 + a1 · log =
Dg
 
D
= −4.1 − 10.1 · log , (2)
40
where a0 and a1 are the regression coefficients and Dg is a
coefficient for the normalization distance.
It is possible, this way, to use two different kinds of GR
data: the original and the range-adjusted ones. The (original
and adjusted) GR data were accumulated in the aforemen-
tioned time interval, then, in order to make them available
on a latitude-longitude grid (with grid spacing of 0.09◦ ); the
remapping procedure (Baldwin, 2000; Accadia et al., 2003)
was also applied.
However, in order to choose the best representative GR
field, it was decided to compare the precipitation retrieved
from the raw and the range-adjusted GR data with the rain
gauge ones. Fig. 3 shows the results of this comparison. A
near zero correlation is found when the original precipitation
data is used in the comparison with the rain gauges (Fig. 3a), (a) RG gridded field vs. non-adjusted GR field
whereas a significantly (taking also into account its confi-
dence interval) higher correlation is obtained by using pre-
cipitation reconstructed by range-adjusted GR data (Fig. 3b).
In this latter case the better agreement between these two
fields appears more evident. Therefore, we shall hereafter
consider as GR data the range-adjusted ones.
Then, the observational analysis (OBSPREC; Fig. 4) is im-
proved by merging the RG gridded field and the GR range-
adjusted data using the following formula for each verifica-
tion grid point:
(GR × GR + RG × RG)
OBSPREC = ; (3)
(GR + RG)
this way, when there is no contribution from RG (over sea,
for instance), OBSPREC is given only by GR; the same is
true for RG, on the grid points where GR is shielded by
mountains, for instance. When estimations by both instru-
ments were available, they were respectively weighted by
their values.

3 Forecast precipitation

The model used in this verification case study is BO- (b) RG gridded field vs. range-adjusted GR
LAM, a hydrostatic primitive-equation model (Malguzzi and field
Tartaglione, 1999; Buzzi and Foschini, 2000). The standard
6 h, 0.5◦ resolution, 60-model-level ECMWF analyses were
first horizontally interpolated onto the 30-km model domain Fig. 3. Precipitation scatterplots of the 24-h rain gauge gridded field
Fig. 3.against
covering the entire Mediterranean region. The outputs of this
Precipitation
the 24-h radar scatterplots of the
field over 73 grid 24-h
points whererain
bothgauge gridded
data were
model were used as initial and boundary conditions (one-wayfield against
available. Correlation (CORR.), and its confidence interval (Fisher, both
the 24-h radar fields over 73 grid points where
nesting) for a domain with a finer grid (10 km) and encom- 1925),
data were bias (BIAS),
available. standard error
Correlation (STD. ERR.),
(CORR.), and mean square error inter-
its confidence
(MSE), root mean square error (RMSE) are
val (Fisher, 1925), bias (BIAS), standard error (STD. indicated. A ERR.),
linear re-mean
passing only the eastern Mediterranean region.
gression fit (solid line) and a 95% confidence ellipse are also shown.
Modelled precipitation was then 24-h accumulated (see square error (MSE), root mean square error (RMSE) are indicated.
Fig. 5) for the same time interval described in the previous
A linear regression fit (solid line) and a 95% confidence ellipse are
section. These data, originally on the BOLAM native grid, also shown.
were remapped (Accadia et al., 2003) on the same grid of the

and adjusted) GR data were accumulated in the aforemen-

 for instance), OBSPREC is given only by GR; the same is
confidence ellipse (see Fig. 6) are related to the higher val-
true for RG, on the grid points where GR is shielded by
mountains, for instance. When estimations by both ues of the observed precipitation, mainly due to GR data; RG
instru-
88ments were available, they were respectively weighted by −1 versus modelled precipitation
did not
M. exceed
Casaioli 39Radar
et al.: mmadjusted
(24h)data.
their values.

4 Casaioli et al.: Radar adjusted data versus modelled precipitation

Fig.4.4. The
Fig. The24-h
24-hobservational
observational precipitation
precipitation contours
contours (mm)
(mm) from
from
were
0600UTCremapped (Accadia
UTC et al.,6 2003) on theobtained
same grid of the tallied upon 2 × 2 contingency tables, summarize in a cate-
06:00 5 March to
to 0600
06:00UTC
UTC March 2003,
6 March 2003, according
obtained accord-
observed
to to
ing Eq.Eq. precipitation field, which was used as verification
3. (3). gorical way possible combinations of forecast and observed
e gridded grid. events above or below selected precipitation thresholds. The
here both set of thresholds, depending on the event magnitude, does
nce inter- not include values greater than 40 mm (24h)−1 .
R.), mean 3 Forecast precipitation However, single case-study verification only by means of
indicated. these scores may lead to unstable results, due to the limits
ellipse are The model used in this verification case study is BO- of the statistical sample. Thus, the CRA analysis (Ebert and
LAM, a hydrostatic primitive-equation model (Malguzzi and McBride,
Fig. 6. As2000),
in Fig. 3,allowing to assess
but for the the spatial
24-h forecast structure
field against the of
24-h
Tartaglione, 1999; Buzzi and Foschini, 2000). The standard the forecast
observational error,
field was
over also
532 performed.
grid points. This object-oriented
6h, 0.5◦ resolution, 60-model-level ECMWF analyses were
Fig. 6. As in Fig. 3, but for the 24-h forecast against the 24-h ob-
method is simply based on pattern matching of two contigu-
foremen- first horizontally interpolated onto the 30-km model domain servational field over 532 grid points.
ous areas, defined as the observed and forecast precipita-
available covering the entire Mediterranean region. The outputs of this tion areas, delimited by a chosen isohyet, which was set to
09◦ ); the model were used as initial and boundary conditions (one-way 0.5 mm (24h)−1 . In a complex rain field (such as the one
l., 2003) nesting) for a domain with a finer grid (10 km) and encom- insetquestion),
of thresholds, depending
it is preferable on thethe
to match event magnitude,
rainfall patterns in- does
passing only the eastern Mediterranean region. not include
stead values greater
of the precipitation than 40thus
maxima, mmthe h)−1 .
(24correlation max-
ative GR Modelled precipitation was the 24-h accumulated 4 (seeMethodology
imization was employed as the CRA pattern-matching crite-
retrieved Fig. 5) for the same time interval described in the previous rion. However,
The CRAsingle case-study
analysis verification
is performed onlythe
by shifting byforecast
means of
the rain section. These data, originally on the BOLAM native grid, these scores may lead to unstable results, due to the limits
Fig. 5.5. The
Fig. The 24-h
24-h BOLAM
BOLAM forecast field contours
contours (mm)
order precipitation
In from
(mm) from
to give
ofHowever,
field both latitudinally
a quantitative
the statistical sample. Thus, the
and longitudinally.
assessment
CRA of the
analysis model’s
(Ebert and skill
in order to eliminate matches that are not sta-
0600 UTC 5 March to 0600
06:00 06:00UTC
UTC66March
March2003.
2003. in predicting
McBride,
tisticallythe event,allowing
2000),
significant, anon-parametric
thresholdto assess
minimum skill structure
the spatial
correlationscores,
value ofsuch as
ETS, BIA, the
betweenforecast
HK,shifted error,
ORSS, was
forecastPODalso performed.
and FAR,
and observations This object-oriented
waswere
imposed. initially
This used.
Visual comparison between forecast and observational method is simply based on pattern matching of two contigu-
value depends on the effective number of independent sam-
analysis (Fig.
observed 6) displays
precipitation a fair
field, agreement,
which was used although These
the cor- scores
as verification ous used
ples
(for
areas, details
defined
in the as the
comparison,
seeobserved
Hanssen
which is aand
andof Kuipers,
forecast
function precipita-
the num-
1965;
relation
grid. is still low. Almost all the pairs outside the 95%
confidence ellipse (seebetween
Fig. 6) are related and
to the
Schaefer,ber
higher val-
1990;
tion grid Wilks,
ofareas, delimited
points 1995;
wherebythe Stephenson,
a chosen
analysisisohyet, 2000),
which
is performed was can
(this setwhich
to are
Visual comparison forecast observational 0.5 mm (24 h) −1 . In a complex rain field (such as the one
change from shift to shift) and the autocorrelation of both the
ues of the
analysis observed
(Fig. precipitation,
6) displays a fair mainly duealthough
agreement, to GR data;
the RG
cor- in question), it is preferable
−1 observed and forecast fields. to match
The thewith
F test, rainfall
a 95%patterns
confi-in-
did not exceed
relation is still 39 mmAlmost
low. (24h) all . the pairs outside the 95%
stead level,
dence of thewasprecipitation
then applied maxima, thus
to assess thethe correlation
statistical max-
signifi-
confidence ellipse (see Fig. 6) are related to the higher val- imization
cance of eachwasmatch
employed as the CRA
(see Panofsky andpattern-matching
Brier, 1958; Xie and crite-
ues of the observed precipitation, mainly due to GR data; RG rion. The
Arkin, CRA analysis is performed by shifting the forecast
1995).
did not exceed 39 mm (24 h)−1 . precipitation field both latitudinally and longitudinally.

5 In Results
orderand conclusions
to eliminate matches that are not statistically
4 Methodology significant, a threshold minimum correlation value between
Itshifted
is nowforecast
possible and
to match the obtained
observations was results into This
imposed. a coher-
value
In order to give a quantitative assessment of the model’s skill ent picture. Model skill scores calculated with
depends on the effective number of independent samples respect to the
in predicting the event, non-parametric skill scores, such as RG
usedgridded
in the analysis
comparison, (i.e., which
only over
is athe island of
function of the
Cyprus)
numberare of
fairly good (Table 1), whereas when including the GR range-
ETS, BIA, HK, ORSS, POD and FAR, were initially used. grid points where the analysis is performed (this can change
adjusted data (i.e., also over the sea) quite poor scores are
These scores (for details see Hanssen and Kuipers, 1965; from shift to shift) and the autocorrelation of both the ob-
obtained (Table 2).
Schaefer, 1990; Wilks, 1995; Stephenson, 2000), which are served and forecast fields. The F test, with a 95% confidence
tallied upon 2×2 contingency tables, summarize in a cate- level, was then applied to assess the statistical significance of
gorical way possible combinations of forecast and observed each 1.
Table match
Skill (see
scorePanofsky
results of and Brier, 1958;
comparison betweenXie
the and
BOLAMArkin,
events above or below selected precipitation thresholds. The forecast
1995). and the rain gauge analysis for the 5 March 2003 event.
Skill score values less or equal zero are not indicated.

Fig. 6. As in Fig. 3, but for the 24-h forecast against the 24-h ob- Threshold BIA ETS POD FAR HK ORSS
 ◦
Table 2. Skill score results of comparison between the BOLAM sults show that shifting 0.27 eastward, the BOLAM forecast
forecast and the observational field (rain gauges + radar) for the gives a better match with the observational analysis (correla-
M. 5Casaioli et al.: Radar
March 2003 event. adjusted data versus modelled tion increases from 0.36 to 0.43).
precipitation 89

Table Threshold
1. Skill scoreBIA results of
ETScomparison
POD between
FAR the HKBOLAMORSS
forecast
mmand(24h)the
−1 rain gauge analysis for the 5 March 2003 event.

Skill score values less or equal zero are not indicated.

5.0 1.284 0.159 0.947 0.262 0.230 0.753
10.0 1.517 0.182 0.922 0.392 0.306 0.762
Threshold
15.0−1 BIA 1.732 ETS 0.160 POD
0.799FAR0.539HK 0.326 ORSS0.631
mm (24 h)
20.0 1.723 0.131 0.638 0.629 0.288 0.531
40.0
5.0 0.243 0.739
1.014 0.1361.000
0.1620.014
0.3330.7500.156 – 0.939
10.0 1.070 0.479 0.965 0.098 0.590 0.957
15.0 1.275 0.347 0.925 0.274 0.501 0.887
20.0 1.370 0.133 0.667 0.513 0.254 0.479
40.0obtaining– good skill
fields, – scores– when– the area – is large
– (and
more than one type of observation is included) is very un-
likely; whereas, it is easy to obtain good skill scores on
smaller arear (in particular when only one type of observation (a) Forecast shifted 0.27◦ E and 0.09◦ N with respect to the
Table 2. Skill score
is included). Thisresults
resultofcould
comparison between
also indicate the BOLAM
a major impact Barnes rain gauge-based analysis
forecast
of theand the observational
observational field (rain
error (e.g., gauges + radar)
GR range-effect, RGfor the
repre-
5 March 2003 event.
sentativity error) on the skill scores when using the combined
observational data set. Although we do not present the results
inThreshold
this paper, itBIAis worth ETS POD that
mentioning FARthe worstHK skill ORSS scores
mm (24 h)−1
were obtained by using in the observational composite (i.e.,
RG 5.0+ GR) original
1.284 GR 0.159data0.947
(rather0.262
than those0.230obtained0.753 by
using10.0range-adjusted
1.517 0.182 GR data).0.922This result0.306
0.392 also appears
0.762 in
Fig.15.0
3(a), where1.732the precipitation
0.160 0.799 scatterplot
0.539 0.326 of the0.631
original
GR 20.0
data versus 1.723 0.131 0.638 0.629 0.288 0.531(cor-
the RG data exibits the worst indicators
relation
40.0 and MSE).
0.243 This confirms
0.136 0.162that0.333
radar range
0.156 adjustment
0.939
is a key procedure for having reliable GR data. Besides, BO-
LAM tends to overforecast precipitation almost everywhere
(see the BIA score in Tables 1 and 2 and the 95% confidence
ellipse in Fig. 6), especially over the sea. This could indi-
5 Results and conclusions
cate that forecast rainfall qualitatively changes from land to
sea, but this may be proven only on the basis of a long-term
It is now possible to match the obtained results into a co-
statistical verification of BOLAM over Cyprus. (b) Forecast shifted 0.27◦ E and 0◦ N with respect to the RG-
herent picture. Model skill scores calculated with respect to GR composite analysis
Furthermore, model skill scores may be penalized by the
the RG gridded analysis (i.e., only over the island of Cyprus)
forecast oversmoothness1 . In fact, many details that are vis-
are fairly good (Table 1), whereas when including the GR
ible in the analysis over sea are not present in the forecast
range-adjusted
field (see in dataFig. (i.e.,
4 thealso overpeaks
rainfall the sea) quitesouth,
located poor scoressouth- Fig.
Fig. 7. The 24-h
24-h BOLAM
BOLAM forecast
forecastfield
fieldcontours
contours(mm)
(mm)fromfrom
are west
obtained (Table 2).
and north-east of the island, respectively, and compare 06:00 UTC 55 March
0600 UTC Marchtoto0600
06:00UTC
UTC6 6March
March2003,
2003,
asas result
result of of
thethe
There are many
with Fig. reasons such
5). However, whichdiscrepancies
could account (andfortheir
sucheffect
dif- CRA
CRA analysis.
ferences
on skillin scores)
skill scores.
cannotFor be example,
unequivocally whenattributed
comparing two
to fore-
fields,
castobtaining
error, sincegoodtheskill scores when
observational errorthe(namely,
area is large the (and
radar An a priori estimate of the impact of the possible observa-
more than one type
range-effect) of observation
may sensibly affect theis estimation
included) of is maximum
very un- tional error
cate that on these
forecast resultsqualitatively
rainfall is not possible, thus we
changes frompropose
land to
likely; whereas,
rainfall peaks. it is easy to obtain good skill scores on a subjective check of the analysis (Fig. 4) against
sea, but this may be proven only on the basis of a long-term the orig-
smallerFor what(inconcerns
arear particular thewhen
CRAonly one type
results, whenofusing
observation
the RG inal and shifted forecast (Figs. 5 and
statistical verification of BOLAM over Cyprus.7(b)). The idea is to
is included). This result could also indicate
gridded analysis for verification, a better agreement a major impact of
(using identify
Furthermore, model skill scores may be penalized byinthe
the qualitative features of the analysis illustrated
Fig. 4 which are less prone to possible observational error,
the correlation
observational error (e.g.,between
maximization) GR range-effect,
observations RGand represen-
forecast forecast oversmoothness1 . In fact, many details that are vis-
fieldserror)
tativity is obtained
on theby shifting
skill scoresthewhen
forecastusingfield
the0.27 ◦
combinedE and and to check them against both the non-shifted and shifted
ible in the analysis over sea are not present in the forecast
◦
0.09 N (seedata
observational Fig.set.
7(a)Although
against Fig.
we 2doand notTartaglione
present theet re- al., model fields. For instance, it is quite improbable that radar
field (see in Fig. 4 the rainfall peaks located south, south-
2005).
sults in thisThis result
paper, quite similar
it is worth to the that
mentioning one the
obtained
worstby us-
skill errors introduce a macroscopic pattern shifting.
west and north-east of the island, respectively, and compare
The gross structure of the rainy area on the island s south-
ing were
scores the RG-GR
obtained composite
by usingfor verification.
in the observational In fact, CRA re-
composite with Fig. 5). and
However, such discrepancies
western side of the non-rainy area on the(and their effect
northwestern
(i.e., RG
1
+ GR) original GR data (rather than those obtained on skill scores) cannot be unequivocally attributed to the
fore-
Numerical models oversmooth precipitation fields with respect side is better matched by the shifted forecasts than by
by using range-adjusted GR data). This result also appears in cast error, since the observational error (namely, the radar
to reality (Chéruy et al., 2004) original one. Over the sea, the check is difficult since a few
Fig. 3a, where the precipitation scatterplot of the original GR range-effect) may sensibly affect the estimation of maximum
data versus the RG data exibits the worst indicators (correla- rainfall peaks.
tion and MSE). This confirms that radar range adjustment is For what concerns the CRA results, when using the RG
a key procedure for having reliable GR data. Besides, BO- gridded analysis for verification, a better agreement (using
LAM tends to overforecast precipitation almost everywhere
(see the BIA score in Tables 1 and 2 and the 95% confidence 1 Numerical models oversmooth precipitation fields with respect
ellipse in Fig. 6), especially over the sea. This could indi- to reality (Chéruy et al., 2004).
90 M. Casaioli et al.: Radar adjusted data versus modelled precipitation
correlation maximization) between observations and forecast
fields is obtained by shifting the forecast field 0.27◦ E and
0.09◦ N (see Fig. 7a against Fig. 2 and Tartaglione et al.,
2005). This result is quite similar to the one obtained by
using the RG-GR composite for verification. In fact, CRA
results show that shifting 0.27◦ eastward, the BOLAM fore-
cast gives a better match with the observational analysis (cor-
relation increases from 0.36 to 0.43).
An a priori estimate of the impact of the possible observa-
tional error on these results is not possible, thus we propose
a subjective check of the analysis (Fig. 4) against the original
and shifted forecast (Figs. 5 and 7b). The idea is to identify
the qualitative features of the analysis illustrated in Fig. 4
which are less prone to possible observational error, and to
check them against both the non-shifted and shifted model
fields. For instance, it is quite improbable that radar errors
introduce a macroscopic pattern shifting.
The gross structure of the rainy area on the island’s south-
western side and of the non-rainy area on the northwestern
side is better matched by the shifted forecast than by the orig-
inal one. Over the sea, the check is difficult since a few ob-
served peaks, probably due to convective activity, are absent
in the BOLAM forecast (the hydrostatic model did not fore-
cast them at all!). However, the position of the main rainy
area, located to the northwest of the island, is better centred
in the shifted forecast than in the original one.
The proposed results are a starting point for future works.
The effect of forecast oversmoothness on the skill scores
could be tested. Besides, the effect of different possible
range adjustment techniques on the observational analysis
could be also studied, in order to estimate and reduce the
observational error and to assess the physical reliability of
the CRA results. Overall, the results point out how much
caution is needed when dealing with rainfall observations
as the “ground truth”. When satellite, rain gauge and radar
data are simultaneously available over a large area, a reliable
data base suitable for precipitation forecast verification can
be built. Otherwise, the model verification task can be quite
difficult, since it cannot be easy to distinguish the effect on
the verification results of the forecast error from the one re-
lated to the observational error. Moreover, even a rough es-
timate of the observational error magnitude cannot be avail-
able. In such a condition, exemplified in the here presented
case study, both analysis and forecast should be regarded as
an error-affected representation of an unknown reality.
Acknowledgements. This research was supported by the EU
Framework 5th Programme – Part A: Environment and Sustain-
able Development Contract EVK2-2002-CT-00155 (VOLTAIRE
project). Authors are grateful to ECMWF for initial and boundary
conditions for BOLAM. Comments and suggestions received
during VOLTAIRE workshops were very useful.
Edited by: V. Kotroni and K. Lagouvardos
Reviewed by: anonymous referee
References
Accadia, C., Mariani, S., Casaioli, M., Lavagnini, A., and Speranza,
A.: Sensitivity of precipitation forecast skill scores to bilinear
interpolation and a simple nearest-neighbor average method on
high-resolution verification grids, Wea. Forecast., 18, 918–932,
2003.
Baldwin, M.: Quantitative Precipitation Forecast Verification Doc-
umentation, Tech. rep., NCEP/EMC, www.emc.ncep.noaa.gov/
mmb/ylin/pcpverif/scores/docs/mbdoc/pptmethod.html, 2000.
Barnes, S. L.: A technique for maximizing details in numerical
weather map analysis, J. Appl. Meteorol., 3, 396–409, 1964.
Barnes, S. L.: Mesoscale objective analysis using weighted time-
series observations, NOAA, National Severe Storm Laboratory,
Norman, OK 73069, tech. memo. ERL NSSL-62, 1973.
Buzzi, A. and Foschini, L.: Mesoscale meteorological features as-
sociated with heavy precipitation in the southern Alpine region,
Meteorol. Atmos. Phys., 72, 131–146, 2000.
Chéruy, F., Speranza, A., Sutera, A., and Tartaglione, N.: Surface
winds in the Euro-Mediterranean area: The real resolution of nu-
merical grids, Ann. Geophys., 22, 4043–4048, 2004.
Ebert, E. E. and McBride, J. L.: Verification of precipitation in
weather systems: determination of systematic errors, J. Hydrol.,
239, 179–202, 2000.
Fisher, R. A.: Statistical Methods for Research Workers, Oliver and
Boyd, Edinburgh (UK), 1925.
Gabella, M., Joss, J., Perona, G., and Michaelides, S.:
Range adjustment for Ground based Radar, derived with
the space-borne TRMM Precipitation Radar, IEEE Transac-
tions on Geoscience and Remote Sensing, 44, 1, 126–133,
doi:10.1109/TGRS.2005.858436, 2006.
Hanssen, A. W. and Kuipers, W. J. A.: On the relationship be-
tween the frequency of rain and various meteorological parame-
ters, Meded. Verh., 81, 2–15, 1965.
Koch, S. E., desJardins, M., and Kocin, P. J.: An interactive Barnes
objective map analysis scheme for use with satellite and conven-
tional data, J. Clim. Appl. Meteor., 22, 1487–1503, 1983.
Malguzzi, P. and Tartaglione, N.: An economical second order ad-
vection scheme for numerical weather prediction, Quart. J. Roy.
Meteorol. Soc., 125, 2291–2303, 1999.
Panofsky, H. A. and Brier, G. W.: Some applications of statistics
to meteorology, Pennsylvania State University, University Park,
1958.
Schaefer, J. T.: The critical success index as an indicator of warning
skill, Wea. Forecast., 5, 570–575, 1990.
Stephenson, D. B.: Use of the “Odds Ratio” for diagnosing forecast
skill, Wea. Forecast., 15, 221–232, 2000.
Tartaglione, N., Mariani, S., Accadia, C., Speranza, A., and Ca-
saioli, M.: Comparison of raingauge observations with modeled
precipitation over Cyprus using contiguous rain area analysis,
Atmos. Chem. Phys., 5, 2147–2154, 2005.
Wilks, D. S.: Statistical methods in the atmospheric sciences: An
introduction, Academic Press, San Diego (USA), 1995.
Xie, P. and Arkin, P. A.: An intercomparison of gauge observations
and satellite estimates of monthly precipitation, J. Appl. Meteo-
rol., 34, 1143–1160, 1995.