Vous êtes sur la page 1sur 8

Urban Water 4 (2002) 137144 www.elsevier.


Performance indicators for urban storm drainage in developing countries

Pete Kolsky


, David Butler

a ^te dIvoire Water and Sanitation Program, World Bank, BP 1850 Abidjan 01, Co Urban Water Research Group, Department of Civil and Environmental Engineering, Imperial College, Imperial College Road, SW7 2BU London, UK

Received 31 January 2001; received in revised form 7 December 2001; accepted 28 February 2002

Abstract This paper describes conceptual and practical aspects of urban storm drainage performance indicators, based on the authors experience in developing countries, particularly India. The paper begins by presenting a general framework of objectives and performance indicators as logical intermediate steps between values and the decisions taken to reect them. The paper then considers practical approaches to performance and indicator measurement, based on eld experience in India. General conclusions about drainage performance indicators are then presented, stressing the challenge of nding indicators which are (1) valid indicators of performance, (2) relatively easy to measure, and (3) helpful to the decision-maker. 2002 Published by Elsevier Science Ltd.
Keywords: Decisions; Developing countries; Drainage; Flooding; Indicators; Objectives; Performance; Values

1. Values, objectives, indicators and decisions Engineers and other environmental professionals are in the business of making decisions to reect the values of society in its interaction with the environment. In this context, social values are abstract notions like truth, beauty, safety, comfort, environmental balance, justice, equity, and freedom. Environmental professionals must, however, deal with more concrete questions of planning permission, pipe size, chemical dose, frequency of drain cleaning, and cost. Linking concrete decisions to abstract values is dicult, and performance indicators are intended to help bridge the gap. Debates about standards, which are usually based upon indicators, reect and illustrate the complexity of this relationship. 1.1. Debates about standards Performance standards for drainage vary widely, and this variation provokes substantial debate. (See, for example, the debates about combined sewer overow (CSO) standards at the 7th International Conference on Urban Storm Drainage in Hannover, 1996 and in Lau,

Butler, & Schutze, 2002). Some variation in standards is inevitable, and reects the wide variation in climatic and socio-economic conditions throughout the world. The debate is not, however, restricted to the appropriate level of a given performance standard (e.g. the frequency of ooding to be tolerated). The argument often revolves around the choice of indicator to reect performance. Are we concerned about the number of CSO incidents, or about the total volume of overow? (Lau et al., 2002). Do we care more about violations of standards for BOD5 , or for those of nutrients or suspended solids? Is anyone measuring bacterial concentrations, and if so, how and why? Debates about standards often work on dierent levels of abstraction. Some may criticise a performance standard as too lax, while others may argue that tightening the standard is irrelevant because the indicator used by the standard does not accurately reect critical aspects of performance. Experts can and do disagree about denitions of acceptable environmental damage, or about how such damage is assessed, or about the choice of appropriate indicator for such damage. 1.2. Levels of abstraction

Corresponding author. E-mail addresses: pkolsky@worldbank.org (P. Kolsky), d.butler@ic.ac.uk (D. Butler). 1462-0758/02/$ - see front matter 2002 Published by Elsevier Science Ltd. PII: S 1 4 6 2 - 0 7 5 8 ( 0 2 ) 0 0 0 1 1 - 0

In clarifying discussion about performance standards and indicators, it may be helpful to consider the levels of


P. Kolsky, D. Butler / Urban Water 4 (2002) 137144

Table 1 Levels of abstraction between values and decisions Level Values Objectives (and performance) Indicators Decisions Working denition Abstract ideals which are sought. Objective manifestation of progress towards values. Examples Truth, beauty, justice, equity, comfort, health, environmental balance, sustainability. Increased number of species, clearer water, reduced concentration of disease-carrying organisms, less frequent anaerobic conditions. E. coli, nutrient, BOD5 concentrations, number, volume of overow events. Choices of pipe size, chemical dose, treatment technology, enforcement decisions, planning permission, etc.

Practically measurable surrogates for performance, ( the satisfaction of objectives). Technical, legal, administrative, economic and other choices made to achieve objectives and progress towards values.

abstraction between values and decisions; one such hierarchy is shown in Table 1. In this paper, values are dened as the ideals we are trying to achieve by a given action. Although it is comforting to believe that at heart we all share most values, the devil is in the detail and the trade-o between values. Much political debate revolves about the balance between freedom and equity, although most of us value both. Some arguments that appear to be about standards and indicators actually reect disagreements about values. This is often the case when a standard is criticised as being too stringent because it will cost too much to achieve. Engineers nd values to be an insucient guide to action. We cannot design wastewater treatment works to deliver environmental balance without a more technical specication of this value. Objectives are dened in this paper as answers to the question How do we know that we have succeeded in promoting a given value? For a wastewater treatment works discharging to a river, we might say, If the river water looked clearer, and the populations of certain species that we value increased, then we would have made progress towards environmental balance. A systems performance in this paper is thus dened as the extent to which the system satises the objectives for which it was created. Technical professionals and the general public can argue about objectives just as passionately as they can argue about trade-os between values. Sometimes the arguments may be about the trade-os between dierent objectives for the same value. For example, do we care more about species diversity or amenity? Do we care more about the number of pollution incidents, or about their severity? Do we care more about euent bacterial concentrations or the formation of trihalomethanes from post-chlorination? While objectives are more concrete than values, they are often dicult to measure. Wastewater treatment plant operators cannot use bio-diversity surveys to operate on a day-to-day basis. Similarly, if a drainage system normally oods only once in ve years, can we really tell that its performance has improved if it oods only once in the next 10 years? Do we want to

wait that long, and how statistically signicant is such a result anyway? These problems have been partially solved by the development of indicators. It is practically impossible to measure the concentrations of Cholera vibrio, Salmonella typhosa, and every other currently known diseasecausing organism in drinking water on a regular basis. It has, however, become relatively straightforward to measure the indicator organism E. coli, and this indicator has been widely successful in advancing the objective of safe drinking water to promote the value of health for all. Performance indicators are thus dened in this paper as practically useful surrogates for the direct measurement of performance. Most standards are based on indicators, (because they can be reasonably measured), rather than on performance itself. Indicators are inherently controversial, precisely because they are imperfect surrogates for the objectives they indicate. The map is not the territory. While E. coli has generally been viewed as a successful indicator for microbiologically safe water, researchers have rightly questioned its value as an indicator for viral contamination, or for the public health risk of contaminated bathing water. Given the diculty of measuring performance directly, and the historical success of some indicators, it is tempting to believe that suitable performance indicators always exist. Indeed, a number of planning and management methodologies (e.g. the Logical Framework used by a number of international development agencies), assume this to be the case. There is, however, no reason to assume a priori that practical low-cost indicators can be developed for every important aspect of performance. Ultimately, we are concerned with indicators because we seek a practical way to obtain relevant data about performance on which to base decisions. These may be engineering decisions about pipe size, chemical dose, detention tank volumes, or enforcement decisions about discharges of unacceptable wastes. Indicators such as the predicted 100-year ood plain level may also inform administrative decisions about the nature and extent of housing development permitted. The validity of each level of the hierarchy depends upon that of the levels above. Engineering decisions are

P. Kolsky, D. Butler / Urban Water 4 (2002) 137144


frequently made to achieve certain results as measured by indicators, but these may or may not accurately reect progress towards achieving higher level objectives and promoting social values. In the design and operation of wastewater treatment works, for example, a large number of decisions are based on euent quality standards. These standards serve as indicators of environmental quality objectives, but do not, in and of themselves, directly measure environmental balance. The extent to which decisions in treatment plant design and operation actually promote the value of environmental balance depends upon the validity of both the indicator (the euent standard) and the objective (the desired environmental conditions in the receiving water) in reecting the social value of environmental balance.

vantages (e.g. low rent, proximity to work and family). Although this work had to be performed towards the end of the eld studies, it broadly conrmed our earlier working denition of performance. Residents explained that drainage would be seen as less of a problem if ooding occurred less often, aected fewer houses, was shallow and slow rather than deep and fast (thus reducing the risk of accidents while walking or cycling, and of damage to property), and lasted less time . . . a few minutes, rather than hours or days. The qualitative research did, however, identify other objectives or performance goals that we had not considered. To the poorest people living adjacent to frequently ooded drainage channels, predictability was a key objective. A major concern mentioned by residents of all four areas related to the predictability of the ooding event . . . In other words, even extensive inundation is bearable if expected . . . Interventions . . . should try to take account of these needs of the community to understand and adapt their coping strategies if necessary. To the vulnerable (whether in Indore or York!) reasonable warning allows a family to ensure its own safety and protect its most valued possessions. The measurement of performance in achieving the objectives of reduced depth, duration, extent and frequency of ooding is conceptually straightforward. Predictability, however, is a more complex objective, both to measure, and to achieve.

2. Implications for urban storm drainage in developing countries 2.1. Objectives and performance Dening the performance of a storm drainage system is not straightforward. What are the objectives of a storm drainage system? How would a community know that a drainage system had improved the quality of its life? The authors of this paper were involved in a threeyear project on drainage performance in developing countries described elsewhere (Kolsky, Parkinson, & Butler, 1996, Chapter 14; Heywood, Kolsky, & Butler, 1997; Kolsky, 1999). Conventional design described in textbooks usually denes storm drainage performance in terms of the frequency of ooding. Given the high frequency of ooding in Indian slums, however we adopted a broader denition of performance to reect the impact of frequent ooding upon the lives of poor slum dwellers. We thus dened storm drainage performance in terms of the frequency, extent, depth and duration of ooding.

2.2. Performance and process indicators Performance itself is often dicult to measure, and performance indicators have evolved largely to address this problem. In many cases, however, even accurate performance data are dicult to use; they may describe the problem, but not what to do about it. Indicators are needed not only to describe performance, but also to point the way towards appropriate decisions. Process indicators, which describe the state of the drainage system (which in turn determines performance) can be much more practical than performance indicators, both in measurement and in use (Table 2). They are, by definition, further removed from performance measurement than performance indicators, and thus involve the risk that changes in process indicators do not reect the desired changes in performance.

Such a denition thus distinguishes between a centimetre of ooding above a single stretch of road for 5 min, and ooding 30 cm deep across a large area for 6 h, even if both types of events occurred 10 times a year. Partly in order to identify suitable drainage objectives and performance criteria from the point of view of slum residents, Stephens, Pathnaik, and Lewin (1994) joined in the project to perform qualitative and participatory research with frequently ooded slum residents in Indore, India. This work established that residents saw frequent ooding as a fact of life or inevitable consequence of living in areas that had certain other ad-


P. Kolsky, D. Butler / Urban Water 4 (2002) 137144

Table 2 Types of performance measures and indicators Type of measurement Performance Examples  Depth  Area  Duration of ooding Advantages  Measures the outcome directlyoften what you want to know Disadvantages  Dicult to measure  Seasonal  Link to necessary decisions often unclear  Relation to outcome may not be clear  May measure symptom, not cause of problem  Relation to outcome may not be clear

Performance indicator

 Solids levels  As-built capacity  Inlet blockage  Frequency of cleaning  Sta time committed to operation  Budget

 Relatively easy to measure  Clearer link to action than performance measurements  Relatively easy and practical  Can be routine  Clear link to action

Process indicator

3. Practical experience from small catchments In our work, we tried a variety of methods to measure performance and performance indicators, (see Kolsky, 1999). Some worked, and some did not. Table 3 and the following text highlight the most successful approaches to both direct measurement of performance, and the measurement of performance indicators. 3.1. Performance measurement 3.1.1. Resident surveys People who have lived in a community for several years often know a great deal about the risks and extent

of ooding in their area. They will denitely remember occasions when water ooded into their homes, and they will have heard from friends and neighbours about how much worse (or better) the problem is a few blocks away. One can usually get a good idea of which areas are worst aected by simply talking to people and asking around. A number of books and articles on both community planning and participatory evaluation (e.g. Feuerstein, 1986; Goethert & Hamdi, 1988) give hints and examples about how best to work with communities in assessing a wide variety of problems, which could easily include ooding. In addition, more detailed work on community perceptions of ooding is possible through application of a variety of qualitative re-

Table 3 Direct measurement of performance Method Resident surveys (systematic interviews of residents regarding ooding) Direct observation during oods Advantages     Low cost Fairly quick to organise Can be done in dry weather Can cover a large area Disadvantages  Depends on memory  Most reliable on depth; less reliable on frequency and duration  More subjective than other methods

 Can observe all of problem in context  Essential for evaluation  Useful check on, and complement to, other data

 May be seasonally limited  Dicult to be at right place at right time: How do you catch the peak?  Unlikely to observe full hydrograph  Can only cover limited area  Dependent upon weather     Only records maximum depth Only valid during season of observation Requires frequent visits Siting is critical

Chalk gauges


Low cost (relative to electronic sensors) Captures oods at any time Useful check on other methods Can cover large areas

Electronic sensors

 Accurate and reliable  24-h/day coverage  Can produce complete hydrographs

 Expensive in money and skill  Can only cover limited number of sites  Require inspection, maintenance, battery checks, and external verication  Requires protection from vandalism

P. Kolsky, D. Butler / Urban Water 4 (2002) 137144


search methods, which do not yield precise numerical results (e.g. average depth of ooding), but rather insight into how people view ooding. These approaches include focus group discussions, individual case studies of aected families, and observation of behaviour before, during and after oods (Stephens et al., 1994). When talking to residents, some simple rules apply: avoid leading questions, ask more than one person, try to be specic. It is probably better to ask rst about the previous years ooding, rather than asking in general, how high does water rise? as the general question really requires residents to perform a pretty complicated averaging exercise in their heads! By contrast, people often remember last years level quite clearly, and the highest level they have ever seen. Similarly, interviewers need to be clear about the reference level of the depth of ooding; does it refer to the middle of the street, or to the household plinth, or to some other point? It is best if the residents show the interviewer a specic place where they remember the high water mark, rather than discussing general depths such as water was hip deep. 3.2. Direct observation during oods Walking around in a ood is one of the best ways to observe performance: which areas are aected, how badly, and the natural drainage routes. However, as a single method of assessing the severity of ooding this approach is demanding, frustrating and limited because eldworkers may not be able to start until the rainy season, cannot be everywhere all the time, can easily miss the most important storm, or the most important part of it, may have many other evaluation objectives to satisfy during a storm (e.g. catchment denition, observation of ow patterns, etc.). Direct observation during oods is much more helpful in seeing how the drainage system as a whole behaves than it can be for making specic performance measurements. Direct observation of pre-painted rulers on telephone poles or walls can provide useful comparisons between storms, and valuable additional data for any detailed computer modelling studies. 3.2.1. Chalk gauges These consist of metre rulers coated with chalk, built into a small protective structure into which oodwaters are free to enter (Fig. 1). As the ood rises, chalk washes o the portion of the ruler below the surface of the

Fig. 1. A chalk gauge (Kolsky, 1999).

water. At the end of the storm, an approximation of the highest level of oodwater can be read at the dividing line between the chalked and unchalked portion of the ruler. The metre stick is then coated again with chalk paste, ready to record the next storms maximum ood level. The box structure protects the metre stick against childrens play and vandalism. In Indore, we established a network of chalk gauges in each of two study catchments to record maximum ood levels. Chalk gauges have the advantages that they cost little, can be distributed across the catchment, and can record maximum ood levels at any time of day or night. In Indore, we found chalk gauges an invaluable check on the validity of computer models for drainage performance; our condence in both the modelling and the validity of the measurements has been boosted by consistent agreement between the two. Chalk gauges have the disadvantage that they do not record the duration of ooding, and they require a regular process of reading and re-painting after each storm. In addition, the gauges need to be carefully sited to provide useful information; it is generally best to site them in low areas where minor events can trigger ooding and thus record a level. Gauges that are too high will simply never record a reading. 3.2.2. Electronic sensors The use of pressure transducers connected to data recording equipment (also known as a logger) oers the most complete data on ood depths. Like chalk gauges, loggers can record data from events occurring at any time of day or night; unlike chalk gauges, however, they can record ood depth over time, so that durations can be computed. Pressure transducers are often combined with electronic velocity gauges to permit discharge measurement in open channels. Thus a typical installation that combines pressure and velocity sensors with recording equipment permits level and ow measurement within


P. Kolsky, D. Butler / Urban Water 4 (2002) 137144

the drain, and may also oer ood level data, depending upon the level to which the water rises. The two major drawbacks of pressure transducers, velocity gauges and data loggers is that not only are they relatively expensive to purchase or rent, but they are also demanding of skilled personnel for installation and use. Like any other measuring device, they must also be checked for accuracy. This is relatively easy for depth, but much more dicult for velocity. In summary, these tools are indispensable for a detailed hydraulic modelling study, but are too expensive and complex for routine use in low-income communities.

for drainage, and the extent to which drainage has or has not been seriously addressed in the catchment. 3.3.2. Solids size distribution All drainage networks contain solids of some form or another. In many cities of the developing world, including Indore, the capacity of many drains is limited by substantial amounts of solid waste and construction debris. We organised the complete cleaning of ve 1 m sections of open drain, and obtained particle size distribution curves for these contents. Comparison with the size distributions found in the industrialised world revealed that the mean particle size by weight was 10 times larger in Indore than in typical distributions found in Britain and Europe (Kolsky & Butler, 2000). The size distribution, combined with measurement of solids levels in the drain, can indicate the nature and extent of drainage capacity reduction from solids deposition.

3.3. Performance indicators Eective performance indicators are a link between performance and action. The development of performance indicators for drainage is in its infancy. While our work in Indore highlighted a number of issues relating performance to design and maintenance, the development and verication of eective indicators for performance requires much more work. Nevertheless, our experience permits us to highlight a number of promising areas. 3.3.1. System drawings Plans and drawings showing locations, sections and slopes of drainage networks provide invaluable information with which to assess the physical capacity of the network. In addition, the existence, condition, and accuracy of such drawings is itself an extremely good indicator of the managerial capacity of those responsible

3.3.3. Solids levels The level of solids in open drains is easily determined, and can quickly reveal what capacity is actually available to manage stormwater ows. (Fig. 2.) Quick surveys can establish whether or not the problem is a serious one, and if so, those locations where the problem is most acute. These measurements can also be repeated to yield the rate of deposition of solids over time, which has a direct bearing upon the required frequency of cleaning. Solids levels in closed drains are more problematic; while a survey of solids levels in manholes can identify gross problems, the solids level in the manhole may well

Fig. 2. Use of the solids scale (Kolsky, 1999).

P. Kolsky, D. Butler / Urban Water 4 (2002) 137144


Fig. 3. Distribution of blockage of street level inlets (Kolsky, 1999).

dier from that in the pipe itself. The inspection of closed drains from the surface using mirrors and powerful lamps can help to identify blockages relatively quickly. 3.3.4. Inlet blockage The degree of kerb or gutter inlet blockage can be estimated and recorded on a rapid tour of the catchment, and can give a quick feel for the state of the existing systems maintenance. This is not precise, and does not describe the degree of blockage within the inlet, but can nevertheless provide helpful information, such as that one-third of the inlets to a network are more than half blocked! (Fig. 3.)

managerial control, but which cannot directly predict or describe performance. While the idea of indicators is appealing, it does not follow that good indicators exist for all aspects of performance. We are well aware that this paper does not oer a clear set of indicators by which drainage performance may be gauged. The paper has, however, tried to share some practical experience in developing performance indicators, while outlining the conceptual diculties that make this a dicult problem. In general, think how indicators will be used before undertaking a major monitoring eort. In some cases it is essential to collect basic data just to know where you are. In general, however, far more data are collected than can ever be used. The eort required to analyse data, and the decisions likely to result from such analysis should be realistically considered before data collection begins. In summary, the map is not the territory. Indicators, by denition, indicate something but are not the thing itself. Performance indicators do not measure performance, or success in promoting values; they are a surrogate for performance measurement. At best, indicators reect a trade-o between validity in representing performance, ease of measurement, and utility in making decisions. Bearing these tradeos in mind when discussing indicators can clarify the issues at stake.

Acknowledgements Fieldwork on indicators described herein was part of Research Project R5477A Performance-Based Evaluation of Surface Water Drainage in Low Income Communities funded by the Engineering Division of the Overseas Development Administration (now the Department for International Development) of the British government. The authors would like to acknowledge the eld support of the sta of the Indore Drainage Evaluation Project, and the organisational support of Professor T.A. Sihorwala and the administration of the Shri Govindram Seksaria Institute of Technology and Science in Indore. The authors also wish to acknowledge the eld support and contribution of Dr. Jonathan Parkinson of GHK Research and Training. This paper is related to the doctoral work of the rst author, who wishes to acknowledge the strong collegial and supervisory support of both Professor Sandy Cairncross and Dr. Ursula Blumenthal at the London School of Hygiene & Tropical Medicine. All opinions expressed herein are those of the authors, and should not be considered to reect the position of the Water and Sanitation Program of the World Bank, or of the Department for International Development.

4. Conclusions This paper presents related conceptual and practical perspectives on the development of performance indicators for urban storm drainage. Although the eld methods described here were used in our research in India, experience in other developing and developed countries leads us to believe that many of the conceptual issues around the meaning, development and use of performance indicators are broadly applicable. In summary, these points are as follows: Indicators lie on the path between values, objectives and decisions. Much debate about performance indicators is actually about values, or about the validity of higher-level objectives and performance criteria. The links between performance and decisions are often unclear. Indicators or measures that describe performance well may not actually be helpful for making decisions. It may be more practical to monitor process indicators, which keep track of activities under


P. Kolsky, D. Butler / Urban Water 4 (2002) 137144 Kolsky, P. J. (1999). Storm drainage: an engineering guide to the lowcost evaluation of performance. London: Intermediate Technology Publications. Kolsky, P. J., & Butler, D. (2000). Solids size distribution and transport capacity in an Indian drain. Urban Water, 2(4), 357 362. Lau, J. T., Butler, D., & Schutze, D. M. (2002). Is combined sewer overow spill frequency/volume a good indicator of receiving water quality? Urban Water, 4(2), 181189. Stephens, C. R., Pathnaik, R., & Lewin, S. (1994). This is my beautiful home: risk perceptions towards ooding and environment in low income communities. London School of Hygiene & Tropical Medicine: London.

Feuerstein, M. T. (1986). Partners in evaluation: evaluating development and community programmes with participants. London: MacMillan. Goethert, R., & Hamdi, N. (1988). Making microplans: a communitybased process in programming and development. London: Intermediate Technology Publications. Heywood, G. M., Kolsky, P. J., & Butler, D. (1997). Modelling drainage performance in an Indian catchment. Journal of the Chartered Institution of Water and Environmental Management, 11(1), 3138. Kolsky, P. J., Parkinson, J. N., & Butler, D. (1996). Third world surface water drainage: the eects of solids on performance, Chap. 14. In D. D. Mara (Ed.), Low-Cost Sewerage (pp. 189214). John Wiley & Sons: Chichester.