The Science of the Total Environment 249 2000.

13]23

Trends in risk assessment and risk management

G.H. EduljeeU
ERM, Eaton House, Wallbrook Court, North Hinksey Lane, Oxford OX2 0QS, UK

Abstract
Environmental risk assessment has matured into a powerful analytical tool, which is finding ever-wider applications
in the arena of policy making and regulation. However, the principal focus of its development to date has been on
the technical challenges of characterising and modelling the environmental behaviour and biological action of
chemicals, whereas issues concerning its broader socio-political context have been generally neglected. Problem
definition, risk analysis and decision making have, therefore, tended to be dominated by experts and by expert
opinion. Fresh insights from the social sciences advocate a pluralistic, inclusive approach, with experts participating
alongside other stakeholders in a consensual decision making process. Adoption of this paradigm has far reaching
consequences for the form and conduct of risk assessment and risk management. Q 2000 Elsevier Science B.V. All
rights reserved.
Keywords: Risk management; Integrated assessment; Consensual decision making; Deliberative processes

1. Introduction
Risk assessment is hardly a new or novel undertaking: as individuals we intuitively analyse,
assess and decide upon risky situations or life
choices with inherently uncertain outcomes as
part of everyday living. However, the adoption of
risk assessment as a formalised analytical process
applied to environmental issues and latterly as a
policy tool to assist regulators in decision-making,

Tel.: q44-01865-384-811; fax: q44-01865-384-848.

E-mail address: ghe@ermuk.com G.H. Eduljee.

is a relatively recent development, when techniques broadly similar to the risk assessments of
today were used in the 1930s to set permissible
occupational exposure limits for chemicals in the
workplace. Since the 1930s, and particularly since
the 1980s, environmental risk assessment has undergone not only a revolution, but also a
counter-revolution that challenges establishment
views on the supremacy of science and infallibility
of experts. This short paper discusses some of
these developments and speculates on the likely
influence of new thinking drawn from the social
sciences on the future form and content of the
process and its outcomes.

0048-9697r00r$ - see front matter Q 2000 Elsevier Science B.V. All rights reserved.
PII: S 0 0 4 8 - 9 6 9 7 9 9 . 0 0 5 0 7 - 0

14

G.H. Eduljee r The Science of the Total En ironment 249 (2000) 13]23

2. Recent developments
2.1. Technical de elopments
Paustenbach 1989. has charted the practice of
environmental risk assessment over the past few
decades. While formal assessments of the risks to
human health in an occupational setting have
been conducted since the 1930s, a systematic and
above all quantitative approach to environmental
risk assessment can be traced to the work of the
US National Research Council 1983., whose
seminal contribution has influenced the conduct
of risk assessment world-wide. In considering a
conceptual framework for the identification and
assessment of risks to human health, the National
Research Council 1983. created a process comprising the following four stages:
1. hazard identification: which chemicals are important and why?
2. exposure assessment: fate and transport of
chemicals, who might be exposed and how?
3. toxicity assessment: determining the numerical indices of toxicity for computing risk;
4. risk characterisation: estimating the magnitude of risk and the uncertainty of the estimate.
The estimates calculated by a risk assessment
are used as a basis for deciding on actions to
eliminate, reduce or otherwise manage the risk
under consideration. This process is termed risk
management. Monitoring and auditing the efficacy of the selected management measure coupled with a feedback loop to the process of risk
assessment results in a cycle of continuous
surveillance and updating of the risk management
decision to take into account new or improved
data, or changing exposure circumstances.
The 1980s and 1990s saw great strides in developing and improving tools to apply to each of the
four stages of risk assessment Paustenbach, 1989,
1995.. Examples include the development of low
dose extrapolation models to elicit the doseresponse characteristics of a chemical, physiologically-based pharmacokinetic PBPK. biological
models, physical and physicochemical models to

characterise the fate, transport, cross-media distribution and uptake of chemicals, and statistical
tools to replace the deterministic treatment of
sensitivity and uncertainty. The advent of computer-assisted modelling and data handling
techniques has transformed the conduct of quantitative human health and environmental risk assessments in the 1980s and 1990s, but by the same
token has also tended to confer a spurious sense
of infallibility and overconfidence in the results of
the assessment, engendering an elitist culture
among experts while excluding and alienating
the recipients of the assessment see below..
The framework of the National Research
Council 1983. and the tools initially developed
for the quantification of human health risks have
subsequently been extended to other environmental problems including ecological risk assessment Suter, 1993. and in the regulatory arena, to
comparative risk assessment and policy analysis
Ministry of Housing, Physical Planning and the
Environment, 1989; US EPA, 1993; Department
of the Environment, 1995; ILGRA, 1999..
2.2. Procedural de elopments
The most influential paradigm for the analysis
of risk has been the framework propounded by
the National Research Council 1983., in which
the process of risk assessment referred to above
leads to the activity of risk management. The
National Research Council 1983. advocated a
separation between risk assessment and risk management. The NRC regarded the process of risk
assessment as an activity conducted by the application of objective science and scientific principles, while risk management was viewed as a
decision making process that entailed. considerations of political, social, economic and engineering information with risk-related information... to
develop, analyze, compare and select. the appropriate regulatory response... The selection
necessarily requires the use of value judgements
on such issues as the acceptability of risk and the
reasonableness of the costs of control. The NRC
regarded the fact that many assessments were so
laden with value judgements and subjective views
of the risk as a problem. Other countries have

G.H. Eduljee r The Science of the Total En ironment 249 (2000) 13]23

subsequently accepted this general premise: for

example, the Health Council of the Netherlands
1995. states that risk assessment is generally an
activity conducted by experts in consultation with
risk managers and government agencies, implicitly echoing the underlying assumptions of objectivity vs. subjectivity held by the NRC, and the
notion that experts are able to understand and
calculate actual or real risks while a lay persons
perception of risk is subjective, value laden, capricious and misperceived see, e.g. Whipple, 1989..
Canada, Australia and New Zealand also follow
the NRC scheme Ontario Ministry of the Environment, 1989; ANZECCrNHMRC, 1992..
This stance has gradually been eroded with the
realisation that the objectivity of experts and their
estimates of risk is a myth; indeed it has been
argued that there is no distinction between perceived risks and actual risks because there are no
risks except perceived risks Shrader-Frechette,
1990; Adams, 1995.. Experts and lay persons alike
are rooted in deeper individual and collective
cultural systems of values and beliefs. Social and
psychological factors as well as cultural conditioning will influence the experts choice of risk
probabilities, weightings, best guess estimates,
the degree of optimism or pessimism injected into
sensitivity and uncertainty analysis, etc. Thus, if
the information an expert puts into a risk assessment is value-laden, the output will by definition
also lose its aura of dispassionate objectivity, and
models used by risk assessors to characterise risk
can no longer be regarded as truth machines
van Asselt et al., 1996..
If the expert risk assessor is no more able to be
detached from the filter of subjective judgement
than a lay person, two important consequences
for the risk assessment and risk management
process follow. Firstly, the objective vs. subjective
distinction between risk assessment and risk management cannot be maintained, and the same
decision making processes envisaged by the NRC
as applying to risk management should also be
applied to the risk assessment process. In other
words, risk assessment should no longer be treated
as the exclusive province of experts. Stakeholder
involvement in its broadest sense should be ap-

15

plied to shape problem definition, scope, conduct

and output of both risk assessment and risk management. As noted by Slovic 1987., the conceptualisation of risk by lay stakeholders. is much
richer than that of the experts and reflects legitimate concerns that are typically omitted from
expert risk assessments. As a result, risk communication and risk management efforts are destined to fail unless they are structured in a twoway process. Rather than being the primary decision-makers, experts become facilitators, bringing
to the table an understanding of how the environment might react to various stressors and of the
limitations of the knowledge base. Secondly, if we
accept that value judgements and socio-political
mores permeate both risk assessment and risk
management, then these often qualitative considerations should be separately and explicitly
incorporated into the analytical and decision
making framework alongside numerical expressions of risk. The title of the report prepared by
the Health Council of the Netherlands 1996. }
Risk is more than just a number } encapsulates
this trend from exclusivity towards inclusivity, and
from a narrow, technocratic viewpoint of risk
centred on what many on the receiving end of
risk assessments view as numerical sophistry towards broader value-focused thinking Keeney,
1992; Tal, 1997..
Whether present decision making systems are
conducive to value-focused thinking remains a
matter of debate, and a large body of literature,
primarily rooted in the social sciences, has evolved
on this subject. In general, the view is that they
are not. Keeney and Raiffa 1976., MacKay 1980.,
Bezembinder 1989. and Stirling 1998. invoked
the work of Arrow 1963. to highlight a fundamental limitation of paradigms that seek definitive outcomes through a purely analytical process
of decision making: there is no single organising
principle that can deliver a unique objective or
rational outcome selected from a range of possible outcomes or choices. Shrader-Frechette 1990.
calls for negotiated risk solutions... that explicitly
recognise the necessity of free, informed consent
to risk among the stakeholders rather than an
imposed risk management solution based on an

16

G.H. Eduljee r The Science of the Total En ironment 249 (2000) 13]23

experts view of risk. The challenge lies in translating individual viewpoints of risk into a social
consensus see Section 3.1..

3. Future trends
3.1. Procedural de elopments
In keeping with the trends outlined in Section
2.2, it is reasonable to anticipate a greater expectation on the part of stakeholders to participate
in and have direct input to environmental decision making. However, the manner in which this
can be achieved remains a subject of debate.
Layfield 1987. takes the view that as in other
complex aspects of public policy where there are
benefits and detriments to different groups, Parliament is best placed to represent the publics
attitude to risk. As noted by HM Treasury 1996.,
government is acting as a guardian of peoples
rights, where the individual at risk enjoys little or
no benefit from the risky activity and where the
power of redress... is very weak. However, HM
Treasury 1996. also notes that the protection of
consumer interests should generally reflect consumers informed and considered preferences.
Whereas the parliamentary system is on one level
the public affirmation of a democratic process, it
also provides very limited opportunities for
genuine stakeholder participation on a broad
range of issues Bezembinder, 1989..
Nevertheless, the stance of Layfield 1987. does
raise the issue of how a strategic national overview
can be maintained within a bottom-up decisionmaking process. In what way should, say, a coherent national transport or waste management
strategy be formulated, while at the same time
respecting the need for participatory decision
making at the local level? The problem is particularly acute in policy areas such as waste management, where tightly drawn system boundaries at
the sub-regional level can result in piecemeal and
less than optimal solutions.
If the premise of stakeholder participation is
accepted, then in principle the process of decision
making should be no different to that employed
on a smaller scale at the local level; individual

preferences should be debated and channelled

into the formulation of national strategy, which in
turn is then implemented at regional and sub-regional levels by the same process. Examples of
this process are provided by Dienel 1989. and
Renn 1999. for Germany, Switzerland and the
US, and by Petts 1995. for the UK. The new
paradigm, therefore, places participatory decision
making centre stage, but even within this general
framework there are different national approaches. For example, in the Netherlands the
Health Council of the Netherlands 1995, 1996.,
while recognising the necessity for participatory
decision making in risk assessment and risk management, depicts public consultation as informing
risk assessment, but not risk management per se
Fig. 1.. Thus, a fundamental decision making
pinch point, deciding on the tolerability of risk,
would appear to lie outwith the mainstream participatory decision making framework, and although the text does emphasise the need for
consultation during every stage of the process,
this often seems to be limited to two participants:
the experts who conduct risk assessments, and
government.
The depiction in Fig. 1 represents a shift from
an earlier framework for risk management in the
context of environmental policy developed in the
Netherlands Ministry of Housing, Physical Planning and the Environment, 1989., which was
overtly technocentric in conception. The Australian perspective on environmental risk Beer
and Ziolkowski, 1995. appreciates the difficulty of
conducting risk assessment in a socio-political
vacuum, but appears not to include a formal
participatory decision making component. A more
inclusive formal model is adopted in other risk
assessment and risk management frameworks
Department of the Environment, 1995; Canadian
Standards Association, 1996.. In an important
contribution to the debate, the National Research
Council 1996. analysed the limitations and reasons for the unsatisfactory outcomes arising from
the technocentric framework of the National Research Council 1983., and advocated a fundamental shift towards a comprehensive stakeholder-based decision making system. The National Research Council 1996. argued that social,

G.H. Eduljee r The Science of the Total En ironment 249 (2000) 13]23

17

Fig. 1. Link between risk assessment and risk management redrawn from the Health Council of the Netherlands, 1995..

ethical and other values should be considered

alongside conventional concerns over health and
ecological risks during problem definition, risk
assessment and decision making for the purposes
of risk management. At each stage, a deliberative
exchange involving the collective consideration of
the issues with continuous feedback, ensures
readjustment and improvement of the process to
allow for a wide variety of viewpoints and outcomes. The National Research Council 1996.
analysis of the conventional risk paradigm leads
logically and unequivocally to the necessity for an
inclusive and broader-based decision making
process. The practical implications of the concept
were further developed by the Commission on
Risk Assessment and Risk Management 1997.,
whose risk management paradigm perhaps best
summarises this approach by explicitly embedding
stakeholder engagement within the entire span of
the risk analysisrmanagement process Fig. 2..
While the sentiments encoded within these latter risk management paradigms reflect current

thinking on the necessity for stakeholder involvement, significant institutional barriers remain to
be overcome before theory can be put into practice. The Health Council of the Netherlands
1996. notes the Dutch governments constitutional duty to make judgements in the public domain about the tolerability of risk and the measures which have to be taken in order to manage
risks and emphasises the need to establish legal
criteria and procedures for evaluating and weighing the different stakeholder points of view. In
most countries this will necessitate a new style of
governance, moving away from the highly centralised and closed decision making systems of
today. The challenge is even greater in the case of
trans-national collaborations such as the EU and
its executive body the EC, which positions another, more remote tier of governance over national decision making institutions.
A large literature exists on techniques for
stakeholder participation in environmental decision making see, e.g. Petts, 1999. but relatively

18

G.H. Eduljee r The Science of the Total En ironment 249 (2000) 13]23

Fig. 2. Framework for risk management redrawn from Commission on Risk Assessment and Risk Management, 1997..

little by way of analysis of the democratic credentials of the decisions themselves and of the influences that are brought to bear on the stakeholders. Arrows Impossibility Theorem has offered a fertile framework for such an analysis in
the workings of government and social welfare.
For example, Sen 1966, 1970. has shown that
majority decisions can satisfy Arrows conditions
for democratic choice if individual preferences
were channelled into a set of collective views.
This provides a theoretical basis for assessing
stakeholder participation techniques discussed in
Petts 1999. as a means of alleviating the difficulties noted by Bezembinder 1989. and Stirling
1998. in translating individual preferences into
societal choices, and has the potential to bring
fresh insights into the dynamics of options appraisal in environmental decision making. Other

issues such as the role of environmental pressure

groups in seeking to influence stakeholder choice
can also be analysed in the context of Arrows
work.
3.2. Risk management
3.2.1. Risk management strategies
One outcome of the Arrows Impossibility
Theorem is that the concept of absolute safety on
the one hand and of broad, generic expressions of
tolerability on the other is also called into question, since a single, unitary expression of tolerability is at odds with reconciling within a unique
analytical framework individual viewpoints of risk,
all of which have equal validity. Under this
paradigm, monolithic statements of tolerability
should be replaced by a system that accommo-

G.H. Eduljee r The Science of the Total En ironment 249 (2000) 13]23

dates greater differentiation of stakeholder values

as well as the inherent variability of the natural
environment. In essence, the discussion on tolerability and development of risk management
strategies is more likely to be based on the principles of ALARA as low as reasonably achievable.
or ALARP as low as reasonably practicable .
than on absolute expressions of risk or tolerability. This trend is evident in national regulatory
systems and in framing EU legislation, either
explicitly UK, the Netherlands, Norway. or implicitly through concepts such as BATNEEC best
available techniques not exceeding excessive cost.
or BAT best available technology. which also
includes a cost-effectiveness element. There is
little doubt that regulation based on the concepts
of ALARA or ALARP is more complex and
potentially more time-consuming than the application of a single national standard, given the
difficulty of defining value-laden terms such as
reasonable and achievable in a manner appropriate to each context.
As to what constitutes reasonableness, the
term necessarily accommodates a range of criteria covering human health, well being of the
ecosystem, economic and social factors, as well as
the concept of fairness. Since it is impossible to
condense these attributes down to a single
numerical representation of risk in any meaningful way, as is often attempted in comparative risk
assessment or options appraisal for the purpose
of ranking, a decision making framework using
the concept of ALARA or ALARP will tend to
retain the various attributes in disaggregated
form, to which would then be applied a range of
empirically derived weighting factors. Stirling
1998. has described the resulting constructs as
political sensitivity maps which would display the
technical, social, economic, etc. attributes of the
options in an essentially neutral manner, without
prejudging their relative merits. Hybrids of such
decision making systems are used presently; for
example, combining weighting factors derived
through stakeholder consultations with estimates
of the effect of a range of environmental stressors
to derive a single numerical representation of the
overall risk posed to the environment. Alternative

19

decision making systems and techniques incorporating numerical and qualitative risk estimates
into a formal deliberative process involving stakeholders need to be developed Apostolakis and
Pickett, 1998; Renn, 1999..
Acknowledging that experts and lay people
share, and are influenced by, the same social and
cultural forces leads to the conclusion that the
community of experts can sometimes present a
range of individual viewpoints that are as heterogeneous and as deeply held as any other group of
stakeholders. Therefore, another direction in
which risk management is likely to develop is in
the increased use of the precautionary principle
in the realisation that science, and experts, may
not provide unequivocal, consensual truths, and
that the further into time and space one has to
extrapolate our current knowledge base, the
greater the inherent uncertainty and lack of
agreement on the nature of the outcome. The
conventional form of the principle states that
preventative action must be taken when there is
reason to believe that harm is likely to be caused,
even when there is no conclusive evidence to link
cause with effect: if the likely consequences of
inaction are high, one should initiate action even
if there is scientific uncertainty.
The precautionary principle encourages action
despite the lack of a complete understanding of
cause or effect. However, there remains the issue
of what action to select out of a range of possible
options, leading to the type of stakeholder involvement and decision making paradigm discussed above. Climate change, a stressor which
has the potential to create serious environmental
impacts but whose mode and timescale of action
is highly uncertain, is a supreme example of the
issues raised by consideration of the precautionary principle Dowlatabadi and Morgan, 1993;
Shlyakhter et al., 1995; Shaub, 1999..
3.2.2. Setting standards
An inclusive decision making paradigm will involve stakeholders not merely in developing the
risk assessment, but also in deciding upon the
tolerability of risks and the standards against
which measurements should be compared see

20

G.H. Eduljee r The Science of the Total En ironment 249 (2000) 13]23

Fig. 1 vs. Fig. 2.. In accommodating a range of

viewpoints as to what is tolerable, the nature of
the standards themselves is likely to change, with
single generic numerical standards progressively
being replaced by a more flexible system, for
example by a range of standards set according to
different circumstances of exposure, as is currently practised in the field of soil protection.
Alternatively, tolerability can be circumscribed by
upper and lower limits rather than described as a
single numerical entity, with risk management
options selected according to a set of organising
principles combining technical, economic and social perspectives, adopted by relevant stakeholders. Thus, issues such as reasonableness,
fairness and practicability as interpreted by
stakeholders in each particular circumstance will
dictate the positioning of a tolerable risk within
the range, which in turn informs the risk management measures adopted. The use of a range of
acceptable lifetime risk of fatality of 1 in 10 4 to
1 in 10 6 as opposed to a single value, typically 1 in
10 6 , is an example of this trend. In the UK the
Health and Safety Executive 1992. defines a region of tolerability and a broadly acceptable
region within the tolerability of risk approach to
the management of risks in the nuclear and industrial sectors, and this concept has also been
applied to the evaluation of the best practicable
environmental option BPEO. under integrated
pollution control legislation Environment
Agency, 1998..
3.3. Methodological issues
In the light of the above discussion, it is of
interest to speculate on likely trends in methodological aspects of risk assessment. Firstly, the
concept of damage as primarily being limited to
the risk to human and occasionally ecological
health needs to be broadened to include a range
of economic and social considerations. The formal inclusion of these additional dimensions of
risk will in turn necessitate a disaggregated, matrix approach to decision making as opposed to
simplification of the risks to a single number or
set of numbers. Since each stakeholder viewpoint

is an equally valid expression of reality, the risk

assessment will involve an analysis of sensitivity
rather than uncertainty per se. The distinction is
important from a philosophical standpoint: sensitivity analysis implies that the values under test
represent certainty and reality, albeit in some
cases with a low probability or frequency of occurrence for example, extreme exposure possibilities or individuals who are particularly susceptible to a chemical insult., whereas uncertainty
analysis reflects the in.completeness of, and the
confidence with which we can apply, our
knowledge base. Present risk assessments are typically based on single-scenario estimation procedures; alternative methodologies need to be developed that formally weave sensitivity and uncertainty analysis into the fabric of the risk assessment and management framework, which would
include stakeholder involvement. Techniques also
need to be developed to compare, assess and
manage dissimilar and unequal risks, as well as
risks that vary over time and differ in severity for
example, acute vs. chronic health effects ..
Secondly, the trend towards integrated risk assessments will continue. By integrated we mean
assessments that allow for multiple exposure
pathways, inter-media transfers of pollutants
together with secondary environmental effects, as
well as assessments that draw into the decision
making framework technical, health, economic,
social and other issues. Environmental quality
standards for substances in air, water, soil, biota
and foods will have to be developed in a manner
that ensures internal consistency and coherence
Van de Meent and de Bruijn, 1995; Eduljee and
Gair, 1997..
Thirdly, risk assessments will need to take
greater note of background risks and of exposure
to low levels of pollutants in order to place in
context the incremental risk imposed by a particular management option, new development or activity and to define the baseline risk prior to the
assessment of management options. These pose
considerable scientific challenges, not least in understanding, distinguishing, disentangling and
characterising the cause and effect chain for a
chemical or other type of stressor from within an

G.H. Eduljee r The Science of the Total En ironment 249 (2000) 13]23

array of confounding intervening influences and

complex interactions such as time delays between
exposure and effect Ashford and Miller, 1998..
This is particularly important in the context of
decisions based on the concepts of ALARA and
ALARP, since the background can often account
for the major proportion of the overall exposure.
Risk assessments will need to take greater account of the cumulative risk caused by exposure
to cocktails of chemicals, often present both in
the new release under consideration and as part
of the background risk.
Fourthly, drawing stakeholders into the risk
assessment process will inevitably focus risk estimation onto actual exposure experienced by specific individuals, as opposed to the hypothetical
individual, critical group or population risks which
are currently the endpoint of virtually all risk
assessments, using generic exposure assumptions.
Individuals potentially exposed to a particular risk
will want reassurance that each will be completely
protected. In practice, complete protection for all
individuals cannot be guaranteed, since every
nuance of human variability and susceptibility
cannot be accounted for either in terms of the
exposure assessment or in terms of defining an
acceptable or tolerable daily dose or environmental standard. Reconciling the concerns of individual stakeholders with the practical realities of
governance will be a major challenge, especially
in dealing with extreme exposure circumstances
and particularly sensitive biological dispositions.
Finally, the move from generic towards more
focused risk assessments will fuel a need for
measurements characterising the activity and exposure patterns of specific susceptible groups to
supplement the default assumptions used in
generic risk assessments, and for techniques that
afford direct measurements of environmental and
biological effects and of small changes in environmental status, to counter our present over-reliance on fate, transport and biochemical modelling with its attendant problems of interpretation. Many complex models appear to be built on
foundations of sand, their outputs theoretical
constructs for example, assumed effects at low
dose extrapolated on the basis of linearised or
other dose response models. that are not

21

amenable to verification by direct measurement.

The identification and development of biomarkers, bioassays and direct toxicity testing procedures would be invaluable in supporting and supplementing modelling procedures. The latter can
never be entirely supplanted; direct measurements provide information on the prior and current status of the environment, while modelling
enables past and present information to be projected into the future. The ability of models to
establish a link between the past, the present and
the future confers a powerful predictive attribute
to risk assessment as a management tool.

4. Conclusions
Paraphrasing John Quincy Adams, risk assessment in the past 50 years seems to have viewed
the subject with too much reference to the nature
of things and not enough to the nature of man.
The realisation that value-laden judgements and
decisions permeate every facet and every stage of
risk assessment and risk management has led to
the development of a new paradigm which demands a pluralistic approach to risk assessment
and risk management and for value-focused decision making Keeney, 1992.. The challenge in the
coming years will be to embed the use of science
in risk assessment and risk management within a
socio-political framework, and to subsume within
the decision making process the nature of things
and the nature of man. This is not to diminish the
status or role of scientists and of experts in environmental decision making. On the contrary, the
new paradigm relies crucially on a better understanding of environmental and biological
processes and on greater sophistication, transparency and rigour in the application of science,
but within a collaborative and consensual decision making framework.
References
Adams J. Risk. London: UCL Press, 1995.
ANZECCrNHMRC Australian and New Zealand Environmental and Conservation Council, National Health and

22

G.H. Eduljee r The Science of the Total En ironment 249 (2000) 13]23

Medical Research Council. Australian and New Zealand

Guidelines for the Assessment and Management of Contaminated Sites. ANZECC. Canberra,1992
Apostolakis GE, Pickett SE. Deliberation: integrating analytical results into environmental decisions involving multiple
stakeholders. Risk Anal 1998;18:621]634.
Arrow K. Social Choice and Individual Values. New York:
John Wiley & Sons, 1963.
Ashford NA, Miller CS. Low-level chemical exposures: a challenge for science and policy. Environ Sci Technol
1998;32:508A]509A.
Beer T, Ziolkowski F. Environmental Risk Assessment: An
Australian Perspective. Supervising Scientist, Commonwealth of Australia. Barton, Australia, 1995.
Bezembinder T. Social choice theory and practice. In: Vlek C,
Cvetkovich G, editors. In social decision methodology for
technological projects. Dordrecht: Kluwer Academic Publications, 1989.
Canadian Standards Association. Introduction to Environmental Risk Assessment Studies. Report No. Z763-96.
Canadian Standards Association. Rexdale, Ontario, 1996.
Commission on Risk Assessment and Risk Management. Final
Report: Risk Assessment and Risk Management in Regulatory Decision Making. Washington DC, 1997.
Department of the Environment. Guide to Risk Assessment
and Risk Management for Environmental Protection. Her
Majestys Stationary Office. London, 1995.
Dienel PC. Contributing to social decision methodology: citizen reports on technological projects. In: Vlek C,
Cvetkovich G, editors. Social decision methodology for
technological projects. Dordrecht: Kluwer Academic Publications, 1989.
Dowlatabadi H, Morgan G. A model framework for integrated
studies of the climate problem. Energy Policy 1993;21:
209]221.
Eduljee GH, Gair AJ. Setting dioxin emission limits for MSW
incinerators: a multimedia exposure assessment framework.
Waste Manage Res 1997;15:335]348.
Environment Agency. Best Practicable Environmental Option
Assessment for Integrated Pollution Control. London: The
Stationary Office, 1998.
Health and Safety Executive, 1992. The tolerability of risk
from nuclear power stations. London: HMSO, 1992.
Health Council of the Netherlands. Not all Risks are Equal.
Publication No. 1995r06E. Committee on Risk Measures
and Risk Assessment. The Hague, 1995.
Health Council of the Netherlands. Risk is More than Just a
Number. Publication No. 1996r03E. Committee on Risk
Measures and Risk Assessment. The Hague, 1996.
HM Treasury. The Setting of Standards. HM Treasury, 28
June 1996. London, 1996.
ILGRA Interdepartmental Liaison Group on Risk Assessment. Risk Assessment Approaches used by UK Government for Evaluating Human Health Effects of Chemicals.
Institute for Environment and Health. University of Leicester, 1999.

Keeney RL. Value-Focused Thinking. Cambridge Massachusetts: Harvard University Press, 1992.
Keeney RL, Raiffa H. Decisions with Multiple Objectives:
Preferences and Value Tradeoffs. New York: John Wiley &
Sons, 1976.
Layfield F. Sizewell B public inquiry: summary of conclusions
and recommendations. London: HMSO, 1987.
MacKay A. Arrows theorem: the paradox of social choice.
New Haven NY: Yale University Press, 1980.
Ministry of Housing, Physical Planning and the Environment.
Risk Assessment in the Context of Environmental Policy.
Annex to the National Environmental Policy Plan. Lower
House of the States General, 1988]1989 session, 21137.
The Hague, Netherlands, 1989.
National Research Council, 1983. Risk assessment in the
Federal Government: managing the process. Washington
DC: National Academy Press, 1983.
National Research Council, 1996. Understanding risk: informing decisions in a democratic society. Washington D C:
National Academy Press, 1996.
Ontario Ministry of the Environment. Guidelines for the
Decommissioning and Clean-up of Sites in Ontario,
Toronto. February, 1989.
Paustenbach D. A survey of health risk assessment. In:
Paustenbach D, editor. The risk assessment of environmental and human health hazards: A textbook of case studies.
New York: John Wiley & Sons, 1989.
Paustenbach D. The practice of health risk assessment in the
United States 1975]1995: how the US and other countries
can benefit from that experience. Hum Ecol Risk Assess
1995;1:29]79.
Petts J. Waste management strategy development: a case
study of community involvement and consensus building in
Hampshire. J Environ Plann Manage 1995;38:519]536.
Petts J. Public participation and environmental impact assessment. In: Petts J, editor. Handbook of environmental impact assessment. Oxford: Blackwell Science, 1999.
Renn O. A model for an analytic-deliberative process in risk
management. Environ Sci Technol 1999;33:3049]3055.
Sen A. A possibility theorem on majority decisions. Econometrica 1966;34:491]509.
Sen A. Collective Choice and Social Welfare. San Francisco:
Holden-Day Press, 1970.
Shaub WM. View from the administrators office. Environ Sci
Technol 1999;33:276A]279A.
Shlyakhter A, Valverde L, Wilson R. Integrated risk analysis
of global climate change. Chemosphere 1995;30:1585]1618.
Shrader-Frechette KS. Perceived risks versus actual risks:
managing hazards through negotiation. Risk-Issues Health
Saf 1990;1:341]363.
Slovic P. Perception of risk. Science 1987;236:280]285.
Stirling A. Risk at a turning point? J Risk Res 1998;1:97]109.
Suter GW. Ecological risk assessment. Boca Raton: Lewis
Publishers, 1993.

G.H. Eduljee r The Science of the Total En ironment 249 (2000) 13]23
Tal A. Assessing the environmental movements attitudes toward risk assessment. Environ Sci Technol 1997;
470A]476A.
US EPA. Guidebook to Comparing Risks and Setting Environmental Priorities. US EPA, Washington DC, 1993.
van Asselt MBA, Beusen AHW, Hilderink HBM. Uncertainty
in integrated assessment: a social scientific perspective.
Environ Model Assess 1996;1:71]90.
Van de Meent D, de Bruijn JHM. A modelling procedure to

23

evaluate the coherence of independently derived environmental quality objectives for air, water and soil. Environ
Toxicol Chem 1995;14:177]186.
Whipple C. Nonpessimistic risk assessment and the minimis
risk as risk management tools. In: Paustenbach D, editor.
In the risk assessment of environmental and human health
hazards: A textbook of case studies. New York: John Wiley
& Sons, 1989.