Académique Documents
Professionnel Documents
Culture Documents
North-Holland
David W. Shimer
California Institute of Technology, Pasadena, CA 91125, USA
Complexity theory provides formal procedures for analyzing problem difficulty. Frank H. Knight in
Risk, Uncertainty and Profit assumed intelligence is finite and stressed the difficulty of solving
problems involving uncertainty. In this paper, a risk decision is a stochastic optimization problem
where the parameters and the functional forms required to determine the optimal decision are
known. An uncertain decision is a stochastic optimization problem where at least one parameter or
functional form must be estimated. Using complexity theory, a valid distinction can be made
between risk and uncertainty which is consistent with Bayesian statistics. From the perspective of
bounded rationality Knight's concepts of consolidation and specilization can be reconciled with the
Bayesians.
1. Introduction
C o m p l e x i t y t h e o r y p r o v i d e s a f o r m a l m e t h o d o l o g y for a n a l y z i n g the
difficulty of solving p r o b l e m s . The p u r p o s e of using c o m p l e x i t y t h e o r y to
i n t e r p r e t F r a n k H. K n i g h t ' s Risk, Uncertainty and Profit (1921/1971) is to o b t a i n
a valid d i s t i n c t i o n between risk a n d u n c e r t a i n t y which is consistent with
B a y e s i a n statistics. Thus, e c o n o m i c theories can be f o r m u l a t e d using c o m p l e x i t y
theory.
K n i g h t c o n s i d e r e d three types of p r o b a b i l i t y : risk, statistical, a n d uncertainty.
T h e risk case, which K n i g h t defined as 'objective' probabilities, dealt with
p r o b a b i l i t i e s with k n o w n p a r a m e t e r s a n d functional forms. F o r all decisions
involving risk, the b u s i n e s s m a n can eliminate his risk t h r o u g h insurance. In
contrast, the u n c e r t a i n t y case, which K n i g h t defined as 'subjective' p r o b a b i l i t y ,
is c h a r a c t e r i z e d by u n k n o w n p a r a m e t e r s o r functional forms which the business-
m a n m u s t e s t i m a t e w i t h o u t a n y p a s t reference u p o n which to base such an
Correspondence to: Alfred L. Norman, Department of Economics, The University of Texas at Austin,
Austin, TX 78712-1173, USA.
estimate. Statistical probability covered the intermediate case where the busi-
nessman had some prior observations upon which to form an estimate.
The development of statistics based on Savage's (1954) axioms has raised
serious questions concerning Knight's three categories. For example, Bayesian
statistics has made Knight's separation of risk and uncertainty into disjoint
decision problems appear to be of dubious value, as there is no distinction
between 'objective' and 'subjective' probabilities in Bayesian statistics and the
completely unknown case can be handled by the construction of a diffuse prior.
Bayesians insist there is no valid distinction between risk and uncertainty [for
example, see Cyert and DeGroot (1987)]. In reply to such criticisms, Bewley
(1986) provided an interpretation of the difference between risk and uncertainty
based on modifying Savage's axioms.
In this paper the authors propose an alternative interpretation of Knight.
Uncertainty for Knight creates the greatest of logical difficulties both in forming
the estimates and in making decisions based on such estimates. This paper
focuses on the second of the two difficulties. A risk decision is defined as
a stochastic optimization problem where the parameters and the functional
forms required to determine the optimal decision are known. And an uncertain
decision is defined as a stochastic optimization problem where at least one
parameter or functional form must be estimated. The difficulties in making risk
and uncertain decisions will be defined in terms of computational complexity. In
this alternative interpretation, the relevant question becomes: 'Is the computa-
tional complexity of an uncertain decision greater than or equal to the computa-
tional complexity of the corresponding risk decision?'
To address this question, a simple monopoly model is developed in section 2.
The use of a monopoly model permits the separation of the computational
complexity issues of production decisions from the computational complexity
issues of business strategy in the game theory models of oligopoly. In the model
the parameter of the production function is known in the case of risk and
unknown in the case of uncertainty. For the case of known and unknown
parameters, Knight's three types of probability will be defined in a manner
consistent with Bayesian statistics. This model will be formulated to demon-
strate that uncertain decisions are difficult to solve even in the simplest case of
uncertainty, a single unknown parameter which can be estimated using a conju-
gate Bayesian distribution.
To consider the computational complexity of the two optimization problems
requires a model of computational complexity. The Traub, Wasilkowski, and
Wo~niakowski (1988) real number computational model based on the concept
of information-based complexity is presented in section 3. This model provides
a framework to study both exact and approximate solutions of problems. The
difficulty in solving the risk and uncertainty monopoly problems is considered
in section 4. The monopoly problem with risk can be solved exactly in three
operations for a time horizon of arbitrary length whereas the same problem with
A.L. Norman and D.W. Shimer, Risk, uncertainty, and complexity 233
2. The model
In this section we shall construct a model to demonstrate that uncertain
decisions are more difficult to solve than risk decisions. As was previously
defined an uncertain decision is a stochastic optimization problem where at least
one parameter or functional form must be estimated. We wish to show that even
in the simplest case of uncertainty, a single unknown parameter, difficulties arise
in solving the uncertain decision. This is true even in the simplest case of
Bayesian estimation, a conjugate distribution.
In order to construct such a model a Bayesian interpretation of Knight's three
probability types is required. A good starting point for this task is a review of
Knight's original definitions:
(3) Estimates. 'The distinction here is that there is no valid basis of any kind for
classifying instances. This form of probability is involved in the greatest
234 A.L. Norman and D. W. Shimer, Risk, uncertainty, and complexity
where qt is the tth observation of net output, xt is the tth level of the production
process, fl is the unknown scalar parameter, and et is the tth unobserved
disturbance term which represents such factors as rejects or exogenous influen-
ces such as the weather. The et are i.i.d, normal with mean zero and known
variance one. The use of a normal disturbance, which implies instances of
negative net output, is needed in order to obtain a conjugate distribution. We
wish to show that our results hold even in the simplest case of Bayesian statistics
that of a conjugate distribution. Instances of negative net output can occur with
a crop failure in agriculture or plant failure in other processes where the output
in one period is a feedstock in the next period's production.
Given a normal prior on fl at time t = l, the prior information on fl at time t is
a normal distribution N(mt, ht), where mt is the mean and ht is the precision. The
mean and precision have the following difference relationships:
mt = ( m t - l h t - 1 + q t - l x t - 1 ) / h t . (3)
In case 1 the agent has either been given precise knowledge of fl or has
observed eq. (1) a countable number of times such that his prior on fl has
asymptotically converged, using eqs. (2) and (3) above, to N(fl, ~ ). This inter-
pretation is consistent with Knight as he gives rolling a perfect die as an example
of a priori probability in Chapter VI and mentions 'or from statistics of past
experience' in Chapter VII. In Chapter VIII, Knight calls case 1 'objective
probability' or 'risk'. The case 2 definition is a simplification of Knight's
definition of case 2.
The use of a subjective prior for case 3 is consistent with Knight's alternative
definition of case 3 as 'subjective probability' or 'uncertainty' given in Chapter
VIII. While the subjective prior could have been modeled as complete ignorance
using an improper uniform prior, with prior belief p(fl) oc constant, the authors
choose to define this case as an agent who 'knows little' in order to obtain
a well-defined decision problem. For a discussion of alternative representations
of ignorance see Zellner (1971).
In defining case 3 Knight uses the enigmatic phrase 'there is no valid basis o f
any kind f o r classifying instances'. Bewley (1986) has proposed an interpretation
of Knight based on modifying Savage's (1954) axioms. In this paper the authors
prefer to interpret Knight in a manner consistent with standard Bayesian
methodology. The Bayesian interpretation of the three types represents a con-
tinuum from little information to asymptotically converged estimates and pro-
vides a,a integrated framework for objective and subjective probability.
In dealing with these three probability situations Knight asserts two types of
difficulties created by uncertainty. The first difficulty is 'the formation of the
estimate'. In this paper the authors have deliberately chosen a conjugate distri-
bution to make the difficulty of forming an estimate as simple as possible. In the
case of uncertainty, the decision maker will have the difficulty of generating
a locally uniform prior in order to formulate a well-defined decision problem.
The second difficulty is 'the estimation of its value'. This difficulty can be
interpreted as being the difficulty of making decisions based on estimates of the
parameters. In his book, Knight gives as an example of the second type of
difficulty a businessman's decision whether or not to increase the capacity of his
firm.
In order to analyze the second difficulty, a decision model must be created.
Assume the decision maker is a monopolist who faces an inverse demand
function
p, = a - dq,, (4)
J r = sup H r ( x r ) ,
xT
where
IIr(x r) = E v'-lp,(x~)q,(x,)lqt-l,x '-1 , (6)
t
3. Information-based complexity
If we assume the standard definition of rational agents, namely that the agent
can exactly solve an arbitrary optimization problem instantaneously without
cost, then there is n6 difference between risk and uncertainty. While Knight
A.L. Normanand D.W. Shimer, Risk, uncertainty, and complexity 237
assumed that the behavior of economic agents is rational in the sense of being
purposeful, he also assumed that human intelligence is finite and emphasized the
difficulties of solving uncertainty problems. In this paper we shall define finite
intelligence in terms of computational complexity. The first task in analyzing the
computational complexity of risk and uncertainty is to define the appropriate
model of computation.
Two major branches of computational complexity are information-based
complexity and combinatorial complexity. The former deals with the difficulty of
approximate solutions of problems where information is partial, noisy, and
costly. The latter deals with problems that can be solved exactly in a finite
number of computations and where information is complete, exact, and costless.
For an example of combinatorial complexity analysis in economic theory see
Norman (1987). As will become apparent later, uncertainty problems are gener-
ally best analyzed using information-based complexity, whereas many risk
problems can be analyzed using combinatorial complexity. In this paper we
need a computational model which can be used to study both information-based
and combinatorial complexity. For this purpose the appropriate model is the
information-based computational model of Traub, Wasilkowski, and
Wo~niakowski (1988). In this model of computation all arithmetic and combina-
torial operations are assumed to be done with infinite precision.
Let the problem sets for OPT1 and OPT2-3 be designated Fl,r and F23,r,
respectively. These problem sets are
S t ( f ) = sup I I r ( x r ) . (8)
xT
c~: N ( f ) - ~ ~ , (10)
on which problem element we choose. In the worst-case setting the error and
cost of approximation are defined over all problem elements as follows:
T
177. = sup ~ rt-l(aflxt - dfl2x 2 - d). (14)
Xt t=l
x,* = a / 2 d B . (15)
Proof The solution (15) needs to be computed only once and requires two
multiplications and one division. Thus there exists a 0-approximate algorithm
whose Cost(T) = 3. Any 0-approximate algorithm will have a Cost(T) > 3. By
Definitions 1 and 2, the 0-computational complexity of OPT1 is T . |
am2
x * - 2d(m z + h 2 1 ) , (16)
a2m 2
Jl(ql) = 4d(m2 + h2 l) d. (17)
a2((mihl + q x x i ) / h x ) 2
Ji(ql) -- 4 d ( [ ( m i h l + q i x l ) / ( h l + x2)] 2 + (hi + x2) - i ) - d, (19)
E[-QI(q')I - d, (20)
LQ2(ql)J
where Ql(ql) and Q2(ql) are quadratic forms in the normal variable ql. As
pointed out by Aoki (1967), this expectation cannot be carried out explicitly to
give an analytic closed expression. For a discussion of the integration issue see
Moses (1971).
Now consider the determination of x*. To simplify the discussion of the
determination of x* consider
In this form the optimization problem is a search to find x* where the adaptive
information operator N ( f ) returns Fll(xi) for each xl selected in the search.
Since (20) cannot be analytically integrated, any finite integration scheme with
finite cost C will have a radius of approximate information, r(Np) > 0. This
implies:
where in this case [3t is an i.i.d, random coefficient. The case of risk for this
problem is where the first two moments of fit are known. In this case the optimal
decision is
complexity of this is transfinite. The point is one does not need to assume that
the distribution of fit is unknown to obtain transfinite 0-computational com-
plexity.
In the examples in this section we have specified uncertainty examples which
have a single unknown parameter for a specific purpose. We want to show that
even in the simplest case of conjugate Bayesian distributions uncertainty leads
to intractable profit maximization problems. But to what extent are these results
generalizable? The sample problems are a simple examples of linear quadratic
control problems. For variations of linear quadratic control problems see Chow
(1975). The 0-computational complexity of many linear quadratic control prob-
lems with known coefficients is known to be low-order polynomial [Norman
and Jung (1977)]. The 0-computational complexity of linear quadratic control
problems with random coefficients models with known first two moments is also
known to be low-order polynomial, whereas the computational complexity of
linear quadratic models with unknown coefficients is known to be transfinite
[Norman (1994)]. Thus the results are immediately generalizable to the case of
linear quadratic control models with the uncertain decision specified as one or
more unknown coefficients.
5. Nonlinear models
Let us consider a very simple nonlinear generalization of (1):
For the purpose of discussion, we have assumed that the nonlinear production
function is linear in fl so that the updating relationships for fl in the case of
uncertainty become
h, = hi_ x + 9 ( x t - 1) 2 , (24)
The complexity of the risk case depends on the difficulty in solving this
nonlinear version of (14): As was the case with the linear model, because there
are no dynamics, solution of the T-time-period problem can be partitioned into
T single-period problems with the same solutions:
H t = sup Z t , (27)
Xt
244 A.L. Norman and D. W. Shimer, Risk, uncertainty, and complexity
where
Z, = r ' - l (aflg(x,) - d~2g(x,) 2 - d) . (28)
6. Concluding remarks
In Part Three of Risk, Uncertainty and Profit Knight details methods which
economic agents use to reduce uncertainty. It is important to note that Knight
A.L. Norman and D. 14/. Shimer, Risk, uncertainty, and complexity 245
uses the word uncertainty to cover the three probability cases previously
discussed. Case 1 is risk which is defined as measurable uncertainty, and case 3 is
uncertainty or, more precisely, unmeasurable or true uncertainty. Thus, the
word uncertainty has two meanings. When Knight discusses ways to reduce
uncertainty, he is using uncertainty in the broad sense to cover the three cases.
Knight defines six methods to reduce uncertainty: 'The two fundamental
methods of dealing with uncertainty, based respectively upon reduction by
grouping and upon selection of men to 'bear' it, are 'consolidation' and 'special-
ization', respectively'. The other methods for reducing uncertainty are 'control of
the future', 'increased power of prediction', 'diffusion', and 'the possibility of
directing industrial activity more or less along lines in which a minimal amount
of uncertainty is involved'. In this section we shall focus of presenting an
interpretation of consolidation and specialization which reconciles Knight with
the Bayesians.
First, let us consider how to reconcile Knight with the Bayesians concerning
specialization. Knight problems for which only subjective probabilities are
available, case 3, are much more difficult to solve than those involving objective
probabilities, cases 1 and 2. Human ability to solve such problems varies and the
market selects those people with superior problem-solving ability to become
successful entrepreneurs. In the Bayesian optimization example presented in the
previous sections, the complexity is transfinite for both cases 2 and 3. Also,
rational behavior assumes economic agents exactly optimize, which precludes
differences in ability. This apparent conflict can be resolved by making the
comparison on the basis of bounded rationality.
From the perspective of modern terminology Knight was a proponent of
bounded rationality [Simon (1975)]; he was concerned with the problem of how
finite intelligence was able to deal with a world of infinite variety. Knight states:
'The ordinary decisions of life are made on the basis of'estimates' of a crude and
superficial character. In general the future situation in relation to which we act
depends upon the behavior of an infinitely large number of objects, and is
influenced by so many factors that no real effort is made to take account of them
all, much less estimate and summate their separate significances.' In discussing
a manufacturer considering an investment decision, he states: 'He 'figures' more
or less on the proposition, taking account as well as possible of the various
factors more or less susceptible of measurement, but the final result is an
'estimate' of the probable outcome of any proposed course of action.'
Similarly, long before formal complexity analysis Bayesians have been aware
that Bayesian optimization problems are hard to solve. One bounded rational
approach which has now been studied for over two decades is how to incorpo-
rate simplifying assumptions to obtain a more computationally tractable prob-
lem. One approach is to separate the estimation problem from the optimization
problem. This approach is known as passive learning and in the case of the
example reduces the complexity from transfinite to linear; see Norman (1994).
246 A .L. Norman and D.W. Shimer, Risk, uncertainty, and complexity
life insurance has the most precise estimates, whereas sickness and accident
insurance the least. Our Bayesian probability model is consistent with Knight if
we equate accurate classification as the formulation of a nonexperimental model
without specification error and also equate instances with observations.
Knight points out insurance is unavailable for many business decisions: 'The
typical uninsurable (because unmeasurable and this because unclassifiable)
business risk relates to the exercise of judgment in making of decisions by the
business man.' In such situations Knight states: 'The possibility of thus reducing
uncertainty by transforming it into a measurable risk through grouping consti-
tutes a strong incentive to extend the scale of operations of a business establish-
ment.' We shall argue that the incentive for such consolidation is improved
expected performance.
Consider the example monopoly model presented in the previous sections. To
discuss the benefits of Knightian consolidation assume there are ten such firms
each with the same unknown value of fl and each with a monopoly in its local
market. Since each has a monopoly in its local market, the incentive to combine
is not to form a cartel. To understand the incentive from the perspective of
estimation and control theory assume a case 3 probability model, where each
firm starts with a locally uniform prior on the distribution of the unknown
parameter. Now, in the spirit of bounded rationality, assume each firm uses
a 'heuristic' certainty equivalence strategy. This means that each period each
firm uses the prior mean offl in (15) to compute an approximation of the optimal
decision. After each decision has been executed and observed each firm updates
its estimate of fl using (2) and (3). From the perspective of ten firms operating
independently, each firm only has its own observations to update its estimate of
fl because such information would be proprietary. From the perspective of
a consolidated firm the corporate headquarters would collect the observations
each period from the ten plants and combine them using (2) and (3) to obtain
a better estimate. Now let us assume a weak sufficiency condition, that the
production of the ten plants is positive each period. As the precision of the
combined estimate is greater than the precision of the separate estimates,
expected performance would be better, because the combination would obtain
a better estimate of the true, but unknown control law for each plant.
It should be noted that the benefits of this type of Knightian consolidation
could have been obtained by a research and development consortium without
the actual consolidation of the firms. This type of consolidation is explored more
fully in Norman (1993).
References
Aoki, Masanao, 1967, Optimization of stochastic systems (Academic Press, New York, NY).
Bewley, Truman F., 1986, Knightian decision theory: Part 1, Cowles Foundation discussion paper
no. 807 (Cowles Foundation, New Haven, CT ).
Blackwell, David, 1965, Discounted dynamic programming, Annals of Mathematical Statistics 36,
226-235.
Blackwell, David, D. Freedman, and M. Orkin, 1974, The optimal reward operator in dynamic
programming, Annals of Probability 2, 926-941.
Chow, Chee-Seng and John N. Tsitsiklis, 1989,The complexity of dynamic programming, Journal of
Complexity 5, 466-488.
Chow, Gregory C., 1975, Analysis and control of dynamic economic systems (Wiley, New York,
NY).
Cyert, Richard M. and Morris H. DeGroot, 1987, Bayesian analysis and uncertainty in economic
theory (Rowman & Littlefield, Totowa).
Easley, David and N.M. Keifer, 1988, Controlling a stochastic process with unknown parameters,
Econometrica 56, 1045-1064.
Knight, Frank H., 1971 (original 1921), Risk uncertainty and profit (University of Chicago Press,
Chicago, IL).
MacRae, E.C., 1972, Linear decision with experimentation, Annals of Economic and Social
Measurement 4, 437-447.
A.L. Norman and D.W. Shimer, Risk, uncertainty, and complexity 249
Moses, Joel, 1971, Symbolic integration: The stormy decade, Communications of the ACM 14,
548-560.
Norman, Alfred L., 1987, A theory of monetary exchange, Review of Economic Studies 54, 499-517.
Norman, Alfred L., 1993, Informational society: An economic theory of discovery, invention and
innovation (Kluwer Academic Press, Boston, MA).
Norman, Alfred L., 1994, On the complexity of linear quadratic control, European Journal of
Operations Research, forthcoming.
Norman, Alfred L. and Woo S. Jung, 1977, Linear quadratic control theory for models with long
lags, Econometrica 45, 905-917.
Ortega, J.M. and W.C. Rheinboldt, 1970, Iterative solution of nonlinear equations in several
variables (Academic Press, New York, NY).
Savage, Leonard, 1954, The foundations of statistics (Wiley, New York, NY).
Simon, H.A., 1957, Models of man (Wiley, New York, NY).
Shreve, Stephen E. and Dimitri P. Bertsekas, 1978, Alternative theoretical frameworks for finite
horizon discrete-time stochastic optimal control, SIAM Journal of Control and Optimization 16,
953-978.
Thiemann, J.G.F., 1985, Analytic spaces and dynamic programming: A measure-theoretic approach
(Centrum voor Wiskunde en Informatica, Amsterdam).
Traub, J.F., G.W. Wasilkowski, and H. Wo~niakowski, 1983, Information, uncertainty, complexity
(Addison-Wesley, Reading, MA).
Traub, J.F., G.W. Wasilkowski, and H. Wo~niakowski, 1988, Information-based complexity (Aca-
demic Press, Boston, MA).
Zellner, Arnold, 1971, An introduction to Bayesian inference in econometrics (Academic Press,
New York, NY).