Solving problems by searching

This chapter has introduced methods that an agent can use to select actions
in environments
that are deterministic, observable, static, and completely known. In such
cases, the agent can
construct sequences of actions that achieve its goals; this process is called
search.
Before an agent can start searching for solutions, a goal must be identified
and a welldefined
problem must be formulated.
A problem consists of five parts: the initial state, a set of actions, a
transition model
describing the results of those actions, a goal test function, and a path cost
function.
The environment of the problem is represented by a state space. A path
through the
state space from the initial state to a goal state is a solution.
Search algorithms treat states and actions as atomic: they do not consider
any internal
structure they might possess.
A general TREE-SEARCH algorithm considers all possible paths to find a
solution,
whereas a GRAPH-SEARCH algorithm avoids consideration of redundant paths.
Search algorithms are judged on the basis of completeness, optimality,
time complexity,
and space complexity. Complexity depends on b, the branching factor in
the state
space, and d, the depth of the shallowest solution.
Uninformed search methods have access only to the problem definition.
The basic
algorithms are as follows:
Breadth-first search expands the shallowest nodes first; it is complete,
optimal
for unit step costs, but has exponential space complexity.
Uniform-cost search expands the node with lowest path cost, g(n), and is
optimal
for general step costs.
Depth-first search expands the deepest unexpanded node first. It is
neither complete
nor optimal, but has linear space complexity. Depth-limited search adds a
depth bound.
Iterative deepening search calls depth-first search with increasing depth
limits
until a goal is found. It is complete, optimal for unit step costs, has time
complexity
comparable to breadth-first search, and has linear space complexity.
Bidirectional search can enormously reduce time complexity, but it is not
always
applicable and may require too much space.
 Informed search methods may have access to a heuristic function h(n)
that estimates
the cost of a solution from n.
The generic best-first search algorithm selects a node for expansion
according to
an evaluation function.
Greedy best-first search expands nodes with minimal h(n). It is not
optimal but
is often efficient.

4 beyond classical search

This chapter has examined search algorithms for problems beyond the
classical case of
finding the shortest path to a goal in an observable, deterministic, discrete
environment.
Local search methods such as hill climbing operate on complete-state
formulations,
keeping only a small number of nodes in memory. Several stochastic
algorithms have
been developed, including simulated annealing, which returns optimal
solutions when
given an appropriate cooling schedule.
Many local search methods apply also to problems in continuous spaces.
Linear programming
and convex optimization problems obey certain restrictions on the shape
of the state space and the nature of the objective function, and admit
polynomial-time
algorithms that are often extremely efficient in practice.
A genetic algorithm is a stochastic hill-climbing search in which a large
population of
states is maintained. New states are generated by mutation and by
crossover, which
combines pairs of states from the population.
154 Chapter 4. Beyond Classical Search
In nondeterministic environments, agents can apply ANDOR search to
generate contingent
plans that reach the goal regardless of which outcomes occur during
execution.
When the environment is partially observable, the belief state represents
the set of
possible states that the agent might be in.
Standard search algorithms can be applied directly to belief-state space to
solve sensorless
problems, and belief-state ANDOR search can solve general partially
observable
problems. Incremental algorithms that construct solutions state-by-state
within a belief
state are often more efficient.
Exploration problems arise when the agent has no idea about the states
and actions of
 its environment. For safely explorable environments, online search agents
can build a
map and find a goal if one exists. Updating heuristic estimates from
experience provides
an effective method to escape from local minima.

5 adversarial search
We have looked at a variety of games to understand what optimal play
means and to understand
how to play well in practice. The most important ideas are as follows:
A game can be defined by the initial state (how the board is set up), the
legal actions
in each state, the result of each action, a terminal test (which says when
the game is
over), and a utility function that applies to terminal states.
In two-player zero-sum games with perfect information, the minimax
algorithm can
select optimal moves by a depth-first enumeration of the game tree.
The alphabeta search algorithm computes the same optimal move as
minimax, but
achieves much greater efficiency by eliminating subtrees that are provably
irrelevant.
Usually, it is not feasible to consider the whole game tree (even with alpha
beta), so we
190 Chapter 5. Adversarial Search
need to cut the search off at some point and apply a heuristic evaluation
function that
estimates the utility of a state.
Many game programs precompute tables of best moves in the opening and
endgame so
that they can look up a move rather than search.
Games of chance can be handled by an extension to the minimax algorithm
that evaluates
a chance node by taking the average utility of all its children, weighted by
the
probability of each child.
Optimal play in games of imperfect information, such as Kriegspiel and
bridge, requires
reasoning about the current and future belief states of each player. A simple
approximation can be obtained by averaging the value of an action over each
possible
configuration of missing information.
Programs have bested even champion human players at games such as
chess, checkers,
and Othello. Humans retain the edge in several games of imperfect
information, such
as poker, bridge, and Kriegspiel, and in games with very large branching
factors and
 little good heuristic knowledge, such as Go.
6 Constraint satisfaction problem
Constraint satisfaction problems (CSPs) represent a state with a set of
variable/value
pairs and represent the conditions for a solution by a set of constraints on the
variables.
Many important real-world problems can be described as CSPs.
A number of inference techniques use the constraints to infer which
variable/value pairs
are consistent and which are not. These include node, arc, path, and k-
consistency.
Backtracking search, a form of depth-first search, is commonly used for
solving CSPs.
Inference can be interwoven with search.
The minimum-remaining-values and degree heuristics are domain-
independent methods
for deciding which variable to choose next in a backtracking search. The
leastconstraining-
value heuristic helps in deciding which value to try first for a given
variable. Backtracking occurs when no legal assignment can be found for a
variable.
Conflict-directed backjumping backtracks directly to the source of the
problem.
Local search using the min-conflicts heuristic has also been applied to
constraint satisfaction
problems with great success.
The complexity of solving a CSP is strongly related to the structure of its
constraint
graph. Tree-structured problems can be solved in linear time. Cutset
conditioning can
reduce a general CSP to a tree-structured one and is quite efficient if a small
cutset can
be found. Tree decomposition techniques transform the CSP into a tree of
subproblems
and are efficient if the tree width of the constraint graph is small.

Chapter 3
7 logical agents
We have introduced knowledge-based agents and have shown how to define
a logic with
which such agents can reason about the world. The main points are as
follows:
Intelligent agents need knowledge about the world in order to reach good
decisions.
Knowledge is contained in agents in the form of sentences in a
knowledge representation
language that are stored in a knowledge base.
 A knowledge-based agent is composed of a knowledge base and an
inference mechanism.
It operates by storing sentences about the world in its knowledge base, using
the
inference mechanism to infer new sentences, and using these sentences to
decide what
action to take.
A representation language is defined by its syntax, which specifies the
structure of
sentences, and its semantics, which defines the truth of each sentence in
each possible
world or model.
The relationship of entailment between sentences is crucial to our
understanding of
reasoning. A sentence entails another sentence if is true in all worlds
where
is true. Equivalent definitions include the validity of the sentence
and the
unsatisfiability of the sentence .
Inference is the process of deriving new sentences from old ones. Sound
inference algorithms
derive only sentences that are entailed; complete algorithms derive all
sentences
that are entailed.
Propositional logic is a simple language consisting of proposition
symbols and logical
connectives. It can handle propositions that are known true, known false, or
completely
unknown.
The set of possible models, given a fixed propositional vocabulary, is finite,
so entailment
can be checked by enumerating models. Efficient model-checking inference
algorithms for propositional logic include backtracking and local search
methods and
can often solve large problems quickly.
Bibliographical and Historical Notes 275
Inference rules are patterns of sound inference that can be used to find
proofs. The
resolution rule yields a complete inference algorithm for knowledge bases
that are
expressed in conjunctive normal form. Forward chaining and backward
chaining
are very natural reasoning algorithms for knowledge bases in Horn form.
Local search methods such as WALKSAT can be used to find solutions. Such
algorithms
are sound but not complete.
Logical state estimation involves maintaining a logical sentence that
describes the set
of possible states consistent with the observation history. Each update step
requires
 inference using the transition model of the environment, which is built from
successorstate
axioms that specify how each fluent changes.
Decisions within a logical agent can be made by SAT solving: finding
possible models
specifying future action sequences that reach the goal. This approach works
only for
fully observable or sensorless environments.
Propositional logic does not scale to environments of unbounded size
because it lacks
the expressive power to deal concisely with time, space, and universal
patterns of relationships
among objects

First order logic

This chapter has introduced first-order logic, a representation language that is far more powerful
than propositional logic. The important points are as follows:
Knowledge representation languages should be declarative, compositional, expressive,
context independent, and unambiguous.
Logics differ in their ontological commitments and epistemological commitments.
While propositional logic commits only to the existence of facts, first-order logic commits
to the existence of objects and relations and thereby gains expressive power.
The syntax of first-order logic builds on that of propositional logic. It adds terms to
represent objects, and has universal and existential quantifiers to construct assertions
about all or some of the possible values of the quantified variables.
A possible world, or model, for first-order logic includes a set of objects and an interpretation
that maps constant symbols to objects, predicate symbols to relations among
objects, and function symbols to functions on objects.
An atomic sentence is true just when the relation named by the predicate holds between
the objects named by the terms. Extended interpretations, which map quantifier variables
to objects in the model, define the truth of quantified sentences.
Developing a knowledge base in first-order logic requires a careful process of analyzing
the domain, choosing a vocabulary, and encoding the axioms required to support the
desired inferences.

Inference in first-order logic

algorithms
for doing it.
A first approach uses inference rules (universal instantiation and
existential instantiation)
to propositionalize the inference problem. Typically, this approach is slow,
unless the domain is small.
The use of unification to identify appropriate substitutions for variables
eliminates the
instantiation step in first-order proofs, making the process more efficient in
many cases.
A lifted version of Modus Ponens uses unification to provide a natural and
powerful
 inference rule, generalized Modus Ponens. The forward-chaining and
backwardchaining
algorithms apply this rule to sets of definite clauses.
Generalized Modus Ponens is complete for definite clauses, although the
entailment
problem is semidecidable. For Datalog knowledge bases consisting of
function-free
definite clauses, entailment is decidable.
Forward chaining is used in deductive databases, where it can be
combined with relational
database operations. It is also used in production systems, which perform
efficient updates with very large rule sets. Forward chaining is complete for
Datalog
and runs in polynomial time.
Backward chaining is used in logic programming systems, which employ
sophisticated
compiler technology to provide very fast inference. Backward chaining suffers
from redundant inferences and infinite loops; these can be alleviated by
memoization.
Prolog, unlike first-order logic, uses a closed world with the unique names
assumption
and negation as failure. These make Prolog a more practical programming
language,
but bring it further from pure logic.
The generalized resolution inference rule provides a complete proof
system for firstorder
logic, using knowledge bases in conjunctive normal form.
Several strategies exist for reducing the search space of a resolution
system without
compromising completeness. One of the most important issues is dealing
with equality;
we showed how demodulation and paramodulation can be used.
Efficient resolution-based theorem provers have been used to prove
interesting mathematical
theorems and to verify and synthesize software and hardware.

Classical Planning
In this chapter, we defined the problem of planning in deterministic, fully observable, static
environments. We described the PDDL representation for planning problems and several
algorithmic approaches for solving them. The points to remember:
Planning systems are problem-solving algorithms that operate on explicit propositional
or relational representations of states and actions. These representations make possible
the derivation of effective heuristics and the development of powerful and flexible
algorithms for solving problems.
PDDL, the Planning Domain Definition Language, describes the initial and goal states
as conjunctions of literals, and actions in terms of their preconditions and effects.
State-space search can operate in the forward direction (progression) or the backward
direction (regression). Effective heuristics can be derived by subgoal independence
assumptions and by various relaxations of the planning problem.
A planning graph can be constructed incrementally, starting from the initial state. Each
 layer contains a superset of all the literals or actions that could occur at that time step
and encodes mutual exclusion (mutex) relations among literals or actions that cannot cooccur.
Planning graphs yield useful heuristics for state-space and partial-order planners
and can be used directly in the GRAPHPLAN algorithm.
Other approaches include first-order deduction over situation calculus axioms; encoding
a planning problem as a Boolean satisfiability problem or as a constraint satisfaction
problem; and explicitly searching through the space of partially ordered plans.
Each of the major approaches to planning has its adherents, and there is as yet no consensus
on which is best. Competition and cross-fertilization among the approaches have
resulted in significant gains in efficiency for planning systems.

Planning and Acting in the real world

The main points:
Many actions consume resources, such as money, gas, or raw
materials. It is convenient
to treat these resources as numeric measures in a pool rather than try to
reason about,
say, each individual coin and bill in the world. Actions can generate and
consume
resources, and it is usually cheap and effective to check partial plans for
satisfaction of
resource constraints before attempting further refinements.
Time is one of the most important resources. It can be handled by
specialized scheduling
algorithms, or scheduling can be integrated with planning.
Bibliographical and Historical Notes 431
Hierarchical task network (HTN) planning allows the agent to take
advice from the
domain designer in the form of high-level actions (HLAs) that can be
implemented in
various ways by lower-level action sequences. The effects of HLAs can be
defined with
angelic semantics, allowing provably correct high-level plans to be derived
without
consideration of lower-level implementations. HTN methods can create the
very large
plans required by many real-world applications.
Standard planning algorithms assume complete and correct information
and deterministic,
fully observable environments. Many domains violate this assumption.
Contingent plans allow the agent to sense the world during execution to
decide what
branch of the plan to follow. In some cases, sensorless or conformant
planning can be
used to construct a plan that works without the need for perception. Both
conformant
and contingent plans can be constructed by search in the space of belief
states. Efficient
representation or computation of belief states is a key problem.
 An online planning agent uses execution monitoring and splices in
repairs as needed
to recover from unexpected situations, which can be due to nondeterministic
actions,
exogenous events, or incorrect models of the environment.
Multiagent planning is necessary when there are other agents in the
environment with
which to cooperate or compete. Joint plans can be constructed, but must be
augmented
with some form of coordination if two agents are to agree on which joint plan
to execute.
This chapter extends classic planning to cover nondeterministic
environments (where
outcomes of actions are uncertain), but it is not the last word on planning.
Chapter 17
describes techniques for stochastic environments (in which outcomes of
actions have
probabilities associated with them): Markov decision processes, partially
observable
Markov decision processes, and game theory. In Chapter 21 we show that
reinforcement
learning allows an agent to learn how to behave from past successes
and failures.

Knowledge representation
By delving into the details of how one represents a variety of knowledge, we
hope we have
given the reader a sense of how real knowledge bases are constructed and a
feeling for the
interesting philosophical issues that arise. The major points are as follows:
Large-scale knowledge representation requires a general-purpose
ontology to organize
and tie together the various specific domains of knowledge.
A general-purpose ontology needs to cover a wide variety of knowledge and
should be
capable, in principle, of handling any domain.
Building a large, general-purpose ontology is a significant challenge that
has yet to be
fully realized, although current frameworks seem to be quite robust.
 We presented an upper ontology based on categories and the event
calculus. We
covered categories, subcategories, parts, structured objects, measurements,
substances,
events, time and space, change, and beliefs.
468 Chapter 12. Knowledge Representation
Natural kinds cannot be defined completely in logic, but properties of
natural kinds can
be represented.
Actions, events, and time can be represented either in situation calculus or
in more
expressive representations such as event calculus. Such representations
enable an agent
to construct plans by logical inference.
We presented a detailed analysis of the Internet shopping domain,
exercising the general
ontology and showing how the domain knowledge can be used by a shopping
agent.
Special-purpose representation systems, such as semantic networks and
description
logics, have been devised to help in organizing a hierarchy of categories.
Inheritance
is an important form of inference, allowing the properties of objects to be
deduced from
their membership in categories.
The closed-world assumption, as implemented in logic programs, provides a
simple
way to avoid having to specify lots of negative information. It is best
interpreted as a
default that can be overridden by additional information.
Nonmonotonic logics, such as circumscription and default logic, are
intended to capture
default reasoning in general.
Truth maintenance systems handle knowledge updates and revisions
efficiently.