Vous êtes sur la page 1sur 41

# Informed strategies

Lecture Plan

Search Methods GEIST Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Informed search strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

1 / 28

Informed strategies

## Blind vs. informed search strategies

Search Methods GEIST Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Blind search methods You already know them: BFS, DFS, UCS et al. They dont analyse the nodes in the fringe; they simply pick the rst one (going down or sideways). Informed search methods They try to guess which node in the fringe is most promising.... ...by using an evaluation function, also known as the heuristic. The heuristic is based on additional knowledge about the search space.

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

2 / 28

Informed strategies

## Best-rst search Search Methods GEIST Informed strategies

Best-rst search algorithm family General approach: picks node for expansion based on evaluation function, f (n). We dont know goal is best; if we knew, it wouldnt be a search at all. Heuristic function h(n) is the key component of f (n). Heuristic functions h(n) = estimated cost of the cheapest path from node n to the goal node. If n is the goal node, h(n) = 0.

Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

3 / 28

Informed strategies

## Best-rst search Search Methods GEIST

Heuristic functions
Choosing an evaluation function If we know the locations of cities, we know the straight-line distances between them. hSLD straight-line distance from n to Bucharest.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

71

75

Zerind

151

87
Iasi

140
Sibiu

92 99
Fagaras Vaslui Rimnicu Vilcea

118 80
Timisoara

111 70

Lugoj

97 146

Pitesti

211

142 98

101 138

85

Hirsova

Urziceni

75
Dobreta

86
Bucharest

120
Craiova

90
Giurgiu Eforie

Straightline distance to Bucharest Arad 366 Bucharest 0 Craiova 160 Dobreta 242 Eforie 161 Fagaras 178 Giurgiu 77 Hirsova 151 Iasi 226 Lugoj 244 Mehadia 241 Neamt 234 Oradea 380 Pitesti 98 Rimnicu Vilcea 193 Sibiu 253 Timisoara 329 Urziceni 80 Vaslui 199 Zerind 374

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

4 / 28

Informed strategies

## Best-rst search Search Methods GEIST

Heuristic functions
Admissible heuristic Never overestimates the cost to reach the goal, e.g. h(n) h (n) (where h (n) is the real cost to get from n to goal). Is by nature optimistic. Monotonic (consistent) heuristic For every node n and every successor n of n (generated by any action a), estimated cost of reaching the goal from n is no greater than the stop cost of getting to n plus estimated cost of reaching the goal from n : h(n) c(n, a, n ) + h(n ). This is a form of general triangle inequality: each side of a triangle cannot be longer than sum of two other sides.
GEIST (KA AGH) Search Methods

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

1 czerwca 2010

5 / 28

Informed strategies

Greedy search Search Methods GEIST Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Greedy search
Greedy best-rst search Only takes into account the heuristic, e.g. f (n) = h(n).

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

6 / 28

Informed strategies

Greedy search Search Methods GEIST Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Greedy search
Greedy best-rst search Only takes into account the heuristic, e.g. f (n) = h(n).

Sibiu 253

Timisoara 329

Zerind 374

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

6 / 28

Informed strategies

Greedy search Search Methods GEIST Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Greedy search
Greedy best-rst search Only takes into account the heuristic, e.g. f (n) = h(n).

Sibiu

Timisoara 329

Zerind 374

Fagaras 176

Rimnicu Vilcea

193

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

6 / 28

Informed strategies

Greedy search Search Methods GEIST Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Greedy search
Greedy best-rst search Only takes into account the heuristic, e.g. f (n) = h(n).

Sibiu

Timisoara 329

Zerind 374

Fagaras

Rimnicu Vilcea

193

Sibiu 253

Bucharest 0

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

6 / 28

Informed strategies

Greedy search Search Methods GEIST Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Greedy search

Greedy search properties Completeness: No can get stuck in loops. Time complexity: O(b m ), but depends on quality of heuristic. Space complexity: O(b m ) keeps all nodes in memory. Optimality: No.

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

7 / 28

Informed strategies

## A* search Search Methods GEIST

A* search
A* search algorithm Idea: avoid expanding paths that are already expensive. f (n) = g (n) + h(n), where:
g (n) cost so far to reach n, h(n) estimated cost from n to goal (heuristic), therefore f (n) estimated total cost of path through n to goal.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

8 / 28

Informed strategies

## A* search Search Methods GEIST

A* search
A* search algorithm Idea: avoid expanding paths that are already expensive. f (n) = g (n) + h(n), where:
g (n) cost so far to reach n, h(n) estimated cost from n to goal (heuristic), therefore f (n) estimated total cost of path through n to goal.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Sibiu 393=140+253

Timisoara 447=118+329

Zerind 449=75+374

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

8 / 28

Informed strategies

## A* search Search Methods GEIST

A* search
A* search algorithm Idea: avoid expanding paths that are already expensive. f (n) = g (n) + h(n), where:
g (n) cost so far to reach n, h(n) estimated cost from n to goal (heuristic), therefore f (n) estimated total cost of path through n to goal.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Sibiu

Timisoara 447=118+329

Zerind 449=75+374

Fagaras

Rimnicu Vilcea

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

8 / 28

Informed strategies

## A* search Search Methods GEIST

A* search
A* search algorithm Idea: avoid expanding paths that are already expensive. f (n) = g (n) + h(n), where:
g (n) cost so far to reach n, h(n) estimated cost from n to goal (heuristic), therefore f (n) estimated total cost of path through n to goal.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Sibiu

Timisoara 447=118+329

Zerind 449=75+374

Fagaras

Rimnicu Vilcea

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

8 / 28

Informed strategies

## A* search Search Methods GEIST

A* search
A* search algorithm Idea: avoid expanding paths that are already expensive. f (n) = g (n) + h(n), where:
g (n) cost so far to reach n, h(n) estimated cost from n to goal (heuristic), therefore f (n) estimated total cost of path through n to goal.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Sibiu

Timisoara 447=118+329

Zerind 449=75+374

Fagaras

Rimnicu Vilcea

Bucharest 450=450+0

Craiova

Pitesti

Sibiu

591=338+253

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

8 / 28

Informed strategies

## A* search Search Methods GEIST

A* search
A* search algorithm Idea: avoid expanding paths that are already expensive. f (n) = g (n) + h(n), where:
g (n) cost so far to reach n, h(n) estimated cost from n to goal (heuristic), therefore f (n) estimated total cost of path through n to goal.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Sibiu

Timisoara 447=118+329

Zerind 449=75+374

Fagaras

Rimnicu Vilcea

Bucharest 450=450+0

## Craiova 526=366+160 Bucharest 418=418+0

Pitesti

Sibiu 553=300+253

591=338+253

Craiova

Rimnicu Vilcea

615=455+160 607=414+193

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

8 / 28

Informed strategies

## A* search Search Methods GEIST

A* optimality
Lets suppose that... ...a suboptimal goal G2 has been generated. Let n be an unexpanded node on a shortest path to the optimal goal G .
Start

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

G2

Proof f (G2 ) = g (G2 ), since h(G2 ) = 0 g (G2 ) > g (G ), since G2 is suboptimal g (G ) f (n), since h is admissible Since f (G2 ) > f (n), A* will never select G2 for expansion
GEIST (KA AGH) Search Methods 1 czerwca 2010 9 / 28

Informed strategies

## A* with consistent heuristic

Node expansion order If the heuristic is consistent, A* expands nodes in order of increasing f value. A* gradually adds f -contours of nodes; contour i has all nodes with f = fi , where fi < fi+1 .
O N I
380

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Z A S F
400

T L M D

R P H U
420

B E G

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

10 / 28

Informed strategies

## A* with inconsistent heuristic

If h is inconsistent: As we go along the path, f may sometimes increase. A* doesnt always expand nodes in order of increasing f value, as it may nd lower-cost paths to nodes it already expanded. A* should re-expand those nodes; however, the Graph-Search algorithm will not re-expand them.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

This example is due to Prof. Dana Nau. GEIST (KA AGH) Search Methods 1 czerwca 2010 11 / 28

Informed strategies

## A* with inconsistent heuristic

If h is inconsistent: As we go along the path, f may sometimes increase. A* doesnt always expand nodes in order of increasing f value, as it may nd lower-cost paths to nodes it already expanded. A* should re-expand those nodes; however, the Graph-Search algorithm will not re-expand them.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

This example is due to Prof. Dana Nau. GEIST (KA AGH) Search Methods 1 czerwca 2010 11 / 28

Informed strategies

## A* with inconsistent heuristic

If h is inconsistent: As we go along the path, f may sometimes increase. A* doesnt always expand nodes in order of increasing f value, as it may nd lower-cost paths to nodes it already expanded. A* should re-expand those nodes; however, the Graph-Search algorithm will not re-expand them.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

This example is due to Prof. Dana Nau. GEIST (KA AGH) Search Methods 1 czerwca 2010 11 / 28

Informed strategies

## A* with inconsistent heuristic

If h is inconsistent: As we go along the path, f may sometimes increase. A* doesnt always expand nodes in order of increasing f value, as it may nd lower-cost paths to nodes it already expanded. A* should re-expand those nodes; however, the Graph-Search algorithm will not re-expand them.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

This example is due to Prof. Dana Nau. GEIST (KA AGH) Search Methods 1 czerwca 2010 11 / 28

Informed strategies

## A* with inconsistent heuristic

If h is inconsistent: As we go along the path, f may sometimes increase. A* doesnt always expand nodes in order of increasing f value, as it may nd lower-cost paths to nodes it already expanded. A* should re-expand those nodes; however, the Graph-Search algorithm will not re-expand them.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

This example is due to Prof. Dana Nau. GEIST (KA AGH) Search Methods 1 czerwca 2010 11 / 28

Informed strategies

## A* with inconsistent heuristic

If h is inconsistent: As we go along the path, f may sometimes increase. A* doesnt always expand nodes in order of increasing f value, as it may nd lower-cost paths to nodes it already expanded. A* should re-expand those nodes; however, the Graph-Search algorithm will not re-expand them.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

This example is due to Prof. Dana Nau. GEIST (KA AGH) Search Methods 1 czerwca 2010 11 / 28

Informed strategies

A* search Search Methods GEIST Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Properties of A*

A* properties Completeness: Yes, unless there are innitely many nodes with f < f (G ). Time complexity: Exponential, depends on mean relative error of heuristic. Space complexity: Keeps all nodes in memory. Optimality: Yes, if:
for Tree-Search, the heuristic is admissible, for Graph-Search, the heuristic is admissible and consistent.

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

12 / 28

Informed strategies

## Designing heuristics Search Methods GEIST

Heuristic functions
Heuristic function quality An admissible heuristic never overestimates the cost of reaching the goal. It usually underestimates see the SLD heuristic. However, a good heuristic function tries to minimise the gap between its value and the actual cost. Example How would you design a heuristic for the 8-puzzle?
7 5 8 3
Start State

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

4 6 1

5 1 4 7

2 5 8
Goal State

3 6

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

13 / 28

Informed strategies

Designing heuristics Search Methods GEIST Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Heuristic functions
Example How would you design a heuristic for the 8-puzzle?
7 5 8 3
Start State

4 6 1

5 1 4 7

2 5 8
Goal State

3 6

8-puzzle: two possible heuristics h1 (n): number of tiles not in their places h2 (n): sum of Manhattan distances of all tiles from their places

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

14 / 28

Informed strategies

## Designing heuristics Search Methods GEIST Informed strategies

Heuristic functions
Example How would you design a heuristic for the 8-puzzle?
7 5 8 3
Start State

4 6 1

5 1 4 7

2 5 8
Goal State

3 6

Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

15 / 28

Informed strategies

## Designing heuristics Search Methods GEIST

Heuristic functions
8-puzzle: two possible heuristics h1 (S) = 6 h2 (S) = 4 + 0 + 3 + 3 + 1 + 0 + 2 + 1 = 14 Heuristic domination If h2 (n) h1 (n) for all n, then h2 dominates h1 . Therefore, h2 is always better than h1 . Performance comparison Number of nodes expanded if solution is at depth d : d 14 24 IDS 3 473 951 54 000 000 000 A*(h1 ) 539 39 135 A*(h2 ) 113 1 641

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

16 / 28

Informed strategies

## Designing heuristics Search Methods GEIST Informed strategies

Heuristic functions
Heuristic domination If h2 (n) h1 (n) for all n, then h2 dominates h1 . Therefore, h2 is always better than h1 . Hybrid heuristics If ha and hb are admissible heuristics, then h(n) = max(ha (n), hb (n)) is admissible and dominates both ha and hb . Deriving admissible heuristics Admissible heuristics can be derived from the exact solution cost of a relaxed version of the problem.

Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

17 / 28

Informed strategies

## Designing heuristics Search Methods GEIST

Heuristic functions
Deriving admissible heuristics Admissible heuristics can be derived from the exact solution cost of a relaxed version of the problem. Example: 8-puzzle For the 8-puzzle problem: If the rules are relaxed so that a tile can move anywhere, then h1 (n) (number of misplaced tiles) gives the exact cost. If the rules are relaxed so that a tile can can move to any adjacent square (regardless of whether its occupied), then h2 (n) (sum of Manhattan distances) gives the exact cost.
7 5 8 3
Start State

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

4 6 1

5 1 4 7

2 5 8
Goal State

3 6

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

18 / 28

Informed strategies

## Iterative improvement algorithms

Iterative improvement For many problems, the solution is important, not how we got there the path is irrelevant. State space: set of complete congurations; nd optimal conguration (e.g., TSP) or conguration satisfying constraints (e.g., timetable). Iterative improvement: keep current state, try to improve it. Example: Travelling Salesman Problem Start with any complete tour, then try to switch pairs.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

19 / 28

Informed strategies

## Iterative improvement algorithms

Example: n-queensExample: n-queens
Put nPut n queens n anboard n board two that noon thequeens queens on an on n n with no so queens two same conict on same row, column, or diagonal row, column or diagonal. Move a queen to reduce number of conicts

Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## Move a queen to reduce number of conicts.

h=5

h=2

h=0

Almost always solves n-queens problems almost instantaneously for very large n, e.g., n = 1million
GEIST (KA AGH) Search Methods 1 czerwca 2010 20 / 28

Informed strategies

## Hill-climbing Search Methods GEIST Informed strategies

Hill-climbing search
Hill-climbing ascent/descent search

Hill-climbing (or search algorithms. Also called local greedy gradient ascent/descent)

Like climbing Mt. Everest in thick fog, with amnesia. Like climbing Everest in thick fog with amnesia
function Hill-Climbing( problem) returns a state that is a local maximum inputs: problem, a problem local variables: current, a node neighbor, a node current Make-Node(Initial-State[problem]) loop do neighbor a highest-valued successor of current if Value[neighbor] Value[current] then return State[current] current neighbor end

Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

21 / 28

Informed strategies

## Hill-climbing Search Methods GEIST

Hill-climbing search
objective function

global maximum

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## shoulder local maximum "flat" local maximum

current state

state space

Hill-climbing algorithm can get stuck on: local maxima, ridges, ats.
GEIST (KA AGH) Search Methods 1 czerwca 2010 22 / 28

Informed strategies

## Hill-climbing Search Methods GEIST Informed strategies

Hill-climbing search
Variants of hill-climbing search If an algorithm (e.g., standard hill-climbing) only selects moves which improve the solution, it is bound to be incomplete. Random-start algorithms are complete, but very ineective. Stochastic hill-climbing picks one of the better successors at random. First-choice hill-climbing generates random successors until it nds one better than the current state. Random-restart hill-climbing conducts a series of searches, starting with random start states. It is complete with probability close to 1, because it must eventually generate the goal state.

Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

23 / 28

Informed strategies

## Hill-climbing Search Methods GEIST

Simulated annealing
Background In metallurgy, annealing is the process used to temper or harden metals and glass by heating them to a high temperature and then gradually cool them. This allows the material to coalesce into a low-energy crystalline state. The algorithm Selects move at random. If the move improves the solution, it is made every time. If the move worsens the solution, the probability if it is made is determined upon:
the amount E by which the solution is worsened, the so-called temperature T , which is decreased in every iteration.a
a That way, the bad moves are more likely to be allowed at the start, and become less likely as the temperature decreases.

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

24 / 28

Simulated annealing
Idea: escape local maxima by allowing some bad moves but gradually decrease their size and frequency
function Simulated-Annealing( problem, schedule) returns a solution state inputs: problem, a problem schedule, a mapping from time to temperature local variables: current, a node next, a node T, a temperature controlling prob. of downward steps current Make-Node(Initial-State[problem]) for t 1 to do T schedule[t] if T = 0 then return current next a randomly selected successor of current E Value[next] Value[current] if E > 0 then current next else current next only with probability e E/T
Chapter 4, Sections 34 8

Simulated annealing

Informed strategies

Hill-climbing Search Methods GEIST Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

25 / 28

Informed strategies

## Local beam search

Local beam search Idea: keep k states instead of 1; choose top k of all their successors. Not the same as k individual searches run in parallel. Searches that nd good states invite others to join them. Threads that do not provide good results are discarded. Problem Often, all k states end up on same local maximum. Solution: stochastic local beam search Choose k successors at random, but still be biased towards the good ones.

Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

## GEIST (KA AGH)

Search Methods

1 czerwca 2010

26 / 28

Informed strategies

## Hill-climbing Search Methods GEIST

Genetic algorithms
Genetic algorithms Variant of stochastic beam search. Successor states generated by combining two parent states (rather than modifying a single state). We have k randomly generated states, called the population. Each state is caled an individual. Each individual is represented as a string over a nite alphabet. Steps in GA
1

Informed strategies Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Evaluate tness of each individual in population, as probability of being selected for reproduction. Perform selection, taking tness into account. Perform crossover within pairs (at random point). Introduce random mutation to allow for stochastic exploration.
GEIST (KA AGH) Search Methods 1 czerwca 2010 27 / 28

2 3 4

Informed strategies

## Hill-climbing Search Methods GEIST Informed strategies

Genetic algorithms
24748552 32752411 24415124 32543213
24 31% 23 29% 20 26% 11 14%

Pairs

## 32748552 24752411 32752124 24415411

CrossOver Mutation

## 32748152 24752411 32252124 24415417

Best-rst search Greedy search A* search Designing heuristics Iterative improvement algorithms Hill-climbing

Fitness

Selection

Search Methods

1 czerwca 2010

28 / 28