Vous êtes sur la page 1sur 35

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

CSE 481
ARTIFICIAL INTELLIGENCE
ADVERSARIAL SEARCH AND GAMES

Conventional Search

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

In conventional search

There is no adversary to consider


A solution is a method for finding goal
Heuristic techniques
Could find an optimal solution but not always

Games: Adversarial Search

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

Games are adversarial

A solution involves strategy that specifies a move for every


possible opponent reply
We can find an exact solution
Using search with alternating player moves and assuming an
opponent playing perfectly
The search explores the game tree to the conclusion and selects
the move with the guaranteed best outcome

Time limits may force an approximate solution


A heuristic evaluation function
Replaces the utility function

It evaluates the goodness of a game position


Examples: chess, checkers etc.

The Game Prototype

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

Formulating games as search problems


1. Initial State
The board position + the player to move

2. Successor Function
A list of (move, state) pairs specifying legal moves

3. Terminal Test
Detects a terminal state (i.e., detects if the game is finished)

4. Utility Function
Returns a numerical value for terminal states
Possibly as simple as 1 for win, 0 for lose, and -1 for draw

Game Search & Game Trees

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

From the initial state and successor function

We generate the game tree


Each turn is 1 ply (a layer of nodes representing states)

We use the search tree to determine the next move:


1. Search to terminal states
2. Use their utility function values + assumptions about opponent
moves
3. Pick the optimal next move

Example: a complete tree for an example game called


NIM (http://illuminations.nctm.org/ActivityDetail.aspx?ID=140)

A very small game: NIM Game Rules

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

Consider a game called NIM

It is played by two players

In this game a number of stones are placed in a pile

In each players turn, player can remove either 1 or 2


stones from the pile

The player who removes the last stone wins

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

A very small game: NIM Game Representation

Consider the game tree for


the 5 stone version of NIM.

[VAL] represents the first


player (We) node or MAX
node

(VAL)
represents
the
Opponent node or MIN
node

VAL is the current number


of stones in the pile

We start the game

A very small game: NIM Node Evaluation

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

If a node corresponds to MAX


move and the value is [0], then
that node indicates a Lose for
MAX and a utility value of 0 is
assigned to this node

Similarly a node with (0) is a


Win for MAX and a utility value
of 1 is assigned to this node

To propagate these values back


up the list from bottom up as
follow:

if we are evaluating a MAX node, find


the maximum of the utility function of
all branches going out from this node
if we are evaluating a MIN node find
the minimum of the utility function of
all branches going out from this node

Optimal Strategies: minimax

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

minimax values for a simple 2-ply example

here's the game tree


with terminal values from a utility function

Optimal Strategies: minimax

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

10

minimax values for a simple 2-ply example

now we back up the values for min nodes

Optimal Strategies: minimax

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

11

minimax values for a simple 2-ply example

then we back up values for max node

Optimal Strategies: minimax

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

12

The optimal play assumption

Optimal play for MAX


Assumes that MIN also plays optimally
i.e. maximizes the worst-case outcome for MAX

What if the assumption doesn't hold


What if MIN does not play optimally?
In this case, MAX will do even better

Extending minimax

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

13

Pruning the game search tree

The number of game states is


Exponential in the number of moves

Thus, for real non-trivial games


There are just too many nodes

So, how to avoid examining every node in the tree?


We can still compute the correct minimax decision without
expanding all the nodes

We can prune the game tree by not expanding portions that


wouldn't make any difference (- pruning algorithm)

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

14

- pruning example

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

15

- pruning example

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

16

- pruning example

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

17

- pruning example

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

18

- pruning example

- Pruning Algorithm

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

19

Idea: if current path is already worse than some


other known path, stop expanding it.

Improves the speed of the mini-max search


algorithm

is the max lower bound of all solutions. i.e. the value of


the best move so far at any choice point on MAX path
is the min upper bound of all solutions. i.e. value for the
best move so far at any choice point on MIN path

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

20

- Pruning Algorithm Example


=-
=+

=-
=+
=-
=+
=-
=+

MAX
MIN
MAX
MIN
MAX

1) Setup phase: Assign to each left-most internal node of the tree,


variables: = -, = +

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

21

- Pruning Algorithm Example


=-
=+

=-
=+
=-
=+
=-
=3
3

MAX
MIN
MAX
MIN
MAX

2) Look at first computed final configuration value. Its 3. Parent is a MIN node, so
set the (min) value to 3

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

22

- Pruning Algorithm Example


=-
=+

=-
=+

MIN

=3
=+
=3
=+

=-
= 33
3

MAX

MAX
MIN
MAX

3) Look at next value, 5. Since parent is a MIN node, we want the minimum of
3 and 5 which is 3. Parent MIN node is done fill (max) value of its parent MAX
node. Always set for MAX nodes and for MIN nodes. Copy the state of the MAX parent
node into the second unevaluated MIN child

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

23

- Pruning Algorithm Example


=-
=+

=-
=+

MIN

=3
=+
=3
=2

=-
= 33
3

MAX

MAX
MIN
MAX

4) Look at next value, 2. Since parent node is MIN with =+, 2 is smaller, change

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

24

- Pruning Algorithm Example


=-
=+

=-
=+
=3
=+ 3
=-
= 33
3

MAX
MIN

=3
=2
2
2

MAX
MIN
MAX

5) Now, the MIN parent node has a max value of 3 and MIN value of 2. The value of the 2nd child
does not matter. if it is >2, 2 will be selected for MIN node. If it is <2, it will be selected for MIN node,
but since it is <3 it will not get selected for the parent max node. Thus, we prune the right sub-tree of
the MIN node. Propagate MAX value up the tree

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

25

- Pruning Algorithm Example


=-
=+
=-
=3
=3
=+ 3
=-
= 33
3

MIN
=-
=3

=3
=2
2
2

MAX

=-
=3

MAX
MIN
MAX

6) MAX node is now done and we can set the value of its parent and propagate node
state to sibling sub-trees left-most path

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

26

- Pruning Algorithm Example


=-
=+
=-
=3
=3
=+ 3
=-
= 33
3

MIN
=-
=3

=3
=2
2
2

MAX

=-
=3
10

MAX
MIN
MAX

7) The next node is 10. 10 is not smaller than 3, so state of parent does not change. We
still have to look at the 2nd child since is still

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

27

- Pruning Algorithm Example


=-
=+

MAX

=-
=3
=3
=+ 3
=-
= 33
3

=3
=2
2
2

MIN
=4
=3
4
=-
=3
4

MIN

10

MAX

MAX

8) The next node is 4. Smallest value goes to the parent MIN node. MIN sub-tree is
done, so the parent MAX node gets the (max) value from the child. Note that if the
MAX node had a 2nd sub-tree, we can prune it since >

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

28

- Pruning Algorithm Example


=3
=+

MAX

=-
=3 3
=3
=+ 3
=-
= 33
3

=3
=2
2
2

=3
=+
=4
=3
4
=-
4 = 3
10

=3
=+
=3
=+

MIN

MAX
MIN
MAX

9) Continue propagating value up the tree, modifying the corresponding and values.
Also, propagate the state of root node down the left-most path of the right sub-tree

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

29

- Pruning Algorithm Example


=3
=+

MAX

=-
=3 3
=3
=+ 3
=-
= 33
3

=3
=2
2
2

=3
=+

MIN

=4
=3
4
=-
4 = 3

=3
=+
2
=3
2 = 2

MIN

10

MAX

MAX

10) Next value is a 2. We set the (min) value of the MIN parent to 2. Since no other
children exist, we propagate the value up the tree

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

30

- Pruning Algorithm Example


=3
=+

MAX

=-
=3 3
=3
=+ 3
=-
= 33
3

=3
=2
2
2

=3
=2

MIN

=4
=3
4
=-
4 = 3

=3
=+
2
=3
2 = 2

MIN

10

MAX

MAX

11) We have a value for the 3rd level MAX node, now we can modify the (min) value of
the MIN parent to 2. Now, we have a situation that > and thus the value of the
rightmost sub-tree of the MIN node does not matter, so we prune the whole sub-tree

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

31

- Pruning Algorithm Example


=3
=+

=-
=3 3
=3
=+ 3
=-
= 33
3

=3
=2
2
2

MAX
=3
2=2

=4
=3
4
=-
4 = 3
10

=3
=+
2
=3
2 = 2
2

MIN

MAX
MIN
MAX

12) Finally, no more nodes remain, we propagate values up the tree. The root has a value of 3 that
comes from the left-most child. Thus, the player should choose the left-most childs move in order to
maximize his/her winnings. The result is the same as with the original minimax algorithm, but we did not
visit all nodes of the tree

Comments on - Pruning Algorithm

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

32

Pruning does not change the eventual results though


entire sub-trees can be pruned
Its effectiveness depends partly on

The order of examining successors


Perfect ordering would be best successors first which allows
earliest pruning

With "perfect ordering" time complexity is O(bm/2)

The branching factor reduces to sqrt(b)


From an alternative view, given - pruning and a fixed
amount of time we can look twice as far ahead as with simple
minimax

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

33

Dealing with games with large game trees

What to do when the game tree is too large

Complete search is not feasible


We can't guarantee optimal play
Replace the utility function with a heuristic evaluation
function that assigns an estimate to intermediate nodes
without going all the way down to the terminal nodes

Non-Optimal Game Trees

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

34

Heuristic evaluation function

Evaluation function should


1. order terminal states as the utility function would
2. be economical to compute
3. when applied to non-terminals
Be highly correlated with the actual chance of winning
This introduces uncertainty about the outcome
This uncertainty is due to computational limitations
The game is still characterized by complete information

Non-Optimal Game Trees

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

35

Cutoff depth and heuristic evaluation function

The simplest cutoff is a fixed depth limit


Chosen to allow a move choice within the time limit

A more robust approach


Uses iterative deepening
Selects its move from deepest search completed in time