AI in poker game

Introduction
• Poker is not a completely solved game analytically because it is
extremely hard to consider every opponent strategy.

• One of the earliest attempts of game theory to analyze poker: 1944

when Von Neumann and Oskar Morgenstern wrote ‘Theory of Games
and Economic Behavior ‘
The Von Neumann Model of Poker
• The von Neumann poker model is an extremely simplified version of
poker consisting in only two players and a limited set of rules and
actions.

• Yet, even with the simplifications, it still takes some effort to solve the
game
Simplified Von Neumann Poker Rules
• The game has two players with very limited actions:
• Each player antes $1 into the pot
• Each player is then “dealt” a random number between 0 and 1
• Actions: check, raise or fold
• Player 1 goes first.
The game tree for player 1
• B is the bet amount

• Note how restricted this game is

compared to actual poker.
Neither player can choose a
variable betting amount, and
there is no chance to re-raise.

• What is the solution to this

game?
Solving the game by backwards induction

Player 2 Player 1
• Can either ‘call’ or ‘fold’ • Can either bet, check… or bluff
• [0 – c) ->fold • [b – 1] -> bet
• [c – 1] -> call • (a – b) -> check

• Clearly there is no benefit for • …[0 – a) -> bet (bluffing)

player 2 to bluff.
Poker - 5 card draw Probabilities
History

• In 1494, Fra Luca Paccioli released his work Summa de arithmetica,

geometria, proportioni e proportionalita which was the first written
text on probability.
• Probability theory was one of the last major areas of mathematics to
be developed; its beginnings are usually dated to correspondence
between the mathematicians Fermat and Pascal in the 1650's
concerning some interesting problems that arose from gambling.
Probabilities of being dealt a hand
Deepstack
• First win of AI against humans at Poker Texas Hold 'em
• AI for Poker is way harder to implement because unlike Chess and Go,
there is a lot of random involved in the game ( the cards received by
each player and the cards on the table)
• A clear strategy can't be calculated and we have to use "intuition:
Chess/Go vs Poker
• "The aim of traditional game-playing AIs is to calculate the possible
results of a game as far as possible and then rank the strategy options
using a formula that searches data from other winning games. The
downside to this method is that in order to compress the available
data, algorithms sometimes group together strategies that don’t
actually work." (Michael Bowling, a computer scientist at the
University of Alberta in Edmonton, Canada)
Implementation details for Deepstack
• avoids abstracting data by only calculating ahead a few steps rather
than an entire game
• program continuously recalculates its algorithms as new information
is acquired
• when the AI needs to act before the opponent makes a bet or holds
and does not receive new information, deep learning steps in
• researchers required the program to solve more than 10 million
randomly generated poker game situations
Test for Deepstack
• a pool of 33 professional poker players selected by the International
Federation of Poker
• 4 weeks, the players challenged the program to 44,852 games of
heads-up no-limit Texas Hold ‘em
• researchers found that DeepStack’s final win rate was 486 milli-big-
blinds per game (milli- big-blind = one-thousandth of the bet required
to win a game ; 10 times that of what professional poker players
consider a sizable margin)
Deepstack and Libratus
• The results were almost the same with Libratus(AI Poker
from Carnegie Mellon University in Pittsburgh)
• Libratus: 20-day competition – beat top 4 players of poker in the
world over the course of 120,000 hands
• Libratus has no deep learning, so it has to solve the entire game
• Deepstack has a lot less need for computation power so it can run on
a laptop
From poker to the world
• business implications of the technology
• AI that can understand imperfect information scenarios could help
determine what the final sale price of a house would be for a buyer
before knowing the other bids,
• "While poker is a step more complex than perfect information games,
it's still a long way to go to get to the messiness of the real world."
Bibliography
• https://mindyourdecisions.com/blog/2015/01/20/game-theory-
tuesdays-von-neumann-poker/
• http://www.chrissnijders.com/eth/Ferguson.pdf
• http://www.sciencemag.org/news/2017/03/artificial-intelligence-
goes-deep-beat-humans-poker