Example Attributes Goal

Alt Bar Fri Hun Pat Price Rain Res Type Est WillWait
X1 Yes No Yes Yes Full $ No No Thai 10-30 Yes
X2 Yes No Yes No Full $$$ No Yes French >60 No
X3 No Yes No Yes Some $$ Yes Yes Italian 0-10 Yes
X4 No Yes No No None $ Yes No Burger 0-10 No
X5 No No No Yes Some $$ Yes Yes Thai 0-10 Yes
X6 No Yes Yes No Full $ Yes No Burger >60 No
X7 Yes Yes Yes Yes Full $$$ No Yes Italian 10-30 No
X8 No No No No None $ No No Thai 0-10 No
X9 Yes Yes Yes Yes Full $ No No Burger 30-60 Yes
X 10 Yes No No Yes Some $$$ No Yes French 0-10 Yes
X 11 Yes No No Yes Full $ No No Thai 30-60 No
X 12 No Yes No No Some $ No No Burger 0-10 Yes

A set of examples X1,…, X12 for the restaurant domain is shown in Figure 18.3.
The positive examples are ones where the goal WillWait is true (X1, X3,....) and
negative examples are ones where it is false (X2, X4, ...).
Figure 18.2 A decision tree for deciding whether to wait for a
table.
Figure 18.4 Splitting
the examples by
testing on
attributes.
In (a) Patrons is a
good attribute to
test first;
in (b) Type is a poor
one; and
in (c) Hungry is a
fairly good second
test, given that
Patrons is the first
test.
Figure 18.8 The “optimal” decision tree induced from the 12-example training
set using ID3
But if we actually look at this tree we can
perhaps see the “logic” in the tree
Patrons=some OR
Patrons=full and Hungry=no and Type=french OR
Patrons=full and Hungry=no and Type=burger OR
Patrons=full and Hungry=no and
Type=Thai and
Fri/Sat=Yes
 Decision Tree
• decision trees represent disjunctions of conjunctions
Outlook

Sunny Overcast Rain

Humidity Yes Wind

High Normal Strong Weak

No Yes No Yes

(Outlook=Sunny  Humidity=Normal)
 (Outlook=Overcast)
 (Outlook=Rain  Wind=Weak)
Representing knowledge as logic
• Restaurant example from handout (based on
chapter 18).
Alternate(X1) ^ ¬Bar(X1) ^ Fri(X1) ^
Hungry(X1) .... Estimate(10-30,X1)
-> WillWait(X1)
• But this is just one example. We would
need to encode all 12 examples and join
them with disjunction.
Representing knowledge as logic
• But THAT suggests we know only those 12 cases.
In fact, the book has proposed that we know more
through the construction of the decision tree
(Figure 18.6)…
• As Logic…
ForAll x    WillWait(x) <=>
Patrons(x,Some)
V  Patrons(x,Full) ^ Hungry(x) ^ Type(x,French)
V  Patrons(x,Full) ^ Hungry(x) ^ Type(x,Thai) ^ Fri(x)
V  Patrons(x,Full) ^ Hungry(x) ^ Type(x,Burger)
 Learning over time
• How do you evolve knowledge over time
when you learn little bit by little bit?
– Abstract version – the “Frinkle”
 The Question
• The Question
– How can we build this kind of representation
over time?
• The Answer
– Rely on the concepts of false positives and false
negatives
 The idea
• False Positive
– An example which is predicted to be positive but whose
known outcome is negative
– The problem is that our hypothesis is too general.
– The solution is to add another condition to our hypothesis.
• False Negative
– An example which is predicted to be negative but whose
known outcome is positive
– The problem is that our hypothesis is too restrictive.
– The solution is to remove a condition to our hypothesis [or
to add disjunction]
Let’s try this using the restaurant
problem based on my handout
• Start with X1
H1: ForAll x WillWait(x) Alt(x)
Let’s try this using the restaurant
problem based on my handout
• Consider X2 given the current hypothesis
H1: ForAll x WillWait(x) Alt(x)
• This yields a result of WillWait but we see
that the known outcome is false.
• This is a false positive… add a condition
H2: ForAll x WillWait(x) Alt(x) ^ Hun(x)
Let’s try this using the restaurant
problem based on my handout
• Consider X3 given the current hypothesis
H2: ForAll x WillWait(x) Alt(x) ^ Hun(x)
This yields a result of NOT WillWait but we
see that the known outcome is true.
• This is a false negative… remove a
condition
H3: ForAll x WillWait(x) Hun(x)
Let’s try this using the restaurant
problem based on my handout
• Notice that the next 3 cases all are correct
given our current hypothesis
H3: ForAll x WillWait(x) Hun(x)
Let’s try this using the restaurant
problem based on my handout
• Consider X7 given the current hypothesis
H3: ForAll x WillWait(x) Hun(x)
This yields a result of WillWait but we see
that the known outcome is false.
• This is a false positive… add a condition
H4: ForAll x WillWait(x) Hun(x) ^ NOT
Cost(x,$$$).
Let’s try this using the restaurant
problem based on my handout
• X8 and X9 both are correct given our
current hypothesis.
H4: ForAll x WillWait(x) Hun(x)
Let’s try this using the restaurant
problem based on my handout
• Consider X10 given the current hypothesis
H4: ForAll x WillWait(x) Hun(x) ^ NOT Cost(x,$$
$).
This yields a result of NOT WillWait but we see that
the known outcome is true.
• This is a false negative… remove a condition.
• Except, we don’t really have a condition we can
remove. Thus, add disjunction.
H4: ForAll x WillWait(x) Hun(x) ^ NOT Cost(x,$$
$).
Let’s try this using the restaurant
problem based on my handout
• H4: ForAll x WillWait(x) Hun(x) ^
NOT Cost(x,$$$).
• Except, we don’t really have a condition we
can remove.
• Thus, add disjunction.
H5: ForAll x WillWait(x) 
Hun(x) ^ NOT Cost(x,$$$) V
Hun(x) ^ Cost(x,$$$) ^ Patrons(x,Some)