Rule Based Design

How to Design Rules

Identification Tree
 Introduction to Rule-Based Systems
• a set of if-then statements, but provide the basis for so-
called “expert systems” which are widely used in many fields
• the knowledge of an expert is encoded into the rule set.
When exposed to the same data, the expert system AI will
perform in a similar manner to the expert.
• Rule-based systems are a relatively simple model that can
be adapted to any number of problems.
• Overall, rule-based systems are really only feasible for
problems for which any and all knowledge in the problem
area can be written in the form of if-then rules and for
which this problem area is not large. If there are too many
rules, the system can become difficult to maintain and can
suffer a performance hit.
 Introduction to Rule-Based Systems
• To create a rule-based system for a given problem,
you must have (or create) the following:
– A set of facts to represent the initial working memory. This
should be anything relevant to the beginning state of the
system.
– A set of rules. This should encompass any and all actions
that should be taken within the scope of a problem, but
nothing irrelevant. The number of rules in the system can
affect its performance, so you don’t want any that aren’t
needed.
– A condition that determines that a solution has been found
or that none exists. This is necessary to terminate some
rule-based systems that find themselves in infinite loops
otherwise.
 Rule Conflicts
There are several different strategies, but here are a
few of the most common:
• First Applicable: If the rules are in a specified order, firing the first applicable
one allows control over the order in which rules fire. This is the simplest
strategy and has a potential for a large problem: that of an infinite loop on
the same rule. If the working memory remains the same, as does the rule-
base, then the conditions of the first rule have not changed and it will fire
again and again. To solve this, it is a common practice to suspend a fired
rule and prevent it from re-firing until the data in working memory, that
satisfied the rule’s conditions, has changed.
• Random: Though it doesn’t provide the predictability or control of the first-
applicable strategy, it does have its advantages. For one thing, its
unpredictability is an advantage in some circumstances (such as games for
example). A random strategy simply chooses a single random rule to fire
from the conflict set. Another possibility for a random strategy is a fuzzy
rule-based system in which each of the rules has a probability such that
some rules are more likely to fire than others.
 Rule Conflicts
• Most Specific: This strategy is based on the number of
conditions of the rules. From the conflict set, the rule with
the most conditions is chosen. This is based on the
assumption that if it has the most conditions then it has the
most relevance to the existing data.
• Least Recently Used: Each of the rules is accompanied by a
time or step stamp, which marks the last time it was used.
This maximizes the number of individual rules that are fired at
least once. If all rules are needed for the solution of a given
problem, this is a perfect strategy.
• "Best" rule: For this to work, each rule is given a ‘weight,’
which specifies how much it should be considered over the
alternatives. The rule with the most preferable outcomes is
chosen based on this weight.
Knowledge Representation
 Semantic Net

In this sample semantic network, Zoom is a feline; a feline is a mammal; a mammal

is an animal. Zoom chases a mouse; a mouse is a mammal; a mammal is an animal.
Zoom eats fish; fish is an animal. The relations are written on the lines: is a, is an,
eats, chases. The nodes (circles) are ‘objects’.
Semantic Tree
Decision Tree
Identification Tree
 Identification Trees (ID Trees)
• an ID tree is a decision tree in which all
possible divisions is created by training the
tree against a list of known data. The purpose
of an ID tree is to take a set of sample data,
classify the data and construct a series of test
to classify an unknown object based on like
properties.
 Training ID Trees
• First, the tree must be created and trained. It must
be provided with sufficient labeled samples that
are used to create the tree itself. It does this by
dividing the samples into subsets based on
features. The sets of samples at the leaves of the
tree define a classification. The tree is created
based on Occam’s Razor, which (modified for ID
trees) states that the simplest (smallest) tree, that
is consistent with the training samples, is the best
predictor. To find the smallest tree, one could find
every possible tree given the data set then
examine each one and choose the smallest. But, it
could be time consuming
 How to create smallest tree
• At each node, pick a test such that branches are close
to same classification
• Split into subsets with the least disorder
• Find which of these tests minimizes the disorder
THEN
• Until each leaf node contains a set that is homogenous
or is near homogenous
• Select a leaf node that is non-homogenous
• Split this set into two or more homogenous subsets
to minimize disorder
 Since the goal of an ID tree is to generate
homogenous subsets, we want to calculate how
non-homogenous the subsets each test creates. The
test that minimizes the disorder is the one that
divides the samples into the cleanest categories.
Disorder is calculated as follows:
Average disorder = Σb (nb/nt) * (Σc -(nbc/nb) 2log(nbc/nb))

Where:
nb is the number of samples in branch ‘b’
nt is the total number of samples in all branches
nbc is the total of samples in branch b of class c
 Mengubah ID Trees jadi Rules
• Once an ID tree is constructed successfully, it
can be used to generate a rule-set, which will
serve to perform the necessary classifications
of the ID tree. This is done by creating a single
rule for each path from the root to a leaf in
the ID tree.
 Pruning Unnecessary Conditions
• If there are conditions of that rule that are inconsequential to the
outcome, discard them thus simplifying the rule (and thus
improving efficiency). This is accomplished by proving that the
outcome is independent of the given condition. Events A and B are
independent if the probability of event B does not change given
that event A occurs. Using Bayes Rule:

P(B | A) = P(B)

• This states that the probability of event B given that event A occurs
is equal to the probability that event B occurs by itself. If this holds
true, then event A does not effect whether or not event B occurs. If
A is a condition and B is a result, then A can be discarded without
affecting the rule.
• If two or more rules share the same end result, you may be able to replace them with
a rule that fires in the event that no other rule is fired:
• if (no other rule fires) then (execute these common actions)

• If there is more than one such group of rules, replace only one group. Which one is
determined by some heuristic tiebreaker. Two such tiebreakers follow:
• Replace the larger of the two groups. If group A has six rules which share a common
result and group B only has five, replace the larger group A with will eliminate more
rules and simplify the rule base the most.
• Replace the group with the highest average number of rule conditions. While more
rules may remain, the rules that remain will be more simple as they have fewer
conditions. For example, given the rules:
• if (x) and (y) and (z) then (A)
• if (m) and (n) and (o) then (A)

• vs.

• if (p) then (Z)

• if (q) then (Z)
• You would want to replace the first set with:
• if (no other rule fires)
• then (A)
Forward Chaining
Backward Chaining
Identification Tree Training
 ID Tree Training
Diketahui: Observasi
Langkah-langkah menghasilkan Rules
1. Bentuk pohon identifikasi berdasarkan masing-
masing atribut
2. Hitung Average-Disorder untuk setiap pohon
3. Urutkan dari kecil ke besar
4. Kembangkan pohon dengan AD terkecil sampai
setiap cabang homogen (AD = 0)
5. Hasilkan Rules berdasarkan pohon ID
6. Optimalkan Rules
Id Trees
 Disorder Calculation
1. Size_Disorder
= Σb (nb/nt) * (Σc -(nbc/nb)2log(nbc/nb))
= (4/8) * ((-(2/4) 2log (2/4)) + (-(2/4) 2log (2/4)))
+ ((1/8) * 0) + ((3/8) * 0)
= 0.5

Merah = cabang1 ; Hijau = cabang 2; Biru =

cabang 3
 Disorder Calculation
2. Color_Disorder
= Σb (nb/nt) * (Σc -(nbc/nb)2log(nbc/nb))
= (3/8) * ((-2/3) 2log (2/3)) + (-(1/3) 2log (1/3)) + (3/8) *
((-2/3) 2log (2/3)) + (-(1/3) 2log (1/3)) + ((2/8) * 0)
= 2 * (3/8) * ((-2/3) * (-0.58496)) + (-(1/3) * (-1.58496))
= (6/8) * (0.38997 + 0.52832)
= (6/8) * (0.91829)
= 0.688 = 0.69
Merah = cabang1 ; Hijau = cabang 2; Biru = cabang 3
 Disorder Calculation
3. Weight_Disorder
= Σb (nb/nt) * (Σc -(nbc/nb)2log(nbc/nb))
= (2/8) * ((-1/2) 2log (1/2)) + (-(1/2) 2log (1/2)) +
(3/8) * ((-2/3) 2log (2/3)) + (-(1/3) 2log (1/3)) +
(3/8) * ((-2/3) 2log (2/3)) + (-(1/3) 2log (1/3))
= 0.25 + 0.344 + 0.344
= 0.938 = 0.94
Merah = cabang1 ; Hijau = cabang 2; Biru =
cabang 3
 Disorder Calculation
4. Rubber_Disorder
= Σb (nb/nt) * (Σc -(nbc/nb)2log(nbc/nb))
= (5/8) * ((-3/5) 2log (3/5)) + (-(2/5) 2log (2/5)) + (3/8) * 0
= 5/8 * ((-3/5) * (- (1.58496 – 2.32193)) +
((-2/5) * (- (1 – 2.32193)))
= 5/8 * ((-3/5) * (- 0.73697) + (-2/5) * (- 1.32193))
= 5/8 * ( 0.442+ 0.528)
= 5/8 * 0.97
= 0.606 = 0.61
Merah = cabang1 ; Hijau = cabang 2;
Hitungan dengan Rumus Logaritma
• Log (A.B) = Log A + Log B
• Log 1/A = - Log A
• A Log B = Log B / Log A
• Since Size is the lowest disorder, we take that
one and further break down any
unhomogenous sets. There is only one, those
of the ‘small’ branch. The following trees are
resulting from the further division of the
size=small test.
Pohon Hasil Break Down
 Pohon Akhir
The Rubber test splits the remaining samples into
perfect subsets with 0 disorder. Therefore, our final,
simplest ID tree which represents the data is:
 Building RB from Identification Trees
R1: IF (size = large)
THEN (ball bounce)
R2: IF (size = medium)
THEN (ball does not bounce)
R3: IF (size = small) AND (rubber = no)
THEN (ball does not bounce)
R4: IF (size = small) AND (rubber = yes)
then (ball bounce)
Eliminating unnecessary rule conditions
The last two rules have two conditions each.
Consider, for example, the first of these, R3:
R3: IF (size = small) AND (rubber = no)
THEN (ball does not bounce)

Looking at the probability with event A =

(size=small) and event B = (ball does not bounce)
P(B|A) = (3 non rubber balls do not bounce / 8 total) = 0.375
P(B) = (3 non rubber balls / 8 total) = 0.375
P(B|A) = P(B) therefore B is independent of A
If we were to eliminate the first condition:
size=small, then this rule would trigger for
every ball not made of rubber. There are 3
balls not made of rubber. They are 2, 3 and 8 –
none of these bounce. Because none bounce,
the size does not affect this and we can
eliminate that condition.
IF (rubber = no)
THEN(ball does not bounce)
Examining the next condition, the probability for A =
(rubber=no) and B the same:
P(B|A) = (2 small balls do not bounce / 8 total) = 0.25
P(B) = (4 small balls / 8 total) = 0.5
P(B|A) does not equal P(B) therefore A and B are not
independent

If you eliminate the next condition in the same rule,

(rubber = no) this triggers for every small ball. Of the
small balls, two bounce and two do not. Therefore, the
rubber does affect if they bounce or not and cannot be
eliminated. The small balls bounce only if they are
rubber.
Now, the next rule with two conditions:
R4: IF (size = small) AND (rubber = yes)
THEN (ball does bounce)

Examining the probabilities: A = (size=small); B =

(ball bounce)
P(B|A) = P(2 small balls bounce / 8 total) = 0.25
P(B) = P(4 small balls / 8 total) = 0.5
P(B|A) does not equal P(B) therefore A and B are
not independent.
If we eliminate the first rule, it fires for all rubber balls. Of the
five rubber balls, two are small and both bounce. Of the
other three, one bounces and two do not. For this rule,
(size=small) is important.
On to the next condition. Examining the probabilities: A =
(rubber=yes); B = (ball bounce)
P(B|A) = P(3 rubber balls bounce / 8 total) = 0.375
P(B) = P(5 rubber balls / 8 total) = 0.625
P(B|A) does not equal P(B) therefore A and B are not
independent
Eliminating the second fires for all small balls. Of the four
small, two bounce and two do not. Again, the condition is
significant and cannot be dropped. Therefore this rule must
stay as it is.
 Eliminating unnecessary rules
R1: IF (size = large)
THEN (ball does not bounce)
R2: IF (size = medium)
THEN (ball bounce)
R3: IF (rubber = no)
THEN (ball does not bounce)
R4: IF (size = small) AND (rubber = yes)
THEN (ball bounce)
Of these, we have two sets of rules, each set shares a
common result. The first group consists of rules R1 and
R3. The second consists of rules R2 and R4.
We can eliminate one of these sets and replace it
with the rule:
if (no other rule fires)
then (perform these common actions)
Both sets have the same number of rules, 2, but
the second set has more conditions than the first.
So, we’ll eliminate the second set and replace it
with:
if (no other rule fires)
then (ball bounce)
 Our final rule set
R1: IF (size = medium)
THEN (ball does not bounce)
R2: IF (rubber = no)
THEN (ball does not bounce)
R3: IF (no other rule fires)
THEN (ball bounce)