Fuzzy Inference

Fuzzy inference
Cold Warm Hot
17 20 26 29
Cloudy Partial Sunny
30 50 100
Fuzzyfication Implication
Low Medium High
48 Defuzzyfication
. .
Fuzzy inference systems

Knowledge base
Database
Rule base
Crisp Inference Crisp

Input Fuzzifier Defuzzifier Output
Fuzzy Engine Fuzzy
Input Output
Fuzzyfier: translates crisp inputs into fuzzy values

Inference engine: applies reasoning to compute fuzzy outputs
Defuzzyfier: translates fuzzy outputs into crisp values
Knowledge base: defines rules and membership functions
Carlos Andres Pena Reyes

. Logic Systems Laboratory − Swiss Federal Institute of Technology .
. .
Network−like view of a fuzzy system

Low
Pressure
&
High High
OR
Gas
&
Cold Low
OR
Temp.
&
Hot
Carlos Andrés Peña−Reyes

. Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne .
. .
Operational parameters
Low
Pressure
&
High High
OR
Gas
&
Cold Low
OR
Temp.
&
Hot
Membership function values

. .
Connective parameters
Low
Pressure
&
High High
OR
Gas
&
Cold Low
OR
Temp.
&
Hot
Weights
Consequents
Antecedents Rules

. .
Structural parameters
Low
Pressure
&
High High
OR
Gas
&
Cold Low
OR
Temp.
&
Hot
Number of
{ Relevant variables
Rules
Membership functions

. .
Logical parameters
Low
Pressure
&
High High
OR
Gas
&
Cold Low
OR
Temp.
Hot
& Reasoning mechanism
Fuzzy operators
Membership function types
Defuzzification method
. .
Parameters of a fuzzy system

Class Parameters Component
Reasoning mechanism
Inference engine
Fuzzy operators
Logic
Membership function types Fuzzi- and defuzzifier
Defuzzification method Defuzzifier
Relevant variables
Knowledge base
Structural Number of membership functions
Number of rules
Antecedents of rules
Rulebase
Connection Consequents of rules
Rule weights
Operational Membership function values Database
Carlos Andrés Peña-Reyes

. Logic Systems Laboratory - Swiss Federal Institute of Technology Lausanne .
. .
The general modeling problem

What do you know about the modeled system?
i.e. what is predefined and what looked for?
Search space
How do you search?

i.e. do a well suited and/or well known technique exists?
Search method
Have you preferences or restrictions to the model?

i.e. do issues like size, speed, or simplicity matter?
Constraints

. .
Search space in fuzzy modeling

The number of parameters is too high to perform a full search,
some parameter pre−definition is thus required.
According with the searched parameters we can have:
Operational parameters:
Knowledge base Knowledge tuning. P1 P2 P3 P4
Database
Connective parameters:
If V1 is Low AND ....
Rule base Behavior learning.
Structural parameters: R1, ... , Rn
Fuzzifier
Inference
Defuzzifier Structure learning. f1, ... , fm
Engine
Logical parameters:
System design. and, or, not, ...

. .
Search methods: Fuzzy modeling techniques
Knowledge engineering
"Classic" identification methods
Machine learning approaches
Neuro−fuzzy systems
Evolutionary fuzzy modeling techniques

. .
Usual constraints in fuzzy modeling

What is the fuzzy system expected to do?
− Classification: Classification performance, quadratic error.
− Control: Dynamic response, adaptability, robustness, etc.
− Diagnostic: Overall performance, sensitivity, specificity
− Data mining: Completness, complexity.
How is the system expected to do it?

− Speed: Real−time constraints, computing resources.
− Size: Available memory, computing platform.
Who is going to interact with the system?

− Interpretability: Allowed complexity.
− Availability: Continuity of explanations (time to provide them)

Fuzzy modeling: direct approach
Domain Knowledge
expert engineer
Fuzzy
model
Design loop
Validation loop
This approach is also called knowledge engineering

Fuzzy modeling: data-driven approaches
Domain data Building algorithm

Fuzzy
model
Design loop
Validation loop
Domain expert
This approaches are also denominated

knowledge discovering
. .
Interpretability−related constraints
Rule−specific MFs are not allowed

all rules share the same MFs
Orthogonal MFs with well defined

null and unity membership

. .
Interpretability considerations: semantic criteria

Semantics: the study of meanings
● Distinguishability: Each linguistic label has semantic meaning

● Number of elements: Compatible with human capabilities
● Coverage: Any element belongs to at least one fuzzy set
● Normalization: At least one element has unitary membership
● Complementarity: For each element, the sum of memberships is one
Cold Cool Warm Hot Cold Cool Warm Hot
1 1
Temperature Temperature
0 0

. .
Interpretability considerations: syntactic criteria

Syntax: the way in which linguistic elements are put together
● Completeness: for any input, at least one rule must fire

● Rule−base simplicity: Set of rules as small as possible
● Rule readability: small number of conditions in rule antecedents
● Consistency: rules firing simultaneously must have similar consequents
R7 R8 R9 RB RB
R4 R5 R6 R4 R5 RA R5 RA
R1 R2 R3 R1 R2 R0

. .
Strategies to satisfy interpretability criteria

● Linguistic labels shared by all rules
● Normal, orthogonal membership functions
● Don’t care conditions
● Default rule
Cold Warm Hot RB
R5 RA
R0
17 20 26 29

. .
Fuzzy modeling: some data−driven approaches
Identification−based Fuzzy system Neuro−fuzzy systems
Human Logic
design Structural
Estimation Connective
algorithm Operational Fuzzy system
ANN−like training algorithm
Constructive−learning
Fuzzy system
Structural Connective Operational
Data

Neuro-Fuzzy Systems:
Review and Prospects
Detlef Nauck
Faculty of Computer Science

University of Magdeburg
Universitaetsplatz 2
D-39106 Magdeburg, Germany
Detlef.Nauck@cs.uni-magdeburg.de
http://fuzzy.cs.uni-magdeburg.de
N
SF EURO
UZZY
Dr. Detlef Nauck EUFIT‘97 -1-
Concurrent ”Neuro-Fuzzy Models“
Not a neuro-fuzzy model in the narrow sense, because the neural net does not
modify the fuzzy system
NN modifies the output from the fuzzy system, or
NN modifies/produces the input to the fuzzy system
The overall system is not completely interpretable
Easy to implement, e.g. used in consumer products that adapt to the
customer's needs/preferences/environments. N
SF EURO
UZZY
Cooperative Neuro-Fuzzy Models
A neural network is used to determine

membership functions (e.g. Nomura),
fuzzy rules(e.g. Pedrycz/Card),
or rule weights (e.g. Kosko).
After this the fuzzy system works without the neural network. N
SF EURO
UZZY
Hybrid Neuro-Fuzzy Systems
control output
external
reinforce-
ment determine
THEN
error S
IF
Hybrid
Neuro-Fuzzy
Controller
system state
An architecture that can be intertpreted in terms of a neural network with fuzzy weights,
or in terms of a fuzzy system.
E.g.: A neuro-fuzzy controller trained by reinforcement learning
(like GARIC [Berenji/Khedkar] or NEFCON [Nauck/Kruse])
N
SF EURO
UZZY
Our View of Neuro-Fuzzy Systems
I.
A neuro-fuzzy system is a fuzzy system that is trained
by a learning algorithm (usually) derived from neural
network theory.
The heuristic learning procedure operates on local

information, and causes only local modifications in the
underlying fuzzy system.
The learning process is not knowledge based, but data

driven.
N
SF EURO
UZZY
Neuro-Fuzzy Systems (contd.)
II.
A neuro-fuzzy system CAN be viewed as a special 3-layer feed-
forward neural network.
The units in this network use t-norms or t-conorms as activation
functions. The hidden layer represents fuzzy rules. Fuzzy sets are
encoded as (fuzzy) connection weights.
This view of a fuzzy system illustrates the data flow within the
system (data and error signals), and its parallel nature.
However, this neural network view is NOT a prerequisite for

applying a learning procedure, it is merely a convenience.
N
SF EURO
UZZY
Foundational Model: A Generic Fuzzy Perceptron
Y1 Y2
1
The connections are
0 weighted by fuzzy sets
R1 R2 R3
1
The connections are
weighted by fuzzy sets
0
X1 X2
N
SF EURO
UZZY
Learning in Neuro-Fuzzy Systems
Learning algorithms must determine:
fuzzy rules
fuzzy sets
(sometimes additional parameters are used, e.g. rule weights, etc. Semantics!)
Learning paradigms:
supervised learning: function approximation, classification
reinforcement learning: control
Learning fuzzy rules:
cluster oriented approaches
structured sample space (rules on a grid)
Learning fuzzy sets:
gradient descent methods (only Sugeno-type fuzzy systems)
heuristics based on backpropagation
N
SF EURO
UZZY
Approaches to Learning Fuzzy Rules
Cluster-oriented approaches:
combinations with RBF networks (only for certain Sugeno-type models)
or fuzzy clustering:
search for hyperellipsoidal clusters, obtain fuzzy sets by
projecton (errors occur), problems with interpretation.
Fuzzy RuleNet, Fuzzy Min-Max networks:
search for hyperrectangular clusters, obtain fuzzy sets by
projection (no errors), interpretation can be difficult.
Combinatorical approaches:
FuNe, WINROSA:
provide initial fuzzy sets, test combinations of several variables,
pick best rules, can provide small rule bases, computationally expensive.
Structured data space approaches:
NEFCLASS, NEFCON, NEFPROX:
pick rules from a grid (data space is structured by initial fuzzy sets),
simple and fast but limited, no problems with interpretation.
N
SF EURO
UZZY
Artificial Evolution
A metaphor of natural evolution
T T T T S S S X X X A At As C O A As C O A As C O
1 2 3 4 1 2 3 1 2 3 1 1 1 1 1 t2 2 2 2 t3 3 3 3
Set of parameters
Artificial evolution
Natural evolution
ADN
Individual ( Fuzzy) System
M
O
D
E
L
Environment = Selection Performance = Selection

How do you evolve fuzzy systems?
26
T T T T S S S X X X A A A C O A A C O A A C O
40
91
Evaluate A A C O A A C O A A C O
systems
X A
70
X X
S S
T S
T T
T
19
T
T
T
T
S
S
Initialize
S T T T T S S S X X X A A A C O A A C O A A C O
X
X
X
A
A
56
A
Select
C
O
A
A
population
C
O
A
A
C
O
individuals
Parents
Crossover
Mutation
Offspring
What do you evolve in fuzzy systems?
17 20 26 29 30 50 100 0 50 100 H S H O O W P M A C C L O
Linguistic variables Rules

17 20 26 29 30 50 100 0 50 100 H S H O W P M A C C L O
Numeric values Symbolic values

Continuous Discrete
Interdependent
Simultaneous search
How to deal efficiently with them?

. .
Evolutionary knowledge tuning (database)
X2 - Fixed rule base (completness)

Big
Q3
R3 R6 R9
- Rules of type:
Normal
if X1=Low and X2=Normal then Output = Ci

Q2
R2 R5 R8
Small
- Knowledge is tuned by evolution,

Q1
R1 R4 R7
which searches for membership function values
X1
P1 P2 P3
- Genome encodes values P, Q, and C

Low Mid High (3*P + 3*Q + 9*C) * 5 bits = 75 bits

. .
Evolutionary behavior learning (rule base)

- Number of rules exploses rapidly
- Two strategies for reducing this number:
Don’t care conditions and default rule
- Evolution can be used to find a minimal (or fixed size) rule base,
- Three main approches to evolutionary behavior learning
Michigan Pittsburgh Iterative Rule Learning

Individual = One rule Individual = Entire system Evolution finds the best rule
(rule base or knowledge base)
Population = Rule base Incremental construction of
(i.e. fuzzy system) Population of systems the knowledge base
R2
R1
R3
Ri
R2
R1
R3
Ra, Rb, Rc ...
Ri

. .
Evolutionary behavior learning: An example

- 4 input variables, 5 membership functions per variable
{Tiny, Small, Normal, Big, Huge}
- Space of 625 rules (1295 including don’t care conditions)

IF V1 is Tiny AND ... AND V4 is Normal then Out = Huge
- Evolution searches for a subset of N rules (fixed by the designer)

- Genome encodes rules: Antecedents and consecuents
N rules * 15 bits
R1 R2 .... Ri .... Rn
15 bits
A1 A2 A3 A4 C
5 * 3 bits
5 functions + 3 don’t care

. .
Evolutionary knowledge base learning

(Knowledge base = rule base + database)
Parameter Modeling Usual Type of Fuzzy system

class type quantity values attribute
Connective Behavior learning 10 - 1000 Symbolic Rule base
Operational Knowledge tuning 10 - 1000 Real-valued Database
Critical issues for applying evolution:
- Parameter representation
- Tight interdependency
- Size of the search space
- Computation requirements

. .

- A basic approach: Single population, single evolution
Example: Breast cancer diagnosis problem (Peña and Sipper 99)

- 9 inputs, 1 output, 2 membership functions per variable
- A simple genetic algorithm searches for the knowledge base
- Genome encodes: Rule antecedents and membership function parameters
9 Variables * 6 bits Nr rules * 18 bits
V1 V2 .... Vi .... V9 R1 R2 .... Ri .... Rn Low High
6 bits 18 bits
P d A1 .... Ai .... A9
3 bits 3 bits 9 antecedents * 2 bits

P d
Sample ru;e: IF V1 is Low and V4 is High THEN Diagnostic is Benign

. .

- A variation: Single population, double evolution
Example: Evolving fuzzy rule based classifiers with GA–P (García et al. 99)
- Genome encodes: Complete rule base and membership function parameters
- A simple genetic algorithm searches for the database
- The rule base is evolved using genetic programming
1 0 0 1 0 0 0 1 1 0 0
Numeric part: Database Symbolic part: Rule base

. .

- Hybrid learning: Evolved rule base, learned database
Example: Breast cancer diagnosis (J.-F. Philagor, student project SPG, 1999)
- Evolution searches for a fixed-size rule base
Genome encodes rules: Antecedents and consequent
- Database is tuned using a neuro-fuzzy approach
A fuzzy self-organizing map searches for P and Q values
X2
R1 R3
N rules * 19 bits
Normal Big
Q3
R2
R1 R2 .... Ri .... Rn
Q2
19 bits
Small
Q1
A1 A2 ... A9 C
X1
9 * 2 bits + 1 bit P1 P2 P3
Low Mid High

. .
The Wisconsin Breast Cancer Database
The features
The test
The database

. .
Proposed Fuzzy System
R1: if (V1 is Low) and (V2 is High) and ... and (V9 is Low) then (output is Benign)
R2: if (V1 is Low) and (V2 is Low) and ... and (V9 is None) then (output is Benign)
...
...
...
...
...
else (Output is Malignant)
Low High Low Benign
.....
Low Low None Benign
.....
V1 V2 V9
Malignant
P1 P1+d 1 P2 P2+d 2 P9 P9+d 9

. .
Genome encoding
V1 V2 .... Vi .... V9 R1 R2 .... Ri .... Rn

6 bits 18 bits
P d A1 .... Ai .... A9
3 bits 3 bits 9 antecedents * 2 bits
Ai = 1 (Benign)
P = [1;8]
Ai = 2 (Malignant)
d = [1;8]
Ai = 0 or 3 (Variable not assigned)
Total genome length = 54 + 18Nr bits

. .
Fitness function
Fc : Classification performance,
the most important performance measure
Fv : Number of variables
F = Fc + a* Fv + b*Fe measures the interpretability
Fe : Quadratic error
selection pressure to fine tune parameters

. .
Results: Classification performance (Number of variables)
Setiono Taha Peña This

Rules Setiono (96) Liu (96) Ghosh (97) Sipper (98) work (99)
1 95.42% (2) 96.35% (3) 97.07% (4)
2 96.65% (7) 97.36% (3)
3 97.14% (4) 97.21% (4) 97.80% (4.7)
4 97.80% (4.8)
5 96.19% (1.8) 97.51% (3.4)
Learned Boolean rules Evolved fuzzy rules

. .
The best single-rule system
Low Low Low Low Benign
v1 v2 v6 v8
Malignant
IF the clump of cells is not very thick,
AND the cell’s size is quite uniform,
AND there are few bare nuclei,
AND nucleoli are not highly abnormal,
THEN the case is benign;
ELSE the case is malignant.

. .
Proposed approach: two elements

1 A system model: Fuzzy systems
2 A building algorithm: Cooperative coevolution
Database Fuzzy
CoCo

. .
Fuzzy modeling: a coevolvable problem
Parameter Modeling Usual Type of

class type number values
Logical System design 3 − 10
Structural (size) Structure learning 5 − 20 Integer
Connective (rules) Behavior learning 10 − 1000 Symbolic
Operational (labels) Knowledge tuning 10 − 1000 Real−valued
The required solutions can be very complex, These features render

they can be decomposed in distinct components, fuzzy modeling an
represented by different types of values, adequate target for
and which are very interdependent. COOPERATIVE
COEVOLUTION

. .
Fuzzy CoCo:
A cooperative coevolutionary approach to fuzzy modeling
Membership functions Rules Two evolutionary algorithms

searching for:
Evaluation Evaluation membership functions
and rules.
Selection Selection Advantages:

− Divide−and−conquer strategy
− Better search power
Modification Modification − Lesser computational cost
− More−flexible setup

.
Fitness evaluation in Fuzzy CoCo

1. Cooperators for generation g MFs Rules
are selected from generation g−1 Fitness
both fitness−dependent and
randomly Cooperators
Rules MFs
g−1
Cooperators
Generation
Fitness
2. Individuals are combined with cooperators
g to form fuzzy systems.
3. These fuzzy systems are evaluated, and
Selected cooperators individual fitness is then calculated.

. Logic Systems Laboratory − Swiss Federal Institute of Technology Lausanne
. .
Interpretability strategies in Fuzzy CoCo

● Shared membership functions: reinforced by the existence of a separate species
● Normal, orthogonal membership functions: constrained representation
● Don’t care conditions: encourage shorter rules
● Default rule: guarantees complete coverage of the input space
● Linguistic fitness: when used, increases selective pressure for interpretability
Cold Cool Warm Hot RB

1
R5 RA
R0
Temperature
0

. .
Fisher’s Iris Data
Case SL SW PL PW Class
The database
The variables
Features Classes
(1) SL Sepal length (1) setosa 1 5.1 3.5 1.4 0.2 Setosa
(2) SW Sepal width (2) versicolor 2 4.9 3.0 1.4 0.2 Setosa
(3) PL Petal length (3) virginica
(4) PW Petal width 150 5.9 3.0 5.1 1.8 Virginica

. .
Iris: Variable analysis

. .
Iris proposed solution: Controller−type
Input Fuzzy Subsystem Class Stair−function Class

Knowledge base
Database
Subsystem
Rule base
Inference
estimation
Crisp Defuzzifier Crisp
Fuzzifier Engine
Input Fuzzy Fuzzy Output
Input Output
The fuzzy subsystem estimates a continuous "class" value
The selection unit approximates it to the nearest class

. .
Iris controller−type: Proposed Fuzzy System

R1: if (SL is A11) and (SW is A12) and (PL is A13) and (PW is A14) then (output is Class1)
R2: if (SL is A21) and (SW is A22) and (PL is A23) and (PW is A24) then (output is Class2)
...
...
...
...
...
Rn: if (SL is An1) and (SW is An2) and (PL is An3) and (PW is An4) then (output is Classn)
else (Output is Class0)
Low High Low setosa
.....
1 2 3
Medium Low None versicolor
.....
1 2 3
SL SW PW
virginica
P11 P21 P31 P12 P22 P32 P14 P24
1 2 3

. .
Iris proposed solution: Classifier−type
Input Fuzzy Subsystem µ (setosa) Maximum and

Class
Knowledge base Threshold Subsystem
Database
µ (versicolor)
Rule base
Crisp Inference
Defuzzifier Crisp
µ (virginica)
Fuzzifier Engine
Input Fuzzy Fuzzy Output
Input Output
The fuzzy subsystem estimates a continuous membership

value for each class
The selection unit chooses the most active class

. .
Iris classifier−type: Proposed Fuzzy System

R1: if (SL is A11) and ... and (PW is A14) then (setosa is Yes),(versicolor is No),(virginica is No)
R2: if (SL is A21) and ... and (PW is A24) then (setosa is No),(versicolor is Yes),(virginica is Yes)
...
...
...
...
...
Rn: if (SL is An1) and ... and (PW is An4) then (setosa is No),(versicolor is No),(virginica is Yes)
else (setosa is No),(versicolor is Yes),(virginica is No)
setosa versicolor virginica

Low Low
.....
No Yes No Yes No Yes
Medium None
.....
SL PW No Yes No Yes No Yes
P11 P21 P31 P14 P24
No Yes No Yes No Yes

Iris: the genomes
. .
Membership functions Rules (Controller/Classifier)

SL SW PL SW
P R1 ... Ri ... Rn Co
3 x 5 bits 10/11 bits 2/3 bits
P1 P2 P3 A1 .... A4 Ci
5 5 5 4 * 2 bits 2/3 bits
Genome length = 60 bits Genome length = 10/11*Nr + 2/3 bits

. .
Iris: Fuzzy CoCo set−up

1. Fitness function
the most important
Fc * Fmb
F= { (Fc + a*Fv) * Fmb
Fm : 1 − mse (mean square error)
encourages not−so−bad errors
measures the interpretability
2. Fuzzy CoCo parameters

Population size {60, 70}
Maximum generations 500 + 100*Nr
Crossover probability 1
Mutation probability {0.02, 0.05, 0.1}
Elitism rate {0.1, 0.2}
"Fit" cooperators 1
Random cooperators {1, 2}

. .
Iris results: classification (average rule)

Simple GA FuGeNeSys Fuzzy CoCo
Controller Rules Shi et al (1999) Russo (1998) ICANNGA’01
2 99.33% (2)
3 100 % (1.7)
4 98 % (2.6) 100 % (2.5)
5 100 % (3.3)
Constructive Learning Methods Neurofuzzy Fuzzy CoCo

Rules Hong (00) Wu (99) Hung (99) ICANNGA’01
Classifier
2 98 % (1.5)
3 96.2 % (4) 99.33% (2.3)
4 97.4 % (4) 99.33% (2)
8 97.3 % (2)

. .
Iris controller−type: A three−rule system
SL SW PL PW Class
SL SW PL PW Class
SL SW PL PW Class
Class

. .
Iris classifier−type: A three−rule system
SL SW PL PW setosa versic. virgin.
setosa versic. virgin.

. .
Breast cancer diagnosis: the WBCD problem
The database
The test
Fuzzy Subsystem Threshold Subsystem Diagnostic

Proposed
Input Appraisal
solution
malignant
benign

. .
The genomes
Membership functions Rules

V1 V2 .... Vi .... V9 R1 ... Ri ... Rn Co

6 bits 19 bits 1 bit
P d A1 .... A9 Ci
3 bits 3 bits 9 * 2 bits 1 bit
Ai = 1 (Low)
Ci = 1 (Benign)
P = [1;8] Ai = 2 (High)
Ai = 2 (Malignant)
d = [1;8] Ai = 0 or 3 (None)
Genome length = 54 bits Genome length = 19*Nr + 1 bits

. .
Fuzzy CoCo set−up

Fitness function
the most important performance measure
F = Fc − a* Fv
measures the interpretability
Fuzzy CoCo parameters

Population size [30−90]
Maximum generations 1000 + 100*Nr
Crossover probability 1
Mutation probability [0.02−0.3]
Elitism rate {0.1−0.6]
"Fit" cooperators 1
Random cooperators {1,2,3,4}

. .
WBCD results: classification (longest rule)
Rules NeuroRule Fuzzy−genetic Fuzzy CoCo − IEEE TFS 2001

Setiono (2000) AIM 1999 Average Best
1 97.36% (4) 97.07% (4) 97.36% (4.0) 97.36% (4)
2 97.36% (3) 97.73% (3.9) 98.54% (5)

3 98.10% (4) 97.80% (6) 97.91% (4.4) 98.54% (4)
4 97.80% (−) 98.12% (4.2) 98.68% (3)
5 98.24% (5) 97.51% (−) 98.18% (4.6) 98.83% (5)
6 98.10% (−) 98.18% (4.3) 98.83% (5)
7 97.95% (−) 98.25% (4.7) 98.98% (5)
Learned Boolean rules Evolved fuzzy rules

. .
Two−rules evolved system
if (v1 is Low) and (v3 is Low) and (v5 is Low) then (output is Benign)
if (v1 is Low) and (v4 is Low) and (v6 is Low) and (v8 is Low) and (v9 is Low) then (output is Benign)
else (output is Malignant)
v1 v3
v4 v5 v6 v8 v9
Classification rate = 98.54%

. .
Computing requirements
Fuzzy GA: Single population (Peña & Sipper 99)
Number of fitness evaluations = Np * Gmax

200 * (2000 + 500Nr)
Single-rule systems: 500.000 fitness evaluations
Five-rule systems: 900.000 fitness evaluations
Fuzzy CoCo: Cooperative coevolution (CEC-2000)
Number of fitness evaluations = 2 * Np * Gmax * (Ncf + Ncr)

32.000 * (1000 + 100Nr) {worst case, Ncr=3}
Single-rule systems: 352.000 fitness evaluations
Five-rule systems: 480.000 fitness evaluations

. .
The problem: mammography interpretation
mammogram COBRA system:

reading computer−assisted
protocol case interpretation
Database
{ 516 readings
187 malignant (pos)
329 benign (neg)
biopsy
recommendation

. .
COBRA system: internal view
Web−based user interface
Reading form Reading

Fuzzy system Threshold unit Biopsy
Malignancy
input appraisal Proposal
Diagnostic decision unit
Database

Understanding the database
. .
Variable type Number

Binary 4
Continuous 3
Discrete 8

. .
Genome encoding for linguistic labels
Ri: if (v1 is Ai1) and (v2 is Ai2) and (v3 is Ai3) and ... and (v15 is Ai15) then (output is Ci)
Low High None Benign Malignant
V1 V3 ..... V15
P1 P’1 P3 P’3 P15 P’15
Binary variables (e.g., V2):

Pi P’i not encoded
Continuous variables (e.g., V1):
V1 V2 .... Vi .... V15
3 var. x 2 par. x 7 bits = 42 bits
Discrete variables (e.g., V3):
8 var. x 2 par. x 4 bits = 64 bits
DB
Total genome length = 106 bits

. .
Genome encoding for rules

A4 A5 A6 A7 A8 A9 if Sr = 0
A1 A2 A3 + Radiological
Clinical
A10 A11 A12 A13 A14 A15 if Sr = 1
22 2 2 2 2 2 1 1
Ac1 Ac2 Ac3 Ar1 Ar2 ... Ar6 Sr C
20
20 1
R1 ... Ri ... Rn Co
RB Total genome length = 20 x Nr +1

. .
Performance measures and fitness function

TP
Sensitivity TP + FN
Basic fitness (Fbase)
Sensitivity +α Specificity
Specificity TN 1+α
TN + FP
TP + TN Accuracy reinforcement
Accuracy
TP+TN+FP+FN
Fbase + β Accuracy
1+β
TP (note: done only if Accuracy > 0.7)
PPV
TP + FN

. .
Fuzzy CoCo results on 65 runs

25
Average per class
22
Nr Fitness Reff Vr
20 10 0.8754 9.17 2.52
15 0.8786 12.03 2.62
20 0.8934 14.15 2.59
15 14 14 25 0.8947 15.78 2.76
10
Best individual
Nr Fitness Reff Vr
5 10 0.8910 9 2.22
5 4
3 15 0.8978 12 2.50
1 2 20 0.9109 17 2.41
0 25 0.9154 17 2.70
0.82 0.83 0.84 0.85 0.86 0.87 0.88 0.89 0.9 0.91 0.92

. .
Performance of two selected systems
17−rule 9−rule
Measure Figure Ratio Figure Ratio
Sensitivity 99.47% 186/187 98.40% 184/187
Specificity 68.69% 226/329 64.13% 211/329
Accuracy 79.84% 412/516 76.55% 395/516
PPV 64.36% 186/289 60.93% 184/302

. .
The 9−rule system with two different thresholds
Threshold = 2 Threshold = 3
Measure Figure Ratio Figure Ratio
Sensitivity 100.0% 187/187 98.40% 184/187
Specificity 63.22% 208/329 64.13% 211/329
Accuracy 76.55% 395/516 76.55% 395/516
PPV 60.71% 187/308 60.93% 184/302

COBRA system: reading form
. .


Fuzzy Inference

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Fuzzy Inference

Transféré par

Droits d'auteur :

Formats disponibles

Fuzzy inference

Cold Warm Hot

Cloudy Partial Sunny

Low Medium High

Fuzzy inference systems

Crisp Inference Crisp

Fuzzyfier: translates crisp inputs into fuzzy values

Carlos Andres Pena Reyes

Network−like view of a fuzzy system

Carlos Andrés Peña−Reyes

Membership function values

Carlos Andrés Peña−Reyes

Carlos Andrés Peña−Reyes

Carlos Andrés Peña−Reyes

Parameters of a fuzzy system

Carlos Andrés Peña-Reyes

The general modeling problem

How do you search?

Have you preferences or restrictions to the model?

Carlos Andrés Peña−Reyes

Search space in fuzzy modeling

Structural parameters: R1, ... , Rn

Carlos Andrés Peña−Reyes

Search methods: Fuzzy modeling techniques

"Classic" identification methods

Machine learning approaches

Evolutionary fuzzy modeling techniques

Carlos Andrés Peña−Reyes

Usual constraints in fuzzy modeling

How is the system expected to do it?

Who is going to interact with the system?

Carlos Andrés Peña−Reyes

This approach is also called knowledge engineering

Domain data Building algorithm

This approaches are also denominated

Rule−specific MFs are not allowed

Orthogonal MFs with well defined

Carlos Andrés Peña−Reyes

Interpretability considerations: semantic criteria

● Distinguishability: Each linguistic label has semantic meaning

Carlos Andres Pena Reyes

Interpretability considerations: syntactic criteria

● Completeness: for any input, at least one rule must fire

Carlos Andres Pena Reyes

Strategies to satisfy interpretability criteria

Cold Warm Hot RB

Carlos Andres Pena Reyes

Fuzzy modeling: some data−driven approaches

Identification−based Fuzzy system Neuro−fuzzy systems

ANN−like training algorithm

Carlos Andres Pena Reyes

Faculty of Computer Science

A neural network is used to determine

The heuristic learning procedure operates on local

The learning process is not knowledge based, but data

However, this neural network view is NOT a prerequisite for

Individual ( Fuzzy) System

Environment = Selection Performance = Selection

Linguistic variables Rules

Numeric values Symbolic values

How to deal efficiently with them?

Evolutionary knowledge tuning (database)

X2 - Fixed rule base (completness)

if X1=Low and X2=Normal then Output = Ci