Vous êtes sur la page 1sur 20

Intelligent

Systems
Robert J. Schalkoff
Clemson

JONES

AND BARTLETT PUBLISHERS

Sudbury,
BOSTON

University

TORONTO

Massachusetts
LONDON

SINGAPORE

Principles,

Paradigms,
Pragmatics

and

Contents

Preface

Introduction to
1.1

1.2

1.3

xxv

Intelligent Systems

(IS)?

What Is Intelligent Systems

1.1.1

Key Topics

1.1.2

IS: A Practical Definition?

1.1.3

Some

Significant Preliminary Questions

1.1.4

IS: A

Psychology-Based Viewpoint

1.1.5

Two

Key

or

Themes

Approaches

Events and

to IS

History

Development

of Artificial

Intelligence/Intelligent Systems (AI/IS)

....

1.2.1

A Timetable of

1.2.2

Grand Research Challenges in Computer Science and Engineering Related

Key

An Elusive Model Yields

Events

an

f>

Elusive Goal

to

IS

7
!)

1.3.1

The Mind and

1.3.2

Puzzle

1.3.3

Data

1.3.4

Semantic Computing

11

1.3.5

IS and Game

II

(Versus?)

Solving (Riddles)
Versus Knowledge

the Brain
and

E>

10

Intelligence

11

Theory

1.4

Distinguishing Characteristics of IS Problems

12

1.5

The Turing Test

14

1.6

Where Is the Field of IS

Tools for IS

1.7

Quantitative

1.8

Organization of the Text

1.9

Practice

First
2.1

2.2

Steps

or

Introductory

in IS:

14

Going?

Development

15
16

Exercises

17

Representation, Ontologies, and Obtaining Expertise

Representation

in IS

19
19

and Related Definitions

2.1.1

Knowledge Representation

2.1.2

Human

2.1.3

Declarative Versus Procedural

Reasoning: Classes, Categories, Concepts,


Representations

and

19

Chunking

19
20
20

Tools for Representation

vi

Contents

2.2.1

Discrete Mathematics:

2.2.2

Digraphs

2.2.3

Semantic Net

2.3

The Semantic Web:

2.4

Structuring

2.5

2.6

Using Relations

and

Properties

20

to Semantic Nets

21

Examples

Concepts

21

and

the

Relationships,

Not Just Data

or

Words

Frames and

Representation:
Object-Oriented Representations
Slots, "is-a," and Inheritance

2.4.1

Frame

2.4.2

Object-Oriented Approaches

Ontologies: Concept

24

26

Background and Motivation

2.5.2

What Is

2.5.3

Ontologies and Knowledge Bases

27

2.5.4

Standardization of

27

The

an

Ontology

Protege Ontology

and

26

Why

Ontology

Do I Care?

26

Formats

Editor

30

Protege Examples

31

Alternate Representation Formats Using Protege


Ontological Reasoning and "Common Sense" Implementations

33

2.7.1

CYC

33

2.7.2

Common Sense

(OpenCYC)

(MIT)

33

35

Expert Systems

35

2.8.1

What Is

2.8.2

Rule-Based

2.8.3

How

2.8.4

The

2.8.5

The

an

Long

Expert System?
Implementations

35

38

Does It Take to Become

Appeal

of

an

Expert?

38

Expert Systems

39

Expert System Knowledge Engineering Process


Production Systems for Expert System Implementation

2.8.6
2.9

23

24

and Tools

2.6.2

2.8

....

2.5.1

2.6.1

2.7

22

Exercises

39
42

42

Search and Computational Complexity in IS

45

3.1

Why

45

3.2

The
3.2.1

3.3

Do We Care about IS Problem

Computational Complexity?
Concept of a System State Space
System State Representation

3.2.2

The

3.2.3

Satisfying Goals

3.2.4

Nondeterminism: Unreachable States

State

Space

via State

45
46

Manipulation

with

Operators

47
47

Computational Complexity
3.3.1
Complexity Functions
3.3.2

3.4

System

45

48

48

An Example of the State Space and IS Problem

Search

Complexity

49
51

3.4.1

To Search

3.4.2

Problem

3.4.3

Definition of

3.4.4

State-Space Graphs

52

3.4.5

Search Costs and Heuristics

53

or

Not to Search

51

Representation
a

Search Problem

51

(P)

52

Contents

3.4.6
3.4.7

Graphical Representations for Search


Example: A Graphical Representation

3.4.8

From

3.4.9
3.5

3.6

3.7
3.8

Another

Example:

3.5.1

Example

3.5.2

Propagation

Graphical

The

of Search and

Explosion

53

55

Representations

55

Representation of Search

57

State-Space

57

Paths

57

of Operator Choices

59

Heuristics

59

3.6.1

What Is

3.6.2

Definitions

59

3.6.3

Applications

59

3.6.4

Sample

3.6.5

Why

3.6.6

The

3.6.7

Machine

Heuristic?

60

IS Heuristics

60

Use Heuristics?
and

Power, Application,
Learning

The Computational

Properties of

61

Heuristics

61

of Heuristics

Informed Versus Uninformed

("Blind")

62

Search

62

Cost of Search

Computational Complexity of Forward Propagation


Backward State Propagation
The

3.8.2

Common Search

of States

Other Search

65
66

3.11

Exercises

Approaches (Informed Search)

3.10

63

65

Brute Force

3.9.2

63

65

Algorithms

("Uninformed") Algorithms
Search
Systematic
Strategies

3.9.1

68

Constraint Satisfaction
4.1

53
of State

Searching the Problem State Space

3.8.1

3.9

Graph

to Tree

vii

Problems,

71

Part 1

Nonstructural Constraint Satisfaction Problems

71

(CSPs)

71

4.1.1

4.1.2

Another CSP Example: Numerical Constraint Satisfaction

72

Formal Definition

73

4.1.3

Simple Introductory Example: Map Coloring

Prepositional Satisfiability (SAT)

73

Problems

4.2

CSPs

4.3

SAT: Definition, Properties, and Solutions

74

4.3.1

/c-SAT

74

4.3.2

SAT Solutions

74

4.3.3

SAT

74

as

Properties

(Somewhat) Obvious

Solution to CSPs: Generate-and-Test

4.4

4.5

Constraint Programming and

4.6

The
4.6.1

4.6.2

Role for

Problem

Labeling (Assignment)
Labeling Complexity

4.6.3

Image Labeling Example


CSP Solutions in Prolog

4.6.4

Adding Unary

4.6.5

Adding

Constraints

Global Constraints

Prolog

in CSPs

(GAT)

75
75

76
76
76

77
78

79

viii

Contents

Adding Optimization

4.6.6
4.7

The
4.7.1

Background

4.8

and

Prolog-Based

4.7.2

to the Formulation

82

Problem

Map Coloring

85

85

History

Solution

Another Framework for CSP

Approaches
Assignment Problems and Alternative Solution

Approaches

88

4.8.1

A General

4.8.2

Enforcing

Representational Framework

4.8.3

Iterative CSP Solutions via Constraint

88

Object Label Compatibility

89
Discrete

Propagation: Introducing

Relaxation
4.9

4.10

Investigating

90

Solution Efficiencies

98

4.9.1

Considering

4.9.2

GAT

99

4.9.3

GAT with Heuristic Guidance

99

4.9.4

Backtracking

4.9.5

Constraint Propagation

Search

98

99
100

4.10.2

Continuous/Probabilistic Formulations
"Relaxing" and Extending Several Aspects of
An Algorithm for Propagating P,:(A)

4.10.3

Examples and Extensions

Extension to

4.10.1

4.11

Summary

4.12

A More

100
Discrete Relaxation

100
102
103
108

4.12.1

Comprehensive CSP: Electric Power System


Background and Motivation

4.12.2

Objectives

4.12.3

Simple

4.12.4

Problem Formulation

4.12.5

4.12.6

Prolog Description
Breaker Backup

4.12.7

Detailed

Protection

Analysis

108

108
108

Power Protection

of

Background

109

109

System Topology

110
Ill

Prolog Description Derivation

Listing

of Overall

4.12.9

Sample

Results

4.12.10

Summary: Expert System

4.12.8

Ill

Prolog Program

117

120
for Electric Power

System Protection

Interpretation

86

121

4.13

Extensions

121

4.14

Exercises

121

CSPs,

Part 2: Structural

Understanding
5.1

Introduction

5.2

What

5.3

Natural

5.4

Early

Types

to Natural

Topics

Language
129

129
of Problems Have Structural Constraints?

Language

NL

Approaches Leading

and Related

and Structure

Understanding Efforts

129

132
132

5.4.1

McCarthy's Proposal

132

5.4.2

ELIZA

133

Contents

Natural

5.5

5.6

Example:

5.5.2

The Phase Structure

5.5.3

L(G)

5.5.4

Grammar Modes

5.5.5

Grammar

Types

Syntax, Semantics,

5.5.7

Graphical Aids

138

and Context

139
13!)

Recognition Using Prolog

140

5.6.1

The Initial Grammar Model

5.6.2

Grammar Gl

5.6.3

Sample

5.6.4

Showing Structure and Extending the Language

142

5.6.5

Enforcing Meaningful Semantics

144

Feature Structure-Based

Simple

140
140

Results for

5.8

Simple

Grammar

Parse to Structure to

141

148

Meaning

Representation

and Manipulation

149

Motivations

149

5.8.2

Feature Structures

5.8.3

Feature Structure-Based

5.8.4

Feature Structure Representation Enhancements

157

5.8.5

Rules

158

5.8.6

Lattices of Feature Structures

158

5.8.7

Lattice

159

as

and Basic

(FS)

149

Properties

153

Manipulation

Feature Structures

Processing

Directions

Exercises

From

6.3

138

Sentence Generation and

5.9

136
137

and Productions

5.5.6

From

6.2

(Rewrite Rules)

(String) Grammar

138

5.7

6.1

136

Sentence Formation As Productions

5.5.1

5.8.1

Language Understanding: Grammars

ix

161

Logic-Based Chaining

to Production

167

Systems

Conceptual Background and Motivation


Logic in IS Representation and Manipulation

167

The Role of
6.2.1

Inference and the

6.2.2

Other

Logic-Based

Logic

Vocabulary

of

168

Families

169

Underpinnings of Rule-Based

6.3.1

The Implication Connective

6.3.2

An

6.3.3

Rule-Based

6.3.4

Business Rules and

Implication-Based IS

Rule

Chaining Systems

169
170
170

Representation Examples

Engines

and Inference:

171

Simple Modus Ponens

Implication

6.3.6

Modus Ponens with Variables

6.3.7

The

6.3.8

Resolution

6.3.9

Mechanical Implementations of Logic Can Lead

Chaining,

6.5

Implementing

Directions,

Rule-Based

and Potential

(MP)

172
172

Concept and Process of Unification


(Proof by Refutation): An Alternative

Inference

169

Example

6.3.5

6.4

167

Logic

172
to

to

MP

Illogical Results

Complexities

in

Chaining

Chaining

178
178
180

6.5.1

The Role and Design of the System Inference Engine

6.5.2

Computational

Elements of Rule-Based

176

Chaining

and

180

Examples

181

Contents

6.6

Extensions to the

Measures,

6.7

Rule

6.6.2

Possible Conflict Resolution

6.6.3

Conflict Resolution

6.6.4

Conflict Resolution Computational Aspects and Tradeoffs

181

Strategies

182

Examples

183

185

6.7.1

185

6.7.2

Programming Paradigms and Production Systems


A More General Viewpoint: Rules Trigger Actions

6.7.3

Production Systems Architecture

186

6.7.4

Features of Rule-Based Production

6.7.5

Production

Exercises

Systems

7.5

187

Systems:

The Next

188
189

Step

190
190

The C Language Integrated Production System

7.4

185

System Properties
Decomposable Production Systems

Production

7.3

185

The Production System Paradigm

6.9

7.2

Resolution

181

The Conflict Set

6.8

7.1

Selection, Conflict

6.6.1

6.7.6

Simple Chaining Paradigm:

and the Conflict Set

(CLIPS)

195

CLIPS: Conceptual Background and Motivation

195

7.1.1

CLIPS History

195

7.1.2

CLIPS Structure

195

7.1.3

CLIPS Nomenclature

196

7.1.4

Simplified CLIPS Execution Cycle

196

7.1.5

CLIPS Documentation

196

7.1.6

Introductory

7.1.7

Logging

7.1.8

CLIPS

CLIPS

Examples

197

CLIPS Session

201

Help

202

Selected Syntax, Constructs, and Examples

202

7.2.1

Predominant CLIPS Constructs

202

7.2.2

A Third

ExamplePrelude

to

an

Expert System

210

CLIPS Conflict Resolution and the agenda


7.3.1
Controlling Production System Computation

210

7.3.2

The agenda

212

7.3.3

User-Defined Rule Salience

213

7.3.4

the Conflict Set

Forming

7.3.5

Available CLIPS Conflict Resolution

7.3.6

Conflict Resolution Examples

Another Expert System

210

(agenda)

215

Strategies

Application Example:

215
216

"Tenure"

216

7.4.1

An Application Domain

216

7.4.2

The

219

7.4.3

Initial CLIPS

Development

Process

7.4.4

Expert System Development Example


Enhanced CLIPS Development of the Previous Example

7.4.5

A CLIPS

7,5.1

Motivation

Planning Example: The Monkey


CLIPS Efficiency and the Rete Algorithm

and Bananas

219
221

(MAB)

Problem

...

225
226
226

Contents

7.5.2

Computational Complexity

7.5.3

The Computational Model

7.5.4

The''Traditional"

7.5.5

Rete

7.5.6

Productions and

7.5.7

How Is Rete Implemented?

7.5.8

Exploring

7.5.9

Writing

Efficient CLIPS

Beyond

Rete

7.5.10

227
of Production

227

System Operation

229

(I) Background
Data: Who's In

the Rete

Charge?

22!)

231

Algorithm

with CLIPS

234
23(i

Programs

238
238

Embedded CLIPS

7.6

7.7

7.6.1

Embedding

7.6.2

Sample

Production

CLIPS

238

System

Embedding (Using C)

238
241)

Descendants of CLIPS
7.7.1

227

Viewpoint

wm

240

JESS

7.7.2

Clips

7.7.3

Soar

xi

in Java

242
242

7.8

CLIPS Web Resources and Downloads

242

7.9

Exercises

242

Extended Production

System Representation

and

Manipulation Approaches,
253

Including Agents
8.1
8.2

8.3

Modular Production System Development Approaches

Implementing

Representations

253

in CLIPS

25 1
254

8.2.1

Introduction

8.2.2

Module Definition

8.2.3

Specifying

8.2.4

Examples

8.2.5

Module

8.2.6

Example: Changing Focus

251

Syntax

the Module for


of Module

25 I

Construct

254

Specifications: Different Knowledge Domains

25(i

Visibility

257
Production

Object-Oriented (00) Representation,

Systems,

and CLIPS COOL

....

258

8.3.1

COOL Class

Syntax

258

8.3.2

Sample Class Hierarchy in COOL


Examining the COOL Classes

258

8.3.3

8.4

Modular

Creating

8.3.5

Integrating Classes, Instances, and Productions in COOL

Protege, OO-Based Ontologies, CLIPS,


8.4.1
Protege Representation Files

and COOL

8.4.2

Sample Ontology

8.4.3
8.4.4

259
262

Instances

8.3.4

Graphical Representation

of the

262
266

266
266

Importing

*.pont Files into CLIPS

266

Importing

*.

pins Files into CLIPS

266

the

into CLIPS

8.4.5

Loading

8.4.6

Combining CLIPS Productions with the Protege Sample Ontology

8.4.7

Instance-Set

8.4.8

Protege, fuzzyCLIPS,

Protege Ontology

Queries

in COOL
and COOL

270

270
271

272

xii

Contents

8.5

8.6

IS

Representation/Knowledge

8.8

8.9

272

Consistency

Redundancy

272

8.5.2

Conflicts

272

8.5.3

Subsumed

Conditions

Unnecessary

8.5.5

Circular Rules

Nonmonotonic

272

Subordinate Rules

or

8.5.4

8.6.1

8.7

Base

8.5.1

273
273

273

Logic

Definition

273
of Situations

8.6.2

Examples

8.6.3

Soar and Nonmonotonic Reasoning

274

8.6.4

Nonmonotonic

274

Reasoning

with Time

Involving

Nonmonotonic

Reasoning

Formalisms

Reasoning

(Temporal Logics

and

Operators)

275

8.7.1

Characteristics of Temporal Reasoning

275

8.7.2

Subdivisions of Time

275

8.7.3

Time Intervals and Points

8.7.4

Possible

276
of RWT

Implementations

276

IS Collaboration: Blackboards
8.8.1

Blackboard Control

8.8.2

Typical Sequence

279
280

of Blackboard

Operations

Prom Production

Systems to Distributed,
an Agent?

Autonomous

280

Agents

281

8.9.1

What Is

8.9.2

Definitions

281

8.9.3

An

282

8.9.4

An

281

Example: NL Understanding

Agent
Agent Example: Network Monitoring

282

8.9.5

Agent Characteristics
8.10 Basic Agent Structures and Types of Agents

283

283

8.10.1

Generic Structure

283

8.10.2

Agent Types
BDI Agents

284

8.10.3
8.10.4

Theoretical

8.10.5

285

8.10.6

Agents and Rule-Based Systems


Living Organisms As Agents

8.10.7

Potential Applications for Agents

285

8.10.8

Multiple,

8.10.9

Software

8.10.10

274

Agent

284

Components of

Mobile

an

Agent

Agents
a Design Paradigm

and Distributed

285

285

Computing

Agents As

Software Standards

286
287
287

8.10.11 Tools for Agent-Based Development

287

8.10.12

288

Merging Multiple Agents

and Blackboards

8.11

References

288

8.12

Exercises

288

Soar
9.1

293
Soar Background

293

9.1.1

293

Soar History

Contents

IS Design and

9.1.2
9.1.3
9.2

9.3

Soar

Significant

294

Implementation Using Soar

Elements of Soar

and

Philosophy

Syntax, Computational Architecture,

and

xiii

Pragmatics

Examples

295

296
29(i

9.2.1

Soar

9.2.2

States and

Memory Architecture

9.2.3

Variables in Soar

297

9.2.4

Comments and Constants

298

9.2.5

Hierarchical Representations and

9.2.6

Production

Working Memory (wm)

Memory (pm)

9.2.7

Started with

Getting

9.2.8

Arithmetic in Soar

Digging Deeper

in Soar

297

298

Objects

and Production

Syntax

(Very) Simple Example

298
299
303

into Soar

303
303

9.3.2

Long-Term Knowledge in Soar


Simplified Soar Execution Cycle

9.3.3

Preference

305

9.3.4

Operator Preference

9.3.5

SOAR

9.3.6

Steps

9.3.7

States and State Elaboration

307

9.3.8

General Operator Definition

307

9.3.9

307

9.3.10

I-Support Versus O-Support


Example #2: Operator-Based State Initialization

9.3.11

Soar Operator Life Cycle

308

9.3.1

9.3.12
9.3.13

304

Memory

305

Preference

and Semantics

Operator
Syntax
Soar Operator Preference Resolution

in

305
306

308

Example #3: Operator


Example #4: Goal State Checking

309

View of the Soar Execution

313

Preference Use

309

9.4

Expanded

9.5

Another Soar Example: Revisiting the Image Labeling Problem

314

9.6

An

316

Example

9.6.1
9.6.2
9.7

9.8

of

Using

Soar for

"Lightbulb" Diagnosis
Operator-Based

Diagnosis

Soar Formulation and Results

9.7.1

Simple

Text

9.7.2

Communication Between Soar

An

9.8.3

Soar's

9.8.4

9.8.5

321

Output

Impasses, Subgoaling,
9.8.2

and

Learning

9.8.6

(Chunking)

Handling Impasses:

The

Soar

322

324
Resolution

Impasse
Impasse (Tie) Example

Chunks

321

322

Impasse

Debugging
9.9.1

Agents and the Outside World

Soar Is Different

Library
and Learning

Representations
Soar Visual Debugger

325

Activity

Available

Default

9.9

317
321

Input/Output

Why

317

Formulation

Soar

9.8.1

Cycle

325

Impasse

Resolution Productions in the


327
327
327
327

Contents

xiv

9.9.2
9.10

9.11

328

Additional Debugging Techniques

Soar, Ontologies,

and Other

Development

Aids

329

9.10.1

Background

329

9.10.2

Herbal

329

Soar "Under the Hood": So You

Really

Want to

Develop Embedded, Autonomous


329

Agents?
A Note About Different Soar

9.11.2

The Soar

9.11.3

The Soar Kernel

330

9.11.4

Agents and the Soar Kernel


C++Interfaces to Soar Using SML

330

9.11.5

10

Platforms/Implementations

9.11.1

Computational

329

Architecture

330

334

9.12

Soar Resources and References

344

9.13

Exercises

347

Representing and Manipulating Uncertainty in IS, Parti: Confidence


Factors, Probability,
10.1

Networks,

and Multivalued

353

Logic

353

10.1.1

Representing

10.1.2

Expressing Uncertainty in Statements


Approaches to Uncertainty

10.1.3
10.2

Belief

Introduction
and

Manipulating Uncertainty

353
353
354

Use and Limitations of Probability in IS

354

10.2.1

Conditional

354

10.2.2

10.2.3

Bayes Rule
Independence

10.2.4

Limitations of the Probabilistic Approach

10.2.5

A Simple Example of Probability Limitation Leading Up

Probability

355
355
356
to

Confidence

Factors

10.3

10.4

10.5

356

10.2.6

Joint

10.2.7

Bayes Rule

From

Probability (Conjunction) Propagation

Probability

for Information Fusion

to Confidence

357
358

Factors

359

10.3.1

Introduction

359

10.3.2

Confidence Factors

360

10.3.3

A Heuristic

10.3.4

360

10.3.5

Comparison of the Heuristic Approach with Probability


Positive and Negative Reinforcement in the Heuristic Approach

10.3.6

Extension of the Heuristic

362

10.3.7

Confidence Factors Are Implemented in CLIPS

Approach

360

Approach

to

OR Formulations

Belief Networks

361

362
362

10.4.1

Definitions

362

10.4.2

BN Example and Use

363

Multivalued

Logic

364

10.5.1

Review of Two-Valued Logic

10.5.2

Extension to Multivalued

364

10.5.3

364

10.5.4

Logics
Simple Heuristic Three-Valued Logic System Example
Formalizing MVL: Truth Sets and Truth Values

364

365

Contents

10.6
11

10.5.5

"Weak" Connectives in Multivalued

10.5.6

Using Multivalued Logic for Inference

and
11.1

36G

Logic

369

Exercises

Representing

370
and

Manipulating Uncertainty

in

IS,

Part 2:

Fuzzy Systems
373

FuzzyCLIPS

Fuzzy Sets and Fuzzy Logic

373

11.1.1

373

Examples

of

Fuzzy Concepts, Connotations, and Use

11.1.2
11.2

11.2.2
11.2.3

11.3.2
11.3.3

11.6

11.7
12

Membership

Functions

Membership

Resulting
The Extension Principle
Fuzzy System

375
Functions

375

from Set-Theoretic

37!)

Operations

379

Structure

380

380

Major Components

Linguistic
Fuzzy

380

Variables

Antecedents and Rules

381

381

Fuzzy System Design


11.4.1
Principal Tasks

Procedures

381

11.4.2

Fuzzy Computational

11.4.3

The Plethora of

11.4.4
11.5

Representing

Sets with

General
11.3.1

11.4

374

Sample Fuzzy Applications


Fundamental Fuzzy System Concepts

11.2.1

11.3

Mechanisms

382
383

Fuzzy CRI and Defuzzification Strategies


Applying Compositional Rules of Inference (CRI)

383

Fuzzy System Application (Control) Example

384

11.5.1

The Plethora of Defuzzification Techniques

385

11.5.2

Fuzzy Approach Shortcomings

387

Exploring Uncertainty

and

Fuzzy Concepts

with CLIPS

387
387

11.6.1

Introduction to FuzzyCLIPS

11.6.2

Implementing Confidence

11.6.3

Implementing Fuzzy Sets and Reasoning in CLIPS

Factors in

388

FuzzyCLIPS

391
413

Exercises

417

Planning in IS
12.1

xv

Introduction

417
418

12.1.1

Path

12.1.2

Blocks World

12.1.3

Planning

12.1.4

12.1.5

Planning Representations
Representation Choice: Explicit Representation

12.1.6

The Frame Problem in Planning

420

12.1.7

Developing

421

12.1.8

Plan Generation As

12.1.9

Planning

12.1.10

Cycles

Planning

418

Related to

Production

System

419
419
of

Everything

Representation
a

12.1.11 Unreachable States

421

Control Problem

Difficulties: Local Actions and Global

Allowing
419

Derivable Information

or

Objectives

422
422

423

Contents

xvi

12.2

12.1.12 Trade-offs

424

Representation and STRIPS


12.2.1
Operator Characterization in STRIPS
12.2.2 STRIPS Blocks-World Example

424

12.2.3
12.3

12.4

Another

Simple

Plan Generation

427

12.3.2

An Algorithm for FSP

427

Cataloging (Remembering)
The

Triangle

Example:

and

Re-Using

428

Plan

428

Table

Triangle

Table for

and Use of the

Properties
Planning, Abstraction,

430

Blocks-Moving Problem

Triangle

Table

430
432
432

12.5.2

Details, Details, and Defeating Search Complexity


Planning in Abstraction Space

12.5.3

Planning Using Subgoals Generated

Parallel Actions in

and

Subgoals

12.5.1

12.8

426

Algorithms

12.4.3

12.7

426

Generate-and-Test

12.4.2

12.6

425

(Robot) Example

STRIPS

12.3.1

12.4.1

12.5

424

from Abstract

433

435

Space

436

Planning

436

12.6.1

More Is Better?

12.6.2

Extending Operator Representations

to

Multiagents

or

Parallel Actions

....

CLIPS
Planning Implementations
12.7.1 The Sample Problem
12.7.2

The Desired Solution

440

12.7.3

Sample CLIPS Solutions

438
440
449

Soar

Planning Implementations Using


A Simple "Warm-Up" Problem: Using
12.8.1

Single Block and

Checkerboard
12.8.2

More Elaborate

450

Planning (and Soar) Using

4 Checkerboard and Two


457

Blocks
12.9
13

488

Exercises

493

Biologically-Inspired Computing and IS: Neural Networks (Part 1)


13.1
Conceptual Background and Motivation
13.2
13.3

Relationship
Biology
13.3.1

and ANN

Physical

493
494

Building Blocks

(Biological)

Neurons

494

Biological Mechanisms into Artificial Unit Characteristics

13.3.2

Abstracting

13.3.3

Two-Part Unit Models: Activation and Squashing

Sigmoid (Logistic) Squashing Function


Sigmoid Derivative
Multilayer Feedforward Structure, Architectures,

13.3.4

13.5

The

495
495

497
and Notation

Training
13.5.1 Training Considerations

Overview of the Generalized Delta Rule

....

496

The

the MLFF Network: The Generalized Delta Rule

13.5.2

493

of IS to ANNs

13.3.5
13.4

436
438

in

497

501
501

(GDR)

501

Contents

13.6

13.7

13.5.3

A Formulation

13.5.4

Output

13.5.5

A Modified Procedure for Hidden Units: The GDR

Unit

50 !

Introducing Delta
Corrections

Weight

50(i
507

MLFF ANN:

Internal/Hidden Layers and Network Mapping Ability


Many Hidden Layers Are Needed?

13.6.1

How

13.6.2

A Theoretical Result

Comprehensive

for Continuous Activation Functions

MLFF ANN

and

Training

xvii

Application Examples

513
513
51 I

51 1

13.7.1

Fruit Classification

511

13.7.2

Blocks Movement

516

13.7.3

Learning

an

A/D

Converter Function

518

13.8

Resources

51 i)

13.9

References

52(1

13.10 Exercises
14 Neural Networks

520
Networks and IS

(Part 2): Recurrent

Applications

527

14.1

Introduction

527

14.2

Basic Parameters and Recurrent Network Design

527

14.2.1

Network Parameters

528

14.2.2

Network

14.2.3
14.3

14.4

14.5

14.7

and Network

Weight Storage Prescription

529
and Behavior

530
531

Capacity

14.3.1

Weight Prescriptions

531

14.3.2

Additional Characterizations of the

532

14.3.3

Network

Capacity

Storage Prescription

Estimation

Recurrent Network

533
and Example CAM

Applications

14.4.1

Design Procedures
General Design Procedure

14.4.2

Another CAM

14.4.3

Storing and Accessing Stable States


CAM Example: Association of Simple Two-Dimensional

Recurrent Networks:

Example: Design

Energy Analysis

14.5.2

Example:

Summary

of the Generalized

4 Network

533
533

of

Simple Hopfield

Network
534

Patterns

Energy Function Characterization


or "Why Hopfield Networks Work"

14.5.1

14.5.3
14.6

Dynamics
Determining and Quantifying Network States

Energy

Function

Hopfield

Network

536
536
538
539

Equations

540

Recurrent ANN Constraint Satisfaction and Optimization Applications

540

14.6.1

Optimization

540

14.6.2
14.6.3

Mapping Constraints and Objectives


Other Interpretations of E

14.6.4

The Overall

14.6.5

Unit Characteristics and

and CSP Problems

Design

Example: Partitioning

Process

Weights

of Sets

Statement,

541
541
542
542
543

14.7.1

Problem

14.7.2

Problem Representation

543

14.7.3

Constraints

544

14.7.4

Network

Energy

Case

into Recurrent Networks

#1

Functions

543

545

xviii

14.8

Contents

14.7.5

Solving

14.7.6

Looking at W

for

14.7.7

Case

Weights

547

#2: Optimization

Measure and Associated E

State

Traveling

and the

Example:
Representation
Embedding Constraints

Constraints and

Mapping

14.8.3

Unit Characteristics and

14.8.4

The Role of the

14.8.5

Discrete

14.8.6

Sample Results
Other Sample CSP Applications

The Psychology

into E

Objectives

549

550

Weights
Hopfield Unit Bias

or Continuous

548
549

14.8.2

14.8.7

547

Salesman Problem

in E

14.8.1

14.9

546

and Biases

550

Unit Models

551
551
551

of "Recall" and ANN Bidirectional Associative

Memory (BAM)
551

Structures

14.9.1

The Order of

14.9.2
14.9.3

Discrimination, Association, and Principles


Relevant Principles of Association

14.9.4

BAM Evolution from Hopfield (Totally

14.9.5

Architecture

556

14.9.6

Training and Connections


BAM Examples

556

14.9.7

14.9.8

BAM

as

551

Thought

Partitioned

of Connection

(James)

552
552

Recurrent)

Structure

554

558

Bipolar Hopfield

and Other Connection Matrices

562

14.10 References

563

14.11 Exercises

564

15 Neural Networks

(Part 3): Self-Organizing Systems

569

15.1

Biological

15.2

Self-Organization via Clustering

569

15.2.1

Determining

569

15.2.2

Clustering Similarity

15.2.3

Clustering Complexity

15.2.4

Clustering Algorithm

15.3

Justification

The c-Means

569

"Natural Clusters"
Measures

570
570

Parameters

571

Algorithm

571

15.3.1

Algorithm Description

15.3.2

ID c-Means

571
and Visual

Simple
Example
Example: Clustering Digits (0,1,2,..., 9)
Self-Organizing Feature Maps (SOFM)

Interpretation

15.3.3
15.4

572

574
574

15.4.1

Introduction

574

15.4.2

Unit

574

15.4.3

Defining Topological Neighborhoods ("Bubbles")


Network Learning Algorithm

575

15.4.5

Network Coordinate SystemsTopological and Weight Spaces

577

15.4.6

Algorithm Properties

and Discussion

577

15.4.7

Example

15.4.8

Validation and

15.4.4

Topologies

Simple

1-D SOFM

Interpretation

576

578
of the

Resulting

SOFM

580

Contents

15.4.9
15.5

15.6

15.7

16

Another

Example:

2-D SOFM

(NG) Self-Organizing
Algorithm

The Neural Gas

Application

NG

15.5.2

How Is the NG Sorted

15.5.3

Engineering

15.5.4

NG Example

15.5.5

Validation of NG

15.5.G

Neural Gas Statistical Characterization

Growing

Neural Gas

Digits

581

Network

15.5.1

and

to 11 X 8

xix

583
583

Array Used?

Pragmatic

58-1

Concerns

58-1
585

Example

585
587

(GNG)

588

15.6.1

Algorithm Description

588

15.6.2

GNG

589

Example

"Batch" SOFM and NG

Approaches

589

15.7.1

Batch SOFM

590

15.7.2

Batch NG

590

15.8

References

591

15.9

Exercises

591

Learning in IS
16.1

595

Introduction to Machine
16.1.1

Learning (ML)

595

Preface

595

16.1.2

Where Do We Integrate Learning in

16.1.3

Learning

about

an

IS?

595

Learning

596

16.1.4

16.2

16.3

Learning History and Psychology


16.1.5 Learning Definition(s) and Examples of Automated Learning Paradigms
Case-Based Reasoning and Learning
of

598
599

Components

16.2.2

The Four "R"s of CBR

600

16.2.3

Case Representation Structure and Determining Case Similarity

601

16.2.4

Issues with CBR

601

16.2.5

Case-Based

Reasoning and the U.S. Legal System


Learning through Description Generalization/Specialization

602

16.3.1

602

16.3.3

System

16.2.1

16.3.2

600

and Specific-to-General

General-to-Specific (G-S)
Representations and Descriptions
The Training Sets

(S-G) Approaches

603

16.3.5
16.3.6

Description

16.3.7

Specific

16.3.8
16.3.9

Modification

(S-G) ("Opening"
Description) Approach
General-to-Specific (G-S) ("Narrowing" the Description) Approach
the

2-D Geometric

Example: Learning
Reasoning

16.3.11 Generalization

Learning

and

Operators

Decision Trees

603
605

605

to General

16.3.10 Inductive

602

603

Consistency and "Covering"


Algorithmic Goals for Generalization/Specialization-Based Learning

16.3.4

16.4

CBR

597
.

Descriptors

Truth/Falsity

Preservation

605
606

607
608
609

612

xx

Contents

16.4.1

Synopsis of

the ID3

613

16.4.2

Approach
Sample Exemplars (ID3 Input Data)

16.4.3

The Decision Tree: Definition and Tree

613

Components
(Information) Entropy

16.4.4

The General Concept of

16.4.5

The ID3

16.4.6

ID3

16.4.7

An ID3

16.4.8

Another ID3 Handworked

16.4.9

Using ID3 for Learning Block Movement Strategies


Learning, Training Data, and Generalization

613
614

615

Algorithm

618

Algorithm Examples

Application: Learning

621

Musical Structures

Example:

Results and Ramifications

629

16.4.10 ID3
16.4.11
16.5

16.6

Converting

632

Rule Set

640
640

Exemplars

640

16.5.2

ID3 Results

640

16.5.3

Validation of the Tree

16.5.4

Determination of

641

Resulting Classification

Rules

642

Exercises

642

Algorithms, Swarm Intelligence,

17.2

Genetic

Computation

649

Algorithms

649

650

17.2.2

Applications
Chromosomes, Genes, the Genome,

17.2.3

GA as

17.2.4

Critical Aspects of

17.2.5

Genetic Algorithm Simulation Parameters

Evolutionary Computing
649

Biology Suggests
17.2.1

and Other

in IS

17.1

17.5

16.5.1

Concepts

17.4

Decision Tree into

Integrating and Implementing ID3, Generalization, and Rule Determination

17 Genetic

17.3

625

Search and
a

and

Genotypes
Optimization Technique
Genetic

(Programming)

and

Phenotypes

650
651

Solution

653

654

Simple Genetic Algorithm Example

656

17.3.1

Problem Formulation

656

17.3.2

Objective
Selection, Crossover,

657

17.3.3
17.3.4

Sample Results

17.3.5

Other Simulation Parameters and Examples

670

17.3.6

Extension:

675

Function

and Mutation

Modifying

658
659

the Fitness Function

Applying Genetic Approaches

to the

Traveling

Salesman Problem

17.4.1

The TSP

17.4.2

Significance and Computational

17.4.3

TSP Representational Issues and Approaches

(TSP)

679
680

Complexity

of the TSP

680
681

Application of Genetic Algorithms to ANN Design and Training


17.5.1
Topology Optimization and/or Weight Optimization

683

17.5.2

The

684

17.5.3

Blueprint-Based Crossover

684

17.5.4

Sample

685

Blueprint Representation
Blueprint-Based Representation Results

684

Contents

17.6

Fusing Learning

and

Genetic Application: Meta-Algorithm Development

Problem Motivation

685

17.6.2

Background

680

Dynamic Algorithm Graph Representation and Manipulation


17.6.4 The Algorithm Graph Data Structure Representation

686

17.6.5

Genetic Recombination of

688

17.6.6

Crossover

Graphs

687
688

17.7

A Genetic

17.8

Swarm

Intelligence

694

17.8.1

Natural Behavior

694

17.8.2

Definition

695

17.8.3

Population-Based Processing Approaches

17.8.4

Ant

17.8.5
17.8.6
17.8.7
17.8.8
17.9

Approach

to

Optimizing

gcc

A.2

A. 3

695

a
Algorithms (How
Path)
Developing and Applying an Artificial Ant Colony Algorithm
Return to the Traveling Salesman Problem

695

Solving the TSP Using the Ant Colony Metaphor


Computational Shortcomings of the Artificial Algorithm

697

Real Ant Colonies Find

References

Minimum

and the ACS

697
697

698
700
700

Fundamentals of Discrete Mathematics


A.l

691

Compiler Options

17.10 Exercises

685

17.6.1

17.6.3

xxi

Sets

705

705

A. 1.1

Basic

A. 1.2

Operations

Concepts
on

705
Sets

706

Relations

706

A. 2.1

Properties

707

A.2.2

Relations As Functions

707

Graphs

708

A.3.1

Subgraphs

A.3.2

Directed

Graphs

A.3.3

Directed

Graphs and Intraset

A. 3.4

Trees

Fundamentals of

708

(Digraphs)

708
Relations

708
709

Prolog

B. l

Introduction

B.2

Predicates, Clauses, Facts, Rules,

711
711
and Goals

711

B. 2.1

Variables

712

B.2.2

Goals

712

B.2.3

Examples

713

B.2.4

The

Anonymous Variable

714

B.2.5

714

B.2.6

Prolog Uses Term Unification


Prolog "help" Notation

B.2.7

Recursion in

716

Prolog

716

xxii

B.3

Contents

B.2.8

Variable Bindings and

B.2.9

Prolog and

B.2.10

Predicates

Lists in

B.4

B.5

Arithmetic
to

Verify

the

717

Type

of

Term

List Representation

B.3.2

A List Example
List

Prolog

717

718

Membership

via Predicate "member"

Databases

718

B.4.1

Prolog Dynamic Databases

718

B.4.2

assert and retract

718

Backtracking
B.5.2

The

and the cut

Concept

Logic

in

of

()

Negation

719
as

Failure

719

719

Prolog

B.5.3

B. 7

718

Prolog

B.5.1

B.6

717
717

Prolog

B.3.1
B.3.3

717

is

Prolog and Modus Ponens (MP)


Parsing (Grammatical/Structural Recognition) and Prolog
B.6.1
Using Grammar R.ule Notation (the LGN)
B.6.2
A Significant LGN Utility: Automatic Parsers

719

B.6.3

Adding Variables in the LGN

721
722

720

720
721

Prolog

References and

B.7.1

Books

722

B.7.2

Web References

722

Pragmatics

Fundamentals of Linear Algebra

723

C. l

Elementary

Matrices

C.2

Vectors

724

C.3

Linearity

725

C.4

Inner and Outer Products and Applications

726

C.5

Measures of

Space

727

C.6

Differentiation of Matrices and Vectors

728

C.7

The Chain Rule

729

C. 8

Gradient Descent-Based Optimization Procedures

Similarity

723

in Vector

729

Fundamentals of

Lisp

733

D. l

The Roots of

Lisp

733

D.2

First

Principles
D.2.1
Programs
D.2.2

D.3

D.4

The

733
and Data Are Lists

Lisp Top-Level Loop (EVAL)

Common Lisp Building Blocks


D.3.1

s-Expressions (or Forms)

D.3.2

Some

D.3.3

Basic Function

D.3.4

Special Symbols

References

Special Forms
Groups

733
734
734

and Evaluation

734
735
735

736
737

Contents

D.4.1

Implementations

D.4.2

Online Common

xxiii

737

Lisp

Resources and References

738

Bibliography

739

Index

755

Vous aimerez peut-être aussi