Introduction to Artificial Intelligence

People have been fascinated with the concept of artificial intelligence for as long as we
have talked about intelligence.

Some milestones in the discussion of AI

Turing Test:
The Chess-playing robot of the 19th century
The ELIZA software program.
The AI FAQ Section 7
"Bots" or other agents that work over the Internet

What defines AI? - Discuss.

Is the game of Life AI? (cellular automata) Why or why not? What is the role of
randomness?

Some examples include chess playing systems, pattern recognition (voice, vision etc.),
irritability, reproduction (software worms), software psychology (ELIZA1), machine
learning, imprecision, cooperation.

All these, when we study them in detail seem to be just interesting machines, not truly
intelligent, except perhaps to the uninitiated. What is the difference between a
sophisticated mathematical model and a learning system? Can we make a mail delivery
mobile unit (robot) that could operate in a normal work environment with minimal
programming?

What is humanity's Frankenstein Complex? - Discuss

"I want to build a robot that is proud of me."

The intersection of robotics and AI. (muscle and mind). The Robocup contest and all the
other autonomous robot contests all focus around aspects of this problem. AI applies to
autonomous robots because all other robots by definition have a human operator
providing the intelligence.

What constitutes an intelligent system? How about the Turing Test? Suggest your own
test for intelligence. Can an author write a book about a protagonist that is more
intelligent than the author? Are all our efforts so far merely stepping stones or is this an
activity that is doomed from the outset?

Discuss: Can an author write a book about a character that is more intelligent than the
author? What principles for developing intelligent systems can we postulate from this
discussion?
To dig a little deeper http://cogsci.soton.ac.uk/~harnad/Papers/Py104/searle.comp.html

What is the current state of AI research and application?

What's the difference between "classical" AI and "statistical" AI? (From the FAQ)

Statistical AI, arising from machine learning, tends to be more concerned with
"inductive" thought: given a set of patterns, induce the trend. Classical AI, on the other
hand, is more concerned with "deductive" thought: given a set of constraints, deduce a
conclusion. Another difference, as mentioned in the previous question, is that C++ tends
to be a favorite language for statistical AI while LISP dominates in classical AI.

A system can't be truly intelligent without displaying properties of both inductive and
deductive thought. This lends many to believe that in the end, there will be some kind of
synthesis of statistical and classical AI.

Demonstration of learning. The 20 stick game.

How did you arrive at a solution? What are the characteristics of the solution?

What kind of learning is needed in each of the following circumstances:

1. A man is looking at a photograph of a person and says.
"Brothers and sisters have I none.
This man's father is my father's son.
Who am I."

2. A little Indian and a big Indian are hiking through the desert each armed with a bow
and arrow. The little Indian is the big Indian's son but the big Indian is not the little
Indian's father. Who is the big Indian?

Problem 1 - Deductive; problem 2 - Inductive

References

Turing test ( http://cogsci.ucsd.edu/~asaygin/tt/ttest.html )

ELIZA http://i5.nyu.edu/~mm64/x52.9265/january1966.html

Weizenbaum, J., "ELIZA -- A computer program for the study of natural

language communication between man and machine", Communications of the
ACM 9(1):36-45, 1966.
ELIZA is a classic program that parses English sentences and responds with psycho-
babble generalities to keep the conversation going. Some people became convinced that
they were talking to a real psychologist and even after the program was explained to them
continued to believe that ELIZA's psychological help was meaningful and helpful.
AI FAQ http://www.faqs.org/faqs/ai-faq/general/part1/

The Robocup page http://www.robocup.org/02.html

Appendices: Comments on the Artificial Intelligence discussion above

A. * What is the difference between a sophisticated mathematical model and a learning

system? A model will only reflect your current understanding of the problem. A learning
algorithm will discover new insights or structures within the problem that you were not
aware of. A generic ANN will model a process without your knowing the equations to
describe it. Note that most ANNs cannot then describe their "reasoning" they can only
give answers based on their learning.

B. Can an author ...

* Need a standard of measurement. Note that optimization algorithms require us to
measure success without necessarily being able to model or define the process by which
the output is obtained. IE the measurement of the output is (and should be) independent
of the process that creates the output.
* The standard may be multi-faceted ("Seven Kinds of Smart" etc.), or, more
pragmatically, may be measured only in dimensions relative to the problem. IE we
produce "idiot-savants"
* A math genius can solve a problem (E.G. what is the square root of 173579)
instantaneously. I will take longer and will probably use a calculator, but I will still get an
answer. How do you suppose Conan Doyle wrote the Sherlock Holmes mysteries? Can
we define (create) a human-like intelligence that is an idiot-savant. IE has enormous
capability in some limited areas of human intelligence but is severely handicapped in
others.

C. When will computers overtake human beings?

* Problem: Some expressions of Moore's Law suggest that integrated circuit density (and
by implication, computing capability) in a computer doubles every 18 months. From our
current understanding of physiology human neural networks have about 1010 nodes and
10100 interconnections. The current state of the art for artificial ANNs is about 106 nodes.
Assuming that intelligence is simply a function of the number of learning nodes and that
Moore's Law applies, how soon will computers have the same learning power as human
beings? Assuming that each interconnection is represented by one unit of memory (say
one byte) and that Moore's law applies to memory size, how soon will we be able to
connect up sufficient interconnections?

AI - Techniques

Artificial Neural Networks and Fuzzy logic

Machine Intelligence a.k.a. artificial intelligence, a.k.a. computational intelligence

What constitutes an intelligent machine? What are the limitations on machine
intelligence? We have seen at a very simple level how difficult it is to make a simple
fork-lift cope with a controlled environment. How can a system cope with a much less
controlled environment?

How do systems deal with approximation and learning?

Definition of AI: The ability of a system to perceive changes in its environment and to
develop a plan and produce appropriate reactions. A system that exhibits learning and
elements of reasoning (generalization, discovery, association, abstraction)
Generalization: From several examples of birds can identify a generic creature as a bird
or not-bird.
Creativity: Can produce new meaningful artifacts from raw materials.
Association: Shown one thing (car) can find another thing in memory that is similar
(truck)
Abstraction: From specific examples can generate a rule or method. EG watch objects
falling => Law of gravity.

OK now let's switch from philosophy to engineering.

Some notes on Fuzzy Logic and Artificial Neural Networks

Artificial Neural Networks (ANN) a.k.a. Machine Learning (CS ???)

ANNs are
either an artificial intelligence human-like learning system
or
a non-linear, statistical, parametric, multi-dimensional optimization algorithm,
or both

Many models and algorithms - favorite occupation of researchers - develop a new

algorithm.

The famous ones

Perceptron learning/back propagation

Artificial Neural Networks (ANN): ANNs imitate human thinking. They are capable of
learning relationships and patterns in a way that is analogous to organic memory. Some
of the significant characteristics of this learning include
* Learning to approach a correct answer with repeated examples or learning to group
similar items together.
* Generalization. From a set of typical examples the NN system can identify all patterns
of the same type. EG if shown various samples of birds it can recognize birds it has
never seen before.
* Content recall. If shown part of a pattern the NN can recall the original pattern or type.
* Fault and noise tolerance. NNs can recognize learned material even if the material is
obscured by interference of various kinds.
Neural Networks find applications in situations where relationships or patterns cannot
easily be described because they are nom-linear, ill-defined, time-variant multiple-
input/multiple-output and so on. The neural network can learn from a "teacher" or can
organize information into similar groups by itself. In either case the network can then be
used with inputs or combinations that it has never seen before and can produce outputs
according to its learned memory.

The inherent structure of an ANN is massively parallel. All the input neurons receive
information together, react together and pass the information on the next layer, eventually
passing it on to the output layer. For example a graphic figure can be presented to an
ANN where each pixel represents an input neuron. The picture is processed until the
output layer produces one or more outputs identifying the picture. The parallelism of NNs
partially explains why a human being whose neurons process approximately once per
millisecond can outperform a computer in many tasks despite the fact that the computer
can process (of the order of) once per nanosecond. The computer operation are processed
serially. Humans are estimated to have approximately 1010 neurons with 10100
interconnections all being processed simultaneously.

Artificial neural networks have found applications in image recognition, medical

diagnosis and industrial control. They are best used where it is desirable to model an
unknown process without extensive mathematical analysis of the process.

Introduction to ANNs (Microsoft Word)

ANNs and fuzzy logic are often used together. The neural network is used to identify
important criteria and the fuzzy logic uses these criteria to control the process.

ANN Review Questions

1. Consider the curve fitting analogy of ANNs. How many inputs to the ANN are
represented by a simple graph of a curve being fit to a line? The function surface or error
surface is an N-dimensional hyperplane. What defines the number of dimensions of the
hyperplane?

2. Sketch a simple ANN. What are the essential three layers of the ANN. Simple ANNs
are frequently described as having two layers because the inputs have no processing
associated with them. Does the number of layers limit the number of inputs and outputs?

3. What are the implications of multi-input, multi-output systems like ANNs? Compare
them to statistical correlation methods and closed loop controllers (E.G. PID).

4. Nodes in ANNs usually include a function such as a sigmoid (1/(1+e^-net) or

tanh(net). What properties or abilities does this function give the ANN? Give one
example of a system to be learned where this property is necessary.
5. ANNs learn to recognize or model unknown systems. Where is this learned knowledge
stored?

6. One of the most popular forms of ANN is known as Back-Propagation. What is being
propagated backwards?

7. For some time after ANNs were first conceived they did not develop because of a
couple severe limitation in learning. Linear networks could not model which famous
example problem? Non-linear networks had problems with which aspect?

8. What can be learned about the underlying physical properties of the system being
modeled after the ANN is trained?

9. What is training set data and and test set data and how much of each is required (%)?

10. Assume you are using an ANN to predict plastics injection molding quality. You will
use four different temperature and three different pressure signals and a flow rate signal
to predict the breaking strain and amount of flash in the finished molded item. How many
inputs and how many outputs would the ANN require.

11. How many hidden nodes would the ANN require for the previous problem? (Hint:
Trick question)

Fuzzy Logic:
Concept: Many control ideas are understood well intuitively by human operators but are
difficult to model for the purpose of creating an algorithmic control system. This is
particularly true where control decisions are expressed in terms of generalized linguistic
variables. ("Use full-power heating until it gets quite hot. If it gets very hot, turn down
the heat a little until the reaction is fairly stable"). "Fuzzy" words like "quite hot," "very
hot," and "turn down the heat a little" are difficult to convert into definite ("crisp")
values. If the line between "hot" and "very hot" is taken to be, say, 120 degrees then that
implies that 119.9999 is not "very hot" and 120.00001 is "very hot." Fuzzy logic provides
a technique to quantify the "softness" in these variables and to make definite decisions on
the basis of softly defined variables. Fuzzy logic control describes the control algorithm
in terms of linguistic variables and sets of rules. Fuzzy logic, based on fuzzy sets
provides a formal and structured way to convert these rules into a control scheme. Fuzzy
logic is a superset of Boolean logic. In Boolean logic something can be '1' ("hot") or '0'
("not hot"). In fuzzy logic a fuzzy bit can assume any value, called degree of
membership, between '0' and '1'.
Application domain: "In general fuzzy logic is best applied to non-linear, time-variant,
ill-defined systems." (Shetty & Kolk) Systems that can be represented by simple
equations or sets of equations should be modeled using those equations and controlled
using classical control techniques. Fuzzy logic will find applications where an expert is
available who can describe in words what needs to be done to control a particular
process. This expert's instructions can then be converted to a fuzzy logic rule set and
applied to the same problem without the need for extensive analytical and mathematical
modeling of the system

Example. Generate a set of rules in terms of linguistic variables. The four steps of fuzzy
logic application are then Fuzzification (calculate membership in the fuzzy functions),
Inference (MIN activation of each rule), Composition (Max activation of each output by
each active rule), De-fuzzification. (centroids and singletons etc.) In practice some time
will be spent "tuning" the system after it is built.

Implementation: The real-time processing is straight-forward and is usually configured to

only require ordinary arithmetic operations. This allows it to be fast and to be executed
on very simple processors. With many fuzzy logic packages available for industrial
applications many more applications are including this technique. This should no longer
be considered as an esoteric scientific concept but rather as available application-oriented
technology. See the reference on fuzzy logic for more detail

Applications of fuzzy logic:

* Spell-checker to suggest the most likely word to replace the mis-spelled one.
* Control of Japanese subway system
* Camera exposure control

Fuzzy Logic Review Questions

1. What are "fuzzy" and "crisp" values? Describe today's temperature in fuzzy and crisp
terms.

2. What is a linguistic variable?

3. Is Boolean logic a subset of fuzzy logic or is fuzzy logic a subset of Boolean logic or
do they partially intersect or is there no intersection between the two systems?

4. What is a membership function? What are common membership functions (describe in

words or sketch). Is there a membership function that describes Boolean logic?

5. In Boolean logic NOT(1)=0 and NOT(0)=1. What is the meaning of the NOT
operation in fuzzy logic?

6. What problems in Mechatronics could you use fuzzy logic to solve? Pick one example
problem, define linguistic variables and a rule set for that application.

7. Earlier we noted that PID loops were single-input-single-output (SISO) and that neural
networks were (in general) multiple-input-multiple-output MIMO. Furthermore PID
loops are intended for 2nd order linear differential systems (systems describable in terms
of 2nd order linear differential equations E.G. a capacitor-inductor or a spring-mass
system) and neural networks were intended for non-linear systems in general. How would
you describe fuzzy logic in these terms?

Reference:

FAQs for AI, ANN and fuzzy logic http://www-

2.cs.cmu.edu/Groups/AI/html/faqs/ai/top.html

1. ANN links http://www.emsl.pnl.gov:2080/proj/neuron/neural/

Fuzzy logic links:

1. Fuzzy logic newsgroup and FAQ

2. Fuzzy logic links page http://www.abo.fi/~rfuller/fuzs.html