SE-402:

Artificial Intelligence

Dr. Mustansar Ali Ghazanfar

Mustansar.ali@uettaxila.edu.pk

9/2/2013 1
Lectured by
Dr. Mustansar Ali Ghazanfar
Mustansar.ali@uettaxila.edu.pk

Course leader

TA: Engr. Sobia

9/2/2013 2
Structure
1 Lectures:
In a week

This course
20 credits - 20% CS, 80% exam
Keep an eye on web page and emails!

9/2/2013 3
AI
What are you expecting?

Previous experience?

How about coding?

Well use Java. Others use MATLAB
Self-study on both of these is a good idea!

9/2/2013 4
Resources
AI:
Artificial Intelligence: A modern approach
(Stuart Russell and Peter Norvig)
Collective Intelligence in Action
(Satnam Alag)
Web
Lots of good resources, but be a bit careful
Library
Books, Journals, Conference Proceedings
9/2/2013 5
Quick outline - lectures
Introduction to AI, History, Agent-based
paradigm

Intelligent Algorithms: Search, Genetic

Algorithms, Adversarial Search, Constraint
Satisfaction Problems

Machine Learning: Instance Learning, NNs,

Decision Trees

9/2/2013 6
Topics covered
Introduction
Making Decisions
Fast search, Constraint satisfaction
Modeling Uncertainty, Bayes nets
Decision theory
Machine learning: Perceptron, kernels
Applications
9/2/2013 7
Quick outline - lectures
Agent-based paradigm, AI in multi-
agent systems
Natural Language Processing
Recommender Systems

9/2/2013 8
Quick outline - labs

I dont think we have one, officially,

BUT we will have some lab work
(course work)

9/2/2013 9
AI
1967, Marvin Minsky:
"Within a generation ... the problem of creating 'artificial
intelligence' will substantially be solved

So how far away are we?

9/2/2013 10
Throughout the course
Focus on LEARNING
Think about the topics bigger picture

Not just learning techniques

Need to understand their application

AI is a good topic to read around

Try to be scientific in your approach

9/2/2013 11
Wooo
We are going to start

Questions?

9/2/2013 12
Outline
What is AI?

9/2/2013 13
 Outline

Definition
Defining AI & Intelligence

Interdisciplinarity of AI

The State of the Art

History
Agents
Turning test
Frame problem
Flavours
Strong vs. weak

Neat vs. scruffy

9/2/2013 14
14
Imagined AI

9/2/2013 15
Imagined AI

9/2/2013 16
Real AI?

Need help with our

website? Need information
about travelling by train?
Just 'Ask Lisa', our new
virtual assistant.

9/2/2013 17
 AI at Southampton

Scientists test gait biometrics

New Research Could Lower

Language Barriers Across
Europe

Small, cheap,
swarming robots
unveiled
ECS disaster simulation system wins
championship prize
9/2/2013 18
AI at UET-Taxila

9/2/2013 19
 What is AI
AI has proved notoriously difficult to define, reflecting the
difficulty of defining natural intelligence:

In 1921, the Journal of Educational Psychology asked fourteen

leading experts . . . to provide their definitions . . .[they] got 14
different answers back. (Pfeifer and Scheirer 1999, p. 6)

One influential attempt at defining natural intelligence is dueto

van Heerden (1968):

Intelligent behavior is: to be repeatedly successful in

satisfying ones psychological needs in diverse, observably
different, situations on the basis of past experience.

But how can one define psychological needs scientifically?

9/2/2013 20
Some Notions of Intelligence
Some of the 14 different notions were:

The ability to carry on abstract thinking (Terman)

. . . ability to learn to adjust . . . to the environment
(Colvin)
. . . ability to learn to adapt . . . adequately to new
situations in life (Pinter)
The capacity to acquire capacity (Woodrow)
The capacity to learn or profit by experience (Dearborn)
A biological mechanism by which the effects of acomplexity
of stimuli are brought together . . . (Peterson)

9/2/2013 21
 Defining AI
Q. What is artificial intelligence?
A1. It is the science and engineering of making intelligent
machines, especially intelligent computer programs. It is
related to the similar task of using computers to
understand human intelligence, but AI does not have to
confine itself to methods that are biologically observable.
John McCarthy, 2007.

A2. Its aim is to build intelligent entities.

Russell & Norvig, AIAMA2e

9/2/2013 22
Defining AI
An early attempt at definition is Artificial Intelligence is the
science of making machines do things that would require
intelligence if done by men (Minsky 1968, p. v).
This view remains current Nilsson (1998, p. 1) writes that
Artificial intelligence, broadly (and somewhat circularly)
defined, is concerned with intelligent behavior in artifacts.
Yes, these are circular!
How about:
AI is the attempt to mechanise thought by technological
means. ?
But then what is thought?!

MORAL: definition of AI is not easy!

9/2/2013 23
 What is Intelligence?
Insight Social Ability

Knowledge
Common Sense

Wisdom Language

Foresight Sharpness
IQ
Logic Cleverness
Rationality Creativity
9/2/2013 24
What is Intelligence?

9/2/2013 25
Robotics
Robotics
Part mech. eng.
Part AI
Reality much
harder than
simulations!
Technologies
Vehicles
Rescue
Soccer!
Lots of automation
In this class:
We ignore mechanical aspects
Methods for planning
Methods for control

9/2/2013 26
Vision (Perception)
Object and character recognition
Scene segmentation
3D reconstruction
Image classification

9/2/2013 27
Logic
Logical systems
Theorem provers
NASA fault diagnosis
Question answering
Methods:
Deduction systems
Constraint satisfaction
Satisfiability solvers

9/2/2013 28
Games
May, '97: Deep Blue vs. Kasparov
First match won against world-champion
Intelligent creative play
200 million board positions per second!
Open question:
How does human cognition deal with the search space
explosion of chess?
Or: how can humans compete with computers at all??
1996: Kasparov Beats Deep Blue
I could feel --- I could smell --- a new kind of
intelligence across the table.

9/2/2013 29
Decision Making
Scheduling, e.g. airline routing, military
Route planning, e.g. google maps
Medical diagnosis
Automated help desks
Fraud detection
Spam classifiers
Web search engines
Movie and book recommendations
Lots more!

9/2/2013 30
 Intelligent Brain

Brains (human minds) are

very good at making
rational decisions (but not
perfect)
Brains are to intelligence
as wings are to flight
Brains arent as modular
as software

9/2/2013 31
Rational Decisions
rational :
Rational: maximally achieving pre-defined goals
Rational only concerns what decisions are made
(not the thought process behind them)
Goals are expressed in terms of the utility of
outcomes
Being rational means maximizing your
expected utility
A better title for this course would be:
Computational Rationality

9/2/2013 32
 Rational Agents
An agent is an entity that
perceives and acts.
A rational agent selects
actions that maximize its
utility function.
Characteristics of the
percepts, environment,
and action space dictate
techniques for selecting
rational actions.

9/2/2013 33
Rational Agent (contd ..)

9/2/2013 34
Rational Agent (contd..)
This course focuses on:
General AI techniques for a variety of

problem types
Learning to recognize when and how a

new problem can be solved with an

existing technique

9/2/2013 35
Interdisciplinary AI
Art
Cybernetics Music
Biology Linguistics
Neuroscience Literature
Psychology
Mathematics
Alife A.I. Computer Science
Robotics Operations Economics
Control Theory Research Philosophy
Engineering
Logic

9/2/2013 36
Exercises

A computer can only do what the programmer

tells it to do. So a computer cannot be intelligent.
Is this statement true?

9/2/2013 37
 State of the Art
Chess: Deep Blue In our dreams

Robots: NASAs Remote Agent Pilotless Vehicles

Vehicles: ALVINN drove a van Wise Oracles

across the US
Artificial Friends
Medical Diagnosis

Logistics: Persian Gulf crisis

Language: PROVERB solves

crossword puzzles
9/2/2013 38
Unintentional!

Henry Squirrel was thirsty. He walked over

to the river bank where his good friend Bill
Bird was sitting. Henry slipped and fell in
the river. Gravity drowned.
The End!

9/2/2013 39
 History of AI

Pre-History
Early-AI

Milestones and

Movements
Today?

John McCarthy (1967- )

9/2/2013 40
 Pre-History: Tributaries of AI
Philosophy Logic, methods of reasoning, phil. of mind,
learning, language, rationality

Mathematics Formal notation & proof, algorithms,

computation, undecidability, intractability

Neuroscience Physical substrate for mental activity

Psychology Perception, motor control, experimentation

Engineering Building (fast) computing devices

Control Theory Feedback, homeostasis, stability, optimality

Linguistics Knowledge representation, grammar

Economics Utility, decision theory

9/2/2013 41
 Turing and Bletchley Park
During WWII, Alan Turing worked on
code-breaking at Bletchley Park.
Used heuristic search to translate Nazi
messages in real time
With others, e.g., Jack Good and Don
Michie, he speculated on machine
intelligence, learning

Much of this remained secret until after the war.

The military has retained a strong interest in AI ever since

9/2/2013 42
The Turing Test
Turing, 1950: I propose to consider the question, Can
machines think? . . . (except he didnt!)
It is a given that humans are intelligent.
Turing proposed an imitation game in which a player
communicating via a teletype with a man and a woman in
remote, hidden rooms was required to tell which was which.
(One could lie, the other had to tell the truth.)
Now swap a computer for one of the humans. By extension, if
we cannot tell which is human and which is machine, the
machine must be intelligent.
Several variants of the Turing test.
Usual one these days is to determine if a (single) conversational
entity is a human or a computer.

9/2/2013 43
The Turing Test

9/2/2013 44
 The Turing Test: problems

Equates intelligence with human abilities.

Equates intelligence with linguistic abilities.
Observer may be gullible and/or unwilling to declare
its a computer for fear of being wrong.
How to limit the domain of discourse?
How long do we continue the test?

Some interesting reading:

PRO: B. J. Copeland (2000) The Turing test, Minds and
Machines, 10(4), pp. 519539.
ANTI: R.M. French (1990) Subcognition and the limits of
the Turing test, Mind, 99(393), pp. 5365.
9/2/2013 45
 The Birth of AI (1956)
The Dartmouth Workshop brings together 10
top minds on automata theory, neural
nets and the study of intelligence.
Conjecture: every aspect of learning or any other feature of
intelligence can in principle be so precisely described that a machine
can be made to simulate it
Ray Solomonoff, Oliver Selfridge, Trenchard More, Arthur Samuel,
John McCarthy, Marvin Minsky, etc.
Allen Newell and Herb Simons Logic Theorist
For the next 20 years the field was dominated by these participants.

9/2/2013 46
 AI Winter
Collapse in AI research (1966 - 1973)
Progress was slower than expected.

Unrealistic predictions.

Some systems lacked scalability.

Combinatorial explosion in search.

Fundamental limitations on techniques and

representations. Seymour Papert
Minsky and Papert (1969) Perceptrons. (1928- )

9/2/2013 47
 AI Revival (1969-1970s)
Exploiting encoded domain knowledge
DENDRAL (Buchanan et al. 1969)

First successful knowledge intensive system.

MYCIN diagnosed blood infections (Feigenbaum et al.)

Introduction of uncertainty in reasoning.

Increase in knowledge representation research.

Logic, frames, scripts, semantic nets, etc.,

9/2/2013 48
 Connectionist Revival (1986- )
Parallel distributed processing (Rumelhart & McClelland 86)

Multi-level perceptrons and backpropagation learning

Language, reasoning, perception, control + a little mystery

Robust behaviour, graceful degradation

No representations? Sub-symbolic AI

90s: Elman pioneers layered recurrent nets

90s: Fully recurrent networks and robot control (e.g., Beer)

Ultimately neural networks as data-mining, statistics

9/2/2013 49
 Intelligent Crow

Once upon a time there was a dishonest fox

and a vain crow. One day the crow was
sitting in his tree, holding a piece of cheese in
his mouth. He noticed that he was holding
the piece of cheese. He became hungry, and
swallowed the cheese. The fox walked over
to the crow. The End.
[Shank, Tale-Spin System, 1984]
9/2/2013 50
 Intelligent Agents (1995- )
Combined whole organism perspective with a rational utility-maximising
framework borrowed from economics.

A response to nouvelle AI?

An empty label?

A hybrid? A bolt-hole for formalists? A revolution?

How does an agent act/behave embedded in real environments

with continuous sensory inputs
More on agents next time.

9/2/2013 51
 Data, Data, Everwhere (2000- )
Massive amounts of raw power and raw data fuel advances
in machine learning:

Eigenfaces
Corpus linguistics

Kernel methods

Computational learning theories

Offline vs. Online AI?

Pattern Recognition in a Bucket?

9/2/2013 52
Brief History [1]
197088: Knowledge-based approaches
196979: Early development of knowledge-
based systems
198088: Expert systems industry booms
198893: Expert systems industry busts: AI
Winter
1988: Statistical approaches
Resurgence of probability, focus on uncertainty
General increase in technical depth
Agents and learning systems AI Spring?
2000: Where are we now?
9/2/2013 53
Where are we now?
We do have intelligent systems operating
more less well in certain restricted domains:
Google, agents, robot helpers, biometric
retina scans, in-speech recognition, machine
translation, recommender, systems, spam
filters . . .
Generally, these are based on probability and
learning from data, rather than symbolic
rules. Success critically dependent on doing
one thing well.

9/2/2013 54
 Where are we now?
Quiz: Which of the following can be done at present?
Play a decent game of table tennis?
Drive safely along a curving mountain road?
Drive safely along Telegraph Avenue?
Buy a week's worth of groceries on the web?
Buy a week's worth of groceries at Berkeley Bowl?
Discover and prove a new mathematical theorem?
Converse successfully with another person for an hour?
Perform a complex surgical operation?
Unload a dishwasher and put everything away?
Translate spoken Chinese into spoken English in real time?

9/2/2013 55
 Symbol-Grounding Problem

If intelligence involves thinking thoughts, what makes a

thought about a particular thing?
What is aboutness? What makes mental states
meaningful?
A thought of a cat means cat because:
Stripy (Cat)
it looks like a cat?
it was caused by a cat?

it was brought about by a mechanism with the job of

creating thoughts about cats?
9/2/2013 56
56
 Physical Symbol System
Hypothesis
A physical symbol system has the necessary and
sufficient means for general intelligent action.
Alan Newell and Herbert Simon (1976)

Human intelligence is a kind of symbol manipulation

Machines that manipulate symbols can be intelligent
The PSSH is central to classical AI
It loads symbols and formal symbol manipulation with the
burden of instantiating intelligence.

9/2/2013 57
57
 Could Symbols Really be
Enough?
Classic AI takes thoughts to be like sentences:

The cat is on the mat = On(Cat, Mat)

Shape(Mat,Flat), Colour(Mat,Brown), Tastes(Mat,Bad)

Thought is manipulation of such symbolic expressions. But:

1. Gdel placed formal limits on this kind of system

2. Wittgenstein argued that concepts such as Game or even

Chair are not easily captured by sets of propositions.

Might thought be informal, sub-symbolic or non-algorithmic?

9/2/2013 58
58
 The Frame Problem
In the confined world of a robot, surroundings are not static.
Many varying forces or actions can cause changes or
modifications to it. The problem of forcing a robot to adapt to
these changes is the basis of the frame problem in artificial
intelligence. Information in the knowledge base and the robot's
conclusions combine to form the input for what the robot's
subsequent action should be. A good selection from its facts
can be made by discarding or ignoring irrelevant facts
and ridding of results that could have negative side
effects. (Dennett) (Fischler 304-5)

It began in classical AI as an annoyance in inference systems; if

an event occurs, what attributes of the model need not be
considered in inferring its consequences? The lack of a
satisfactory answer has universally led AI systems to
Combinatorial Explosion.

9/2/2013 59
 The Frame Problem

Dennetts framing:

R1 - Robot
Pull
R1 discovers: (Wagon,Out)
!
- Needs(R1,cell)

- In(cell,house)

- In(cell,bomb)!

- On(cell,wagon)

Boom! 60
9/2/2013 60
 The Frame Problem

We need a new robot:

R1D1: Robot-Deducer

R1D1 discovers:

- Needs(R1D1,cell)

- In(cell,house)

- In(cell,bomb)!

- On(cell,wagon)

Boom! 61
9/2/2013 61
 The Frame Problem

We need another new robot:

R2D1: Robot-Relevant-Deducer

R2D1 discovers:

- Needs(R1D1,cell)

- In(cell,house)

- In(cell,bomb)!

- On(cell,wagon)

Boom! 62
9/2/2013 62
 Flavours of AI
There are a few ways to characterise this distinction:

Super-human intelligence vs. Sub-human intelligence

General intelligence vs. Domain-specific intelligence

Really-real intelligence vs. Mere as-if intelligence*

9/2/2013 63
63
 Flavours of AI
There are a few ways to characterise this distinction, too:

General-Purpose vs. Domain-Specific

Elegant vs. Pragmatic

Ideal vs. Real-World (in-the-head vs. in-the-world?)

Is the real world more like chess? Or football?

9/2/2013 64
64
 Background Reading
AIAMA2e Chapter 1 + cited references
Brooks, R. A. (1991). Intelligence without reason. MIT AI Memo #1293.*
Dupuy, J.-P. (2000). The mechanization of mind. Princeton University
Press.
Husbands, P., Holland, O., & Wheeler, M. (eds.) (2008). The mechanical
mind in history. MIT Press.
Luger, G. F.(ed). (1995). Computation and intelligence: Collected
readings. AAAI/MIT Press.

9/2/2013 65
 Background Reading
AIAMA2e Chapter 1 + cited references
Simon, H. (1996). The sciences of the artificial. 3rd ed. MIT Press.
Hodges, A. (1992). Turing the enigma. Vintage, Random House.
Turing, A. (1950). Computing machinery and intelligence. Mind, LIX
(236): 433-460. [Here]*
Pfeiffer, R. & Scheier, C. (1999). Understanding intelligence. MIT Press.
Posner, M. (ed.) (1993). Foundations of cognitive science. MIT Press.
Varela, F., Thompson, E., & Rosch, E. (1991). The embodied mind. MIT
Press.

9/2/2013 66
 Background Reading
AIAMA2e Chapter 26 + cited references
Boden, M. (ed.) (1990). The philosophy of artificial intelligence. OUP.
cont: McCulloch & Pitts, Turing, Searle, Boden, Newell & Simon, Marr,
Dennett, Sloman, Rumelhart, Clark, Dreyfus, Churchland, Cussins
Boden, M. (ed.) (1996). The philosophy of artificial life. OUP.
cont: Langton, Boden, Ray, Maynard Smith, McFarland, Wheeler, Kirsh,
Clark. Godfrey-Smith, Bedau, Sober, Pattee
Dennett, S. (1991). Consciousness explained. Penguin Press.
Haugeland, J. (ed.) (1997). Mind design II. MIT Press.*
cont: Turing, Haugeland, Dennett, Newell & Simon, Minsky, Dreyfus, Searle,
Rumelhart, Churchland, Fodor, Clark, Brooks, van Gelder
Steels, L. & Brooks, R. A. (eds.) (1994). The artificial life route to artificial
intelligence: Building embodied situated agents. Lawrence Erlbaum.
cont: Varela, Brooks, Steels, Smithers, Mataric, McFarland, Hallam, Pfeiffer,
Kaelbling, Harnad
9/2/2013 67
 Online Resources
Wikipedia Entries

Philosophy of AI Problem of Other Minds

Frame Problem Swamp Man

Symbol Grounding Qualia

Chinese Room Mind-Body Problem

Turing Test

Great Debate: Can Computers Think

9/2/2013 68
 Online Resources
AIMA2e

AIMA2e Resources

Wikipedia Entries

AI

Rationality

Intelligence

Intelligent Agent

9/2/2013 69
 Online Resources
Wikipedia Entries

Philosophy of AI Problem of Other Minds

Frame Problem Swamp Man

Symbol Grounding Qualia

Chinese Room Mind-Body Problem

Turing Test The Framing Problem

Great Debate: Can Computers Think

9/2/2013 70
 Online Resources (cont.)
Eliza State of the Art

AI in the movies Deep Blue

More AI in the movies Remote Agent

McCarthy's "What is AI?" ALVINN

Sloman's "What is AI?" Medical Diagnosis

DART

PROVERB

9/2/2013 71
 Online Resources
Wikipedia Entries

History of AI

Nouvelle AI

Machine Learning

AI Winter

9/2/2013 72
References
[1] MIT lecture slides
[2] Soton lecture slides
[3] Soton agents

9/2/2013 73