CSC3206 Artificial Intelligence

Lecture 1: Introduction

1
My info

Dr. Richard Wong

Room 3.05

ricwtk.github.io/calendar
(check my schedule before emailing to request for consultation)

2
What are the applications of AI?

c
Speech/Language Recognition a

Facial/Object Recognition Medical Imaging

Path planning Robotics

Chatbot Games h

Autonomous Control

3
References for images

a
Image from http://miclab.fee.unicamp.br/blog/presentation-8th-international-workshop-pattern-recognition-neuroimaging
b
Image from https://blogs.aspect.com/dont-confuse-speech-recognition-with-natural-language-understanding/
c
Image from
https://becominghuman.ai/facial-recognition-and-ai-latest-developments-and-future-directions-39d22201d88b
d
Image captured from https://www.youtube.com/watch?v=qXZt-B7iUyw
e
Image from https://blog.markgrowth.com/how-chat-bots-can-help-you-increase-conversion-6561ba0b8ab0
f
Image from https://www.tataelxsi.com/ip-solution/automotive/Autonomai.html
g
Image from https://www.theverge.com/2019/3/6/18222203/video-game-ai-future-procedural-generation-deep-learning
h
Image from https://www.zdnet.com/article/google-io-softbank-maker-of-ai-pepper-robot-has-news-for-u-s-developers/

4
So, what is AI?

AI stands for artificial intelligence.

Artificial
• synthetic; man-made

Intelligence
• R. Sternberg defines intelligence as
Intelligence is the cognitive ability of an individual to learn
from experience, to reason well, to remember important infor-
mation, and to cope with the demands of daily living.1

1
Sternberg, R. J. 1994. In search of the human mind. 395-396. New York, NY: Harcourt-Brace.

5
Intelligence

Intelligence

learn from experience

cope with the demands of daily living (adapt)
reason well
remember important information

There are constant arguments about how we know one (someone or

something) is intelligence.
I Can animals be intelligence?
I If animals are intelligent, how do you measure their intelligence?
I Can computational units be intelligent?

6
Isaac Newton once wrote. . .

If I have seen further it is by standing on the shoulders of Gi-

ants.

. . . to Robert Hooke

7
History of AI

1943 1952 1955 1966 1969 1973 19791980 1987 1995 2001

The birth of AI

Early enthusiasm

A dose of reality

Knowledge-based systems

AI becomes an industry

The return of neural networks

AI adopts the scientific methods

Emergence of intelligent agents

Availability of large data sets

8
The birth of AI (1943-1956)

Warren McCulloch & Walter Pitts (1943)2 worked on AI based on

I knowledge of the basic physiology and function of neurons in the
brain,
I a formal analysis of propositional logic due to Russell and
Whitehead,
I and Turing’s theory of computation.
In their work, artificial neuron is characterised as being “on” or “off”,
stimulated by neighbouring neurons.

2
McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of
Mathematical Biophysics, 5(4), 115–133. https://doi.org/10.1007/BF02478259

9
The birth of AI (1943-1956)

Marvin Minsky & Dean Edmonds at Harvard

I built the first neural network computer (SNARC) in 1950.

Alan Turing
I Turing test (1950)3 is one of the earliest efforts to identify if a
computer can exhibit intelligence.
I A person asking questions via a keyboard to both a person and a
computer in another room. If the interrogator could not tell the
computer apart from the human, the computer has passed the
Turing test and could be perceived as being intelligent.
I Has Turing test been passed? 4
3
TURING, A. M. (1950). I.—COMPUTING MACHINERY AND INTELLIGENCE. Mind, LIX(236), 433–460.
https://doi.org/10.1093/mind/LIX.236.433
4
https://isturingtestpassed.github.io/

10
The birth of AI (1943-1956)

John McCarthy, Marvin Minsky, Claude Shannon, Nathaniel Rochester

I organised a two-month workshop at Dartmouth College (summer
1956).
I the workshop is a study based on the conjecture that “every
aspect of learning or any other feature of intelligence can in
principle be so precisely described that a machine can be made
to simulate it”.
I Newell and Simon proposed a reasoning program, the Logic
Theorist (LT).
The workshop/conference did not lead to any new breakthroughs, but
it introduced the major figures who, with their students and
colleagues, dominated the field of AI in the next 20 years.

11
History of AI

1943 1952 1955 1966 1969 1973 19791980 1987 1995 2001

The birth of AI

Early enthusiasm

A dose of reality

Knowledge-based systems

AI becomes an industry

The return of neural networks

AI adopts the scientific methods

Emergence of intelligent agents

Availability of large data sets

12
Early enthusiasm (1952-1969)

Newell and Simon (those who created the LT) created the General
Problem Solver (GPS).
I imitate human problem-solving protocols
I first program to embody the “thinking humanly” approach

Herbert Gelernter (1959) constructed the Geometry Theorem Prover.

I able to prove theorems that many students of mathematics
would find tricky

13
Early enthusiasm (1952-1969)

Arthur Samuel (1952) wrote a series of programs for

checkers/draughts.
I the programs learned to play at a strong amateur level
I his program quickly learned to play a better game than its creator
disproved the idea that computers can do only what they are told
to

14
Early enthusiasm (1952-1969)

John McCarthy moved to MIT.

I defined the high-level language Lisp, the dominant AI
programming language for the next 30 years.
I published a paper (1958) 5 about the Advice Taker.
a hypothetical program that can be seen as the first complete AI
system
designed to use knowledge to search for solutions to problems
to embody the general knowledge of the world
to allow it to achieve competency in new areas without being
reprogrammed

5
Mccarthy, J. (1959). Programs With Common Sense. Symposium on the Mechanization of Thought Processes, 77–84.
https://doi.org/10.1.1.11.9028

15
Early enthusiasm (1952-1969)

Marvin Minsky moved to MIT in 1958, and supervised a series of

students focusing on microworlds.
I microworlds are limited domains in which specific problems that
require intelligence to solve exist
I James Slagle – SAINT (1963) – to solve closed-form calculus
integration problems typical of first-year college courses
I Tom Evans – ANALOGY (1968) – to solve geometric analogy
problems that appear in IQ tests
I Daniel Bobrow – STUDENT (1967) – to solve algebra story
problems

16
Early enthusiasm (1952-1969)

Neural networks, built on top the work of McCulloch and Pitts (neural
network), and Donald Hebb (Hebbian learning)
I Winograd and Cowan (1963)
I Bernie Widrow (1960, 1962)
I Frank Rosenblatt (1962)

17
History of AI

1943 1952 1955 1966 1969 1973 19791980 1987 1995 2001

The birth of AI

Early enthusiasm

A dose of reality

Knowledge-based systems

AI becomes an industry

The return of neural networks

AI adopts the scientific methods

Emergence of intelligent agents

Availability of large data sets

18
A dose of reality (1966-1973)

Herbert Simon predicted in 1957 that

within 10 years a computer can think like human, and there-
fore would be chess champion, and a significant mathemati-
cal theorem would be proved by machine.

These predictions came true (approximately true) within 40 years

rather than 10.

19
A dose of reality (1966-1973)

The early AI systems failed when tried out on more difficult problems.

20
A dose of reality (1966-1973

First difficulty:
most early programs knew nothing of their subject matter (context)
For eg, translation between English and Russian. The famous
retranslation:

the spirit is willing and the flesh is weak

English → Russian → English

the vodka is good but the meat is rotten

21
A dose of reality (1966-1973)

Second difficulty:
intractability of the problems that AI was attempting to solve
Early AI programs solved problems by trying out different
combinations of steps until the solution was found. This can be
unmanageable with larger problems.

The fact that a program can find a solution in principle does

not mean that the program contains any of the mechanisms
needed to find it in practice.

22
A dose of reality (1966-1973)

Third difficulty:
fundamental limitations on the basic structures being used to
generate intelligent behaviour
Limitation of perceptrons (a simple form of neural network)

23
History of AI

1943 1952 1955 1966 1969 1973 19791980 1987 1995 2001

The birth of AI

Early enthusiasm

A dose of reality

Knowledge-based systems

AI becomes an industry

The return of neural networks

AI adopts the scientific methods

Emergence of intelligent agents

Availability of large data sets

24
Knowledge-based systems (1969-1979)

To solve a problem by using more powerful, domain-specific

knowledge that allows larger reasoning steps and can more easily
handle typically occurring cases in narrow areas of expertise.
In other words, to solve specific, recurring problems in narrow areas
of expertise.
For examples,
I DENDRAL by Buchanan, Sutherland, and Feigenbaum (1969)
to infer molecular structure based on the elementary formula of
the molecule and the mass spectrum
I MYCIN by Feigenbaum, Buchanan, and Shortliffe (1984)
expert system to diagnose blood infections
I SHRDLU by Winograd (1972)
to understand natural language

25
History of AI

1943 1952 1955 1966 1969 1973 19791980 1987 1995 2001

The birth of AI

Early enthusiasm

A dose of reality

Knowledge-based systems

AI becomes an industry

The return of neural networks

AI adopts the scientific methods

Emergence of intelligent agents

Availability of large data sets

26
AI becomes an industry (1980-present)

R1
I first successful commercial expert system
I began operation at the Digital Equipment Corporation (1982)
I configure orders for new computer systems

By 1988, DEC’s AI group developed 40 expert systems

In 1981, Japanese announced the “Fifth Generation” project, a
10-year plan to build intelligent computers running Prolog.
and many more. . .

27
History of AI

1943 1952 1955 1966 1969 1973 19791980 1987 1995 2001

The birth of AI

Early enthusiasm

A dose of reality

Knowledge-based systems

AI becomes an industry

The return of neural networks

AI adopts the scientific methods

Emergence of intelligent agents

Availability of large data sets

28
The return of neural networks (1986-present)

Research on neural networks are divided into two main branches:

I connectionist – creating effective network architectures and
algorithms and understanding their mathematical properties
I symbolic – modelling of the empirical properties of actual
neurons and ensembles of neurons

29
History of AI

1943 1952 1955 1966 1969 1973 19791980 1987 1995 2001

The birth of AI

Early enthusiasm

A dose of reality

Knowledge-based systems

AI becomes an industry

The return of neural networks

AI adopts the scientific methods

Emergence of intelligent agents

Availability of large data sets

30
AI adopts the scientific method (1987-present)

AI was founded as rebellion against the limitations of existing fields

(control theory, statistics, etc.). At this stage, AI is embracing those
fields.
Research focuses on the improvement of existing theories rather than
to propose new ones.
The dominating techniques under such approach are:
I hidden Markov models (HMMs),
I data mining, and
I Bayesian network.

31
History of AI

1943 1952 1955 1966 1969 1973 19791980 1987 1995 2001

The birth of AI

Early enthusiasm

A dose of reality

Knowledge-based systems

AI becomes an industry

The return of neural networks

AI adopts the scientific methods

Emergence of intelligent agents

Availability of large data sets

32
Emergence of intelligent agents (1995-present)

I “-bot”
I search engines
I recommender systems

33
History of AI

1943 1952 1955 1966 1969 1973 19791980 1987 1995 2001

The birth of AI

Early enthusiasm

A dose of reality

Knowledge-based systems

AI becomes an industry

The return of neural networks

AI adopts the scientific methods

Emergence of intelligent agents

Availability of large data sets

34
Availability of large data sets (2001-present)

I More recent studies in AI suggest that “data” is more important

than “algorithm”.
I It’s found that the performance of the algorithm was poorer with
less data.
I “knowledge bottleneck” in AI – the problem of how to express all
the knowledge a system needs – can it be solved by learning
methods with enough data rather than hand-coded knowledge
engineering?

35
History of AI

1943 1952 1955 1966 1969 1973 19791980 1987 1995 2001

The birth of AI

Early enthusiasm

A dose of reality

Knowledge-based systems

AI becomes an industry

The return of neural networks

AI adopts the scientific methods

Emergence of intelligent agents

Availability of large data sets

36
Weak AI versus Strong AI

Weak AI
Any system that exhibits intelligent behaviour is considered as an
example of AI, therefore focusing on the output/performance of the
system.

Strong AI
This branch of AI aims to achieve intelligence by simulating the
biological systems of a human, therefore focusing on the structure of
the system.

37
Who are big in AI?

I Google – TensorFlow, Google Brain, Google DeepMind, Google AI

I Open source/Community – scikit-learn, pybrain, keras, theano
I Facebook – PyTorch
I Microsoft
I Matlab – Toolboxes
I IBM

38
What are the areas in AI?

AI Logic and
Knowledge
Representation

Search Systems

Machine
Learning

39
What are the techniques in these areas of AI?

Knowledge Representation Decision Tree Artificial Neural Network

Logic Fuzzy Logic K-means

Regressions KNN

Logic and Knowledge Representation

Machine Learning

AI

Systems
Search

Production Systems Expert Systems

Uninformed Search Informed Search Search using Games
Fuzzy Systems
Evolutionary Algorithms Swarm Intelligence

40
What are covered in this AI course?

Knowledge Representation Decision Tree Artificial Neural Network

Logic Fuzzy Logic K-means

Regressions KNN

Logic and Knowledge Representation

Machine Learning

AI

Systems
Search

Production Systems Expert Systems

Uninformed Search Informed Search Search using Games
Fuzzy Systems
Evolutionary Algorithms Swarm Intelligence

40
What are covered in CSC3034 Computational Intel-
ligence (CI)?

Knowledge Representation Decision Tree Artificial Neural Network

Logic Fuzzy Logic K-means

Regressions KNN

Logic and Knowledge Representation

Machine Learning

AI

Systems
Search

Production Systems Expert Systems

Uninformed Search Informed Search Search using Games
Fuzzy Systems
Evolutionary Algorithms Swarm Intelligence

40
Lecture Plan for CSC3206

Wk Practical (Mon) Lecture (Mon) Lecture (Thu)

1 P1 Intro L1 Introduction L2 Search

2 P2 Search L3 Search L4 Search

3 × × L5 Search

4 × × L6 Logic and KR

5 × × L7 Logic and KR

6 P3 Search L8 Logic and KR L9 PES

7 P4 Search L10 PES L11 PES

Semester break
8 × × ×

9 × × L12 Machine learning

10 × × L13 Machine learning

11 P5 Machine learning L14 Machine learning L15 Machine learning

12 P6 Machine learning L16 Machine learning L17 exercise

13 P7 Machine learning L18 revision –

14 – – –

KR Knowledge representation PES Production and expert systems

41
Assessment

Coursework 50%
Assignments (2) Due Week 7, 14 30%
Short Quizzes (2) Week 4, 12 10%
Lab test Week 14 10%
Final Examination 50%
Total 100%

42
Labs

Lab sheets are provided through

https://ricwtk.github.io/ai-labs

1. Prepare before lab sessions.

2. The lab will be discussed in the last 45 minutes of the lab
session.

43
Microsoft Team

Sign in with your Sunway imail account

Join the team with code nnbgyk0

44
References

1. Lucci, S. & Kopec, D., 2016. Artificial Intelligence in the 21st

Century. 2nd ed. Mercury Learning and Information.
2. Russell, S. & Norvig, P., 2010. Artificial Intelligence: A Modern
Approach. 3rd ed. Prentice Hall.
3. Jones, M.T., 2008. Artificial Intelligence: A Systems Approach.
Infinity Science Press.
4. Negnivitsky, M., 2005. Artificial Intelligence: A Guide to
Intelligent Systems 2nd ed. Addison Wesley.

45
Any question?

46