Artificial Intelligence

Fall 2013
Kashif Naeem Assistant Professor Department of Computer Science, The Institute of Management Sciences, Lahore

Course Prerequisites
Computer Programming Discrete Structures Vector/Linear Algebra Theory of Automata

Recommended Textbook
Book Title
Artificial Intelligence: A Modern Approach

Edition
3rd Edition

Authors
Russel & Norvig

Publisher
Prentice Hall

Course overview
Introduction and Agents (chapters 1,2) Search (chapters 3,4,5,6) Logic (chapters 7,8,9) Planning (chapters 11,12) Uncertainty (chapters 13,14) Learning (chapters 18,20) Natural Language Processing (chapter 22,23)

Topics
History of AI What is AI? What AI can do? Search

Machine Learning

Graph Search Constraint Satisfaction Games Nearest Neighbors Decision Trees Neural Networks SVM

Knowledge Representation & Interface

Propositional and First-Order Logic Rule-based Systems Natural Languages

History of AI
1940-1950: Early days
1943: McCulloch & Pitts: Boolean circuit model of brain 1950: Turing's Computing Machinery and Intelligence

195070: Excitement: Logic Development!

1950s: Early AI programs, including
Samuel's checkers program, Newell & Simon's Logic Theorist, Gelernter's Geometry Engine

1956: Dartmouth meeting: Artificial Intelligence adopted 1965: Robinson's complete algorithm for logical reasoning E.g., generate plan for driving to the airport 1966: Weizenbaum's Eliza / Turing test

Herb Simon, 1957

It is not my aim to surprise or shock you but the simplest way I can summarize is to say that there are now in the world machines that think, that learn and that create. Moreover, their ability to do these things is going to increase rapidly until in a visible future the range of problems they can handle will be coextensive with the range to which human mind has been applied. More precisely: within 10 years a computer would be chess champion, and an important new mathematical theorem would be proved by a computer.

History of AI
197088: Knowledge-based approaches
196979: Early development of knowledge-based systems 198088: Expert systems industry booms 198893: Expert systems industry busts: AI Winter

1988: Statistical approaches

Resurgence of probability, focus on uncertainty General increase in technical depth Agents and learning systems AI Spring?

2000: Where are we now?

What is AI?
The Science of making machines that:
Think Humanly Think Rationally

Act Humanly

Act Rationally

Definition
The branch of Computer Science that is concerned with the automation of intelligent behavior. (Luger and Stubblefield, 1993) and many other definitions.
You should surf the internet for some definitions and their type illustration in detail?

What can AI do?

Quiz: Which of the following can be done at present? Play a decent game of table tennis? Drive safely along a curving mountain road? Drive safely along Telegraph Avenue? Buy a week's worth of groceries on the web? Buy a week's worth of groceries at Berkeley Bowl? Discover and prove a new mathematical theorem? Converse successfully with another person for an hour? Perform a complex surgical operation? Unload a dishwasher and put everything away? Translate spoken Chinese into spoken English in real time? Write an intentionally funny story?

Acting humanly: Turing Test

Turing (1950) "Computing machinery and intelligence": "Can machines think?" "Can machines behave intelligently?" Operational test for intelligent behavior: the Imitation Game Predicted that by 2000, a machine might have a 30% chance of fooling a lay person for 5 minutes

Anticipated all major arguments against AI in following 50 years

Suggested major components of AI:
knowledge,

reasoning,
language understanding learning

Rational Decisions
Here, the term Rational means: Maximally achieving pre-defined goals It only concerns what decisions are made (not the thought process behind them) Goals are expressed in terms of the utility of outcomes

And finally: maximizing your expected utility

(by designing and developing good Rational Agents) NOTE: A system is Rational if it does the right thing

Intelligent Behavior
Perceiving ones environment Acting in complex environments Learning and understating from experience Reasoning to solve problems and discover hidden knowledge Knowledge applying successfully in new situations Thinking abstractly using analogies Communication with others

Creativity
Ingenuity (originality) Expressiveness Curiosity

Hard / Strong AI
Generally, AI research aims to create AI that can replicate human intelligence completely. Strong AI refers to a machine that approaches or supersedes human intelligence:
If it can do typically human tasks If it can apply a wide range of background knowledge If it has some degree of self-consciousness

Soft / Weak AI
Weak AI refers to the use of software to study or accomplish specific problem solving or reasoning tasks that do not encompass the full range of human cognitive abilities. Example: A chess program such as Deep Blue Weak AI does not achieve self-awareness; it demonstrates wide range of human-level cognitive abilities; a specific problem solver

AIs Goals to pursue

Such definitions give four possible goals to pursue:
1. Think Humanly (Cognitive Science Approach) 2. Think Rationally (Laws of Thought Approach)

3. Act Humanly (Turing Test Approach)

4. Act Rationally (Rational Agent Approach)

Most of AI work falls into category 2 and 4.

1. Cognitive Science: Think Humanly

Machine with Minds (in full and literal sense) Computational Model as to HOW results are obtained. Focus is not just on behavior and I/O but looks at Reasoning Process.
Goal is not just to produce human behavior but to produce a sequence of steps of the reasoning process, similar to the steps followed by a human in solving the same task.

2. Laws of Thought: Think Rationally

The study of the computations that make it possible to perceive, reason and act. Focus is on inference (conclusion) mechanisms that are provably correct and guarantee an optimal solution. Develop systems of representations to allow inferences to be like Socrates is a man. All men are mortal. Therefore, Socrates is mortal.

Goal is to formalize the reasoning process as a system of logical and procedures for inference.
The issue is, not all problems can be solved just by reasoning and inferences.

3. Turing Test: Act Humanly

Focus is on action, and not intelligent behavior A behaviorist approach is not concerned with how to get results but the similarity to what human results are. Goal is to develop systems that are human-like

3. Turing Test: Act Humanly

Example: Turing Test
3 rooms contain: a person, a computer and a interrogator

The interrogator can communicate with the other 2 by teletype

The interrogator tries to determine which is the person and which is the machine The machine tries to fool the interrogator to believe that it is the human and the person also tries to convince the interrogator that is the human If the machine succeeds in fooling the interrogator then conclude that the machine is intelligent.

4. Rational Agent: Act Rationally

Rational behavior: doing the right thing The right thing: that which is expected to maximize goal achievement, given the available information

Doesn't necessarily involve thinking e.g., blinking reflex but thinking should be in the service of rational action

4. Rational Agent: Act Rationally

Tries to explain and emulate intelligent behavior in terms of computation process; that is concerned with the automation of intelligence.

Focus is on systems that act sufficiently is not optionally in all situations

It is passable to have imperfect reasoning if the job gets done. Goal is to develop systems that are rational and sufficient.

Rational agents
An agent is an entity that perceives and acts Abstractly, an agent is a function from percept histories to actions: [f: P* A] For any given class of environments and tasks, we seek the agent (or class of agents) with the best performance Caveat: computational limitations make perfect rationality unachievable design best program for given machine resources

State Description in AI Techniques

Initial state Transition Current state Final state

Action functions causes the change in state. State space is a mathematical model of a physical system as a set of input, output and state variables related to each others. The number of inputs, outputs and states, the variables are expressed as vectors.

AI Techniques
Various techniques are available and are concerned with how we present, manipulate and reason with knowledge in order to solve problems. Example:
Techniques, not all intelligent but used to behave as intelligent
1. Describe and match

2.
3. 4. 5.

Goal reduction
Constraint satisfaction Tree searching Generate and Test (GT)

6.
1. 2. 3.

Rule-Based Systems (RBSs)

Neural Networks (NN) Genetic Algorithms (GA) Reinforcement Learning (RL)

Biology-inspired AI techniques are currently popular

1. Describe & Match

Model is a description of a systems behavior
Finite State Model consists of a set of states (set of input events and the relations between them). Given a current state and an input event you can determine the next current state of model. Computational Model is a finite state machine. It includes a set of states (a set of start states, an input alphabet and a transition function which maps input symbols and current states to a next state).
Representation of Computational System includes start and end state descriptions and a set of possible transition rules that might be applied. Problem is to find appropriate transition rules.

1. Describe & Match

Transition Relation: If a pair state (S, S) is such that one move takes the system from S to S then the transition relation is represented by S => S Sate-Transition System is called deterministic if every state has at most one successor (descendant) and it is called non-deterministic if at least one state has more than one successor. Further example of some possible transitions between states are discussed in Towers of Hanoi puzzle.

Example: Tower of Hanoi: (with 2 disks)

Shortest solution is the sequence of transitions from the top state downward to the lower left.

2. Goal Reduction
Goal-reduction procedure are special case of the procedural representations of knowledge in AI The process involves the hierarchical sub-division of goals in sub-goals, until the sub-goals which have an immediate solution are reached and said goal has been satisfied. An AND/OR tree structure can represent relations between goals and sub-goals, alternate sub-goals and conjoint sub-goals.

2. Goal Reduction
Goal reduction process is illustrated in the form of AND/OR tree drawn upside-down. Goal levels: Higher-level goals are higher in tree and lower level goals are lower in the tree.

Arcs are directed from a higher-to-lower level node represents the reduction of higher-level goal to lowerlevel sub-goal.
Nodes at the bottom of the tree represents irreducible action goals.

Work hard

Work hard

3. Constraint Satisfaction Techniques

Constraint is a logical relation among variables e. g. Circle is inside a square The constrains relate objects without specifying their position; moving any one, the relation is still maintained. Constraint satisfaction is a process of finding a solution to a set of constraints. Constraint Satisfaction Problem (CSP) and its solution
A CSP consists of:
Variables, a finite set X = {x1, . . . xn} Domain, a finite set Di of possible values which each variable xi can take Constraints, a set of values that the variables can simultaneously satisfy the constraint (e.g. D1 != D2)

3. Constraint Satisfaction Techniques

A solution to a CSP is an assignment of a value from its domain to every variable satisfying every constraint; that could be:
One solution, with no preference as to which one,

All solutions,
An optimal, or a good solution Constraint Optimization Problem (COP)

Constraint satisfaction has application in AI programming languages, symbolic computing, computational logic and etc.
(NOTE: Two examples to be illustrated by the students)

4. Tree Searching
Many problems (e.g. goal reduction) can be described in the form of a search tree. A solution to the problem is obtained by finding a path through this tree. A search through the entire tree, until a satisfactory path is found, is called exhaustive search.

Tree Search Strategies

1. Depth-first Search
Assumes any one path is as good as any other path. At each node, pick an arbitrary path and work forward until a solution is found or a dead end is reached.

In the case of a dead end backtrack to the last node in the tree where a previously unexplored path branches of, and test this path.

Tree Search Strategies

Types of Backtracking
Chronological Backtracking:
Undo everything as we move back up the tree to a suitable note.

Dependency Directed Backtracking:

Only withdraw choices that matter (i.e. those on which dead ends depend)

Tree Search Strategies

1. Breadth-first Search
Look for a solution amongst all nodes at given level before proceeding to the next

2.

Hill Climbing
Like depth-first but involving some quantitative decision on the most likely path to follow at each node

3.

Beam Search
Lie breadth-first (level by level) but selecting only those N nodes at each level that are most likely to lead to a solution

4.

Best-first Search
Like beam search but only proceeding from on most likely node at each level.

5. Generate and Test (GT)

The method first guesses the solution and then tests whether this solution is correct, means solution satisfies the constraint.
This paradigm involves two processes:
Generator to enumerate possible solution (hypothesis) Test to evaluate each proposed solution

The algorithm is:

5. Generate and Test (GT)

Disadvantages Not very efficient: generates many wrong assignments of values to variables which are rejected in testing phase. Generator leaves conflicting instantiations and it generates other assignments independently of the conflict. For better efficiency GT approach need to be supported by backtracking approach.
(Example: Opening a combination lock without knowing the combination)

6. Rule-Based Systems (RBSs)

RBSs are simple and successful AI technique.
Rules are of the form: IF <cond> THEN <action>. Rules are often arranges in hierarchies (and/or trees). When all conditions of a rule are satisfied the rule is triggered.
Three components

6. Rule-Based Systems (RBSs)

Working Memory (WM)
Contains facts about the world observed or derived from a rule; stored as a triplet <object, attribute, value> e.g. <car, color, red> The color of my car is red. Contains temporary knowledge about problemsolving session. Can be modified by the rules.

6. Rule-Based Systems (RBSs)

Rule Base (RB)
RB contains rules where each rule is a step in problem solving. Rules are domain knowledge and modified only from outside.

Rule syntax is IF <condition> THEN <action> e.g. IF <(temperature, over, 20)> THEN <add (ocean, swimmable, yes)>
If the conditions are matched to the working memory and if fulfilled then rule may be fired. RB actions are: Add Remove Modify fact(s) to WM; fact(s) to WM; fact(s) to WM;

6. Rule-Based Systems (RBSs)

Interpreter
It is the domain independent reasoning mechanism for RBS. It selects rule from Rule Based and applies by performing action. It operates on a cycle:
Retrieval Refinement Execution - Find the rules that matches the current WM; - Prunes, reorders and resolves conflicts; - Executes the actions of the rules in the Conflict Set, then applies the rule by performing action.

Biology-Inspired AI Techniques