Theory of Computation

Dr. N. R. Ansari

Theory of Computation (Theory of Automata) (Introduction) LECTURE: 1

Dr. N. R. Ansari

Theory of Computation

About Me
Education
2001, MCS Bahria Institute / Peshawar University 2005, M.S. (Software Engineering) NUST 2008, Ph.D. (Computer Engineering) Graz University of Technology, Austria.

Research Interests
Model-based and Qualitative Reasoning
Distributed Computing Verification and Validation

Test Pattern Generation

Theoretical Computer Sciences
Dr. N. R. Ansari

Theory of Computation

About Me
Work Experiences
Permanent Faculty Member, COMSATS IIT Islamabad. Visiting Faculty Member, COMSATS, AIOU Research Assistant, Graz University of Technology, Austria Senior Researcher in a Defence Organization Offshore Post-Doc Technology. Researcher, Graz University of

Contact
nransari@hotmail.com

0331-5260536

Dr. N. R. Ansari

Theory of Computation

Pragmatics
Pre-Requisites
No Pre-Requisite

Text book
Introduction to Computer Theory by Daniel I.A. Cohen

Quizzes
Roughly once every two weeks Marks Distribution Theoretical Course 40 % Mid-term, 40% Final, 20 % Quizzes

Dr. N. R. Ansari

Theory of Computation

Course Objectives
The major objective of this course is to introduce the student to the concepts of theory of computation in computer science. The student should acquire insights into the relationship among formal languages, formal grammars, and automata. Upon successful completion of this course, students will be able to: Apply a number of proof techniques to theorems in language design.

Develop a clear understanding of un-decidability.

Understand the equivalence between Nondeterministic Finite State Automata and Deterministic Finite State Automata.

Understand the equivalence between Context-Free Grammars and Nondeterministic Pushdown Automata. Appreciate the power of the Turing Machine, as an abstract automaton, that describes computation, effectively and efficiently.
Dr. N. R. Ansari

Theory of Computation

Course Outline
(1) Introduction and Languages. (2) Regular Expressions. (3) Finite Automata. (4) Transition Graphs. (5) Kleene's Theorem. (6) Finite Automata with Output. (Moore /Mealy Machines) (7) Regular Languages. (8) Decidability. (9) Context-Free Grammars. (10)Grammatical Format. (11)Pushdown Automata. (12)CFG = PDA. (13)Context-Free Languages. (14)Decidability. (15)Turing Machines. (16)Minsky's Theorem. (17)Computers.
Dr. N. R. Ansari

Theory of Computation

Theory of Computation
The theory of computation is the branch of computer science and mathematics that deals with whether and how efficiently problems can be solved on a model of computation, using an algorithm.
A model of computation is the definition of the set of allowable operations used in computation and their respective costs The field is divided into two major branches computability theory and complexity theory

Computability theory deals primarily with the question of the extent to which a problem is solvable on a computer.

Complexity theory considers not only whether a problem can be solved at all on a computer, but also how efficiently the problem can be solved.
7

Dr. N. R. Ansari

Theory of Computation

Automata Theory

Automata theory is the study of abstract machines and problems which they are able to solve
A typical abstract machine consists of a definition in terms of input, output, and the set of allowable operations used to turn the former into the latter. Automata theory is a step in abstracting your attention away from any particular kind of computer or particular programming language In English we have letter, words and sentences (relationship) Not all collection of letters form a valid word, not all collections of words form a valid sentence

Dr. N. R. Ansari

Theory of Computation

Automata Theory

Same in Computer Languages

Certain character strings are recognized (do, if, While) Certain strings of words are recognizable commands Certain set of commands become a program Its important to adopt a structure, where the decision whether certain smaller units constitute a larger unit is based on explicit rules and not guesswork. It is important that a program compiles whether or not it does what the programmer intended If it compiles Its Valid and machine is responsible for executing the specified instructions Its hard to state all the rules for the language ENGLISH But in programming languages we can insist on precise rules, computers cannot understand informal speech
9

Dr. N. R. Ansari

Theory of Computation

Introduction to defining languages

Today, we will begin with only one finite set of fundamental units out of which we build structures, it is called alphabet.
A certain specified set of strings of characters from the alphabet is called the language.

The strings which are permissible in the language are called words.
We shall allow a string to have no letters. This we call the empty string or null string and denote with the symbol .

Lets start with an alphabet having only one letter, the x

= {x} L1 = {x xx xxx xxxx ..} Or L1 = {xn for n = 1 2 3 4..}

Similarly L2 = {x xxx xxxxx xxxxxxx .} L2 = {x2n+1 for n = 0 1 2 3 4..}

10

Dr. N. R. Ansari

Theory of Computation

Length and Reverse of a string and Palindrome

We define the length of the string to be the number of letters in the string.
If a = xxxx then length(a) = 4 Length(428) = 3

Length() = 0

If a is a word in some language L, then reverse(a) is the same string of letters spelled backward, called the reverse of a. Reverse(xxx) = xxx Reverse(420) = 024

If = { a b} then PALINDROME = { , and all strings x such that reverse(x) = x}

PALINDROME = { a b aa bb aaa aba bab bbb aaaa abba}

11

This language has some very interesting properties which we will examine later. Dr. N. R. Ansari

Theory of Computation

KLEENE CLOSURE

Given an alphabet , we wish to define a language in which any string of letters from is a word, even the null string
This is called closure of the alphabet and denoted by * This notation is sometimes known as kleene star * = L1 = { x xx xxx xxxx.}

If = {x}, then
If = {0 1} then

* = L2 = { 0 1 00 01 10 11 000 001.}
* = L3 = { a b c aa ab ac ba bb bc ca cb cc aaa ..}

If = {a b c} then

Dr. N. R. Ansari

12

Theory of Computation

KLEENE CLOSURE
If S is a set of words, then by S*, we mean the set of all finite string formed by concatenating words from S, where null string is also included.
If S = {aa b} then

S* = { plus any word composed of factors of a and ab}

S* = { b aa bb aab baa bbb aaaa aabb baab bbaa bbbb aaaab ..}

If we wish to modify the concept of closure to refer to only the concatenation of some (non-zero) strings from a set S, we use the notation + instead of * If = {x} then + = {x xx xxx xxxx }

Dr. N. R. Ansari

13

Theory of Computation

Quiz # 0

This Quiz carries 0 marks and will provide you the idea of the quizzes you are going to get in this course.
Exercises at the end of the chapter are important.

Consider the language S*, where S = {ab ba}

Is the string (abbba) a word in this language? What about abababababba

Write out all the words in S* that have seven or fewer letters.

Let S = {ab bb} and let T = {ab bb bbbb}, show that S* = T* Let S = {ab bb} and let T = {ab bb bbb}, show that S* T*
14

Dr. N. R. Ansari

Theory of Computation

We have completed Chapter 1 and Chapter 2

Forget about Chapter 1 (Not important) Next time we will not have any MULTIMEDIA

Dr. N. R. Ansari

15

Theory of Computation

Dr. N. R. Ansari

Theory of Computation (Theory of Automata) (Context Free Grammars) Chapter 12

Dr. N. R. Ansari

16

Theory of Computation

Grammars

Numerical numbers
1, 2, 3, I, II, III,.. ,,,.

Binary, Octal, Hex

Character sets are different, Grammars can be different G = <N, T, P, S> N = {S, A, V, C} // Nonterminal, parts of speech T = {(,), and , not, or, boolean variables} // Terminal, vocabulary P = { S (S) S b SA SV S C S not S } S S and S S S or S

A sentence is a string of symbols generated from the start symbol

not((b1 or b2) and (b2 or b3))
17

Dr. N. R. Ansari

Theory of Computation

Context Free Grammars

We have just studies Context Free Grammar (CFG)

Context free as sentence can be generated in any sequence On the left hand side of production rules we have a non-terminal and on the right hand side we have a sequence of terminal and nonterminals. The language generated by G is the set of all possible sentences that may be generated from the start symbol S. Context-free grammars are important because they are powerful enough to describe the syntax of programming languages.

almost all programming languages are defined via context-free grammars

Formal Definition from BOOK

Dr. N. R. Ansari

18

Theory of Computation

Grammars
Grammars express languages Example: the English language
sentence noun _ phrase article predicate noun

noun _ phrase predicate

verb
a the

article article noun noun

cat dog

verb runs verb walks

Dr. N. R. Ansari

Theory of Computation

Context Free Grammars

A derivation of the dog walks:
sentence noun _ phrase noun _ phrase article the noun the dog the dog noun verb verb walks predicate verb verb

Another grammar (derive ab)

Whats the language?

S aSb S

S aSb ab

S aSb
Dr. N. R. Ansari

S
20

Theory of Computation

Derivation Order 1. S AB 2. A aaA

Leftmost derivation:
1 2

3. A
3 4

4. B Bb 5. B
5

S AB aaAB aaB aaBb aab

Rightmost derivation:

S AB ABb Ab aaAb aab

Dr. N. R. Ansari

S aAB A bBb B A|
Leftmost derivation:

Theory of Computation

S aAB abBbB abAbB abbBbbB abbbbB abbbb

Rightmost derivation:

S aAB aA abBb abAb abbBbb abbbb

Dr. N. R. Ansari

Theory of Computation

More Examples from Book

Derivation Trees

Dr. N. R. Ansari

S AB

A aaA |

Theory of Computation

B Bb |

S AB
S

Dr. N. R. Ansari

S AB

A aaA |

Theory of Computation

B Bb |

S AB aaAB
S

A
a a

B A

Dr. N. R. Ansari

S AB

A aaA |

Theory of Computation

B Bb |

S AB aaAB aaABb
S

A
a a

B A B
b

Dr. N. R. Ansari

S AB

A aaA |

Theory of Computation

B Bb |

S AB aaAB aaABb aaBb

S

A
a a

B A B
b

Dr. N. R. Ansari

S AB

A aaA |

Theory of Computation

B Bb |

S AB aaAB aaABb aaBb aab

Derivation Tree

A
a a

B A B
b

Dr. N. R. Ansari

S AB

A aaA |

Theory of Computation

B Bb |

S AB aaAB aaABb aaBb aab

Derivation Tree

A
a a

B
yield

aab aab

Dr. N. R. Ansari

Theory of Computation

Ambiguity

Dr. N. R. Ansari

E E E | E E | (E) | a a aa
E E
a

Theory of Computation

E E E a E a EE a a E a a*a
E

leftmost derivation

E
a
Dr. N. R. Ansari

E
a

E E E | E E | (E) | a a aa

Theory of Computation

E EE E EE a EE E a aE a aa
leftmost derivation

E E
a

E
a

E
a

Dr. N. R. Ansari

Theory of Computation

E E E | E E | (E) | a a aa
Two derivation trees

E E
a

E E

E E
a

E
a

E
a
Dr. N. R. Ansari

E
a

E
a

E E E | E E | (E) | a is ambiguous:
The grammar string

Theory of Computation

a a a has two derivation trees

E E E

E
a

E
E E

E
a

a
Dr. N. R. Ansari

The grammar

E E E | E E | (E) | a is ambiguous:
string

Theory of Computation

a a a has two leftmost derivations

E E E a E a EE a a E a a*a E EE E EE a EE a aE a aa

Dr. N. R. Ansari

Theory of Computation

Why do we care about ambiguity?

a aa
E E
a

take

a2
E

E
a

E
a
Dr. N. R. Ansari

E
a

E
a

E
a

Theory of Computation

2 22 6
6 E

2 22 8
8 E

2 E 2

4 E

4 E 2 E 2 2 E 2

2 E 2

2 E 2
Dr. N. R. Ansari

2 E 2

Theory of Computation

Ambiguity is bad for programming languages

We want to remove ambiguity

Dr. N. R. Ansari

Theory of Computation

We fix the ambiguous grammar:

E E E | E E | (E) | a
New non-ambiguous grammar:

E E T E T T T F T F F (E) F a

Dr. N. R. Ansari

E E T T T F T a T a T F a F F a aF a aa

Theory of Computation

a aa
T

E E T E T T T F T F F (E) F a
Dr. N. R. Ansari

E T
F
a

T
F
a

F
a

Theory of Computation

Total Language Tree From Book

Dr. N. R. Ansari