CSE 211: Introduction to the Theory of Computation

Md. Shaifur Rahman

Dept. of CSE
Bangladesh University of Engineering & Technology
Week 1 Class 1
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 1 / 29
Outline
1
Necessity of CS Theory Courses
2
What is Theory of Computation?
3
Course Outline
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 2 / 29
Outline
1
Necessity of CS Theory Courses
2
What is Theory of Computation?
3
Course Outline
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 2 / 29
Outline
1
Necessity of CS Theory Courses
2
What is Theory of Computation?
3
Course Outline
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 2 / 29
Why Theory? (again!)
We dont like theory, because:
One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
Where is the output?
How is it related to our career-path?
We like non-theory courses,
We like programming in C, C++, JAVA, Python, Android etc.
The outcome is visible, projects done in these courses brighten our
resume
The skills are necessary for the career-path of a developer
Achieving grades are comparatively easy :)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
Why Theory? (again!)
We dont like theory, because:
One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
Where is the output?
How is it related to our career-path?
We like non-theory courses,
We like programming in C, C++, JAVA, Python, Android etc.
The outcome is visible, projects done in these courses brighten our
resume
The skills are necessary for the career-path of a developer
Achieving grades are comparatively easy :)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
Why Theory? (again!)
We dont like theory, because:
One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
Where is the output?
How is it related to our career-path?
We like non-theory courses,
We like programming in C, C++, JAVA, Python, Android etc.
The outcome is visible, projects done in these courses brighten our
resume
The skills are necessary for the career-path of a developer
Achieving grades are comparatively easy :)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
Why Theory? (again!)
We dont like theory, because:
One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
Where is the output?
How is it related to our career-path?
We like non-theory courses,
We like programming in C, C++, JAVA, Python, Android etc.
The outcome is visible, projects done in these courses brighten our
resume
The skills are necessary for the career-path of a developer
Achieving grades are comparatively easy :)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
Why Theory? (again!)
We dont like theory, because:
One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
Where is the output?
How is it related to our career-path?
We like non-theory courses,
We like programming in C, C++, JAVA, Python, Android etc.
The outcome is visible, projects done in these courses brighten our
resume
The skills are necessary for the career-path of a developer
Achieving grades are comparatively easy :)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
Why Theory? (again!)
We dont like theory, because:
One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
Where is the output?
How is it related to our career-path?
We like non-theory courses,
We like programming in C, C++, JAVA, Python, Android etc.
The outcome is visible, projects done in these courses brighten our
resume
The skills are necessary for the career-path of a developer
Achieving grades are comparatively easy :)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
Why Theory? (again!)
We dont like theory, because:
One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
Where is the output?
How is it related to our career-path?
We like non-theory courses,
We like programming in C, C++, JAVA, Python, Android etc.
The outcome is visible, projects done in these courses brighten our
resume
The skills are necessary for the career-path of a developer
Achieving grades are comparatively easy :)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
Why Theory? (again!)
We dont like theory, because:
One has to memorize a lot of theorems, lemmas, proofs
It is inherently boring!
Where is the output?
How is it related to our career-path?
We like non-theory courses,
We like programming in C, C++, JAVA, Python, Android etc.
The outcome is visible, projects done in these courses brighten our
resume
The skills are necessary for the career-path of a developer
Achieving grades are comparatively easy :)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 3 / 29
Theory vs. Non-theory
Dicult to separate theories from all courses!
Course Theory-bias
CSE 103 (Discrete Math) 90%
CSE 105 (Structured Programming Language) 30%
CSE 201 (Object Oriented Programming Language) 20%
CSE 203 (Data Structure) 80%
CSE 205 (Digital Logic Design) 60%
In this semester:
Course Theory-bias
CSE 207 (Algorithms) 95%
CSE 209 (Digital Electronics and Pulse Technique) 40%
CSE 211 (Theory of Computation) 100%
CSE207 is the fraternal brother of CSE211
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 4 / 29
The Power of Theory
Write a C program that nds all the factors of a 512-bit integer
For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 2
3
3
But wait! 512-bit number!
So the input ranges from 1 to 2
512
1 thats roughly 1.3 10
154
The highest input can be of 155-digits!
Input: 12345676449393993 . . . 930049922 (155 digits)
3 hours gone, your C-program is not coming up with any answer?
Your program worked perfectly for 1, 2, 10, 20 even 50 digit inputs!
So whats wrong??
Is your program erroneous?
Is the computation power of the machine insucient?
Is there a probem in the nature of the input?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 5 / 29
The Power of Theory
Write a C program that nds all the factors of a 512-bit integer
For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 2
3
3
But wait! 512-bit number!
So the input ranges from 1 to 2
512
1 thats roughly 1.3 10
154
The highest input can be of 155-digits!
Input: 12345676449393993 . . . 930049922 (155 digits)
3 hours gone, your C-program is not coming up with any answer?
Your program worked perfectly for 1, 2, 10, 20 even 50 digit inputs!
So whats wrong??
Is your program erroneous?
Is the computation power of the machine insucient?
Is there a probem in the nature of the input?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 5 / 29
The Power of Theory
Write a C program that nds all the factors of a 512-bit integer
For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 2
3
3
But wait! 512-bit number!
So the input ranges from 1 to 2
512
1 thats roughly 1.3 10
154
The highest input can be of 155-digits!
Input: 12345676449393993 . . . 930049922 (155 digits)
3 hours gone, your C-program is not coming up with any answer?
Your program worked perfectly for 1, 2, 10, 20 even 50 digit inputs!
So whats wrong??
Is your program erroneous?
Is the computation power of the machine insucient?
Is there a probem in the nature of the input?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 5 / 29
The Power of Theory
Write a C program that nds all the factors of a 512-bit integer
For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 2
3
3
But wait! 512-bit number!
So the input ranges from 1 to 2
512
1 thats roughly 1.3 10
154
The highest input can be of 155-digits!
Input: 12345676449393993 . . . 930049922 (155 digits)
3 hours gone, your C-program is not coming up with any answer?
Your program worked perfectly for 1, 2, 10, 20 even 50 digit inputs!
So whats wrong??
Is your program erroneous?
Is the computation power of the machine insucient?
Is there a probem in the nature of the input?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 5 / 29
The Power of Theory
Write a C program that nds all the factors of a 512-bit integer
For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 2
3
3
But wait! 512-bit number!
So the input ranges from 1 to 2
512
1 thats roughly 1.3 10
154
The highest input can be of 155-digits!
Input: 12345676449393993 . . . 930049922 (155 digits)
3 hours gone, your C-program is not coming up with any answer?
Your program worked perfectly for 1, 2, 10, 20 even 50 digit inputs!
So whats wrong??
Is your program erroneous?
Is the computation power of the machine insucient?
Is there a probem in the nature of the input?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 5 / 29
The Power of Theory
Write a C program that nds all the factors of a 512-bit integer
For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 2
3
3
But wait! 512-bit number!
So the input ranges from 1 to 2
512
1 thats roughly 1.3 10
154
The highest input can be of 155-digits!
Input: 12345676449393993 . . . 930049922 (155 digits)
3 hours gone, your C-program is not coming up with any answer?
Your program worked perfectly for 1, 2, 10, 20 even 50 digit inputs!
So whats wrong??
Is your program erroneous?
Is the computation power of the machine insucient?
Is there a probem in the nature of the input?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 5 / 29
The Power of Theory (Contd.)
Case: Your program is erroneous
How do you know? The program is not even stopping!
It may indeed stop after 10 hours and show the correct output!
Case: Computation Power of the Machine is insucient
We may use multi-core processors clocked at a very high speed
like: 10-core processor each core clocked at 3 GHz
Can we aord that machine?
What if the number is even bigger? Can the machine scale up to the
input?
Caveat: Even with todays fastest computer, the computation for a
500-digit input will take 5 billion years!
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 6 / 29
The Power of Theory (Contd.)
Case: Your program is erroneous
How do you know? The program is not even stopping!
It may indeed stop after 10 hours and show the correct output!
Case: Computation Power of the Machine is insucient
We may use multi-core processors clocked at a very high speed
like: 10-core processor each core clocked at 3 GHz
Can we aord that machine?
What if the number is even bigger? Can the machine scale up to the
input?
Caveat: Even with todays fastest computer, the computation for a
500-digit input will take 5 billion years!
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 6 / 29
The Power of Theory (Contd.)
Case: Your program is erroneous
How do you know? The program is not even stopping!
It may indeed stop after 10 hours and show the correct output!
Case: Computation Power of the Machine is insucient
We may use multi-core processors clocked at a very high speed
like: 10-core processor each core clocked at 3 GHz
Can we aord that machine?
What if the number is even bigger? Can the machine scale up to the
input?
Caveat: Even with todays fastest computer, the computation for a
500-digit input will take 5 billion years!
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 6 / 29
The Power of Theory (Contd.)
Case: Nature of Input
Some numbers are too big or too small
Are you crazy? 500-digit number? Thats pretty big!
So is:
the number of stars in the universe
the number of electrons in your body
the energy in electron volt when two stars collide
the number of combinatorial choices of pairings of almost half-million
genes in our cell
So, data can be very large or too small? You never can tell
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 7 / 29
The Power of Theory (Contd.)
Case: Nature of Input
Some numbers are too big or too small
Are you crazy? 500-digit number? Thats pretty big!
So is:
the number of stars in the universe
the number of electrons in your body
the energy in electron volt when two stars collide
the number of combinatorial choices of pairings of almost half-million
genes in our cell
So, data can be very large or too small? You never can tell
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 7 / 29
The Power of Theory (Contd.)
Case: Nature of Input
Some numbers are too big or too small
Are you crazy? 500-digit number? Thats pretty big!
So is:
the number of stars in the universe
the number of electrons in your body
the energy in electron volt when two stars collide
the number of combinatorial choices of pairings of almost half-million
genes in our cell
So, data can be very large or too small? You never can tell
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 7 / 29
The Power of Theory (Contd.)
Case: Nature of Input
Some numbers are too big or too small
Are you crazy? 500-digit number? Thats pretty big!
So is:
the number of stars in the universe
the number of electrons in your body
the energy in electron volt when two stars collide
the number of combinatorial choices of pairings of almost half-million
genes in our cell
So, data can be very large or too small? You never can tell
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 7 / 29
The Power of Theory (Contd.)
Case: Nature of Input
Some numbers are too big or too small
Are you crazy? 500-digit number? Thats pretty big!
So is:
the number of stars in the universe
the number of electrons in your body
the energy in electron volt when two stars collide
the number of combinatorial choices of pairings of almost half-million
genes in our cell
So, data can be very large or too small? You never can tell
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 7 / 29
The Power of Theory (Contd.)
Case: Nature of Computation
Can we break down the processing into multiple blocks and distribute
those among multiple computers?
Can we predict the time required by analyzing the nature of input
(input-domain) and the method of computation (algorithm)?
Can we identify certain problems as really hard problems?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 8 / 29
The Power of Theory (Contd.)
Case: Nature of Computation
Can we break down the processing into multiple blocks and distribute
those among multiple computers?
Can we predict the time required by analyzing the nature of input
(input-domain) and the method of computation (algorithm)?
Can we identify certain problems as really hard problems?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 8 / 29
The Power of Theory (Contd.)
Case: Nature of Computation
Can we break down the processing into multiple blocks and distribute
those among multiple computers?
Can we predict the time required by analyzing the nature of input
(input-domain) and the method of computation (algorithm)?
Can we identify certain problems as really hard problems?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 8 / 29
Really Hard Problems
General Tile Fitting Problem
Given a nite set of tiles and an area on a plane, is it possible to cover the
complete area using each type of tiles at least once?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 9 / 29
Really Hard Problems (Contd.)
Sometimes, there is a solution
Sometimes, there is no solution
Sometimes, we cannot tell for sure- if there is a solution! (undecidable)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 10 / 29
Really Hard Problems (Contd.)
There are many problems that cannot be guaranteed to be solved by any
computer! These are called undecidable problems
David Hilbert
Entscheidungsproblem
Give an algorithm A that takes as input
a hypothesis H and answers True or
False based on a nite set of axioms S
A
.
In other word, Algorithm A decides
whether H logically follows from S
A
.
Hypothesis: A machine works without
consuming any energy.
Decision: False (according to the set of
axioms of Physics)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 11 / 29
Really Hard Problems (Contd.)
Alonzo Church & Alan Turing
Alonzo Church and Alan Turing independently proved respectively in 1936
and in 1937 that
Hilberts Entscheidungsproblem is undecidable
This is known as Church-Turing Thesis.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 12 / 29
Really Hard Problems (Contd.)
More undecided problems:
Mortal Matrix Problem
Given a nite set of n n matrices with integer entries, determine
whether they can be multiplied in some order, possibly with repetition, to
yield the zero matrix.
Conjugacy problem
Given two words x and y of a Group G, whether there exists a z such that:
y = zxz
1
*A group is an algebraic structure consisting of a set with one or more
operators such that the result of applying the operators on one or more
elements of the is also an element in the set. For example, the set of
integer I is a group.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 13 / 29
Really Hard Problems (Contd.)
Some problems are hard but still solvable i.e. NOT undecidable
(Euclidian) Travelling Salesman Problem
Given a set of vertices and their coordinates, nd a tour or cycle which
visits each vertex exactly once and is of the minimum length.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 14 / 29
Really Hard Problems (Contd.)
Some problems are hard but still solvable i.e. NOT undecidable
Chromatic Number or Graph Coloring Problem
Given a graph G = (V, E), nd the minimum number of dierent colors
needed for this graph such that two adjacent vertices are not given the
same color.
Total three dierent colors are needed
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 15 / 29
Really Hard Problems (Contd.)
Not all hard problems are related to graphs!
Circuit Satisability problem
Determine whether a given Boolean circuit has an assignment of its inputs
that makes the output true (1).
An assignment for output 1
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 16 / 29
Really Hard Problems (Contd.)
There is relationships among the hard solvable problems, we can compare
their hardness!
Circuit satisability problem
(Circuit-SAT) can be reduced
to Travelling Salesman
Problem (TSP)
If there is a good solution to
Circuit-SAT, there is also a
good solution to TSP
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 17 / 29
Really Hard Problems (Contd.)
Moral of the story
If the given problem is proved to be undecidable or unsolvable, dont
invest your valuable eort to write program to solve it.
If the given problem is solvable but still hard, it is not your programs
fault that the computation is taking too much time.
If the given problem is solvable and not hard, and your program fails
to solve it in the reasonable amount of time, only then invest eort
for improvement of your program!
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 18 / 29
Really Hard Problems (Contd.)
Moral of the story
If the given problem is proved to be undecidable or unsolvable, dont
invest your valuable eort to write program to solve it.
If the given problem is solvable but still hard, it is not your programs
fault that the computation is taking too much time.
If the given problem is solvable and not hard, and your program fails
to solve it in the reasonable amount of time, only then invest eort
for improvement of your program!
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 18 / 29
Really Hard Problems (Contd.)
Moral of the story
If the given problem is proved to be undecidable or unsolvable, dont
invest your valuable eort to write program to solve it.
If the given problem is solvable but still hard, it is not your programs
fault that the computation is taking too much time.
If the given problem is solvable and not hard, and your program fails
to solve it in the reasonable amount of time, only then invest eort
for improvement of your program!
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 18 / 29
Theory of Computation
Theory of computation
The branch that deals with whether and how eciently problems can be
solved on a model of computation, using an algorithm
The eld is divided into three major branches:
Automata Theory
Computability Theory and
Computational Complexity Theory
There are many models of computation:
Turing Machine
Combinatory logic
Markov Algorithm etc.
In our CS couses, we use Turing Machine model.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 19 / 29
Theory of Computation
Theory of computation
The branch that deals with whether and how eciently problems can be
solved on a model of computation, using an algorithm
The eld is divided into three major branches:
Automata Theory
Computability Theory and
Computational Complexity Theory
There are many models of computation:
Turing Machine
Combinatory logic
Markov Algorithm etc.
In our CS couses, we use Turing Machine model.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 19 / 29
Theory of Computation
Theory of computation
The branch that deals with whether and how eciently problems can be
solved on a model of computation, using an algorithm
The eld is divided into three major branches:
Automata Theory
Computability Theory and
Computational Complexity Theory
There are many models of computation:
Turing Machine
Combinatory logic
Markov Algorithm etc.
In our CS couses, we use Turing Machine model.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 19 / 29
Theory of Computation
Theory of computation
The branch that deals with whether and how eciently problems can be
solved on a model of computation, using an algorithm
The eld is divided into three major branches:
Automata Theory
Computability Theory and
Computational Complexity Theory
There are many models of computation:
Turing Machine
Combinatory logic
Markov Algorithm etc.
In our CS couses, we use Turing Machine model.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 19 / 29
Branches of Theory of Computation
Automata Theory
the study of abstract machines ( in our case, Turing Machine) and
the problems that can be solved using these machines
If the abstract machine cannot solve the problem, no computer on
earth, no matter how powerful it is, can ever solve the problem
Topics include: Finite Automaton (FA), Push-down Automaton
(PDA), Turing Machine (TM) etc.
studied in the theory of computation, compiler design, verication
systems, robotics etc.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 20 / 29
Branches of Theory of Computation (Contd.)
Computability Theory
deals with the question of the extent to which a problem is solvable
on a computer
Topics include: Turing Degree (level of unsolvabiltiy), Reducibility
(the ability to transform one hard problem to another) etc.
studied in mathematical logic theories like recursion theory, articial
intelligence, machine learning etc.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 21 / 29
Branches of Theory of Computation (Contd.)
Computational Complexity Theory
deals with classifying computational problems according to their
inherent diculty
Two major aspects are considered: time complexity (how many steps
the algorithm takes) and space complexity (how much memory is
required by the program)
studies complexity classes: P, NP, NPC and relationships among
these classes
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 22 / 29
Course Outline
Course Name: CSE 211 (Theory of Computation)
Credit Hour: 2.0
Number of Class Tests: 3
Course Teachers:
Md. Shaifur Rahman (rst seven weeks before mid-term)
and
Sukarna Barua (after mid-term)
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 23 / 29
Class Outline
Week Day Topics
1
1 Introduction to TOC
2 Deterministic Finite Automaton (DFA)
2
1 DFA (Contd.)
2 Operators of Finite Automaton
3
1 Non-deterministic Finite Automaton (NFA)
2 Equivalence of DFA and NFA
4
1 Regular Expressions
2 Class Test-1
5
1 Regular Expressions (Contd.)
2 Pumping Lemma
6
1 Context-free Grammar (CFG)
2 CFG (Contd.)
7
1 Overall Discussion
2 Class Test-2
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 24 / 29
Textbooks
Main textbook
Introduction to the Theory of
Computation (3rd Edition)
by
Michael Sipser
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 25 / 29
Textbooks (Contd.)
Supporting textbooks
Introduction to Automata Theory,
Languages & Computation
(3rd Edition)
by
Hopcroft et. al.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 26 / 29
Textbooks (Contd.)
Supporting textbooks
Elements of the Theory of
Computation
(2nd Edition)
by
Lewis et. al.
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 27 / 29
Course Website
http://sites.google.com/site/cse211february2013
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 28 / 29
Question?
Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 29 / 29