The Chinese University of Hong Kong Fall 2011

CSCI 3130: Formal languages and automata theory

Turing Machines

Andrej Bogdanov
http://www.cse.cuhk.edu.hk/~andrejb/csc3130
What is a computer?

program
input
computer output

A computer is a machine that manipulates data

according to a list of instructions.
What is a computer?
void hello( String name)
{
print( “Hello, “, name);
}
? Hello, world
world

deep blue
?
E2-E4

?
computer
Turing Machines

head
control
a b b ☐ ☐ …

input blanks

Can both read from and write to the tape

Head can move both left and right
Tape is infinite
Has two special states accept and reject
Example
L1 = {w#w: w ∈{a, b}*}

Strategy:
Read and remember the first symbol abbaa#abbaa

Cross it off (x) xbbaa#abbaa

Read the first symbol past # xbbaa#abbaa

If they don’t match, reject

If they do, cross it off xbbaa#xbbaa
Example
L1 = {w#w: w ∈ {a, b}*}

Strategy:
Look for the first uncrossed symbol xbbaa#xbbaa

Cross it off (x) xxbaa#xbbaa

Read the first uncrossed symbol past # xxbaa#xbbaa

If they match, cross it off, else reject xxbaa#xxbaa

At the end, there should be only xs and #s xxxxx#xxxxx

If not, reject; otherwise, accept.
How Turing Machines operate
current state

q1
a/bL q2
a b a ☐ …

Replace a with b, and move head left

new state

q1
a/bL q2
a b b ☐ …
Formal Definition
A Turing Machine is (Q, , , , q0, qacc, qrej):
– Q is a finite set of states;
–  is the input alphabet not containing the blank symbol ☐
–  is the tape alphabet (  ) including ☐
– q0 in Q is the start state;
– qacc, qrej in Q are the accepting and rejecting state
–  is the transition function
: (Q – {qacc, qrej})   → Q    {L, R}

Turing Machines are deterministic

Configurations
• A configuration consists of the current state, the head
position, and tape contents

configuration
q1
a b a ☐ … abq1a

q1
a/bR qacc

qacc abbqacc
a b b ☐ …
Configurations
• We say configuration C yields C’ if the TM can go
from C to C’ in one step
abq1a yields abbqacc

• The start configuration of the TM on input w is q0w

• An accepting configuration is one that contains qacc;

A rejecting configuration is one that contains qrej
The language of a Turing Machine
• We say M accepts x if there exists a sequence of
configurations C0, C1, ..., Ck where

C0 is starting Ci yields Ci+1 Ck is accepting

The language recognized by M is the set of all

strings that M accepts
Looping
• Something strange can happen in a Turing Machine:
☐/☐R  = {0, 1}
qacc
input: e
q0 This machine
qrej never halts

• Inputs can be divided into three types

qacc qrej

accept reject loop

Halting
• We say M halts on x if there exists a sequence of
configurations C0, C1, ..., Ck where

Ck is accepting
C0 is starting Ci yields Ci+1
or rejecting

A TM M is a decider if it halts on every input

Language L is decidable if it is recognized by a

TM that halts on every input
Programming Turing Machines

Description of Turing Machine: L1 = {w#w: w ∈ {a, b}*}

1 Until you reach #

2 Read and remember entry xbbaa#xbbaa
3 Write x xxbaa#xbbaa
4 Move right past # and past all xs xxbaa#xbbaa
5 If this entry is different, reject
Otherwise
6 Write x xxbaa#xxbaa
7 Move left past # and to right of first x xxbaa#xxbaa
8 If you see only xs followed by ☐, accept
Programming Turing Machines
a/aR L1 = {w#w: w ∈ {a, b}*}
b/bR x/xR

#/#R a/aL
qa1 qa2 a/aL
4 b/bL
x/xR x/xL b/bL

#/#L
q0 #/#R
1 q1 ☐/☐R qacc q2 q3 qrej
8
7

everything else
4
qb1 qb2
#/#R

a/aR x/xR
b/bR

x/xR
Programming Turing Machines
a/aR input: aab#aab
b/bR x/xR
configurations:
#/#R a/aL
qa1
4
qa2
b/bL a/aL q0aab#aab
x/xR x/xL b/bL xqa1ab#aab
#/#R #/#L xaqa1b#aab
q0 q1 ☐/☐R qacc q2 q3
1
8
7 xabqa1#aab
xab#qa2aab
qb1 4
qb2 xabq2#xab
#/#R xaq3b#xab
a/aR x/xR xq3ab#xab
b/bR
q3xab#xab
x/xR xq0ab#xab
Programming Turing Machines
L2 = {aibjck: i × j = k and i, j, k > 0 }

aabbcccc
High-level description of TM:
For every a: aabbcccc
Cross off the same number of bs and cs aabbcccc
Uncross the crossed bs (but not the cs) aabbcccc
Cross off this a aabbcccc
If all as and cs are crossed off, accept. aabbcccc
aabbcccc
 = {a, b, c}  = {a, b, c, a, b, c, ☐} aabbcccc
Programming Turing Machines
L2 = {aibjck: i × j = k and i, j, k > 0 }

Low-level description of TM:

Scan input from left to right to check it looks like aa*bb*cc*
Move the head to the first symbol of the tape
For every a: how do we know?
Cross off the same number of bs and cs how to do this?
Restore the crossed of bs (but not the cs)
Cross off this a
If all as and cs are crossed off, accept.
Programming Turing Machines
Implementation details: aabbcccc

Move the head to the first symbol of the tape

Put a special marker on top of first a aabbcccc


Cross off the same number of bs and cs: aaqbbcccc



Replace b by b aabqbcccc


Move right until you see a c aabbqcccc



Replace c by c aabqbcccc


Move left just past the last b aabbqcccc



If any bs are left, repeat aabbcqccc



aabbqcccc


 = {a, b, c}  = {a, b, c, a, b, c, a, a, ☐}
 
Programming Turing Machines

L3 = {#x1#x2...#xl : xi ∈ {0, 1}* and xi ≠ xj for each i ≠ j}

High-level description of TM: #01#0011#1

On input w,
For every pair of blocks xi and xj in w,
Compare the blocks xi and xj
If they are the same, reject.
Otherwise, accept.
Programming Turing Machines

L3 = {#x1#x2...#xl : xi ∈ {0, 1}* and xi ≠ xj for each i ≠ j}

Low-level description: #01#0011#1

0. If input is e, or has exactly one #, accept.

1. Place a mark on the leftmost #


#01#0011#1

(i.e. replace # by #) and move right

2. Place another mark on next unmarked # #01#0011#1

 

(If there are no more #, accept)

Programming Turing Machines

L3 = {#x1#x2...#xl : xi ∈ {0, 1}* and xi ≠ xj for each i ≠ j}

 

current state: #01#0011#1

3. Compare the two strings to the right


#01#0011#1

of marked #. If there are same, reject

4. Move the right # to the right
 
#01#0011#1

If not possible, move the left # to the

next # and put the right # on the next
If not possible, accept

5. Repeat Step 3
 
#01#0011#1
Programming Turing Machines

L3 = {#x1#x2...#xl : xi ∈ {0, 1}* and xi ≠ xj for each i ≠ j}

 

3. Compare the two strings to the right #01#0011#1


of marked #. If there are same, reject  
#01#0011#1

4. Move the right # to the right

If not possible, move the left # to the accept

next # and put the right # on the next
If not possible, accept
How to describe Turing Machines?
• Unlike for DFAs, NFAs, PDAs, we rarely give
complete state diagrams of Turing Machines

• Usually we give a high-level description:

A recipe about the workings of the Turing Machine

Just like in cooking, practice makes perfect!

Programming Turing Machines
L4 = {〈G〉: G is a connected undirected graph}

Q: How do we feed a graph into a Turing Machine?

A: We represent it by a string, e.g.

(1,2,3,4)((1,4),(2,3),(3,4)(4,2))
1 2
Convention for describing graphs:
(nodes)(edges)
3 4 no node must repeat
edges are pairs (node1,node2)
Programming Turing Machines
L4 = {〈G〉: G is a connected undirected graph}

On input 〈G〉,
0. Verify that 〈G〉 is the description of a graph
(no vertex repeats; edges only go between nodes)

1. Mark the first node of G x x

1 2
2. Repeat until no new nodes are marked:
For each node, mark it if it is attached x x
to an already marked node 3 4

3. If all nodes are marked accept, otherwise reject.

Programming Turing Machines
L4 = {〈G〉: G is a connected undirected graph}

(1,2,3,4)((1,4)(2,3)(3,4)(4,2))

(1,2,3,4)((1,4)(2,3)(3,4)(4,2))

(1,2,3,4)((1,4)(2,3)(3,4)(4,2))
 

1 2 (1,2,3,4)((1,4)(2,3)(3,4)(4,2))
 
(1,2,3,4)((1,4)(2,3)(3,4)(4,2))
 
(1,2,3,4)((1,4)(2,3)(3,4)(4,2))
3 4  
(1,2,3,4)((1,4)(2,3)(3,4)(4,2))
 
(1,2,3,4)((1,4)(2,3)(3,4)(4,2))
 
(1,2,3,4)((1,4)(2,3)(3,4)(4,2))

etc.
  
(1,2,3,4)((1,4)(2,3)(3,4)(4,2))
What’s so great about
Turing Machines?
Hilbert’s list of 23 problems
• Leading mathematician in the 1900s

• At the 1900 International Congress

of Mathematicians, proposed a list of
23 problems for the 20th century
David Hilbert
(1862-1943)
• Hilbert’s 10th problem:

Find a method to tell if an equation like

xyz – 3xy + 6xz + 2 = 0 has integer solutions
A brief history of computing devices

Z3 (Germany, 1941) ENIAC (Pennsylvania, 1945)

PC (1980) MacBook Air (2008)

Computation is universal

In principle, all these

have the same problem-solving ability
The Church-Turing Thesis

Turing Machine

If an algorithm can be implemented on any

realistic computer, then it can be implemented
on a Turing Machine.
The Church-Turing Thesis

Turing Machine
quantum computing cosmic computing
DNA computing
Alan Turing

• Invented the Turing Test to tell

humans and computers apart
• Broke German encryption
machines in World War II
• The Turing Award is the “Nobel
Alan Turing prize of Computer Science”
(1912-1954)

Turing’s motivation: By studying Turing Machines,

we can understand the limitations of real
computers.
Undecidability
• What Hilbert meant to ask was:

Find a Turing Machine to tell if an equation like

xyz – 3xy + 6xz + 2 = 0 has integer solutions

• Building on work of Gödel, Church, Turing, Davis,

Putnam, Robinson, in 1970 Matijasievič showed:

There is no such Turing Machine!

Computer program analysis
public static void main(String args[]) {
System.out.println("Hello World!");
}

What does this program do?

public static void main(String args[]) {

int i = 0;
for (j = 1; j < 10; j++) {
i += j;
if (i == 28) {
System.out.println("Hello World!");
}
}
}

How about this one?

Computers cannot analyze programs!

No Turing Machine can do this:

input: The code of a java program P
Accept if P prints “hello, world”
Reject if not

Significance: It is impossible for computer to

predict what a computer program will do!
How do you argue things like that?

To argue what computers cannot do,

we need to have a precise definition
of what a computer is.

Turing’s answer: A computer is just

a Turing Machine.

1936: “On Computable Numbers, with an

Application to the Entscheidungsproblem”
Section 1. Computing Machines
The Church-Turing Thesis
All arguments [for the CT Thesis] which can be given are
bound to be, fundamentally, appeals to intuition, and for this
reason rather unsatisfactory mathematically.
The arguments which I shall use are of three kinds:
1. A direct appeal to intuition
2. A proof of the equivalence of two definitions
(In case the new definition has greater intuitive appeal)
3. Giving examples of large classes of numbers which are
computable.

1936: “On Computable Numbers, with an

Application to the Entscheidungsproblem”
Section 9. The extent of the computable numbers