Undecidability

There exist problems that cannot be solved using a computer or a Turing machine. The difficulty in
solving such problems is not due to limits of processing power or memory. Such problems are called
undecidable problems. Since Turing machines have infinite memory, they represent the theoretical limits
of computation i.e. they define what all problems any computer can solve. So if Turing machine fails, the
problem ends up undecidable. A popular undecidable problem is the halting problem (determining
whether a Turing machine halts or runs forever).

Decidable problems are those that can be solved using a Turing machine. A decision problem A is
called decidable or effectively solvable if A is a recursive set. A problem is
called partially decidable, semi decidable, solvable, or provable if A is a recursively
enumerable set. Fig I shows the closure of decision problems. A classic example of a
decidable decision problem is the set of prime numbers. It is possible to effectively decide whether a
given natural number is prime by testing every possible nontrivial factor.

Fig I: Closure of decision problems

We can divide problems that can be solved by a Turing machine into two classes: those that have an
algorithm (halts whether or not it accepts its input), and those that are only solved by Turing machines
that may run forever on inputs they do not accept. The latter form of acceptance does not actually solve
the problem, since no matter how long the TM runs, we cannot know whether the input is accepted or not.
Thus, those problems with no algorithm are classified as undecidable, regardless of whether or not they
are accepted by a Turing machine that falls to halt on some inputs.

Undecidable Problems

An undecidable problem is a decision problem for which it is impossible to construct

a single algorithm that always leads to a correct yes-or-no answer — the problem is
not decidable. The language of an undecidable problem is not a recursive set. There
are problems that are often easily expressed but still end up undecidable.

Let us imagine simulating an undecidable program. Here the problem will take unimaginably long time
before we will be eventually getting an output from the program. This uncertainty about when we will be
getting an output makes the problem undecidable. The problem of undecidability can be understood by a
simple example. Everyone knows the famous “Hello World” program.

main()
{
printf(“Hello World”);
}

Now let us introduce the undecidable problem using the above program. The undecidable problem
statement is: “Determine whether a given program prints ’Hello World’”. At first this problem seems to
be easy, and a solution will be to run the program and see the output. If the output is ‘Hello World’ then
output “Yes” otherwise output “No”. Is it as simple as it seems? There are infinite ways one can write a
program that eventually prints ‘Hello World’. Let us consider another program that eventually prints
‘Hello World’.

int exp ( int i, n)

/* computes i to the power n */
{
int ans, j;
ans = 1;
for (j=1; j<=n; j++) ans *= i;
return (ans);
}
main ()
{
int n, total, x, y, z;
scanf ("% d", &n);
total = 3;
while (1)
{
for (x=1; x<=total-2; x++)
for (y=1; y<=total-x-1; y++)
{
z = total - x - y;
if (exp(x, n) + exp(y, n) == exp (z, n))
printf ("Hello World")
}
total + +;
}
}

The peculiarity in this program is that it takes an input n, and looks for the positive integer solutions to the
equations x" + y" = z n . If it finds a set of valid x, y and z values, it prints ‘Hello World’. Otherwise it will
continue searching forever and never print ‘Hello World’.

To understand what this program does, first observe that exp() is an auxiliary function to compute
exponentials. The main program needs to search through triples (x, y, z) in an order to get to the valid
combination satisfying x" + y" = z n . To organize the search properly, we use a fourth variable, total, that
starts at 3 and, in the while-loop, it is incremented by one, eventually reaching any finite integer. Inside
the while-loop, we divide total into three positive integers x, y, and z, by first allowing x to range from 1
to total-2, and within that for-loop allowing y to range from 1 up to one less than what x has not 'already
taken from total. What remains, which must be between 1 and total-2, is given to z. In the innermost loop,
the triple (x, y, z) is tested to see if x n + y" = z n. If so, the program prints ‘Hello world’, and if not, it
prints nothing.

Suppose we give the value 2 for n, then it will eventually find combinations of integers such as total = 12,
x = 3, y = 4, and z = 5, for which x n + yn = z n. Thus, for input 2, the program does print ‘Hello World’.
However, for any integer n > 2, the program will never find a triple that satisfy x n + yn = z n. This
example gives us the notion of undecidability in a problem. Now if we can plan to write a solution to this
undecidable problem i.e. to write a program which takes any program P , run with I as input for P, and
decide whether P prints ‘Hello World’. Then we can prove that no such program exists.

Problem vs. Program

Problem and program are two different concepts when it comes to undecidability. A problem can be
considered as the membership of a string in a language. The number of different languages over any
alphabet of more than one symbol is uncountable. That is we cannot assign integers to all possible
languages such that every integer is assigned to one language. Thus the numbers of problems possible are
infinite.

On the other hand programs, being finite strings over a finite alphabet, are countable. That is, we can
order programs by their length and it is possible to assign numbers to all programs of the same length.
Thus there are infinitely fewer programs than there are problems. If we pick up a random language, it will
most probably be undecidable.

Proof of non-existence of the ‘Hello World’ testing program

Proving that there exists no program that can test the ‘Hello World’ program can be tricky. But we can
achieve this using proof by contradiction. That is we first assume that a ‘Hello World’ tester program
exist, then be contradict this assumption and prove the non-existence.

Assume there is a program H, that takes as input a program P and an input I, and tells whether P with
input I prints ‘Hello World’. Figure A is a representation of what H does.

I
YES
Hello World Tester

NO
P

Figure A: A hypothetical Hello World Tester

In particular, the only output H makes is either to print the three characters ‘YES’ or to print the two
characters ‘NO’. It always does one or the other. If a problem has an algorithm like H, that always tells
correctly whether an instance of the problem has answer ‘YES’ or ‘NO’ then the problem is said to be
decidable. Otherwise, the problem is undecidable. Our goal is to prove that H doesn't exist; i.e., the hello-
world problem is undecidable.

To start the proof by contradiction we make small changes to the program H resulting in a new program
H2 which we show does not exist. The changes made to H are simple transformations that can be done to
any C program. Thus if H2 does not exist then H also do not exist. We start by converting H to a program
H1. H1 can be obtained by changing the output of H. The output ‘NO’ is changed to ‘Hello World’. Thus
the program H1 behaves exactly as H but, instead of ‘NO’ it displays ‘Hello World’. Figure B represents
the modified program H1.

I
YES

H1

Hello
P World

Figure B: H1 similar to H but, says ‘Hello World’ instead of ‘NO’

Now the tricky part comes, and we modify H1 so that both P and I are replaced with a single input P.
Thus H1 tests P with input P itself. For this we store P in an array A and two pointers are provided to mark
positions in A. Thus we get the new program H2 from H1. Figure C shows program H2.

YES

P H2

Hello
World

Figure C: H2 behaves as H1 but, uses its input P as both P and I.

H2 prints ‘Hello World’ when a program P is given as input to P does not prints ‘Hello World’ as its first
output. Similarly H2 prints ‘YES’ when a program P is given as input to P prints ‘Hello World’.

Now we can prove H2 cannot exist and if we succeed in proving that then, H1 cannot exist and H also does
not exist. To contradict the working of H2, give input H2 to itself as shown in figure D.

YES

H2 H2

Hello
World

Figure D: H2 is given as input to H2

Now H2 is both a program and input to that program, i.e. H2 can be imagined as P and H2 is also I. Now
both the program H2 and input H2 comes inside the H2 box shown in figure. Remember the fact H2 box is
different from the program H2 and input H2. The H2 box is only responsible for interpreting the result of
the H2 program when run with H2 as input. Now let us see what can happen inside. The H2 program
inside the box can say either ‘Hello World’ or ‘YES’ (See figure C). Now coming to the H2 box, its
output (see fig D) will depend on what the H2 program inside says. Now we come to the contradictory
part. If the H2 inside says ‘YES’, then H2 box will say ‘Hello World’ (substitute for ‘NO’) because the
program inside said ‘YES’ and not ‘Hello World’. If the H2 inside says ‘Hello World’ (substitute for NO),
then H2 box will say ‘YES’ because the program inside said ‘Hello World’. Thus in both cases H2
program inside the box contradicts the H2 box. And we prove H2 does not exist. Since H2 does not exist,
H1 and similarly H also do not exist. There exists no Hello World tester program.

Reducing one problem to another

Suppose we have proved a problem P1 to be undecidable using contradictory proof as above, it is easy to
show that another problem P2 is undecidable. This may be done by showing that some solution exists for
P2 and this can be applied to solve P1 (which is already proved undecidable). And it contradicts the
assumption that a solution exists for P2. This proof strategy is called reduction of P1 to P2.

Suppose that we know problem P1 is undecidable, and P2 is a new problem that we would like to prove is
undecidable as well. We suppose that there is a program ‘Decide’ represented in Fig. E by the diamond
that prints ‘YES’ or ‘NO’, depending on whether its input instance of problem P2 is in or is not in the
language of that problem. Now we introduce a ‘Construct’ by which we can convert instances of P1
(already proved undecidable) to instances of P2. The construct will convert any string in language P1 to
that in P2, and any string that is not in language of P1 is converted to another string not in language of P2.
And if we have this construction, we can solve P1.

P1 Construct P2 Decide YES

instance instance

NO

Fig E: Solution to P2 used to solve P1

Now we can solve P1 as follows:

1. For a given instance of P1, every string w which may or may not be in the language of P1 can be
converted to equivalent string x corresponding to P2 using the construct.
2. If w is in P1 then x is in P2 and is w is not in P1 then x will not be in P2.

Thus either way the decide function tells the truth about w. But we know already that P1 is undecidable
and there exist no solution to finding membership of string in P1. Thus we have proof by contradiction
that a hypothesized decision algorithm for P2 does not exist and hence P2 is undecidable.
Turing machine acceptance – An Undecidable problem
One of the most important undecidable problem is ‘Does a given TM accepts a given input?’And proving
this problem as undecidable can be relevant in explaining many other undecidable problems which are
unrelated to Turing machines.

Now remember that a Turing machine is said to accept two classes of languages. They are recursive
languages and recursively enumerable (RE) languages. When a Turing machine is implemented for the
acceptance of a recursive or decidable language (subset of RE), the TM will halt whether or not it accepts.
A language L is recursively enumerable if L=L (M) for some Turing machine M. But for recursively
enumerable languages, we cannot guarantee the machine will halt for an input string not in the language.
And outside the RE set, we have the bigger set of the undecidable problems.

To prove

The language consisting of pairs (M, w) is undecidable such that,

• M is a Turing machine (binary coded) with an input alphabet {0, 1}.

• w is a string of 0’s and 1’s.
• M accepts input w.

Preparing the problem

The idea of considering the input alphabet to be binary is that, if we prove this problem to be undecidable,
then a more general problem with TM having any alphabet is surely undecidable. Before we go into the
proof, we need to introduce some form binary coding for representing the Turing machine. The TM has to
be represented only using 1’s and 0’s, regardless of how many states the TM has. Every binary string
should be considered as a TM and if the binary string is not well formed, it will still represent a TM with
no moves. Now we have to enumerate (giving numbers) the binary strings and binary code the Turing
machine.

• Enumerating Binary Strings

The strings w formed from the input alphabet {0, 1} have to be enumerated first. For that, we assign an
integer i corresponding to each sting w. In this enumeration, ε is the first string, 0 is second, 1 is third, 00
is the fourth, and 01 is the fifth and so on. Thus every string can be uniquely identified by a number and
we refer to the ith string as wi.

• Coding Turing Machine

A Turing machine can be binary coded and represented as a string of 1’s and 0’s. And Turing machines
can be enumerated just like the strings were enumerated so that we can refer to the ith Turing machine as
Mi. To represent a Turing machine M = (Q, {0, 1}, Γ, δ, q0, B, F) as a binary string, we must first assign
integers to the states, tape symbols, and directions L and R.

➢ The states are assumed as q1,q2,…,qr for some r. The start state is q1 and the accepting state is q2.]
➢ The tape symbols are assumed as X1,X2,…Xs for some s. X1 will be 0, X2 will be 1 and X3 will be
B (blank). Other tape symbols can be assigned the remaining symbols arbitrarily.
➢ The direction L will be D1 and direction R will be D2.

Once we have established an integer to represent each state, symbol, and direction, we can encode the
transition function δ. Suppose one transition rule is δ (qi, Xj) = (qk, Xl, Dm), for some integers i, j, k, l, and
m. We shall code this rule by the string 0i l0j10k l0l 10m. Notice that, since all of i, j, k, l, and m are at least
one, there are no occurrences of two or more consecutive l's within the code for a single transition. The
code for the entire Turing machine can be represented as a string of coded transition rules separated by
11.

C111C211… Cn-111Cn

Consider a Turing machine: M = ({q1, q2, q3}, {0, 1}, {0, 1, B}, δ, q1, B, {q2}) with 5 consists of the rules:

δ (ql, 1) = (q3, 0, R)
δ (q3,0) = (q1, 1, R)
δ (q3, 1) = (q2, 0, R)
δ (q3, B) = (q3, 1, L)

The codes for each of these rules, respectively, are:

0100100010100
0001010100100
00010010010100
0001000100010010

For example, the first rule can be written as δ (q1, X2) = (q3, X1, D2), since 1= X2, 0 = X1, and R = D2.
Thus, its code is 01102103101102 =, as was indicated above. And the code for M is:

01001000101001100010101001001100010010010100110001000100010010

But for the proof, we need to represent a Turing Machine M and a string w as (M, w). Together M and w
are represented in binary coded form. It is represented as binary string representing M followed by 111
followed by the binary string w. So if M is the Turing machine and w is 1011 then the final
representation of (M, w) is binary coded M followed by 1111011.

• Diagonalization Language

The next objective is to introduce a new class of language called the diagonalization language which is
not recursively enumerable so that there is no TM that accepts it. We know that Mi is the ith Turing
machine represented by binary code wi (the ith binary string). But there are many integers that do not
represent a TM like 11001 which does not begin with a 0 and 0010111010010100 which has three
consecutive 1’s. But we know that an invalid TM code represents a TM with no transitions. That is for
these values of i, Mi halts on any input and L(Mi) is ϕ.

We can define diagonalization language Ld as as the set of strings wi such that wi is not in L(Mi). Here Ld
is a set of strings w such that w is the binary code representation of some Turing machine M and does not
accept when string w is given as an input to itself. To explain the meaning of diagonalization, consider the
table (Fig F) showing the acceptance of turing machines Mi (Y axis) for string wj (X axis). If Mi accepts
wj, the cell (i,j) will have a 1 otherwise it will be 0. The ith row is called the characteristic vector of the
language L (Mi) i.e. the 1’s indicate the strings that are member of the language Mi.
 Fig F: Acceptance of string by Turing machines

Here the diagonal values tell whether Mi accepts wi (here i=j). Now to construct Ld, compliment the
diagonal elements. The complimented diagonal would be 1,0,0,0 and Ld would have w1 (ϕ ) and will not
have w2 (0), w3 (1) and w4 (01) and so on. Thus the characteristic vector of language Ld is the diagonal
and the technique of making the diagonal the characteristic vector of a language is called diagonalization.
Now if we compare the characteristic vector Ld with the rows of the table, it disagrees at least in one
column. So the compliment of the diagonal cannot be the characteristic vector of any Turing machine.

Ld is not recursively enumerable

Theorem: Ld is not a recursively enumerable language. That is, there is no Turing machine that accepts
Ld.

Proof: Suppose Ld were L (M) for some TM M. Since Ld is a language over alphabet {0, 1}, M would be
in the list of Turing machines we have constructed, since it includes all TM's with input alphabet {0, 1}.
Thus, there is at least one code for M, say i; that is, M = Mi.

Now, ask if wi is in Ld.

• If wi is in Ld, then Mi accepts wi. But then, by definition of Ld, wi is not in Ld, because Ld contains only
those wj such that Mj does not accept wj.
• Similarly, if wi is not in Ld, then Mi does not accept wi, Thus, by definition of Ld, wi is in Ld.

Since wi can neither be in Ld nor fail to be in Ld, we conclude that there is a contradiction of our
assumption that M exists. That is, Ld is not a recursively enumerable language.
THE HALTING PROBLEM

In this section we prove one of the most philosophically important theorems of the theory of computation:
There is a specific problem that is algorithmically unsolvable. Computers appear to be so powerful that
you may believe that all problems will eventually yield to them. The theorem presented here demonstrates
that computers are limited in a fundamental way. What sorts of problems are unsolvable by computer?
Are they esoteric, dwelling only in the minds of theoreticians? No! Even some ordinary problems that
people want to solve turn out to be computationally unsolvable. In one type of unsolvable problem, you
are given a computer program and a precise specification of what that program is supposed to do (e.g.,
sort a list of numbers). You need to verify that the program performs as specified (i.e. that it is correct).
Because both the program and the specification are mathematically precise objects, you hope to automate
the process of verification by feeding these objects into a suitably programmed computer. However, you
will be disappointed. The general problem of software verification is not solvable by computer. In this
section and Chapter 5 you will encounter several computationally unsolvable problems. Our objectives
are to help you develop a feel for the types of problems that are unsolvable and to learn techniques for
proving unsolvability. Now we turn to our first theorem that establishes the undecidability of a specific
language: the problem of determining whether a Turing machine accepts a given input string. We call it A
TM by analogy with A DFA and A CFG . But, whereas