Part I: Iterative Algorithms & Loop Invariants

Chapter: 1: Measures of Progress and Loop Invariants

Section: 1.1: The Steps to Develop an Iterative Algorithm
1. Loop Invariants
(a) What is a loop invariant?
Answer: A picture of what must be true at the top of the loop
(b) What is the formal statement that an algorithm is correct for a problem.
Answer: If for every legal input instance, the required output is produced. If the input
instance does not meet the preconditions, then all bets are o. Formally, we express this as
precond) & code
alg
postcond)
This correctness is only with respect to this specications.
(c) What are the formal mathimatical things involving loop invariants that must be proved to prove
if your program exits then it obtains the post condition?
i.
Answer: precond) & code
preloop
loopinvariant)
ii.
Answer: loopinvariant

) & not exitcond) & code

loop
loopinvariant

)
iii.
Answer: loopinvariant) & exitcond) & code
postloop
postcond)
(d) What is the loop invariant for the GCD alg?
Answer: On input a, b), the algorithm maintains two variables x and y such that
GCD(x, y) = GCD(a, b).
(e) What is the loop invariant for a dynamic programming algorithm?
(f) What is the loop invariant for the binary search?
Answer: If the thing being searched for is anywhere, then then it is in this narrowed sublist.
(g) What is the loop invariant for insertion sort?
2. Loop Invariants: In high school you would have learned the algorithm Gaussian Elimination for solving
systems of linear equations and for inverting a matrix. Here we are to think of this algorithm as an
iterative algorithm with a loop invariant. (Dont panic. No linear algebra or memory of the algorithm
is needed.) The input in an n n-matrix M that is invertible and the output is its inverse N, i.e. the
n n-matrix such that M N = I. Here I is the identity matrix, i.e. that for which M I = M.
As a huge hint, the loop invariant is that the algorithm maintains matrices M

and N

for which
M

= M N

. (I recall drawing these two matrices next to each other with vertical line between
them.)
(a) Initializing:
i. For the iterative algorithm meta-algorithm, what is the formal statement that needs to be
proved about the code before the loop?
Answer: You must prove that the initial code establishes the loop invariant. The formal
statement that must be true is precond) & code
preloop
loopinvariant)
ii. Give the code that goes before the loop for this matrix inverting algorithm.
Answer: M

= M and N

= I.
iii. Prove this formal statement for this matrix inverting algorithm.
Answer: The loop invariant M

= M N

follows trivially because M

= M = M I =
M N

.
(b) The main loop:
i. For the meta-algorithm, what are the two things required of each iteration?
Answer: Make progress and maintain the loop invariant.
ii. For the iterative algorithm meta-algorithm, what is the formal statement that needs to be
proved about the code within the loop?
Answer: You must maintain the loop invariant. The formal statement is
loopinvariant

) & not exitcond) & code

loop
loopinvariant

)
iii. Give a few sentences giving the intuition about what the code within the loop for this matrix
inverting algorithm must accomplish and what needs to be proved about it. You do not need
to specify any details of the linear algebra.
Answer: The code in the loop must take two matrices M

and N

and produce two new

matrices M

and N

.
We must prove that if the loop invariant M

= M N

is true for M

and N

, then it is
true for M

and N

, namely M

= M N

.
You do not need to get this, but M

is produced from M

by adding a multiple of one

row to another row in such a way that M

has one more zero elements than M

. N

is
produced by doing the same row operation on N

.
Also progress must be made. You do not need to get this, but the measure of progress
will be the number of zero of elements of M

. This number will go up by at least one

each iteration.
(c) Ending:
i. For the meta-algorithm, what is the formal statement that needs to be proved about the code
after the loop?
Answer: In this step, you must ensure that once the loop has exited you will be able to
solve the problem. loopinvariant) & exitcond) & code
postloop
postcond)
ii. What should the exit condition be for this matrix inverting algorithm?
Answer: Exit when M

= I (or when M

has an all zero row).

iii. Give the code that goes after the loop for this matrix inverting algorithm.
Answer: Set N = N

and return it.

iv. Prove this formal statement for this matrix inverting algorithm.
Answer: By the loop invariant M

= M N

, by the exit condition M

= I, and by the
post code N = N

. It follows that M N = M N

= M

= I. Hence, as required, the

inverse N of M is returned.
Section: 1.2: More About The Steps
Section: 1.3: Dierent Types of Iterative Algorithms
Section: 1.4: Typical Errors
End of Chapter Questions
3. Sums
2
algorithm Eg(n)
precond: n is an integer.
postcond: Prints His.
begin
s = 0
i = 0
loop
loopinvariant)
exit when s n
i = i + 1
s = s +
1
i
put Hi
end loop
end algorithm
(a) Give a good loop invariant.
Answer: s =

i
j=1
1
j
. Initially true with s = i = 0 and maintained with i

= i

+ 1 and
s

= s

+i

= [

j=1
1
j
] +i

j=1
1
j
.
(b) How is the value of s not a reasonable measure of progress for Eg? How is it reasonable? If you
can think of one, what would be a better measure of progress?
Answer: Ocially, we said that the measure of progress should decrease by at least one each
step. This one increases by only
1
i
. The increasing is not a problem. Increasing by
1
i
2
would
be a problem, but it seems increasing by
1
i
is not. We will in fact use s to prove the program
halts. If you want an ocially good measure of progress note that because s = log
e
i, we have
i = e
s
and hence, M(i) = n e
s
= n i decreases from n to 0 by one each iteration.
(c) Give the theta of the running time of Eg.
Answer: Size = log n = s. n = 2
s
. When the program exists s = sum
i
j=1
1
j
= log
e
i +(1)
n. Hence, Time = i = e
n+(1)
= e
2
s
+(1)
= 2
2
(s)
, which is double exponential.
4. (25 marks) The Carnival Coin Game:
You are running a game booth at your local village carnival. In your game you lay out an array of $1
coins on a table. For a $5 charge, anyone can challenge you to a game. In this game, you and your
customer alternately pick coins from the table, either 1 or 2 at a time. If your customer can make you
pick up the last coin, he walks away with all the coins. You graciously allow your customer to go rst.
Being an enterprising sort, you want to arrange the game so that you will always win. To do this, you
write an iterative algorithm CarniCoin(n) that tells you how many coins to pick up on every turn. The
algorithm takes one parameter n, the number of coins on the table at the beginning of the game.
The algorithm is
while there is more than one coin left on the table
The customer picks up 1 or 2 coins.
You do the opposite: if the customer picked up 1, you pick up 2, and vice-versa.
end while
(a) (5 marks) What is the postcondition for your algorithm?
Answer: One coin remains on the table, and it is the customers turn.
(b) (5 marks) What is the loop invariant for your algorithm?
3
Answer: The number i of coins on the table satises i mod 3 = 1.
(c) (5 marks) What precondition must you place on n in order to guarantee that you will win every
time?
Answer: n > 0 and n = 1 mod 3.
(d) (10 marks) Prove the correctness of your algorithm. Be sure to explicitly include all steps required
to prove correctness.
Answer:
The algorithm is
i = n
while i > 1
i = i 3
end while
The loop invariant is maintained because 3 coins are picked up on each iteration, so that the
number of coins on the table, mod 3, is maintained.
The loop invariant is true on entry by virtue of the precondition. As a measure of progress,
we can use the coins remaining on the table, which, as shown above, decreases by 3 on every
iteration, and will therefore meet the exit condition in nite time.
On exit, i 1 and i mod 3 = 1. Since i decreases by 3 every iteration, i > 2. Thus i = 1
and the postcondition is satised.
Chapter: 2: Examples Using More of The Input Loop Invariant
Section: 2.1: Colouring the Plane
Section: 2.2: Deterministic Finite Automaton
Section: 2.3: More of the Input vs More of the Output
5. Loop Invariants (32 marks)
Longest Contiguous Increasing Subsequence (LCIS): The input consists of a sequence A[1..n] of integers
and we want to nd a longest contiguous subsequence A[k
1
..k
2
] such that the elements are strictly
increasing. For example, the optimal solution for [5, 3, 1, 3, 7, 7, 9, 8] is [1, 3, 7].
(a) (5 marks) Provide a tight lower bound on the running time for this problem. Prove that your
answer is indeed a tight lower bound.
Answer: The problem takes at least (n) time because the input needs to be read. Let A
be an arbitrary algorithm. Give it the input consisting of all zero. If it does not read some
integer, change this integer to a 1 (If it is the rst integer change it to -1). It answers the
same longest length in both cases, however, in the rst the longest has length one an in the
second two. Therefore, it is wrong in at least one of these cases.
(b) (5 marks) Specify an appropriate loop-invariant, measure of progress and exit condition for an
iterative solution to this problem. No explanations are necessary.
Answer: LI: We maintain an over all optimal LCIS found so far and the LSIS that we are
currently working on.
A[k
1
. . . k
2
] is an LCIS of A[1 . . . i] and A[p . . . i] is the LCIS of A[1 . . . i] that ends at i.
Note the rst LI is More of the Input, namely if we consider the prex A[1 . . . i] to be the
entire input then we have the solution.
Most people missed the second one, but it is needed if the LI is to be maintained.
Measure of Progress: the number i of elements read. Many gave that i increases. This is
progress, but not a measure.
Exit Condition: When all the elements have been read because i = n.
4
(c) (10 marks) Provide concise pseudo-code for an algorithm LCIS(A,n) that returns the indices
k
1
, k
2
of the LCIS of A[1 . . . n] uses the loop-invariant, measure of progress and exit condition you
specied above. Assume n 1.
Answer:
I thought more about why LI are important to me. It is a life philosophy. It is about feeling
grounded. Most of the code I marked today makes me feel ungrounded. It cycles, but I dont
know what the variables mean, how they t together, where the algorithm is going, or how
to start thinking about it. Loop invariants are about starting my day at home, where I know
what is true and what things mean. Loop invariants are also about returning back full circle
back to my safe home at the end of my day
algorithm [k
1
, k
2
] =LCIS(A,n)
precond: A[1 . . . n] is an array of integers
postcond: A[k
1
. . . k
2
] is an LCIS of A[1 . . . n]
begin
k
1
= k
2
= p = i = 1
loop
loopinvariant: A[k
1
. . . k
2
] is an LCIS of A[1 . . . i] and
A[p . . . i] is the LCIS of A[1 . . . i] that ends at i
i = i + 1
if( A[i1] A[i] ) then
p = i
end if
if (q p) > (k
2
k
1
)
k
1
= p
k
2
= q
end if
end if
end algorithm
(d) (6 marks) Provide an informal argument for how your code is able to maintain the loop invariant.
A formal proof is not required.
Answer: loopinvariant

) & not exitcond) & code

loop
loopinvariant

)
Suppose at the beginning of an iteration, the LI is true for the current values, i.e. A[k

1
. . . k

2
]
is an LCIS of A[1 . . . i

] and A[p

. . . i

] is the LCIS of A[1 . . . i

] that ends at i

.
The rst block of code either extends the LCIS that we are currently working on from
A[p

. . . i

] to A[p

. . . i

+1] or shrinks it to contain only the one element A[i

+1] depend-
ing on whether or not A[i

+1] is bigger than A[i

]. This is achieved by moving its end point

from i

to i

= i

+1 and either leaving its begging at p

= p

or moving it to p

= i

. Either
way, the LI is maintained that A[p

. . . i

] is the LCIS of A[1 . . . i

] that ends at i

.
The second block of code updates the over all LCIS found so far simply by moving it to the
current one, if the current one has grown to be larger. Either way, the LI is maintained that
A[k

1
. . . k

2
] is an LCIS of A[1 . . . i

]
(e) (6 marks) What are the other key steps in proving your algorithm correct? List and provide a
concise proof for each.
Answer:
Establishing the LI: precond) & code
preloop
loopinvariant).
On entry, k
1
= k
2
= p = i = 1. Thus the rst part of the loop-invariant requires that A[1] be
an LCIS of A[1], which is trivially true. The second part of the loop invariant requires that
A[1] is the LCIS of A[1] that ends at location 1, which is also trivially true.
Making Progress: i increments by 1 on each iteration. Program exits when i = n.
Establishing Post-Condition: loopinvariant) & exitcond) & code
postloop

postcond). When i = n, the rst LI gives that A[k
1
. . . k
2
] is an LCIS of A[1 . . . n], which
is the post condition.
5
6. Suppose you start with the number 1 and it costs you one to add two numbers together that you
already have. (Numbers can be reused.) Once you have produced a number, you can use it many
times. Given an integer k as input, design an algorithm to produce k as cheaply as possible. What
does it cost? The space requirement is how many numbers you need to keep at any given time. Try
to get an algorithm with minimum space. Hint 1: Think of the input k as an n = log
2
(k) bit binary
string. (Consider reading it forward and backward). Hint 2: Try More of the Input, More of the
Output, and Narrowing the Search Space. One of these three works fairly easily. (10 bonus point
if you can do it in time T(k) = n + (
n
log n
).)
Answer: You denitely dont want to produce 1, 2, 3, 4, . . . , k by adding 1 each time, but this
would cost k 1 = 2
n
.
The More of the Input technique has the loop invariant state that if we consider the rst i bits of
the input instance to be the entire input instance, then for this we have a solution. As an example,
suppose k was 110101000
2
in binary and i = 4. Then the prex would be k

= 1101
2
in binary.
The loop invariant says that somehow we have produced that value. Each step of this algorithm
is to extend the prex by one bit and then to produce the solution for the longer instance from
the solution for the shorter instance. In our example, the next bit is a zero. Including this gives
the longer instance k

= 11010
2
. Note that this change doubled the value of the instance, namely
k

= 2 k

= k

+ k

. The loop invariant says that we have produced k

. We can produce k

simply by adding this k

to itself. On the other hand, if the next bit is a 1, then k

= 1101
2
would
extend to k

= 11011
2
. This gives k

= k

+ k

+ 1. Given that we have k

and we can keep 1

around, we can easily produce k

. This algorithm does at most two additions for each bit of k.

The space is only two because it only needs to keep 1 and the current k

.
This same algorithm could be thought of recursively. If k is even, then recursively produce
k
2
and
then add this to itself to give k. If k is odd, then produce
k
2
and then compute k =
k
2
+
k
2
+1.
Another useful technique is to simply try to build big numbers as quickly as possible. From 1,
produce 2, 4, 8, ... by adding the last number to itself. Then to produce a arbitrary k, look at the
value k in binary. For example, k = 180 is 10110100
2
in binary because 180 = 128 + 32 + 16 + 4.
This tells us how to combine the powers of 2 that we have produced to give us k. The time of
this algorithm is still at most two times the number of bits in k. However, it requires log k space.
As said, these three algorithms use the same amount amount of time, namely T(n) = 2 log
2
k = 2n.
No algorithm can do better then n = log
2
k additions because no algorithm can more than double
the largest value, and hence after i iterations, the largest value is at most 2
i
.
If you care about the extra factor of 2 between the upper and the lower bound, lets try a little
harder. Let b be a parameter that we will later set to
1
2
log n. The algorithm rst produces
1, 2, 3, 4, . . . , 2
b
by adding 1 each time. This takes time 2
b
. All of the intermediate values are kept.
Note that thinking of each of these values as a binary string, we have produced every possible b
bit string.
The next stage of the algorithm breaks the string of n bits representing k into
n
b
blocks of length
b. The algorithm is the same as the more of the input algorithm given above except that each
iteration considers not just the next bit but the next block. The loop invariant says that we have
produced the value k

formed from the rst i blocks of the instance. To produce value k

formed
from the rst i + 1 blocks of the instance, we must rst multiply k

by 2
b
to add b zeros and
then we must add on the next block. Multiplying by two just requires adding the number by
itself. Hence, multiplying by 2
b
just requires repeating this b times. Adding on the next block
just requires one more addition. This is because in the rst stage of the algorithm we produced
every possible b bit string. We need only add on the correct one. There are
n
b
iterations in this
second stage of the algorithm and each iteration requires b + 1 additions.
The total time of the algorithm is T(n) = 2
b
+
n
b
(b + 1) = n +
n
b
+ 2
b
. Setting b =
1
2
log n gives
T(n) = n + 2
n
log n
+

n n + (
n
log n
).
7. Let G = (V, E) be a directed weighted graph with possible negative weights and k n some integer.
Find an algorithm that nds for each pair of nodes u, v V a path from u to v with the smallest total
6
weight from amongst those paths that contains exactly k edges. Here a path may visit a node more
than once. Do the best you can, but ideally the running time is O(n
3
log n). A hint is that a smallest
weighted path with i+j edges must be composed of a smallest weighted path with i edges and one
with j edges.
Answer: In order to solve the problem on input instance k, the More of the Input technique
suggests solving it rst for smaller integers rst. Suppose for each pair of nodes u, v V , we
already have that w
u,v,i
is the weight of the smallest weighted path from u to v containing exactly
i edges and that w
u,v,j
the same for those containing exactly j edges. Then a smallest weighted
path from u to v containing exactly i+j edges must be composed of a smallest weighted path
with i edges from u to some node b and then a smallest weighted path with j edges from b onto
v. We need to only try all nodes b and see which one works best. This gives the recurrence
w
u,v,i+j
= min
b
(w
u,b,i
+ w
b,v,i
). Doing this for each pair u and v takes O(n
3
) time. We can then
build up paths with k edges in the same way as done in the previous exercise. The total time is
then O(n
3
log n).
Chapter: 3: Abstract Data Types
Section: 3.1: Specications and Hints at Implementations
Section: 3.2: Link List Implementation
Section: 3.3: Merging With a Queue
Section: 3.4: Parsing With a Stack
Chapter: 4: Narrowing the Search Space: Binary Search
Section: 4.1: Binary Search Trees
Section: 4.2: Magic Sevens
End of Chapter Questions
8. (For the Engineers, By Je)
Imagine a wire that over time carries a 0/1 signal. A front-end processor has sampled it at a rate of
1GHz (ie every nano second) and given you an array A of length n. A[i] = 0 if there is a 0 on the
wire at time i nsecs and A[i] = 1 if there is a 1 on the wire at that time. A transition (or edge) is
dened to be a place in time where the signal transitions from one value to another. This may be of
positive polarity, i.e. from 0 to 1 or of negative polarity meaning from 1 to 0. The time of the
transition is the average of these two indexes. For example, if A[7] = 0 and A[8] = 1 then we say that
we time-stamp this transition with the time t = 7.5 nsecs. Our goal, given such an array A is to nd
the time-stamp of one such transition (if it exists). If there are many such transitions in A, then we
can return any one of them.
(a) Give a lower bound that any algorithm that solves this problem, on the worst case input, requires
n time to nd the answer. Imagine this as a game in which some adversary gives you an algorithm
and you must nd an input array A for which this algorithm queries A n times before nding a
transition. Your solution will describe how you will use the actions of the algorithm to produce
the input. (We did this in class to prove that adding two n bit numbers requires (n) time.) Be
clear how you handle the following circular argument. You dont know what input instance to give
the algorithm until you know what the algorithm does. But you dont know what the algorithm
does until you give it an input instance and run it.
Answer: The input instance we consider will either be all zeros or have exactly one one.
We run the algorithm without actually giving it a xed input instance. Each time the
algorithm queries A at some index i, we respond by saying telling it that A[i] = 0. With this
information, the algorithm can continue on just as if it did have an actual input instance.
If the algorithm stops before reading each array location then it cant know whether or not
there is a transition because we will ether give him the all zero instance or the instance with
a one in one of the locations he has not read.
7
(b) Now let us add the additional constraint that at the beginning and end of the time interval , the
wire has a dierent signal, i.e. A[1] ,= A[n]. Note that this assumption ensures that there is at
least one transition, though it could still have many transitions. Design a sublinear time recursive
algorithm which given the 0/1 array A returns the time t of a transition. Be very clear about
your pre and post conditions and that you are giving your friend a smaller instance that meets
the precondition. Though you must be clear what the algorithm is, actual code is not needed.
Give and solve the recurrence relation for its running time.
Answer: The precondition is that we are given an array A and two indexes p and q for which
A[p] ,= A[q]. We query A[m] for m =
p+q
2
. Then we recurse on A[p..m] if A[p] ,= A[m] and
recurse on A[m..q] if A[m] ,= A[q]. By the precondition on our instance, one of these must
be true (otherwise A[p] = A[m] and A[m] = A[q] giving A[p] = A[q].) When q = p + 1, we
have a transition A[p] ,= A[p + 1] at t = p +
1
2
. The recurrence relation for the running time
is T(n) = T(n/2) + 1 = (log n). It is not needed, but the master theorem can be used as
follows. The time for the top level is f(n) = 1 = n0. Hence c = 0. The time for the base
cases is (n
log a
log b
) = (n
log 1
log 2
) = (n0) = 1. Because the top and bottom are the same, we
know that T(n) = (log(n) f(n)) = (log n).
Chapter: 5: Iterative Sorting Algorithms
Section: 5.1: Bucket Sort by Hand
Section: 5.2: Counting Sort (A Stable Sort)
Section: 5.3: Radix Sort
9. (10 marks) RadixSort sorts d-digit numbers:
loop i = 1 to d
stable sort cards with respect to the i
th
least signicant digit.
end loop
At every iteration the following assertion is true: The input is stable-sorted with respect to the i
th
least-signicant digit. Is this a good loop-invariant? Why or why not?
Answer: No this is not a good loop invariant. It is true but not enough to know that the input
is stable-sorted with respect to JUST the i
th
least-signicant digit. It is a more of the input type
of loop invariant. If you consider each number to be just its least-signicant i digits then the list
is sorted stably.
Section: 5.4: Radix/Counting Sort
Chapter: 6: Euclids GCD Algorithm
10. (Amortized Analysis) Suppose we have a binary counter such that the cost to increment the counter
is equal to the number of bits that need to be ipped. For example, incrementing from 100, 111, 111
2
to 101, 000, 000
2
costs 7. Consider a sequence of n increments increasing the counter from zero to n.
Some of these increments, like that from 101, 111, 010
2
to 101, 111, 011
2
costs only one. Others, like
that from 111 . . . 111
2
= 2
log
2
n
1 n to 1000 . . . 000
2
costs O(log n). This one is the worst case.
Given that we generally focus on worst case, one would say that operations cost O(log n). It might be
more fair, however, to consider the average case. If a sequence of n operations requires at most T(n)
time, then we say that each of the operations has amortized cost
T(n)
n
.
(a) Suppose the counter begins at zero and we increment it n times. Show that the amortized cost
per increment is just O(1).
Hint: Let T
x,j
= 1 if incrementing value x requires ipping the j
th
bit. Note

n
x=1

j
T
x,j
=

n
x=1
T
x,j
.
8
Answer: The j
th
bit ips every 2
j
iterations for a total of
n
2
j
ips. The total cost is
T(n) =

n
x=1

j
T
x,j
=

n
x=1
T
x,j
=

log n
j=0

n
2
j
=

log n
j

=0
2
j

. This sum is geometric

and hence is dominated by the largest term. The largest term comes from the rst bit being
ipped each iteration for a total of n times. This gives that T(n) = O(n) and the amortized
cost per operation is
T(n)
n
= O(1).
(b) Suppose that we want to be able to both increment and decrement the counter. Starting with a
counter of zero, give a sequence of n of operations where each is either an increment and decrement
operation, that gives the highest amortized cost per operation (within ). (Dont have the counter
go negative.) Compute this amortized cost.
Answer: First increment the counter from zero to 111 . . . 111
2
= 2
log
2
(n/2)
1
n
2
. Then
have
n
2
operations alternating between incrementing and decrementing the counter between
111 . . . 111
2
and 1000 . . . 000
2
. Each of these costs log
2
(n/2). Hence, the total cost is
T(n) = (
n
2
) +
n
2
log
2
(n/2) = (nlog n) giving an amortized cost of (log n).
(c) To x the problem from part (b), consider the following redundant ternary number system. A
number is represented by a sequence of trits X = x
n1
. . . x
1
x
0
, where each x
i
is 0, +1, or -
1. The value of the number is

n1
i=0
x
i
2
i
. For example, X = 101
2
is 5 because 4 + 1 = 5.
However, X = (1)(1)(1)
2
is also 5 because 4 + 2 1 = 5. Also X = (1)(1)(0)(1)(1)(1)(1) =
64 32 8 + 4 + 2 + 1 = 31.
The process of incrementing a ternary number is analogous to that operation on binary numbers.
One is added to the low order trit. If the result is 2, then it is changed to 0, and a carry is
propagated to the next trit. This process is repeated until no carry results. For example, X =
(1)(1)(0)(1)(1)(1)(1) = 31 increments to X+1 = (1)(1)(0)(0)(0)(0)(0) = 6432 = 32. Note
this is a change of 8421 = 1. Decrementing a number is similar. One is subtracted from the
low order trit. If it becomes -2 then it is replaced by 0, and a borrow is propagated. For example,
X + 1 = (1)(1)(0)(0)(0)(0)(0) decrements to X = (1)(1)(0)(0)(0)(0)(1) = 64 32 1 = 31.
Note that this increment followed by a decrement resulted in a dierent representation of the
number 31 than the one we started with.
The cost of an increment or a decrement is the number of trits that change in the process. Our goal
is to prove that for any sequence n of operations where each is either an increment and decrement
operations, starting with a counter of zero, the amortized cost per operation is at most O(1). Let
T(t) be the total number of trit changes that have occurred during the rst t operations. Let Z(t)
be the number of non-zero trits in our counter after the t operations. For example, if the resulting
counter is X = (1)(1)(0)(1)(0)(1)(1) then Z(t) = 5. We will dene P(t) = T(t) + Z(t) to be
our measure of progress or potential function at time t.
Bound the maximum amount that P(t) can change in any one increment or decrement operation.
Answer: We will prove that each operation increases P(t) by at most 2. Consider any one
operation. Any time a trit changes and causes a carry increases T(t) by one because another
trit has changed and decreases Z(t) by one because this trit then changes from a non-zero
to a zero trit. The total eect is that P(t) = T(t) + Z(t) does not change at all. Only the
last and nal trit change does not cause a carry. It increases T(t) by 1 and can increase Z(t)
by at most 1, because of a possible change from a zero to a non-zero trit. Hence, the total
change in P(t) resulting from one operation is at most 2.
(d) Use induction (or loop invariants) to bound P(t). In doing so, prove that for any sequence n of
operations where each is either an increment and decrement operations, starting with a counter
of zero, the amortized cost per operation is at most O(1).
Answer: The induction hypothesis or loop invariant is that P(t) 2t. Note that P(0) = 0
because no changes have been made yet and all trits are zero. This is maintained because
P(t) increases by at most 2 each operation, namely P(t + 1) P(t) + 2 2t + 2 = 2(t + 1).
Z(n) is not negative. Hence, T(n) = P(n) Z(n) P(n) 2n and the amortized cost is at
most
T(n)
n
2.
9
11. Shifted Streams: You receive two streams of data, A[1], A[2], A[3], . . . and B[1], B[2], B[3], . . .. Once a
number arrives you cannot request it again. You must output a stream of data containing the values
A[1] + B[1], A[2] + B[2], A[3] + B[3], . . .. Once you output it you do not need to store it. There are
three things that make this more challenging.
1) The B data streams is shifted in time by some amount s.
2) You have only a limited amount of memory M[0], M[1], M[2], M[3], . . . , M[m] of which you want to
use as little as possible.
3) You have only a limited amount of time between data numbers arriving.
(a) Give the loop invariant. (I give the rst point.)
i. The numbers A[1], A[2], A[3], . . . , A[t] have arrived.
The numbers B[1], B[2], B[3], . . . , B[t s] have arrived.
ii. What values have been outputted?
Answer: The numbers A[1] + B[1], A[2] + B[2], A[3] + B[3], . . . , A[t s] + B[t s] have
been outputted.
iii. Which values do you want to be remembering in memory? (Use as little memory as possible.)
Answer: The last s values of A are saved. Specically, A[t s + 1], A[t s + 2], . . . A[t].
iv. In which memory cell do you save each such value? (You want to move things as little as
possible.)
Answer: The A data stream is stored in cyclically in s memory cells so that the last s
values are saving. Specically, for i = t s + 1, .., t, value A[i] is stored in M[i mod s].
(b) I taught a few types of loop invariants? Which type is this? Why? Include the denition.
Answer: More of the Input because if you consider a prex of the input stream as the
input, we have produced the output. Similarly, it could be More of the Output.
(c) What code establishes this loop invariant?
Answer: For i = 1, ..s, value A[i] is stored in M[i mod s]. With t = s, the loop invariant is
true.
(d) What code maintains this loop invariant?
Answer: At time t,
Read values A[t] and B[t s] arrive.
Get value A[t s] from memory M[t s mod s] = M[t mod s].
Output the value A[t s] +B[t s].
Store value A[t] is stored in M[t mod s].
(e) In order to prove that the loop invariant has been maintained, what is the formal statement of
what you prove? (No need to prove it.)
Answer: loopinvariant

) & not exitcond) & code

loop
loopinvariant

)
Part II: Recursion
Chapter: 7: Abstractions, Techniques, and Theory
Section: 7.1: Thinking about Recursion
Section: 7.2: Looking Forwards vs Backwards
Section: 7.3: With a Little Help from Your Friends
Section: 7.4: The Towers of Hanoi
Section: 7.5: Check List for Recursive Algorithms
Section: 7.6: The Stack Frame
Section: 7.7: Proving Correctness with Strong Induction
End of Chapter Questions
10
12. (15 marks) Recursion: Dene a Recursive Integer List (RIL) to be a list of a nite number of objects,
where each object is either an integer or is an RIL. For example, L = [3, 2, [4, 3], [[3, 9], [], 8], 7] is an
RIL. You are allowed the following operations:
(a) If L is an RIL and i is an integer, then L[i] is the i
th
object in the list L. For our above example,
L[1] = 3, L[2] = 2, L[3] = [4, 3], L[4] = [[3, 9], [], 8], L[5] = 7, and L[6] returns an error. More over
L[4][1] = [3, 9] and L[4][1][2] = 9.
(b) Similary, if L is an RIL, then L[i..j] is the RIL consisting of the i
th
through the j
th
objects in
the list L. For our above example, L[2..4] = [2, [4, 3], [[3, 9], [], 8]]. More over, L[2..2] and L[2..1]
return the RILs [2] and [].
(c) If L is an RIL, then [L[ returns the number of objects in the list L. For our above example,
[L[ = 5. More over, [L[4][2][ = 0.
(d) If Obj is an object, then (Obj == Integer) returns true if the object is an integer and (Obj ==
RIL) returns true if it is an RIL. In our example, (L[1] == Integer) is true and (L[3] == Integer)
is false. Note that 2 is an integer, while [2] is a RIL containing one object which happens to be
the integer 2.
Your task is to take as input an RIL L and return the sum of all integers any where in L. In our
example, SUM(L) should return 3 + 2 + 4 + 3 + 3 + 9 + 8 + 7 = 39. You may use a loop if you really
must, but it is better if you dont.
Answer:
algorithm Sum(L)
precond: L is an RIL.
postcond: Returns the sum of all integers in L
begin
if([L[ = 0)
return(0)
else
if( L[1] == Integer )
rst = L[1]
else
rst = Sum(L[1])
endif
return(first +Sum(L[2..[L[]))
end algorithm
algorithm Sum(L)
precond: L is an RIL.
postcond: Returns the sum of all integers in L
begin
sum = 0
loop i = 1 . . . [L[
if( L[i] == Integer )
sum = sum + L[i]
else
sum = sum + Sum(L[i])
endif
end loop
return(sum)
end algorithm
algorithm Sum(Obj)
precond: Obj is either an integer or is an RIL.
postcond: Returns the sum of all integers in Obj
begin
if( Obj == Integer )
return( Obj )
else
L = Obj
sum = 0
loop i = 1 . . . [L[
sum = sum + Sum(L[i])
end loop
return(sum)
endif
end algorithm
11
13. (13 marks) Recursion
algorithm A(a, b)
precond: a and b are integers.
postcond: Returns ??
begin
if( a = b ) then
return( 0 )
elseif( a < b ) then
return(A(b, a))
else
return(1 +A(a 1, b) )
endif
end algorithm
algorithm A(a, b)
precond: a and b are integers.
postcond: Returns ??
begin
if( a = b ) then
return( 0 )
elseif( a < b ) then
return(A(b, a))
else
return(A(a 1, b 1) )
endif
end algorithm
(a) Circle the output of this program.
i. a +b
ii. a b
iii. max(a, b);
iv. Does not halt.
Answer: a b
(b) Prove either that this program halts or that it does
not.
Answer: Consider the size of the input to be
ab if a b and ba+1 otherwise. Every friend
gets a smaller instance and if the size is zero than
a base case is reached.
(c) If the program halts, prove that its output is what
you claim.
(a) Circle the output of this program.
i. a +b
ii. a b
iii. max(a, b);
iv. Does not halt.
Answer: Does not halt.
(b) Prove either that this program halts or that it does
not.
Answer: On instance a, b) = 2, 1), the subin-
stances will be 1, 0) , 0, 1) , 1, 2) , . . ..
(c) If the program halts, prove that its output is what
you claim.
Answer: Consider instance a, b). By way of strong induction, assume that A returns a b for
every instance smaller.
If a = b than the result is 0, which is a b.
If a < b than the result is A(b, a) = [b a] = a b.
else the result is 1 +A(a 1, b) = 1 + [(a 1) b] = a b.
Either way, our result is a b.
Chapter: 8: Some Simple Examples of Recursive Algorithms
Section: 8.1: Sorting and Selecting Algorithms
Section: 8.2: Operations on Integers
14. Recursion: Suppose you are given two n digit numbers (base 10), X = x
1
x
2
. . . x
n
and Y = y
1
y
2
. . . y
n
and you want to multiply them.
(a) In one sentence explain what is the running time of the kindergarten algorithm is and why.
Answer: It adds X times which is 10
n
.
(b) In one sentence explain what is the running time of the high school algorithm is and why.
Answer: i, j [n] it computes x
i
y
j
. Hence the time is O(n
2
).
12
(c) In three short sentences describe the slow recursive algorithm for multiplying. (The one whose
time is the same as the high school algorithm.)
Answer: Break X and Y into pieces X = X
1
X
2
and Y = Y
1
Y
2
. Get four friends to multiply
X
1
Y
1
, X
1
Y
2
, X
2
Y
1
, and X
2
Y
2
. Shift their answers and add.
(d) In one sentence, what is the key trick used to reducing the time.
Answer: Reduce the number of friends from four to three.
(e) Give the recurrence relation for this recursive algorithm. Solve it.
Answer: T(n) = 3T(n/2) +O(n) = O(n
c
) where c = log(a)/ log(b) = log(3)/ log(2).
(f) Give all the key ideas in thinking abstractly about a recursive program?
Answer: If your input is suciently small solve it yourself. Otherwise, construct instances
for friends that are smaller then your instance and that meet the preconditions. Without
you worrying how, assume that they can solve their instance. Combine the solutions to their
instances to obtain a solution to your instance.
15. GCD and Finite Fields.
(a) (2 marks) Given integers a and b, what does the generalized GCD algorithm return and with what
post condition.
Answer: a s +b t = g = gcd(a, b)
(b) (2 marks) How is this used to nd the inverse of u mod p, where p is a prime.
Answer: Let a = u and b = p. This gives u s + p t = gcd(a, p) = 1 or u s = 1 mod p.
Hence s is the inverse.
(c) (2 marks) Why are the integers mod n not a eld when n is not prime?
Answer: If a b = n, then a b = 0 mod n. In a eld this should only be true if a or b are
zero.
16. Finite Fields.
(a) Find the inverse of 20 in Z
67
. To show your work, make a table with columns u, v, r, s, and t
and a row for each stack
Answer: Sorry for not doing the table, but the inverse is 10 = 57 because 20 (10) =
200 = 1 3 67 = 1 201.
(b) Given as input I = a, b, p) compute a
b
mod p. The algorithm is in the notes. Do not copy the
algorithm. The section reference (and necessary changes) is sucient. What is the number of bit
operations needed as a function of n = [I[ = log
2
(a) + log
2
(b) + log
2
(p)?
Answer: The algorithm is in Operations on Integers. It requires (log b) multiplications.
All multiplications are mod p and hence require (log p) bit operations for a total of T(n) =
(log b) (log p) = (n
2
).
(c) Given as input I = a, c, p) solve a
b

mod p
c for b. This is called discrete log. What is the best
algorithm that you can come up with in 15min. (Do not cheat and spend more time than this.)
What is the number of bit operations needed as a function of n = [I[ = log
2
(a)+log
2
(c)+log
2
(p)?
Answer: The best know algorithm requires (b) = (2
n
) (b) = (2
n
) multiplications. It
is the bases of multiplications. It is the bases of various cryptographic systems.
Exercise 0.0.1 In friends level of abstracting recursion, you can give your friend any legal instance
that is smaller than yours according to some measure as long as you solve on your own any instance
that is suciently small. For which of these algorithms has this been done? If so what is your measure
of the size of the instance? On input instance n, m), either bound the depth to which the algorithm
13
recurses as a function of n and m or prove that there is at least one path down the recursion tree that
is innite.
algorithm R
a
(n, m)
precond: n & m ints.
postcond: Say Hi
begin
if(n 0 and m 0)
Print(Hi)
else
R
a
(n 1, m)
R
a
(n, m1)
R
a
(n 2, m2)
end if
end algorithm
Change base case condition in the
R
a
to each of the following.
(b) if(n m 0)
(c) if(n m 10)
Answer:
(a) This does not halt following only rst friend because m never gets small.
(b) If n = m are initially odd, then this does not halt following only third friend because n =
m = 1, 1, 3, 5 . . . and the n m stays positive.
(c) Halts. Size(n, m) = n +m gets smaller by at least one and at most 4 for each friend. When
0 Size(n + m) 3 at most one of n and m are negative so we cant get a negative times a
negative gives us a positive and n m is at most 10. Initially, the size is n +m so it will be 0
in at most n+m time steps. Hence, this is a bound on the depth. Note the time is not n m,
not n, and not m.
Section: 8.3: Ackermanns Function
End of Chapter Questions
17. Quick Sort:
(a) [4 apples] Describe the quick sort algorithm using the friends paradigm. Assume that you have
pivot available as a subroutine.
(b) [2 apples] What is the time complexity of the algorithm when the pivot is selected randomly. Give
your best approximation of its recurrence relation. What does it evaluate to?
18. You have a bag of n nuts and a bag of n bolts. The nuts and bolts are all of dierent sizes, but for each
nut there is a matching bolt (and vice versa). What you want to do is to match them. Unfortunately,
you cant compare a nut to a nut or a bolt to a bolt (your eyes are too weak). Instead, what you can
do is to compare a nut to a bolt and see if the bolt is too small, too big, or just right. The problem
can be solved by comparing each nut to each bolt but this would require O(n
2
).
(a) I want you to describe a simple algorithm which uses randomness and recursion. It should be very
similar to Quick Sort which partitions with a pivot. Be sure to mention all the key requirements
of the friends analogy described in class. (Hint: Use the fact that every nut matches some bolt).
(Nine sentences.)
Answer: Choose a random nut. Compare it to each bolt, partitioning them into those that
are smaller, a perfect match, and larger. At least one bolt is a perfect match. Compare it to
each nut, partitioning them into those that are smaller and larger. Match this nut and bolt.
The nuts and bolts that are smaller then this pair must meet the precondition, namely you
have a bag of n

nuts and a bag of n

bolts and for each nut there is a matching bolt (and

14
vice versa). This subinstance is also smaller. Hence, get a friend to match them. Similarly,
those that are larger.
(b) Explain intuitively what the recurrence relation for the expected running time of this algorithm
is. Explain intuitively what the solution for this recurrence relation is.
Answer: We expect each of the two subinstance to be half the size. We expect the larger
of the two to be of size
3
4
n. If this was always the case, the recurrence relation would be
T(n) = T(
3
4
) +T(
1
4
) +n. This evaluates to T(n) = (nlog n) because there will be (log n)
levels of recursion and the sum work in each level will be (n).
(c) Explain intuitively what the worst cast running time of this algorithm is and why.
Answer: In the worst case, the subinstances are always of size n 1 and zero. This gives
T(n) = T(n 1) + n = (n
2
).
19. For many computational problems there is a trade o between the time and the amount of memory
needed. There are algorithms that use minimal time but lots of memory and algorithms that use
minimal memory but lots of time. One way of modeling this is by a game of pebbling the nodes of
a directed acyclic graph (DAG). Each node represents a value that needs to be computed. Having a
pebble on a node represents the fact that that value is stored in memory. Note the game has one pebble
for each of the memory registers that is available to the algorithm. Pebbles can be removed from a node
and placed else where. This corresponds to replacing the value in a register with a newly computed
value. However, there is a restriction. If there are edges from each of the nodes In(v) = u
1
, u
2
, . . . , u
d

to v, then there needs to be a pebble on these nodes u

1
, u
2
, . . . , u
d
before you can put a pebble on v.
This is modeling the fact that you need the values associated with u
1
, u
2
, . . . , u
d
in order to be able to
compute the value associated with v. In contrast, a pebbles can be placed at any time on a leaf of the
DAG because these nodes have no incoming edges. These correspond to the easy to compute values.
The input to the pebbling problem species a DAG and one of its nodes. The goal is to place a pebble
on this node.
u
1
u
2
2
u
1
u childeren and
Have pebbles on both
can place pebble
on parent v.
v
d=2
Get a pebble to the top.
You will be required to solve the problem corresponding to pebbling the following DAG. The DAG
in consideration will have a rectangle of nodes w wide and h high. Here h much smaller than w,
but w is much smaller than 2
h
. For each node v (not on the bottom row), there will be d nodes
In(v) = u
1
, u
2
, . . . , u
d
on the previous row with directed edges to v. Here, d is some small value.
You are to write both a iterative and a recursive algorithm for this problem.
(a) Iterative:
i. Describe an extremely simple iterative algorithm that puts a pebble on the specied node as
quickly as possible. The algorithm may use as many pebbles (memory) as needed, but does
not redo any work. Being an iterative algorithm, what is the algorithms loop invariant?
Answer: This algorithm starts by placing a pebble on each of the nodes on the bottom
row. This can be done because each such node has in-degree zero. The loop invariant
is that after r iterations, there is a pebble on each of the nodes on the r
th
row from the
bottom. The step of the next iteration is to place pebble on each of the nodes on the
r+1
st
row from the bottom. This can be done because each such node v has pebbles on
all of its in-neighbors In(v) = u
1
, u
2
, . . . , u
d
on the previous row. The step ends by
removing the pebbles from the r
th
row so that they can be reused. Note that progress
has been made while maintaining the loop invariant. The exit condition is when r = h at
which time the task of placing a pebble on the top row has been accomplished.
15
ii. How much time do you for this iterative programming algorithm? Given that you are able to
reuse pebbles, how many do you really need?
Answer: The time required is O(hw) and the number of pebbles is 2w.
(b) Recursive:
i. Suppose your task is to place a pebble on some node v which is r rows from the bottom.
Describe a recursive backtracking algorithm to accomplish this task that re-does work as
necessary, but uses as few pebbles as possible. Being a recursive algorithm, describe this
algorithm using the friends paradigm.
Answer: My task is to place a pebble on some node v which is r rows from the bottom.
For each of the d nodes u
i
In(v) with an edge to v, I ask a friend to place a pebble on
node u
i
. Once, there is a pebble on each node in In(v), I place a pebble on node v.
ii. Let Time(r) and Pebbles(r) be the time and the numbers of pebbles used by your algorithm
to go from there being no pebbles on the DAG to placing a pebble on one node r rows from
the bottom. Give and solve a recurrence relation for each of these.
The hardest part of this pebble question is the recurrence relation for the number of pebbles.
Remember to reuse the pebbles.
Answer: The total time used to complete my task is Time(r) = d Time(r 1) + 1,
because each of my d friends uses Time(r1) time and I move one pebble. This evaluates
to Time(r) d Time(r 1) = d
r1
.
The number of pebbles I will need is Pebbles(r) = Pebbles(r 1) + (d 1). Note that I
must leave a pebble on node u
i
before moving on to node u
i+1
, but the rest of the pebbles
can be reused. The time at which I need the most pebbles is when pebbling node the d
th
node in In(v). At this point I need to have d 1 pebbles on the previous nodes in I(v)
and Pebbles(r 1) pebbles that I allow this last friend to use for a total of Pebbles(r) =
Pebbles(r 1) + (d 1) pebbles. This evaluates to Pebbles(r) = (d 1) (r 1) + 1.
(c) Conclusion: Compare and contrast the time and the space used by these two algorithms. Re-
member that w is much bigger than h and d.
Answer: The iterative algorithm uses time w h which is the number of nodes, while the
recursive algorithm uses exponential time. On the other hand, the iterative algorithm uses
time (w) pebbles which is more than the (dh) used recursive algorithm because we were
given that w is much bigger than h and d.
(d) (This question was not asked for, but I am including the answer for students information.) Once
you learn recursive back tracking and dynamic programming, explain how your two pebbling
algorithms correspond to these concepts.
Answer: A recursive backtracking algorithm has an instance to solve and gets his friends to
solve subinstances for him, who in turn get their friends to solve their subinstances for them.
This sets up an entire recursive tree of subinstances to be solved. Some of these subinstances
that are solved by dierent friends friends friends are in fact the exact same subinstances.
Hence, the tree of subinstances can be collapsed into a DAG. Each node represents a subin-
stance that needs to be solved. If the subinstance associated with node v asks friends to
solve the subinstance associated with nodes In(v) = u
1
, u
2
, . . . , u
d
, then there are edges
from each of these nodes to v, specifying that they each need to be solved before v can be
solved. The leaves of the DAG correspond to the easy to compute base cases. The goal is
to pebble the node corresponding to the initial instance of the problem. The recursive back
tracking algorithm gets friends to solve subinstances even if they have been solved before.
This corresponds to the recursive algorithm above. It uses exponential time and memory
proportional to the depth of recursion. The Dynamic Programming algorithm assigns one
friend to each subinstance and solves each once starting the base cases and ending with the
origin instance. This corresponds to the iterative algorithm above. It uses time and memory
proportional to the number of subinstances.
16
v
u
2
u
1
d
=
2
20. Recall the pebbling game from an earlier question. You can freely put a pebble on a leaf. If there are
edges from each of the nodes In(v) = u
1
, u
2
, . . . , u
d
to v, then there needs to be a pebble on these
nodes u
1
, u
2
, . . . , u
d
before you can put a pebble on v.
Consider a rooted tree such that each node (except leaves) has d children pointing at it. The height is
h.
you can place pebble on parent v.
Have pebbles on all d=3 children
u
3
u
2 1
u
v
Get a pebble to the top.
(a) Give the Theta of the total number n of nodes in the tree. Show your math and give the intuition.
Answer: There are d
i
nodes at level i. n =

h
i=0
d
i
= (d
h
), because it is dominated by the
biggest term.
(b) What is the connection between pebbles and memory?
What does a node in the graph represent?
What does an edge in the graph represent?
Answer: Each node represents a value (solution to a subinstance) that needs to be computed.
Having a pebble on a node represents the fact that that value is stored in memory. If there
are edges from each of the nodes In(v) = u
1
, u
2
, . . . , u
d
to v, then there needs to be a
pebble on these nodes u
1
, u
2
, . . . , u
d
before you can put a pebble on v. This is modeling the
fact that you need the values associated with u
1
, u
2
, . . . , u
d
in order to be able to compute
the value associated with v.
(c) What is the minimum amount of time needed to get a pebble to the root even if you have lots of
pebbles? Why?
Answer: The time needed is n = (d
n
), because each node will need to be pebbled.
(d) Briey describe the recursive algorithm for placing a pebble on the root using as few pebbles as
possible. Describe this algorithm using the friends paradigm.
Answer: My task is to place a pebble on some node v which is r rows from the bottom. For
each of the d nodes u
i
In(v) with an edge to v, I ask a friend to place a pebble on node u
i
.
Once, there is a pebble on each node in In(v), I place a pebble on node v.
(e) What is the number of pebbles for this recursive algorithm to get a pebble to the root? Give and
solve the recurrence relation.
17
Answer: The number of pebbles I will need is Pebbles(r) = Pebbles(r 1) + (d 1) =
(d 1) (r 1) + 1. Note that I must leave a pebble on node u
i
before moving on to node
u
i+1
, but the rest of the pebbles can be reused.
(f) What is the running time of this recursive algorithm? Compare it to that for the minimum time.
Answer: The time needed is still n = (d
n
), because each node will need to be pebbled.
(g) Imagine you are tracing out this recursive algorithm. Stop the tracing at the point in time at
which you need the most pebbles. Put an X on the nodes that have pebbles on them at that time.
Chapter: 9: Recursion on Trees
Section: 9.1: Tree Traversals
21. Postx Transversals: Suppose you are deep within the computation of Postx Traversal of a Binary
Tree.
(a) (5 marks) Give the output of a postx traversal of the tree in the next question.
Answer: 7821 9043 65
(b) (3 marks) Who stores a pointer to the current node in the tree?
Answer: The current stack frame stores a pointer to the current node in the tree.
(c) (3 marks) Who stores pointers that allow you to move down a path in the tree?
Answer: The node in the tree stores pointers that allow you to move down a path in the
tree.
(d) (3 marks) Who stores pointers that allow you to move up the tree to the root?
Answer: Stack frames that are pushed on the the stack of stack frames and waiting to nish
their execution store pointers that allow you to move up the tree to the root.
Section: 9.2: Simple Examples
22. (15 marks) Recursion
Design a recursive algorithm that, given a binary search tree T, and a key key, returns two numbers
key

and key
+
equalling the largest key in T less than or equal to key and the smallest key in T
strictly greater than key, respectively. If key is smaller than all keys in T, then key

= is
returned. Similarly, If key is greater than or equal to all keys in T, then or key
+
= + is returned.
Please provide the pseudocode for your algorithm.
Answer:
algorithm Bracket(T, key)
precond: T is a BST, key is a real number
postcond: Returns two numbers key

and key
+
equalling the largest key in T less than key
and the smallest key in T greater than key, respectively. If key is smaller than all keys in T,
then key

= is returned. Similarly, If key is greater than or equal to all keys in T, then

or key
+
= + is returned.
begin
if T = then return (, )
elseif T.key key
(key

, key
+
) = Bracket(N.right, key, m)
key

= max(key

, T.key)
elseif T.key > key
18
(key

, key
+
) = Bracket(N.left, key, m)
key
+
= max(key
+
, T.key)
end
end algorithm
23. Given a binary search tree T, an key k and an integer b, return two sets S
smaller
and S
larger
. S
smaller
is to contain the largest b elements of T that are smaller or equal to k. If [S
smaller
[ < b this is because
there are no more elements in T than this that are smaller than k. Similarly dene S
larger
except it
is looking for b elements that are strictly larger than k. Feel free to use the notations [S[, S = S
1
S
2
and S = S
1
+element.
Recurrence relation and running time.
Argue whether or not the running time is O(depth(T) +b) for every tree T and every value b. If this
time is not quite right, what is a better expression for the time. If the input b is given as a log(b) bit
string then is this time complexity exponential?
Answer: No the time is O(depth(T) +min(n, b))
24. Dene the distance between two nodes x and y in a binary to be the number of edges in the unique
path from x to their lowest common ancestor and then back down to y. For example, the distance
between 7 and 9 is 6 because of the six edges in the path 7, 2, 1, 3, 4, 0, 9). Write a recursive program
that is given a binary tree and outputs two values. The rst, dist, is the maximum distance between
any two nodes in the tree. It also outputs the depth (height) of the tree in edges. The outputs for the
trees rooted at various nodes are given to the right. (Not to be handed in, but check your code on
these inputs.) Also provide a quick paragraph containing the friends explanation of the algorithm.
0
9 8
7
1 4
<dist=2,depth=1>
because of <7,2,8>
<dist=6,depth=3>
<dist=6,depth=4>
because of <7,2,1,3,4,0,9>
because of <7>
<dist=0,depth=0>
because of <7,2,1,3,4,0,9>
because of <7,2,8> or <1,2,8>
<dist=2,depth=2>
6
2
3
5
algorithm MaxDistance(tree)
precond: tree is a binary tree.
postcond: dist is the maximum distance (in edges) between two nodes in tree
and depth is its depth measured in edges.
begin
Answer:
if( tree = emptyTree )
return( 1, 1) )
else
dist
left
, dept
left
) = MaxDistance(leftSub(tree))
dist
right
, dept
right
) = MaxDistance(rightSub(tree))
dist = Max(dept
left
+dept
right
+ 2, dist
left
, dist
right
)
dept = Max(dept
left
, dept
right
) + 1
19
return( dist, dept)) )
end if
end algorithm
Answer: The key idea in a friends algorithm is that we are given a tree as an instance. We give
our left subtree to one friend and our right to another. Because these are smaller valid instances
we can trust our friend to give us the right answer for their instance. Our job is to construct the
correct answer for our instance from their answers. This was all I was looking for in the question.
I, however, will continue giving you my thinking.
It is best to start by pretending that we have a general big tree with two subtrees. Our depth is
determined by the deeper of our subtrees. Our tree is one deeper than this. This gives dept =
Max(dept
left
, dept
right
) + 1. For example, the depth of the tree rooted at node 5 is dept =
Max(4, 0) + 1 = 5.
The maximum distance (in edges) between two nodes in our tree is a little harder. There are two
cases depending on whether the root is involved in the maximum path. If the root is involved in
the maximum path, then our answer would be the sum of distance from the root to the deepest
node x in the left subtree plus that to the deepest node y in the right subtree. This rst distance
is the depth (in edges) of the left subtree plus 1 for the edge from the root to the root of the left
subtree. This gives dist = dept
left
+dept
right
+ 2.
If the root is not involved in the maximum path, then the two nodes x and y are contained in the
same subtree and our answer is then the same as that given by our friend for the maximum path
length in this subtree. This gives dist = Max(dist
left
, dist
right
).
Our answer is the maximum of these two cases. This gives that dist = Max(dept
left
+dept
right
+
2, dist
left
, dist
right
). For example, the dist of the tree rooted at node 5 is dist = Max(3 + 0 +
2, 6, 0) = 6 and the dist of the tree rooted at node 3 is dist = Max(2 + 2 + 2, 2, 2) = 6.
This completes tree with both a left and a right subtree. Before continuing let us try of gure out
answer empty tree. If it is done correctly it can save us from having lots of strange cases. The
depth of the tree rooted at node 2 is 1. The depth of the tree rooted at node 7 is 0. Following
this pattern let us make the depth of the empty tree -1. For the tree rooted at node 7, the longest
path is that from node 7 to node 7. It has one node and dist = 0 edges. For the empty tree, the
longest path has zero nodes and say dist = 1 or dist = edges.
So as to not have lots of cases lets test our code as it is on other examples. If the tree
only has one subtree, like that rooted at node 1, then dept = Max(dept
left
, dept
right
) + 1 =
Max((1), dept
right
) + 1 = dept
right
+ 1 works ne and so does dist = Max(dept
left
+
dept
right
+ 2, dist
left
, dist
right
) = Max((1) + dept
right
+ 2, (1), dist
right
) = Max(dept
right
+
1, (1), dist
right
).
When the tree consists of a single node, dept = Max(dept
left
, dept
right
) +1 = Max((1), (1)) +
1 = 0 and dist = Max(dept
left
+ dept
right
+ 2, dist
left
, dist
right
) = Max((1) + (1) +
2, (1), (1)) = 0.
This covers all cases.
Section: 9.3: Generalizing the Problem Solved
25. (15 marks) Recall that an AVL tree is a binary search tree with the property that every node has a
balance factor of -1, 0, or 1, where its balance factor is the dierence between the height of its left and
its right subtrees. You are to give pseudocode for an recursive algorithm that is given a binary tree
and returns whether or not it is an AVL tree. You do not have to prove its correctness. Also describe
the algorithm using a paragraph includin all required aspects of loop invariants, of recursive/friends,
and of subroutines.
Answer: In addition to whether or not the tree is an AVL tree, the routine will return the height
of the tree. If our instance tree has no nodes than we return that it is an AVL tree of height 0.
20
Otherwise, we ask one friend about the left subtree and another about the right. They tell us the
heights h
left
and h
right
of these and whether they are AVL trees. If either of the subtrees are
not AVL trees or [h
left
h
right
[ > 1, then we return that our tree is not an AVL. Otherwise, we
return that it is an AVL tree with height h = max(h
left
, h
right
) + 1.
algorithm IsAV Ltree (tree)
precond: tree is a binary tree.
postcond: The output indicates whether it is an AVL tree.
It also gives the height the tree.
begin
if(tree = emptyTree) then
return Y es, 0)
else
leftIs, leftHeight) = IsAV Ltree(leftSub(tree))
rightIs, rightHeight) = IsAV Ltree(rightSub(tree))
height = max(leftHeight, rightHeight) + 1.
if( leftIs and rightIs and [leftHeight rightHeight[ 1 ) then
return Y es, height)
else
return No, height)
end if
end if
end algorithm
Section: 9.4: Heap Sort and Priority Queues
26. Binary Trees
(a) Give the denition of a binary tree
Answer: A binary tree is either the empty tree or consists a node with a left and right binary
tree.
(b) What additional requirements do binary search trees have?
Answer: For each node, all nodes in its left subtree are smaller or equal to it and all nodes
in its right subtree are greater or equal.
(c) Given a set of n integers, how long does it take to put them into a binary search tree. Why?
Answer: It takes (nlog n) time because you must sort them.
(d) What additional requirements do Heaps have?
Answer: For each node, all nodes in its subtrees are smaller or equal to it.
(e) Given a set of n integers, how long does it take to put them into a heap. Why?
Answer: It takes (n) time. No more because you dont really sort them. No less because
you do need to look at each integer.
(f) Give two applications of Heaps.
Answer: Heap Sort and Priority Queues
27. Heaps
(a) What is the ordering imposed on the elements of a max heap?
Answer: A nodes value must be greater than or equal to the values of its children.
21
(b) Given a binary maxheap A, provide concise pseudocode for an O(1) algorithm that prints the
biggest 3 elements in the heap in non-increasing order. You may make the following assumptions:
i. The heap is implemented as a linear array.
ii. Operations LEFT(i) and RIGHT(i) return the indices of the left and right subtree of the
node stored at index i of the array.
iii. The heap contains at least 10 nodes.
Answer:
Print A[1]
l = LEFT[1]
r = RIGHT[1]
If A[l] > A[r]
larger = l
smaller = r
Else
smaller = l
larger = r
Print A[larger]
Print max(A[smaller], A[LEFT[larger]], A[RIGHT[larger]])
(c) Consider the problem of determining the smallest element in a binary maxheap. Derive a tight
lower bound for this problem and prove that it is a tight lower bound.
Answer:
One of the smallest elements of a maxheap must be a leaf. (Otherwise, there must be a
non-leaf that is smaller than one of its descendants, which means the tree is not a maxheap.)
Thus it is sucient to search all leaves.
Is it necessary? Suppose there exists an algorithm that does not search all leaves. Give it
a maxheap A and check which leaf it misses. Lets call this leaf i If it reports this A[i] as
the minimum, set the value of another leaf to A[i] 1. The algorithm will be wrong on this
input. If it reports another value A[j] as the minimum, set A[i] = A[j] 1. the algorithm
will be wrong on this input. Thus a correct algorithm must check all leaves.
An n-node maxheap has n/2 leaves. Thus the problem of nding the smallest element in a
binary maxheap is (n).
Section: 9.5: Representing Expressions with Trees
Section: 9.6: Pretty Tree Print
End of Chapter Questions
28. Binary Trees (20 marks)
Consider a binary tree of integers T. Consider a path s from the root of T to any leaf of T. Let m
s
be the minimum element on the path. We dene the maxmin value of T as the maximum m
s
for all
paths s from the root of T to all leaves of T.
Examples:
In Tree (a), the smallest element on the rst path 7, 3, 5) is 3. The smallest element on the second
path 7, 3, 2), is 2. The smallest element on the third path 7, 4, 8) and fourth path 7, 4, 5) is 4.
The largest of these four smallest values 3, 2, 4, 4 is 4.
In Tree (b), the smallest element on the path 7, 2) is 2 and the smallest element on the path
7, 4), is 4. The largest of these two smallest values 2, 4 is 4.
In Tree (c), the smallest element on the path 7, 2) is 2 and the smallest element on the path
7, 9), is 7. The largest of these two smallest values 2, 7 is 7.
22
e)
Return ? Return 7 Return 7 Return 4 Return 4
a)
d) c) b)
1 8
4
6
2
3
7
7
2 4
7
2 9
6 8
4
2
5
3
7
7
8
In Tree (d), there is only one path 7, 8). Its minimum is 7. The maximum of this one value is 7.
(a) (2 marks) What is the maxmin value for Tree (e).
Answer: Value 4.
(b) (7 marks) Let T be the root of a binary tree of integers. Let T.val be the integer-valued key at
the root. Let m
L
be the maxmin value of the subtree rooted at Ts left child. Let m
R
be the
maxmin value of the subtree rooted at Ts right child. Write an expression for the maxmin value
m
T
of the tree rooted at T in terms of T.val, m
L
and m
R
.
Answer: m
T
= max(min(m
L
, T.val), min(m
R
, T.val))
(c) If we want this general code to work for a tree consisting of a single node, what answer should
the empty tree return? Does work? Why? Does work? Why?
Answer: Yes, works. If the tree consists of one node, the root element is returned as
required.
max( min(MaxMin(leftSub(tree)), root(tree)), min(MaxMin(rightSub(tree)), root(tree)) )
= max( min(, root(tree)), min(MaxMin(, root(tree)) )
= max( root(tree), root(tree) ) = root(tree)
No, does not work. If the tree consists only of 7, then there is only one path whose
minimum is 7. However, the following incorrect thing is returned.
max( min(MaxMin(leftSub(tree)), root(tree)), min(MaxMin(rightSub(tree)), root(tree)) )
= max( min(, 7), min(, 7) ) = max(, ) = .
(d) If we want this general code to work for a tree with no left subtree, what answer should the empty
tree return? Does work? Why? Does work? Why?
Answer: No, does not work. If the root is 7, left subtree is empty, and the right is 2, then
there is only one path whose minimum is 2. However, the following incorrect thing is returned.
max( min(MaxMin(leftSub(tree)), root(tree)), min(MaxMin(rightSub(tree)), root(tree)) )
= max( min(, 7), min(2, 7) ) = max(7, 2) = 7.
It mistakenly thinks the of path to the empty tree as a path.
Yes, works. If the left subtree is empty, the following correct thing is returned.
max( min(MaxMin(leftSub(tree)), root(tree)), min(MaxMin(rightSub(tree)), root(tree)) )
= max( min(, root(tree)), min(MaxMin(rightSub(tree)), root(tree)) )
= max( , min(MaxMin(rightSub(tree)), root(tree)) )
= min(MaxMin(rightSub(tree)), root(tree))
(e) Give the entire code which works on every type of input.
Answer:
if(tree = emptyTree) then
return
elseif(rightSub(tree) = emptyTree and leftSub(tree) = emptyTree) then % tree is a leaf
return rootNode(tree)
23
else
return max( min(MaxMin(leftSub(tree)), root(tree)),
min(MaxMin(rightSub(tree)), root(tree)) )
endif
end alg
(f) Give all the key ideas in thinking abstractly about a recursive program?
Answer: If your input is suciently small solve it yourself. Otherwise, construct instances
for friends that are smaller then your instance and that meet the preconditions. Without
you worring how, assume that they can solve their instance. Combine the solutions to their
instances to obtain a solution to your instance.
29.
9 marks: Finding the k
th
Smallest Element: Write a re-
cursive program, which given a binary search tree and an
integer k returns the k
th
smallest element from the tree.
(Code is ne.)
D 15
k=3
B 4
A 8
C 10
kth smallest = 10
Answer:
algorithm Smallest(tree, k)
precond: tree is a binary search tree and k > 0 is an integer.
postcond: Outputs the k
th
smallest element s and the number of elements n.
begin
if( tree = emptyTree ) then
result( NotPossible, 0) )
else
s
l
, n
l
) = Smallest(leftSub(tree), k)
% There are n
l
+ 1 nodes before the right subtree
s
r
, n
r
) = Smallest(rightSub(tree), k (n
l
+ 1))
n = n
l
+ 1 +n
r
if( k [1..n
l
] )then
s = s
l
elseif( k = n
l
+ 1 )then
s = root(tree)
elseif( k [n
l
+ 2..n] )then
s = s
r
else then
s = OutOfRange
endif
result( s, n) )
end if
end algorithm
Chapter: 10: Recursive Images
Section: 10.1: Drawing a Recursive Image from a Fixed Recur-
sive and Base Case Images
Section: 10.2: Randomly Generating a Maze
Chapter: 11: Parsing with Context-Free Grammars
End of Part Questions
24
30. (20 marks) Recall the Tower of Hanoi problem discussed in class. The task is to move n rings, each of
a dierent diameter, from a source peg Source to a destination peg Dest. A third peg Spare may be
used in the process. The rings start o on Source, stacked according to size, with the largest on the
bottom. The rings are moved one at a time, and a ring can never be put on a ring that is smaller than
it. In class we discussed a recursive algorithm to solve this problem:
algorithm TowerofHanoi(n, Source, Spare, Dest)
precond: The n smallest rings are stacked on Source
postcond: The n smallest rings are stacked on Dest
begin
TowerofHanoi(n 1, Source, Dest, Spare)
Move the n
th
-largest ring directly from Source to Dest in one move
TowerofHanoi(n 1, Spare, Source, Dest)
end algorithm
(a) (10 marks) Derive a tight bound on the complexity of the problem (as a function of n).
Answer: The recurrence relation of the optimal solution is
T(n) = 2T(n 1) + 1
= 2 [2T (n 2) + 1] + 1 = 4T (n 2) + 3
= 2 [2 [2T (n 2) + 1] + 1] + 1 = 8T (n 3) + 7
.
.
.
= 2
i
T (n i) + 2
i
1
= 2
n1
T (1) + 2
n1
1
= 2
n
1
(b) (5 marks) Suppose that you are now allowed to move 10 of the disks at a time. How will this
change the asymptotic complexity of the problem (as a function of n)?
Answer: Method 1. T(n) = 2T(n 10) + 1 = O(2
n
10
). My work of 1 is the work of me
moving the last block of 10 largest disks.
Method 2. Put the disks into blocks which you move together. There are
n
10
blocks. This is
just like the normal problem when there are
n
10
disks. This gives T(n) = O(2
n
10
).
(c) (5 marks) Suppose that you are now allowed to move
n
10
of the disks at a time, where n is the
original number of disks. How will this change the asymptotic complexity of the problem (as a
function of n)?
Answer: T(n

) = 2T(n

n
10
) +1 = O(2
10
) = O(1), where n is the original n. (This does not
make a good recursive algorithm, because it should not know the size of the original instance.)
If we put the disks into blocks which you move together, then there are 10 blocks. This is
just like the normal problem when there are 10 disks. This gives T(n) = O(2
10
) = O(1).
(d) The goal is to design an algorithm for Towers of Hanoi with more towers/stacks. Review (3101
notes) the Hanoi alg with three towers. Remember to trust the friend. I move n disks from
tower
source
to tower
destination
using three towers as follows. I ask my friend to move the smallest
n1 disks from tower
source
to tower
spare
. On my own, I move the largest disk from tower
source
to
tower
destination
. Finally, I ask my friend to move the n1 disks fromtower
spare
to tower
destination
.
Let Friend
k
be the following routine that is able to move a stack of disks from a source to a
destination tower using k towers. It has a friend move the nc
k
(n) smallest disks onto a dierent
tower. (Here c
k
(n) is a parameter chosen later.) It has another friend move the c
k
(n) remaining
25
disks to the desired location. Finally, it has yet another friend move the n c
k
(n) smallest disks
onto the desired location. Let T
k
(n) be the required time. For example, the routine described
above is Friend
3
with c
3
(n) = 1.
i. Give the recurrence relation for the required time T
k
(n). Note T
k
(n) will be a function of n,
c(n), and possibly other T
k
.
Answer: The rst friend has all k towers available to him to move the n c
k
(n). Hence,
his time is T
k
(nc
k
(n)). The second friend can only use k 1 towers because we cannot
use the tower holding the smaller disks. Hence, his time is T
k1
(c
k
(n)). The third friends
time is again T
k
(nc
k
(n)). This gives The time required is T
k
(n) = 2T
k
(nc)+T
k1
(c).
ii. Now assume that you must move n disks using four towers. Are you allowed to ask Friend
4
to move n1 disks using four towers? Are you allowed to ask him to move n disks? Are you
allowed to ask Friend
3
to move n disks using three towers? Why?
Answer: Friend
4
is a recursive call and must be given a smaller instance. Hence, he can
move n 1 but not n disks. Friend
3
is a call to a routine that is already written so can
be called on any size instance. If you consider Friend(n, k) to be one recursive routine
on two inputs, then we can consider the instance 2n, k 1) to be smaller than n, k).
iii. Algorithms often run the fastest when you cut the input in half. Hence, let us set c(n) =
n
2
.
Determine its running time T
k(n)
(n). Hint: For an easier approximation ignore the inuence
of k(n).
Answer: The running time is at most T(n) = 3T(
n
2
) = (n
log a
log b
) = (n
log 3
log 2
).
iv. Again consider the algorithms with c(n) =
n
2
. What is the number k(n) of towers needed to
make sure each friend at each level of recursion has a spare tower. Prove this in two ways.
First unwind the algorithm to deterine how many towers is needed. The second way is to
state in the precondition of the problem that in order to move n disks, k(n) towers is needed.
Then prove that the tasks that you give your friends meet this precondition.
Answer: Unwinding we see that there are log
2
n levels of recurssion, each of which uses
an extra tower. Hence, k(n) = log
2
(n) +1 towers are needed. The only tricky case is the
second friend. When moving the
n
2
largest disks, he cannot use the tower containing the
n
2
smallest disks. Hence, he only has log
2
(n) + 1 1 = log
2
(
n
2
) + 1 towers available to
him. However, given he only has
n
2
disks to move, this is exactly the number of towers
he needs to be meeting the preconditions of the problem.
v. Consider the following game. You start with n disks. Each step your remove c(n

) disks from
your current number n

. We need to compute the number of steps h until there are no disks

left. It is easy to see that if c is a constant then h =
n
c
steps are needed. If c(n

) shrinks as
n

shrinks then fewer disks are removed each step and hence more than h =
n
c(n)
steps are
needed. However, until half the disks are gone, at least c(
n
2
) disks are removed each step.
Hence at most h

=
n/2
c(n/2)
levels are needed until half the disks are gone. The total number
of steps is then at most h

log
2
n
i=1
n/2
i
c(n/2
i
)
. If c(n) =
n
2
then give the Theta of this sum.
Again for c(n) = n
1/4
.
Answer: If c(n) =
n
2
, then h

log
2
n
i=1
n/2
i
c(n/2
i
)
=

log
2
n
i=1
2 = 2 log
2
n. If c(n) = n
1/4
,
then the terms in

log
2
n
i=1
n/2
i
c(n/2
i
)
change by some constant factor and hence this sum
is geometric. This means that the sum is dominated by the biggest term, giving h
(
n/2
c(n/2)
) = (
n
c(n)
), which is tight with our lower bound bound.
vi. Solve T
4
as a function of n and c(n). (Assume h =
n
c(n)
). What is the number of base cases
and what is the work at the top level? Ignoring the other levels for a minute, set c(n) to
optimize running time for these two levels. Then when considering all levels, approximate
T
4
(n).
Answer: The time with three towers is T
3
(n) = 2 T
3
(n 1) + 1 = 2
n
. For k = 4,
this is T
4
(n) = 2 T
4
(n c) + T
3
(c) = 2 T
4
(n c) + (2
c
). The number of base
26
cases is 2
h
where h =
n
c(n)
is the number of levels of recursion. The amount of work at
the top level is 2
c
. We set c(n) to balance these two amounts, i.e. 2
n/c
= 2
c
, giving
c(n) =

n. Because there are h levels, each with the same work 2
n/c
= 2
c
, the total work
is T
4
(n) = h2
c
=

n2

n
2

n
.
vii. Repeat this to compute T
5
(n) and T
k
(n). To make it easy ignore low order factors, eg.
nlog n n and nc 2
n
2
n
. For what value of k(n) does this time become a polynomial?
(For this value of k(n) do not expect to get a good approximation because we were ignoring
factors as we went along.) Compare this ti your result in (b).
Answer: Similarly, T
5
(n) = 2 T
5
(nc) +T
4
(c) 2 T
5
(nc) +2

c
. Setting the number
of base cases 2
n/c
to be equal to top level work 2
c
1/2
, gives c = n
2/3
and T
5
(n) 2
n
1/3
.
Suppose by induction T
k
(n) 2
n
1/(k

2)
). Then T
k
(n) = 2 T
k
(n c) + T
k1
(c)
2 T
k
(n c) + 2
c
1/(k3)
. Setting the number of base cases 2
n/c
to be equal to top level
work 2
c
1/(k3)
, gives c = n
(k3)/(k2)
and T
k
(n) 2
n
1/(k2)
. For k = log
2
(n) + 2, note
that 2
log
2
(n)
= n and hence n
1/ log
2
(n)
= 2 and T
k
(n) 2
n
1/(k2)
= 2
n
1/ log
2
(n)
= 2
2
= 4.
Clearly this is a bad approximation of the time, but it does say that this time is now
likely a polynomial.
Part III: Optimization Problems
Chapter: 12: Denition of Optimization Problems
Chapter: 13: Graph Search Algorithms
Section: 13.1: A Generic Search Algorithm
31. (10 marks) Graph searching:
(a) (2 marks) Briey describe an algorithm which is given a directed graph and a destination node t
and outputs all nodes v for which there is a path from node v to t. You can use as a subroutine
any algorithm discussed in class.
Answer: Turn the edges around and call a generic search algorithm (e.g., BFS) to output all
nodes reachable from t.
(b) (8 marks) Let G be a directed graph where each node of G also has a value:e.g., node 1 might
have value 100, node 2 might have value 50, etc. Given an O([E[ + [V [log[V [) time algorithm
that computes, for every node, the maximum value reachable from that node. For instance, if G
is strongly-connected (every nodes is reachable from every node), then for every node this will be
the maximum value in the entire graph. Hint: one worthwhile preprocessing step is to sort nodes
by value.
Answer: First modify your algorithm for Part (b) so that it marks nodes once they are
handled, and doesnt explore marked nodes on subsequent searches. Now sort the nodes in
decreasing order of value (([V [log[V [) time). Starting with the most valuable node, run
your modied search algorithm to nd all nodes that can reach this most valuable node, and
record this maximum value for each of these nodes. Now repeat the search from nodes of
lesser value in descending order. Since vertices are only handled once, this search step takes
T([E[, [V [) = ([E[), so the total time required is T([E[, [V [) = ([E[ +[V [log[V [).
Section: 13.2: Breadth-First Search/Shortest Paths
Section: 13.3: Dijkstras Shortest-Weighted Paths Algorithm
32. Fixed edge on a shortest path
Given a directed, weighted graph G with non-negative weights (weights can be zero), vertices s, t and
an edge (u, v) in G:
27
(a) Describe (in English or pseudo-code) how to determine whether (u, v) occurs on every path of
minimal cost from s to t.
Answer: First use Dijkstras algorithm to compute the cost d of the shortest path from s to
t. If d = , then there is no path of minimal cost from s to t, and it is true that (u, v) occurs
on every min cost path from s to t.
Now consider the graph G

obtained from G by removing the edge (u, v). Use Dijkstras

algorithm again to compute the cost d

of the shortest path from s to t in G

. If d ,= d

return
true, otherwise return false.
To see that the algorithm works, rst assume that the edge (u, v) does appear on every min-
cost path from s to t in G. That is, (u, v) appears on every path of cost d from s to t. Hence
the min-cost path found on G

cannot be of cost d, because it does not contain (u, v). Hence

d

,= d, and the algorithm returns true in this case.

If there is a min-cost path from s to t not containing (u, v), then the same path would be a
shortest path in G

, so d

= d, and the algorithm returns false in this case.

The algorithm calls Dijikstras algorithm twice, and can be implemented in O(E log V ).
(b) Describe (in English or pseudo-code) how to determine whether (u, v) occurs on some path of
minimal cost from s to t.
Answer: Once again, we run Dijkstras algorithm from s to compute the weights of the shortest
paths d(s, t) and d(s, u). If d(s, t) = , then there is no min cost path from s to t, and it is false
that there is such a path through (u, v).
If d(s, t) < , we run Dijkstras algorithm from v to compute d(v, t). We return true if d(s, t) =
d(s, u) +w(u, v) +d(v, t), and false otherwise.
To show that the algorithm works, suppose that there is a min-cost path p = s . . .
. .
p1
u
v . . .
. .
p2
t through (u, v). We know that a subpath of a shortest path must be a shortest
path, hence p
1
is a shortest path from s to u, and p
2
is a shortest path from v to t. We have
w(p) = d(s, t), w(p
1
) = d(s, u) and w(p
2
) = d(v, t). So d(s, t) = w(p) = w(p
1
) +w(u, v) +w(p
2
) =
d(s, u) +w(u, v) +d(v, t), and the algorithm returns true in this case.
In the opposite direction, suppose that the algorithm returns true. Then we have d(s, t) = d(s, u)+
w(u, v)+d(v, t). Denote by p
1
a shortest path from s to u, and p
2
a shortest path from v to t. Then
w(p
1
) = d(s, u), and w(p
2
) = d(v, t). Take the path p = s . . .
. .
p1
u v . . .
. .
p2
t. p obviously
contains the edge (u, v), and w(p) = w(p
1
) +w(u, v) +w(p
2
) = d(s, u) +w(u, v) +d(v, t) = d(s, t),
so p is a min-cost path containing (u, v), which proves that there exists such a path.
The algorithm calls Dijikstras algorithm twice, and can be implemented in O(E log V ).
Section: 13.4: Depth-First Search
Section: 13.5: Recursive Depth-First Search
Section: 13.6: Linear Ordering of a Partial Order
End of Chapter Questions
33. Given a directed graph and an edge u, v) give an algorithm to nd (if possible) a cycle that contains
u, v). Prove that the algorithm works.
Answer: To nd a cycle use depth rst search starting with the edge u, v). If the node u is
refound while the nodes u and v are still on the stack then the contents of the stack u, v, . . . ,
and then back to u forms a cycle. Conversely, suppose there is a cycle u, v, . . . , w, u. Before v is
removed from the stack, w must be found and the edge w, u) handled. Hence, the node u is will
be refound and the cycle returned.
28
Exercise 0.0.2 (See solution in Section ??) A Boolean circuit can be represented as an acyclic di-
rected graph in which every node is labeled with one of 0, 1, AND, OR, NOT, and edges are called
inputs. See Figure 1. Nodes with labels 0 or 1 (input nodes) have no edges (inputs) going into them,
nodes with label NOT have one input, and nodes with labels AND and OR can have two or more
inputs. The unique node of out-degree 0 is called the output node. Give an algorithm to evaluate a
circuit, producing the value of the output node according to the usual rules. The recursive algorithm
in Example ??.1 for evaluating an expression can solve this problem. Give some thought to why this
algorithm takes linear time on a tree and exponential time on a graph. Then come up with an O([E[)
time algorithm that uses topological sort. The exponential recursive algorithm is called Recursive Back-
tracking and the linear time algorithm is called Dynamic Programming. Try to understand these ideas
on your own and then read about them in Chapters ?? and ??.
NOT
y
AND AND
OR
y x
x
NOT
y=1 x=0
AND AND
NOT
OR
NOT
Figure 1: A circuit for (x AND y) OR(x) OR (x AND y) and a graph representing it. With x = 0 and
y = 1 this evaluates to (0 AND 1) OR(0) OR (0 AND 1) = (0) OR(1) OR (0) = 1
0.0.2 Run topological sort on the graph to nd a total order of the nodes. Computing the value of the nodes
in this order, starting with the input nodes and ending with the output node. By the denition of the
total order, when it is time to compute the value of a node, all the nodes inputs will already have
been computed.
Chapter: 14: Network Flows and Linear Programming
Section: 14.1: A Hill Climbing Algorithm with a Small Local
Maximum
Section: 14.2: The Primal-Dual Hill Climbing Method
Section: 14.3: The Steepest Ascent Hill Climbing Algorithm
Section: 14.4: Linear Programming
End of Chapter Questions
34. Network Flow: Consider the following ow network with source s and terminal (or sink) t. The labels
on edges are of the form f/c

where f is the current ow and c is the capacity of the edge. This ow

is feasible.
0/2
d c
4/9
5/5
7/17
5/7
6/6
t
2/2
3/3
2/2
6/10
b
a
s
b
a
s
t
c d d c
t
s
a
b
(a) Trace one iteration of the network ow algorithm. Use the nodes in the middle g to give the
augmentation graph for this ow. Make the necessary changes to the left gure to give the new
ow.
29
(b) Prove that there is no ow that is better than this current ow. (Use the right gure if you like.)
Answer: The value of this ow value is f(s, a) + f(s, c) = 6 + 8 = 14.
The min-cut (U, V ), where U = s , a , c , d. The min-cut capacity is
c(a, b) + c(d, b) + c(d, t) = 7 + 2 + 5 = 14. No ow can exceed this cut.
Augmenting Path
2
0/2
0/2
New Flow
5
5
V
U
9
8
7
1
2
6
3
8+2=10
2
Augmenting Graph
6
2 2
Cut
5
4
6+2=8
4
2 3
2
7
10
5
2
6
Augmenting Graph Given Flow
b
a
s
t
c d
b
a
s
t
c d
s
a
b
6/10
2/2
3/3
2/2
t
6/6
5/7
7/17
5/5
4/9
c d
d c
5/5
6/6
t
2/2
2/2
b
a
s
8/10
6/9
1/3
7/7
9/17
b
a
s
t
c d
7/7
d c
5/5
t
2/2
b
a
s
35. 7 marks: Network Flows
(a) 3 marks: In class, what was used to prove that when Ford-Fulkersons network ow algorithm
stops with a ow, it has the optimal ow?
Answer: The algorithm provides a matching min cut that acts as an upper bound
(b) 4 marks: Starting with the ow given below, complete the network ow algorithm.
36. (3 marks) Flows
Recall that the ow of a network is dened as the ow out of the source. Suppose you have a network
where ow conservation does not apply (i.e., for nodes other than the source and sink, the ow in may
not equal the ow out). Would the maxow-mincut theorem still apply? Why or why not?
Answer: The maxow-mincut theorem would no longer apply. The max ow is equal to the sum
of the capacities leaving s. This is also the value of a cut. Hence, the min cut is smaller than
equal to this. Consider a graph with edge s, v) having f = c = 2 and v, t) having f = 0 and
c = 1. Then the max ow is 2 and the min cut is 1.
37. (15 marks) Suppose you are given a real-valued network ow instance, an optimal ow for it, and an
edge u, v). Though the current ow through the edge u, v) does not exceed its capacity, the owner of
this edge has bribed the authorities to decrease the ow through this edge as much as possible while
maintaining the validity and the optimality of the overall ow. You are to design an ecient algorithm
to achieve this. Explain your algorithm using the following instance as an example. A formal proof of
correctness is not required.
s
5/5
7/7
2/10
t
5/11
Optimal Flow
2/6
u
v
w
Answer:
30
There are at least two reasonable solutions to this problem.
Solution 1.
The algorithm constructs in the normal way the augmenting graph for the current ow. Clearly,
because the current ow is optimal, there is no path from s to t. Instead nd a cycle in the
augmentation graph that contains the edge v, u). Find the smallest augmenting value on this
cycle. Modify the current ow along this cycle as in the network ow algorithm. Repeat this
process until there is no such cycle. To nd the cycle, use depth rst search starting with the
edge v, u). There is a cycle containing this edge if and only if the node v is refound while the
nodes v and u are still on the stack.
6/10
1/5
1/11
v
u
w
v
u
w
v
u
6/6
4 4
7
2
2
8
5
6
5
s
t
Augmentation Graph Augmentation Cycle New Optimal Flow
v
s 7/7
t
t
s
Solution 2.
Denote the original ow network as G and the optimal ow on this network as f

. Our algorithm
constructs a modied ow network G
0
in which the capacity of edge u, v) is set to 0. (While
by convention we normally include only edges with positive capacity, we make an exception in
this case for the sake of the proof.) The algorithm then uses an ecient maxow algorithm (e.g.,
Edmonds-Karp) to recompute the optimal ow f
0
for the modied network. We then form a
nal network G

in which the capacity of u, v) is set to f

f
0
. Running an ecient max ow
algorithm on this network computes the optimal ow of value f

with minimum ow on u, v).

The proof is as follows. First suppose that f
0
= f

, i.e., u, v) is not required to achieve the

maximum ow. Then clearly the optimal solution is to set the capacity of u, v) to f

f
0
= 0,
to ensure no ow on u, v).
Otherwise, f
0
must be less than f

. Since the original ow network G has a max ow of f

, by
the max-ow min-cut theorem, the capacity of the minimum cut of G is also f

. Similarly, the
capacity of the minimum cut of G
0
is f
0
. Since the only dierence between G

and G
0
is the
capacity of edge u, v), u, v) must be in all min cuts of G
0
, and all cuts that do not contain u, v)
must have capacity of at least f

. By increasing the capacity of u, v) by f

f
0
, we increase
the capacity of all min cuts of G
0
to f
0
+ f

f
0
= f

, and any max ow solution will entail a

ow of f

f
0
through u, v). If we increase the capacity of u, v) by less, than the capacity of
these min cuts will be less than f

. Thus f

f
0
is the minimum ow on u, v) that achieves an
optimal network ow.
38. Once your hill climbing algorithm nds an optimal solution for the given Network Flow instance,
describe how you would take steps that move you around on top of this plateau from one optimal
solution to another. As with a hill climbing step, this step will look at the augmentation graph of the
current ow. Clearly, however, you will not be able to nd an st- path in this augmentation graph.
What do you look for instead? (If you are really bold, attempt to prove that every optimal solution
can be reached with such steps.)
Answer: Same solution for the last question.
39. Short Answers
(a) (2 marks) What is the purpose of considering the Dual of a Linear Program?
(b) (2 marks) What is the Steepest Accent version of the Network Flows algorithm?
(c) (2 marks) What is the measure of progress for this Steepest Accent Network Flows algorithm that
we used and how much does it change each iteration.
31
40. Recall the Network Flow problem. Suppose that in addition to edge capacities, we wanted to allow
vertex capacities too. For example, we might want to say that vertex u can have at most 5 units of
ow passing through it and vertex v can have at most 3. Show how to convert the maximum ow
problem with both node and edge capacities to the standard maximum ow problem that is not too
much larger. In other words, imagine we had an instance with both kinds of constraints, and we wanted
to solve it using a black-box program that only solved the standard edge-capacity problem.
Answer: Replace each node u in the network with two nodes u
0
and u
1
and a directed edge
u
0
, u
1
) with capasity equal to the capasity of the node. Replace each directed edge u, v) in the
network with the directed edge u
1
, v
0
) with the same capasity. It is easy to see that any ow
in the original network translates to a ow in the new network and visa versa. My answer here
should be a little longer but I think you have it.
Chapter: 15: Greedy Algorithms
Section: 15.1: Abstractions, Techniques and Theory
Section: 15.2: Examples
Subsection: 15.2.1: Example: The Job/Event Scheduling Prob-
lem
Subsection: 15.2.2: Example: The Interval Cover Problem
Subsection: 15.2.3: Example: The Minimum-Spanning-Tree
Problem
41. The fractional-knapsack problem is the same as the integer-knapsack problem except that there is
a given amount of each object and any fractional amount of each it can be put into the knapsack.
Develop a quick greedy algorithm for this problem.
42. [2 + 2 + 4 = 8 marks] Greedy Algorithms:
(a) Trace out Minimal Spanning Tree on the following example. Darken the selected edges.
42
99
75
89
85
56
27
97
16
81
61
12 53
95
67
64
84
Answer:
42
99
75
89
85
56
27
97
16
81
61
12 53
95
67
64
84
32
(b) DFS can be used to nd a Minimum Spanning Tree (MST) of a given connected graph, if all edges
have equal weight.
Answer: T
(c) In general, what action does a greedy algorithm take each iteration?
Answer: Commits or rejects the next best object.
(d) What is the generic loop invariant for a greedy algorithm?
Answer: There exists an opt solution consistent with choices made so far.
(e) Briey explain how we prove that this loop invariant is maintained.
Answer: Massage the opt solution consistent with previous choices into one consistent with
the new choice as well.
(f) In both Dynamic Programming and Greedy Algorithms the question for the little bird needs to
be answered. What is the dierence between how this is done in these two settings?
Answer:
i. In Dynamic Programming all options are tried. The best of the best for each of these
options is taken. In a Greedy Algorithm one choice is made and committed to.
ii. In Dynamic Programming the choice is about whether or not to take the last object, while
in a Greedy Algorithm the best object is taken.
43. Bottleneck Spanning Trees (25 marks)
A bottleneck spanning tree of an undirected graph G is a spanning tree of G whose largest edge weight
is minimum over all spanning trees of G.
(a) (10 marks) Prove that a minimum spanning tree is a bottleneck spanning tree.
Answer: Let T = (V, E
T
) be an MST of G = (V, E), and let T

= (V, E
T
) be a bottleneck
spanning tree of G. Suppose T is not a bottleneck spanning tree. Then there exists an edge
(x, y) E
T
: w(x, y) > w(x

, y

)(x

, y

) E
T
.
Consider a cut (X, Y ) of the graph G, x X, y Y , such that (x, y) is the only edge in T that
crosses the cut. This cut partitions T into two subtrees T
X
and T
Y
. Since (x, y) / T

, there
must be at least one other edge (x

, y

), x

X, y

Y , crossing the cut such that (x

, y

) T

and (x

, y

) / T.
The edge (x

, y

) forms a cycle with the path from (x

to y

) in T. Thus we can form a new

spanning tree

T by replacing x, y by (x

, y

) in T. Since w(x

, y

) < w(x, y), w(

T) < w(T).
However, this is not possible, since T is a minimum spanning tree. Thus our supposition that
T is not a bottleneck spanning tree must be wrong: if T is a minimum spanning tree, it must
also be a bottleneck spanning tree.
(b) (15 marks) Your company needs to lease access to a communications network linking a large
number of Canadian cities. The network can be modeled as a connected, undirected graph with
vertices representing cities and edges representing direct communication links between cities. Each
edge provides a maximum data rate, or bandwidth. You need to be able to move data from any
one of the cities to any other, but you do not necessarily need every link. Since you will be taried
according to the number of links you use, you want to minimize the number of links you lease. In
other words, you seek a spanning tree of the graph.
Since your company provides broadband services, you wish to maximize bandwidth. On any
path connecting two cities in the network, the bandwidth of the path is limited by the minimum
bandwidth edge on the path. Your goal is to determine the spanning tree that connects each
pair of cities by a maximum bandwidth path. Your tree will have the property that adding any
additional edges from the graph will not produce a path with higher bandwidth.
Design an algorithm to solve this problem and prove that it is correct.
33
Answer: We assign each edge in the graph G a weight equal to the inverse bandwidth. Then
we compute (using, e.g., Kruskals or Prims algorithm) a minimum spanning tree for this
graph. We claim that the resulting tree will have the desired properties.
The proof is similar to the proof for Part (a). Let T = (V, E
T
) be an MST of G = (V, E).
Consider an arbitrary pair of vertices x, y V , and the path p connecting these vertices in
T. Let (x
m
, y
m
) p denote the minimum bandwidth (bottleneck) edge in p. Consider a cut
(X, Y ) of the graph G, x, x
m
X, y, y
m
Y , such that (x
m
, y
m
) is the only edge in T that
crosses the cut. This cut partitions T into two subtrees T
X
and T
Y
.
By way of contradiction, suppose that p is not a maximum bandwidth path in G. That means
that there exists another path p

G from x to y of greater bandwidth. Since (x

m
, y
m
) / p

,
there must exist a dierent edge (x

, y

) p

, x

X, y

Y crossing the cut (X, Y ), such

that (x

, y

) / T. This edge forms a cycle with the path from x

to y

in T. Thus a new
spanning tree

T can be formed by replacing (x
m
, y
m
) by (x

, y

) in T.
Note that since the bandwidth of (x

, y

) is greater than the bandwidth of (x

m
, y
m
), w(x

, y

) <
w(x
m
, y
m
). Thus w(

T) < w(T). This is a contradiction, since T is a minimum spanning tree.

Thus our supposition that p is not a maximum bandwidth path must be wrong: all paths in
T must be maximum bandwidth paths.
End of Chapter Questions
44. ** The question is already in the notes in some form. Here is an answer. Truth is that I did not like
it as much as I thought I would. **
We proved that the greedy algorithm works for the making change problem, when the denominations
of the coins are 25, 10, 5, and 1, but it does not work when the denominations are 4, 3, and 1 cent.
More generally, suppose the coin denominations are c
1
> c
2
> . . . > c
r
= 1 (order taken by the greedy
algorithm). An interesting problem is determining whether the greedy algorithm works for these. A
complete answer to this question is too hard. However, we can answer some questions about this.
(a) Suppose for each i, each coin c
i
is an integer multiple of the next smaller c
i+1
, eg 120, 60, 12, 3, 1.
This question will proof that for every set of coins with this property and for every input amount
to make up, that the greedy algorithm works.
Add: Suppose you have a second coin worth 1, a minute coin worth 60, an hour coin worth
360 a day coin worth 24*360. If you are making up some amount with the minimum number
of coins, will you ever have 73 minute coins? If you add up the minutes and the second in the
optimal, what is the biggest this sum will be?
i. Consider such coin denominations. Consider an amount to be made with these coins. Consider
an optimal solution optS
LI
held by the fairy god mother. For each i [1, r], let n
i
be the
number of coins of denomination c
i
. Prove a useful property of n
i
.
Answer: We will prove that for each i [2, r], the number n
i
of coins of denomination c
i
is in the range [0,
ci1
ci
1]. By way of contradiction, if n
i

ci1
ci
, then we could remove
ci1
ci
coins of denomination c
i
and replace them with one coins of denomination c
i1
. The
total value does not change because
ci1
ci
c
i
= 1 c
i1
. The total number of coins
decreases by
ci1
ci
1 > 0. This contradicts that what the fairy god mother holds is an
optimal solution. We are using here that fact that
ci1
ci
is an integer.
ii. Again, consider the optS
LI
held by the fairy god mother that has the property stated in the
last question. For each i [2, r + 1], let a
i
=

j[i,r]
n
j
c
j
be the total value of the coins
with denomination c
i
or smaller. Prove by induction on i, some useful property of a
i
.
Answer: We will prove that for each i [2, r + 1], that a
i
c
i1
1. The base case,
when i = r + 1, must have n
i
= 0 coins with denomination c
i
or smaller because there
are no coins with denomination c
r+1
or smaller. Now by way of induction, assume that
a
i+1
c
i
1. By denition of a
i
, we have that a
i
=

j[i,r]
n
j
c
j
= n
i
c
i
+a
i+1
. By our
34
previous question and by the induction assumption, this is at most [
ci1
ci
1] c
i
+[c
i
1] =
c
i1
1. Hence, the inductive hypothesis is true for i. By way of induction, it is true for
all i [2, r + 1].
iii. Suppose that the fairy god mother holds an optimal optS
LI
consistent with all the choices
made by the Algorithm so far. Suppose the algorithm in the next iteration commits to a
coin of denomination c
k
. Explain how to massage optS
LI
into optS
ours
and prove that this
is a valid solution, is consistent both with the choices made previously by the algorithm and
with this new choice, and is optimal. Might there be two dierent optimal solutions to this
problem?
Answer: We will prove that the number n
i
of each denomination in the optimal solution
is unique by showing that except for may be swapping one coin of denomination c
i
for
another of the same denomination, we do not need to massage the fairy god mothers
solution optS
LI
at all.
Let n

i
be the number of coins of denomination c
i
in the fairy god mothers solution optS
LI
excluding those that the algorithm has committed to so far. Our goal is to prove that
n

k
1, i.e. that optS
LI
contains a coin of denomination c
k
that has not previously been
committed to by the algorithm. We massage optS
LI
into optS
ours
by replacing this c
k
with the one that the algorithm committed to.
Let A be the remaining amount required by the algorithm, i.e. the total amount required
by the input minus the value of the coins committed to so far. Because the algorithm did
not commit to a coin with denomination c
k1
but did commit to a coin with denomination
c
k
, we know that c
k
A < c
k1
. Because the remaining amount for the algorithm and
the adversary is the same, we have that A =

i[1,r]
n

i
c
i
.
Because A < c
k1
, we know that know that the fairy god mother is not holding any large
coins that we do not know about, i.e. for i k 1, we have that n

i
= 0.
Let a

k+1
=

j[k+1,r]
n

j
c
j
be the total amount of the coins with of denomination
c
k+1
or smaller in optS
LI
excluding those that have been committed to. By the previous
question, we have a
k+1
c
k
1. Hence, these coins are not enough to make up the
required A c
k
. We can conclude that the fairy god mother, just like the algorithm,
must be holding another c
k
coin.
(b) When the coins are 4, 3, and 1. The problem seems to be that the 4 and the 3 coin are too close.
Suppose for each i, each coin denomination is at least twice the previous, i.e. c
i
2c
i+1
. One
might then argue that if a c
1
coin could be taken that it should be taken, otherwise at least two
c
2
needs to be taken instead.
i. If possible, proof that for every set of coins with this property and for every input amount
to make up, that the greedy algorithm works. Hint: Follow similar steps as done in the last
question.
Answer: It is not possible. See the next question.
ii. If possible, give a set of coins with this property and an input amount for which the greedy
algorithm does not work. Hint: Try three coin denominations c
1
, c
2
, and c
3
= 1. Before
choosing the values for c
1
and c
2
decide how many of each coins will be in the greedy solution
and how many will be in the optimal solution. Write down some requirements on the values
c
1
and c
2
and then solve for them.
Answer: How silly of me. 2a asks about 25, 10, 1. It is a ne counter example. Maybe
you will nd my answer interesting.
The greedy algorithm will commit to one of the largest coins c
1
. Then there is not enough
amount remaining to take a c
2
. Hence, many 1 coins will be needed to make up the rest.
Use n to denote this number. The optimal, needing to be dierent will not take a c
1
but
will take three c
2
coins. The reason for three is that we know that c
1
is at least twice c
2
.
This gives that Amount = c
1
+n = 3c
2
. Because the greedy did not take c
2
, we have that
n < c
2
. Because the greedy solution is worse, we have that 1 +n > 3. Our requirements
on the coin denominations are that c
1
> 2c
2
and that c
2
> 2c
3
= 2.
35
To get 1 + n > 3, set n = 3. To get n < c
2
, set c
2
= n + 1 = 4, which as needed is more
than twice c
3
= 1. Subbing this into c
1
+ n = 3c
2
gives c
1
= 9 which as needed is more
than twice c
2
= 4. Amount = c
1
+n = 3c
2
= 12.
If the coins are 9, 4, 1, each coin is more than twice the previous and the greedy algorithm
does not work for the amount 12. The greedy takes 9 + 1 + 1 + 1 while the optimum is
4 + 4 + 4.
45. (20 marks) Minimum Spanning Trees
*** The MST on this graph is in text. Changing weights is in text exercise 16.2.3 ExLabel 57 ***
(a) (2 marks) Prove that, in general, an undirected graph may have more than one minimum spanning
tree (MST).
Answer: This graph has 3 MSTs.
1
1 1
(b) (6 marks) Prove that, if all weights of an undirected graph G are unique, then the MST of the
graph is unique.
Answer: Let T be an MST of G produced by Kruskals algorithm. Suppose there is also a
dierent MST T

. Since T and T

have the same number of edges, there must be at least one

edge (u, v) in T

but not in T. Consider a cut (U, V ) of G, u U, v V such that (u, v) is

the only edge in T

that crosses the cut. There must be another edge (x, y) T, (x, y) / T

that also crosses the cut. Note that the procedure followed by Kruskals algorithm guarantees
that (x, y) is a light edge crossing this cut. Augmenting T

with this edge (x, y) creates an

alternate path from u to v in T

through (x, y) that forms a cycle with the edge (u, v). Thus
a new spanning tree T

can be formed by replacing (u, v) in T

with (x, y). Since the weights

are unique, w(x, y) < w(u, v), thus w(T

) < w(T

), which means T

could not have been an

MST. Thus T must be the unique MST of G.
(c) (4 marks) On the graph below, darken the edges that are part of the MST. (Pencil recommended.)
42
99
75
89
85
56
27
97
16
81
61
12 53
95
67
64
84
Answer:
36
42
99
75
89
85
56
27
97
16
81
61
12 53
95
67
64
84
(d) (2 marks) Suppose we decide to change the weight 56 to any real number from to +. What
is the interval of values that it could be changed to without making your MST invalid?
Answer: The weight 56 can be changed to any number in the half-open interval (, 81].
(e) (2 marks) Answer the same question for the edge with weight 85.
Answer: The weight 85 can be changed to any number in the half-open interval [61, ).
(f) (4 marks) State the general principal that determines the range over which the weight of an edge
can be changed without invalidating an MST.
Answer: The weight of an edge (u, v) that is part of an MST can be lowered by any amount
without invalidating the MST. Deleting (u, v) partitions the graph into two sets of nodes.
Currently (u, v) must be the lightest of the edges that crosses this cut. Its weight can be
raised as long as this is the case. Hence, it can be raised to equal the weight of the second
lightest such edge. Said another way, it can also be raised to equal the weight of the lightest
edge that is not in the MST, but if added to the MST would form a cycle with (u, v) in the
MST.
Similarly, the weight of an edge (u, v) that is not part of an MST can be raised by any amount
without invalidating the MST. Add (u, v). This creates a cycle. Currently (u, v) must be the
heaviest edge in that cycle. Its weight can be lowered as long as this is the case. Hence, it
can be lowered to equal the weight of the second heaviest such edge. Said another way, it can
also be lowered to equal the weight of the heaviest edge (that is in the MST and would form
a cycle with (u, v) in the MST if (u, v) were added to the MST.
46. (MST) Our goal is to nd a shortest path from node s to node t in an undirected graph. Our algorithm
of choice is to nd a minimal spanning tree and then to return the path from s to t in this tree. Give
a graph with n nodes for which the ratio between the length of the path returned and the length of
the optimal path is as bad as possible.
Answer: The graph has a single edge from s to t with weight 1. This is the optimal path. There
is also a path with n 1 edges from s to t where each edge has weight 1 . The MST will take
all the edges in this path because they are the lightest. It will not take the single edge from s to t
because it creates a cycle. Hence, the algorithms solution is of weight (1 ) (n 1). One could
prove that this is the worst case.
47. Unique Solution/Proles: Given an undirected graph with weights on the edges, a solution to the MST
problem is a subset of the edges that forms a spanning tree. Dene the prole of a solution to be
multi-set consisting of the weights of the edges in the solution sorted so that we no longer know which
edges they came from. For example, if the solution consists of three edges with weights 2, 7, and 2.
37
Then the prole would be [2, 2, 7]. Prove that every input instance has a unique optimal prole, i.e.
every optimal solution has the same prole. Go on to prove that if the instance graphs has a dierent
weight on each edge, that the graph has a unique MST. Hint: Watch the optimal solution held by the
fairy god mother.
Answer: Consider a computation of the greedy algorithm on some instance graph. We proved
that it returns an optimal solution, which we will denote with optS
greedy
. Let P
greedy
be its
prole. I claim that every optimal solution for this graph has this same prole. To prove this
consider any optimal solution. Lets call it optS
other
and its prole P
other
. We must prove that
P
other
= P
greedy
. To do this, we will trace out our proof that the greedy algorithm works. To
begin we give optS
other
to the fairy god mother to witness the fact that the loop invariant is true
when the greedy algorithm rst gets to the top of the loop, i.e. that there is an optimal solution
consistent with every thing committed to so far by the algorithm. We know that optS
other
is
consistent with this because the algorithm has not committed to any edges yet. Each iteration,
the prover instructs the fairy god mother to make a change to her optimal solution so that it
remains a valid optimal solution and so that it remains consistent with everything committed to
by the greedy algorithm. In the end, she holds the same optimal solution optS
greedy
produced by
the greedy algorithm. Now lets pay attention to the prole of the optimal solution held by the
fairy god mother. The prover tells the fairy god mother to remove one edge and replace it with
another. We proved that the weight of the added edge cant be larger than that of the removed
edge. Also it cant be smaller or else what the fairy god mother had before was not optimal.
Hence, the weights of these swapped edges must have been the same. This mean that the prole
that she has does not change. If the prole never changes it follows that her rst prole P
other
is
the same as each of her intermediate proles P
LI
which is the same as her nal prole P
greedy
.
This concludes the proof. If the instance graphs has a dierent weight on each edge, then the
unique optimal prole uniquely species the edges in the optimal solution.
48. Greedy Algorithms: You are standing on the bank of a river that you must cross by jumping on stepping
stones which are spread in a line across the river. Though you cant jump far, you want to jump on
as few stones as possible. An instance to the problem species d
0
, d
1
, d
2
, . . . , d
n
), where d
0
= 0 is
your location, d
i
for i [1, .., n1] is a real number giving the distance in meters that the i
th
stone is
from you, and d
n
is the distance of the opposite shore. Assume that 0 = d
0
d
1
d
2
. . . d
n
. A
solution is the subset S [0, .., n] of the stones that you jump on. Locations 0 and n must be included.
The constraint is that if you jump from stone i to stone j then d
j
d
i
1. The cost of a solution,
which is its size [S[, should be minimized.
(a) (3 marks) What action does your greedy algorithm take each iteration? (Imagine that you are
crossing a river on stones. How would you choose which of the stones to jump on? (Or if lost,
what actions does a general greedy algorithm take each iteration?)
Answer: Suppose that i is the last stone jumped on. In a greedy way, jump on the stone j
that is as far from i as you can jump, i.e. the largest j for which d
j
d
i
+ 1.
(b) (3 marks) What is the loop invariant for your (and any) a greedy algorithm?
Answer: There exists an opt solution consistent with choices made so far.
(c) (10 marks) Explain how we prove that this loop invariant is maintained. Who constructs what?
How is it constructed? What needs to be proven about it? Prove these things. (Again if lost
answer these things for a general greedy algorithm.)
Answer: The fairy good mother is holding an optimal solution consistent with the choices
made so far, i.e. she has a set of stone that she jumps on which includes the stones that
algorithm has jumped on up to and including stone i.
The prover tells her how to modify her solution to be with the new choice as well. He says
to let j

be the next stone after stone i that she jumps on. He tells her to replace j

with j.
This is still consistent with the previous choices made by the algorithm because we only
changed stones after i. It is also consistent with the new choice.
38
It is still optimal because its cost is the same.
It is still valid. Only two jumps have changed. That from i to j and that from j to the next
node k that is the fairy god mothers solution. The jump from i to j is valid because the
algorithm does it. Because j is the furthest stone reachable from i and because j

is reachable
form i, it follows that d
j
d
j
. The jump from j to k must be valid because it is no longer
than the jump from j

to k and the fairy god mother does this.

The existence of this proves that the loop invariant is still true after going around the loop.
49. When asked to explain how we prove that this loop invariant is maintained, the following answers were
given. What, if anything is wrong with them.
(a) We prove it is consistent, optimal, and valid.
(b) We must prove that the choices made by the algorithm are consistent, optimal, and valid.
(c) the algorithms solution
(d) What the algorithm has done so far is optimal.
(e) If you consider the instance to be the part of our instance that we have seen so far, then we have
an optimal solution.
(f) The Prover compares the Fairy Godmothers optimal solution to what the algorithm has done.
Answer:
(a) Dont say it without saying what it is.
(b) We tell the fairy god mother to change her solution OPT
LI
into OPT
ours
. It is OPT
ours
, the
choices made by the algorithm, that we must prove are consistent with the choices made so
far by the algorithm, valid, i.e. has no conicts within itself, and optimal, i.e. there is not a
better valid solution out there.
(c) Dont say the algorithms solution. The algorithm have made some commitments already,
but this does not yet constitute a valid solution.
(d) Dont say What the algorithm has done so far is optimal. Optimal is a property of full
solutions and we dont have one yet.
(e) The More of the Input Loop Invariant does not work here. Saying that what you have
is optimal within this part of the instance is only a local property. It wont necessarily
guarantee global optimality. Eg. When looking for the best path from s to t, taking the best
edge out of s, may give you the best path to this node u, but this is not part of the best path
to t.
(f) The prover is unable to see the Fairy Godmothers optimal solution.
50. (20 marks) (See matching DP algorithm) You are robbing a warehouse lled with sealed boxes. Each
box is specied by w
i
, c
i
), where w
i
is how heavy the box is and c
i
is its capacity in terms of how
much weight can sit on top of it. You must put all of the boxes on the truck, but the base of the truck
is small. Hence, you must stack all the boxes into one tall pile. Your job is to determine in which order
to stack the boxes so that nothing breaks. The rule is that for each box i, the sum of the weights of
the boxes j above it cannot exceed the amount that can be placed on box i. More formally, if S
i
is the
set of boxes to be stacked on box i, then i,

jSi
w
j
c
i
. Assume that the truck itself has innite
capacity. A solution is consists of a permutation of all of the boxes. It is valid (and of value one) if
every box can be put on the truck in this order with nothing breaking. Otherwise, the solution is not
valid (and has value zero). Your goal is to nd a valid solution.
Being a robber, you are greedy. You run into the warehouse and grab the best box based on some
greedy criteria. After grabbing the box, you put it on top of the stack you are building.
(a) (5 marks) One obvious algorithm is to grab the box with the largest weight w
i
, because clearly
you want the heavy boxes on the bottom. Prove that this greedy algorithm may fail to nd a
valid solution when one exists.
39
Answer: Putting the heaviest box on the bottom may be a mistake. Suppose w
1
= 3, c
1
= 1,
w
2
= 2, and c
2
= 3. For this instance, there is a valid solution (Box 1 on Box 2), but the
greedy algorithm fails, because the heaviest box (Box 1) lacks sucient capacity for Box 2.
(b) (5 marks) Another obvious algorithm is to grab the box that can hold the most weight c
i
, because
clearly you want these on the bottom of the stack. Prove that this greedy algorithm also fails.
Answer: Putting the largest c
i
box on the bottom may be a mistake. Suppose that w
1
= 1,
c
1
= 3, w
2
= 100, and c
2
= 2. For this instance, there is a valid solution (Box 1 on Box 2),
but the greedy algorithm fails, because the box with largest capacity (Box 1) lacks sucient
capacity for Box 2.
(c) (6 marks) A third obvious algorithm is to grab any box that can support the set of boxes that
remain after this box is taken. Note that a box does not have to support itself. Prove that if such
a box exists, then the box with maximum sum w
i
+c
i
satises this constraint.
Answer: Let S be the set of boxes remaining. Suppose box i has the largest sum w
i
+ c
i
.
Suppose that box j can support the set of boxes S j. We prove that it follows that box
i can hold the set of boxes S i. By the second statement c
j


kSj
w
k
. Adding w
j
to both sides yields c
j
+ w
j


kS
w
k
. Combining this with the rst statement gives that
w
i
+ c
i
c
j
+ w
j


kS
w
k
. Subtracting w
i
from both sides yields c
i


kSi
w
k
as
required.
(d) There may be many boxes that can support the set of boxes that remain after this box is taken.
(The last question proves that one such box is the one with largest w
i
+ c
i
, but there may be
others.) How do we know which of these choices will lead to a overall solution that stack all of the
boxes? This third greedy algorithm does not worry about this, but just grabs any one that has
this property. We will no prove that this greedy algorithm always gives a correct way of stacking
the boxes (assuming one exists).
i. (5 marks) What is the loop invariant for your greedy algorithm?
Answer: There exists an optimal solution consistent with what the algorithm has done
so far.
ii. (25 marks) Prove that the algorithm maintains this loop invariant. Make sure that you specify
all steps.
Answer: Assume that there exists an optimal solution consistent with what the algorithm
has done during the rst t steps. Let optS
LI
be one such optimal solution. (You can
assume the fairy god mother is holding it.)
The algorithm chooses box some box i that can support the set of boxes that remain after
this box is taken. Perhaps the fairy god mother took box j at this step instead. This
means that box i must be higher up in her stack. Tell her to take box i out of the stack
and insert it into the t + 1
st
place, directly below box j. We need to prove that this is a
consistent and valid solution.
optS
LI
was consistent with the rst t boxes and we maintain this consistency by not
changing these boxes.
It was valid. We have to prove that it is still valid. Nothing changes for the rst t boxes
and things either stay the same or get better for box j and those above it because if they
supported box i, they no longer do. What remains is to show that box i does not break
from this new load. However, be denition box i can support the set of boxes that remain
after i box is taken.
(e) (5 marks) The correctness of this algorithm depends on the fact that it only needs to work if it is
possible to take all of the boxes. Prove that if the goal is to take as many boxes as possible then
greedily taking the box with the largest w
i
+c
i
could be a mistake.
Answer: Suppose that w
1
= 4, c
1
= 1, w
2
= 2, c
2
= 2, w
3
= 2, and c
3
= 2. This criterion
would lead to selecting box 1 rst, and only that box could be loaded. Selecting either box 2
or 3 rst would allow the other to be loaded as well.
40
Chapter: 16: Recursive Backtracking
Section: 16.1: Recursive Backtracking Algorithms
Section: 16.2: The Steps in Developing a Recursive Backtrack-
ing
Section: 16.3: Pruning Branches
Section: 16.4: Satisability
Section: 16.5: Scrabble
End of Chapter Questions
Chapter: 17: Dynamic Programming Algorithms
Exercise 0.0.3 Shortest Prex: The input consists of an array A[1..n], with each element from
0, 1, 2, 3. The output is a matrix B, such that for each x, y, z [n..n], B[x, y, z] = i when A[1..i]
is the shortest prex of A that has x more 1s than 0s, y more 2s than 0s, and z more 3s than 0s.
The running time must be O(n). You may assume that memory is initially zeroed.
Answer: Start with the matrix B[n..n, n..n, n..n] all zero. Scan the array from left to right
maintaining the number of each of the values in the prex A[1..i]. Let x be the number more 1s
than 0s, y the number more 2s than 0s, and z the number more 3s than 0s. If B[x, y, z] = 0,
set B[x, y, z] = i. The running time must be O(n).
Exercise 0.0.4 Repeat the same, but memory initially has arbitrary values. The trick is to use a data
structure other than an matrix so that it can still be done in time O(n).
Answer: Store in a set the tuples x, y, z) that have such an i. The abstract data types described
in the notes states that this can be done in O(1) time each.
Exercise 0.0.5 Longest Balanced Substring: Again, the input consists of an array A[1..n], with each
element from 0, 1, 2, 3. The output now is a longest contiguous subarray A[i..j] with equal number of
0s, 1s, 2s, and 3s. The running time must be O(n). Hint: Use the solution to Exercise 0.0.3.
Answer: Let X be the number more 1s than 0s in the entire array A, Y the number more
2s than 0s, and Z the number more 3s than 0s. Scan the array from right to left maintaining
the number of each of the values in the postx A[j..n]. Let x
2
be the number more 1s than
0s, y
2
the number more 2s than 0s, and z
2
the number more 3s than 0s in A[j..n]. If the
substring A[i+1..j 1] is the have the same number of each value, then the prex A[1..i] must
have dierences x
1
= Xx
2
, y
1
= Y x
2
, z
1
= Z z
2
. B[x
1
, y
1
, z
i
] gives the smallest i for which
this is case. This gives the longest balanced substring A[i+1..j 1] for this j. Take the longest
overall values of j.
Exercise 0.0.6 Longest #0#1 = 2 mod 5 Block: This time the input consists of an array A[1..n],
with each element 0 or 1. The output is a longest contiguous subarray A[i..j] whose number of occur-
rences of 0s minus its number of occurrences of 1s is 2 mod 5. Design and analyze an incremental
algorithm to solve this problem in O(n) worst-case time. Hint: A similar technique works as did for
Exercise 0.0.5.
Answer: Below we oer two solutions. They are essentially saying the same thing.
Solution 1: We dene modality of a 0/1 array X to be the number of 0-bits minus the number
of 1-bits in X modulo 5, and we denote it with M(X). For instance, M([1, 1, 0, 1, 0, 1]) = (2
4) mod 5 = +3. We will call a 0/1 array with modality 2 feasible. So, we are trying to nd the
longest feasible subarray (LFS) of A[1..n].
The algorithm we develop below is an example of an iterative incremental algorithm where we need
to strengthen the loop invariant (i.e., the inductive hypothesis), so that we can maintain it from
41
one iteration to the next, and at the end it is strong enough to imply the desired postcondition.
As we scan through array A, we not only maintain the longest feasible subarray of the scanned
portion, but we also maintain additional information which is given by the auxiliary index-array
B[0..4] as explained below.
We use the following facts about prexes of array A:
Fact 1: For any indices 1 j < i n we have M(A[j+1..i]) = M(A[1..i]) M(A[1..j]) mod 5.
Corollary 1: Subarray A[j+1..i] is feasible if and only if M(A[1..j]) = M(A[1..i]) 2 mod 5.
Fact 2: Longest feasible subarray of A[1..i] is either the longest feasible subarray of A[1..i 1]
or the longest feasible sux of A[1..i], whichever is longer.
The following is the algorithm based on the above ideas. IndexRange eventually denotes the
index range of LFS of array A. IndexRange = [null] means A has no feasible subarray.
algorithm LongestFeasibleSubarray(A, n)
precond: A[1..n] is an array of n 0/1 bits.
postcond: A[IndexRange] is the longest feasible subarray of A[1..n]
begin
3 for m = 0..4 do B[m] +
4 MaxLength 0; Modality 0; IndexRange [null]
5 for i = 1..n do
6
loopinvariant:
(i) A[IndexRange] = LFS(A[1..i 1]) of length MaxLength
(ii) For each m 0..4, A[1..B[m]] is the shortest prex of A with modality m.
If such a prex does not exist, then B[m] = +.
(iii) Modality = M(A[1..i 1])
10 if( A[i] = 0 ) then Modality Modality + 1 mod 5 else Modality Modality 1 mod 5
11 if( B[Modality] = + ) then B[Modality] i
12 j B[Modality 2 mod 5]
13 if( i j > MaxLength ) then
14 MaxLength i j
15 IndexRange [j+1..i]
end if
end for
16 return IndexRange
end algorithm
Proof of Correctness:
preCond & codeA loop invariant:
Clearly LI(i), LI(ii) and LI(iii) hold for i = 1.
loop invariant

& not exitCond

& codeB loop invariant

:
Suppose LI(i), (ii), and (iii) hold just before an arbitrary iteration i. We need to show they
hold at the end of that iteration, that is, at the beginning of iteration i+1. LI(iii) and line
10 imply that LI(iii) is maintained correctly. Line 11 maintains the validity of LI(ii), since
extending the prex A[1..i1] to A[1..i] can only aect array B at the entry m = M(A[1..i]) =
Modality. If B[Modality] is already not +, then we should leave it intact; it already
indicates the shortest prex with modality Modality. However, if B[Modality] is +, then
clearly A[1..i] is the shortest prex with modality Modality. Lines 12-15 guaranty LI(i) is
maintained by using Fact 2 and Corollary 1 and LI. Let m = Modality2 mod 5. Line 12 sets
j = B[m]. Corollary 1 and LI(ii-iii) imply that A[j+1..i] is the longest feasible sux of A[1..i]
at lines 13-15. This, LI(i), and Fact 2 imply that the longer of A[IndexRange] and A[j+1..i] is
the longest feasible subarray of A[1..i]. (Note that on line 12, if j = B[Modality 2 mod 5] =
+, then that and LI(iii) imply that A[1..i] does not have a feasible sux. It would also
42
make the if-condition of line 13 fail. This is the reason for leaving the undened entries of B
to be +.)
loop invariant & exitCond & codeC postCond:
We exit the for-loop at the end of iteration i = n. This can be interpreted as the beginning
of iteration i = n+1. That condition and LI(i) imply the Postcondition. So, we see that to
get to the postcondition we need LI(i), but to maintain LI(i) during the iterations, we need
to strengthen LI by the added information provided in LI(ii-iii).
Termination:
The algorithm clearly takes O(n) time.
Solution 2: The following is another almost identical solution that has the small advantage that
it is closer to the hint to read section 12.1.1 of [HTA]. As before, we read one item at a time.
After reading A[1..i], we maintain not only the longest feasible subarray of the scanned portion,
but we also maintain the longest one from amongst those that ends in A[i], because though short
now, it might be extended to a longest one. However, when we are in the middle of working on
a subarray, its modality so far might not be 2. Hence, for each c [0..4], we let j
c
be such that
A[j
c
..i] is the longest subarray ending in A[i] that has modality c. If such a subarray does not
exist then j
c
= nil. Initially i = 0 and j
0
= 1, because there is such a subarray of length zero with
modality zero, but for other c, j
c
= nil. We maintain the loop invariant as follows. If we read
A[i+1] = 0, then the longest subarray ending in A[i] that has modality c extends to the longest
subarray ending in A[i+1] that has modality c+1. Hence j
c+1 mod 5
= j
c
. Similarly, if we read
A[i+1] = 1, j
c1 mod 5
= j
c
. If after doing this we have j
0
= nil, then we change this to j
0
= i+2
because A[i +2..i +1], though of length zero is the longest subarray ending in A[i +1] that has
modality 0. If the longest one A[j
2
..i] with modality 2 that we are working becomes longer than
our longest, then this new one becomes the longest. In the end, we have the longest.
Section: 17.1: Start by Developing a Recursive Backtracking
Section: 17.2: The Steps in Developing a Dynamic Program-
ming Algorithm
Section: 17.3: Subtle Points
Exercise 0.0.7 (See solution in Section ??) Two Friends Per Bird Answer: Now suppose that in the
bird and friends version of the dynamic programming algorithm, when given an instance of size n,
the bird is asked a question with K possible answers and then two friend each are asked to solve a
subinstance that is never any bigger than size
n
2
. (See Chains of Matrix Multiplications). Again
briey describe what needs to changed so that the dynamic program also outputs the number of optimal
solutions.
Answer: Given a birds answer k, I ask two friends to solve a subinstance and then take these
two solutions to produce my solutions. Let num
k,1
be the number of dierent optimal solutions
that the rst of my friends might give me and num
k,2
be the number that the second might give.
There are then num
k
= num
k,1
num
k,2
ways for me to combine one solution from the rst and
one solutions from the second to give me an optimal solution consistent with this birds answer k.
Generally, these are all distinct solutions. Again the number of optimal solutions to my problem
is num =

k{k | optCost
k
=optCost}
num
k
.
Exercise 0.0.8 (See solution in Section ??) Now let Num(n) denote the maximum number of optimal
solutions that any dynamic program as described in the last question might output given an instance
of size n. Give the recurrence relation for Num(n). In order to solve it, assume that Num(n) = b
n1
for some constant b and solve for b.
43
Answer: num =

k{k | optCost
k
=optCost}
num
k,1
num
k,2
. The maximum this can be is
Num(n) =

k[1..K]
Num
_
n
2
_
Num
_
n
2
_
= K Num(
n
2
)
2
. Plugging in Num(n) = b
n1
(and Num
_
n
2
_
= b
n
2
1
) gives that
Num(n) = K Num
_
n
2
_
2
b
n1
= K
_
b
n
2
1

2
b
n1
= K b
n2
b = K.
Giving that Num(n) = K
n1
. It is interesting that is basically what we had before.
Subsection: 17.3.1: The Question For the Little Bird
Subsection: 17.3.2: Subinstances and Subsolutions
Subsection: 17.3.3: The Set of Subinstances
Subsection: 17.3.4: Decreasing Time and Space
Subsection: 17.3.5: Counting The Number of Solutions
Subsection: 17.3.6: The New Code
End of Chapter Questions
Chapter: 18: Examples of Dynamic Programs
Section: 18.1: The Longest Common Subsequence Problem
Section: 18.2: More of the Input Iterative Loop Invariant
Perspective
Section: 18.3: A Greedy Dynamic Program: The Weighted
Job/Activity Scheduling Problem
Section: 18.4: The Solution Viewed as a Tree: Chains of Matrix
Multiplications
Section: 18.5: Generalizing the Problem Solved: Best AVL Tree
Section: 18.6: All Pairs Shortest Paths with Negative Cycles
Section: 18.7: All Pairs Using Matrix Multiplication
Section: 18.8: Parsing with Context-Free Grammars
Section: 18.9: More Examples
End of Chapter Questions
51. You are very lucky to have a time machine bring you the value each day of m stocks for the next T days.
The input to your problem consists of a table Price(t, s) which gives for each t [1..T] and s [1..m]
the price of buying one share of stock s on day t. Buying stocks costs an overhead of 3%. Hence, if
you buy p dollars worth of s on day t, then you can sell them on day t

for p (1 0.03)
Price(t

,s)
Price(t,s)
.
You have one dollar on day 1 and must sell all your stock on day T. How should you buy and sell to
maximize your prots?
Because you know exactly what the stocks will do, there is no advantage in owning more than one
stock at a time. To make the problem easier, assume that at each point time there is at least one
stock not going down and hence at each point in time you alway own exactly one stock. A solution
will be viewed a list of what you buy and when. More formally, a solution is a sequence of pairs
t
i
, s
i
), meaning that stock s
i
is bought on day t
i
and sold on day t
i+1
. (Here i 1, t
1
= 1 and
t
last+1
= T.) For example, the solution 1, 4) , 10, 8) , 19, 2)) means that on day 1 you put your one
dollar into the 4
th
stock, on day 10 you sell all of this stock and buy the 8
th
stock, on day 19 you
sell and buy the 2
nd
stock, and nally on day T you sell this last stock. The value of this solution is

i
_
(1 0.03)
Price(ti+1,si)
Price(ti,si)
_
= 1 (1 0.03)
Price(10,4)
Price(1,4)
(1 0.03)
Price(19,8)
Price(10,8)
(1 0.03)
Price(T,2)
Price(19,2)
.
Three dynamic programming algorithms occure to me.
44
(a) The most obvious one has mT subinstances (nodes in G) and mT bird answers (degree of nodes
in G) given a time of O(m
2
T
2
).
(b) The second algorithm decreases the number of bird answers (degree of nodes in G) to m given a
time of O(m
2
T).
(c) The third algorithm instead decreases the number of subinstances (nodes in G) to T given a time
of O(mT
2
).
Clearly the rst is the worst. Whether the second or third is best depends on whether there are more
stocks m, or more time steps T. Answers each of the following questions (except for the rst) three
times, one for each of these algorithms.
(a) Suppose at rst that there is not the 3% overhead for selling stock. Give an easy algorithm for
knowing what and when to buy and sell.
Answer: Since there is not cost in buying and selling, we might as well sell and buy every
day. Hence, each day t sell the stock you have and buy the stock s
t
that makes the biggest
prot
Price(t+1,st)
Price(t,st)
between day t and day t + 1.
(b) Read Edmonds notes chapter 19.9 which describes how to design a dynamic program via reduc-
tions. This has not been covered in either class. Follow the steps in this chapter to reduce this
new stock problem to the longest paths problem.
i. Given the table of prices Price(t, s), how do you map this to a graph G? A path is a sequence
of nodes which will correspond to a sequence of states that you might be in during your stock
buying. Hence, each node of G will represent one such state. However, you what as few nodes
as possible. Hence, you must identify a small set of bench mark states that are of interest.
What are these states/nodes? What is the start node s and the nishing node

t in your
graph?
Answer: In the rst and second algorithms, there is a node u
s,t
for each stock s [1..m]
and each day t [1..T]. Being in this state means that on day t you own stock s.
In the third algorithm, **************************** REWRITE
*********************
If you pause your buying-selling sequence after you buy a stock, then your state would
be determined by the current day t
i
and the stock s
i
that you own. However, we choose
to pause after you sell the stock. Hence, your state is determined by the current day. We
will have a node in the graph for each day t [1..T]. Standing on node t means that it
is day t, you have just sold some stock and you are about buy something else. The start
node s is day 1 and the nishing node

t is day T.
Answer: If you pause your buying-selling sequence after you buy a stock, then your state
would be determined by the current day t
i
and the stock s
i
that you own. However, we
choose to pause after you sell the stock. Hence, your state is determined by the current
day. We will have a node in the graph for each day t [1..T]. Standing on node t means
that it is day t, you have just sold some stock and you are about buy something else. The
start node s is day 1 and the nishing node

t is day T.
ii. What are the edges between the nodes?
Answer: The edges out of node t consist of your options. For each stock s [1..m] and
each day t

[t + 1..T] put an edge from node t to node t

.
iii. What is the length of each edge?
Answer: The length of this edge will be log
_
(1 0.03)
Price(t

,s)
Price(t,s)
_
.
iv. Given the longest path from s to

t in your graph G, how does this tell you what and when
to buy stocks?
Answer: If your i
th
edge in the path goes from node t to node t

and buys and sells stock

s then our i
th
item in our solution will be t
i
, s
i
) = t, s).
45
v. The value optCost
path
of the path is the sum of the lengths of the edges in it. The value of
your resulting stock solution is optCost
stock
as dened above. Show the relationship between
optCost
path
and optCost
stock
Answer: log [optCost
stock
] = log
_

i
_
(1 0.03)
Price(ti+1,si)
Price(ti,si)
__
=

i
log
_
(1 0.03)
Price(ti+1,si)
Price(ti,si)
_
= optCost
path
.
(c) Design for a dynamic programming algorithm for this stock buying problem.
i. What are the subinstances? What is problem is each of these subinstances needing to have
solved? How many subinstances are there?
Answer: For each day t [1..T], there is a subinstance that asks what to buy and sell
so that on day t you sell your last stock and maximize the amount of money you have on
that day. There are T such subinstances.
ii. Given you are to solve one of these subinstances, what do you ask the bird and what do you
give your friend? How many bird answers are there?
Answer: If you end on day t

, ask the bird for the last item t

i
, s
i
) in your solution,
namely what stock s
i
[1..m] you buy last and what day t
i
[1..t

1] you should by it.

She answers t, s) You ask your friend to solve the subinstance that ends on day t. The
number of bird answers is mT.
iii. Give the code
Answer:
algorithm Stocks (Price)
precond: An instance consists of the price Price(t, s) of stock s on day t.
postcond: optSol is an optimal valid schedule of what to buy on what day.
begin
% Table: optSol[t] stores an optimal schedule ending on day t and costSol[t] its cost.
table[1..T] optCost, birdAdvice
% Base Case: The only base case is for the optimal set ending on day t = 1.
Its solution consists of the empty set with value 1.
% optSol[1] =
optCost[1] = 1
birdAdvice[1] =
% General Cases: Loop over subinstances in the table.
for t

= 2 to T
% Solve instance t

).
% Try each possible bird answer.
for each t [1..t

1]
for each s [1..m]
% The bird and Friend Alg: I want to nish on day t

. I ask the bird what stock

s
i
[1..m] I should buy last and what day t
i
[1..t

1] I should by it. She answers

t, s) I ask my friend to solve the subinstance that ends on day t.
% optSol
t,s
= optSol[t] +t, s)
optCost
t,s
= optCost[t]
_
(1 0.03)
Price(t

,s)
Price(t,s)
_
end for
% Having the best, optSol
t,s
, for each birds answer t, s), we keep the best of these
best.
t
max
, s
max
) = a t, s) that maximizes optCost
t,s

% optSol[t

] = optSol
tmax,smax
optCost[t

] = optCost
tmax,smax
46
birdAdvice[t

] = t
max
, s
max
)
end for
optSol = SchedulingWithAdvice (Price, birdAdvice)
return optSol, optCost[T])
end algorithm
52. The input to this problem is a pair of strings A = a
1
, . . . , a
n
) and B = b
1
, . . . , b
m
). A solution is
a sequence of operations to convert A into B. The allowed operations are as follows. For a cost of 3
you can delete any letter. For a cost of 4 you can insert a letter in any position. For a cost of 5 you
can replace any letter by any other letter. The goal is to nd a way to convert A into B which is as
cheap as possible. For example, you can convert A = abcabc to B = abacab via the following sequence:
abcabc at a cost of 5 can be converted to abaabc, which at cost of 3 can be converted to ababc, which
at cost of 3 can be converted to abac, which at cost of 4 can be converted to abacb, which at cost of 4
can be converted to abacab. Thus the total cost for this conversion would be 19. This is almost surely
not the cheapest possible conversion.
(a) As a hint to designing the dynamic programming algorithm, think of the process of conversion
from A into B as an iterative process that goes from the left to the right. Each step the little
bird will tell you for free which operation to take. What would the loop invariant of this process
be? What possible steps would be taken to make progress while maintaining the loop invariant?
Answer: The loop invariant of process of converting A into B is that the letters a
1
, . . . , a
i
)
have already been converted into b
1
, . . . , b
j
). Then this conversion is extended by either
deleting the letter a
i+1
giving a conversion from a
1
, . . . , a
i+1
) into b
1
, . . . , b
j
).
inserting the letter b
j+1
giving a conversion from a
1
, . . . , a
i
) into b
1
, . . . , b
j+1
).
changing the letter a
i+1
into b
j+1
giving a conversion from a
1
, . . . , a
i+1
) into
b
1
, . . . , b
j+1
).
if a
i+1
= b
j+1
, then this letter can be left unchanged giving a conversion from
a
1
, . . . , a
i+1
) into b
1
, . . . , b
j+1
).
(b) You are trying to nd an optimal solution for converting A = a
1
, . . . , a
n
) into B = b
1
, . . . , b
m
).
What small question do you ask the little bird?
Answer: We ask the bird Which is that last type of operation in the optimal conversion?
(c) What subinstance do now ask your friend? How do you construct an optimal solution for your
instance out of the optimal solution for his subinstance? How do construct your cost from his?
Answer:
If the bird answers (k = 1) that the last operation was the deletion of the letter a
n
,
then we know that our optimal solution consists of the optimal way of converting from
a
1
, . . . , a
n1
) into b
1
, . . . , b
m
) followed by this last deletion. Hence, we ask our friend
the subinstance of how to optimally convert a
1
, . . . , a
n1
) into b
1
, . . . , b
m
). Our solution
is simply his followed by this last deletion. Our cost is his plus 3.
If the bird answers (k = 2) that the last operation was the insertion of the letter b
m
,
then we know that our optimal solution consists of the optimal way of converting from
a
1
, . . . , a
n
) into b
1
, . . . , b
m1
) followed by this last insertion. Hence, we ask our friend
the subinstance of how to optimally convert a
1
, . . . , a
n
) into b
1
, . . . , b
m1
). Our solution
is simply his followed by this last insertion. Our cost is his plus 4.
If the bird answers (k = 3) that the last operation was the change of the letter a
n
into
b
m
, then we know that our optimal solution consists of the optimal way of converting
from a
1
, . . . , a
n1
) into b
1
, . . . , b
m1
) followed by this last change. Hence, we ask our
friend the subinstance of how to optimally convert a
1
, . . . , a
n1
) into b
1
, . . . , b
m1
). Our
solution is simply his followed by this last conversion. Our cost is his plus 5.
47
If a
n
= b
m
, then we can automatically assume that the bird answers (k = 4) that the
last operation was to leave this letter unchanged. In this case, we know that our optimal
solution consists of the optimal way of converting from a
1
, . . . , a
n1
) into b
1
, . . . , b
m1
)
followed by this last step. Hence, we ask our friend the subinstance of how to optimally
convert a
1
, . . . , a
n1
) into b
1
, . . . , b
m1
). Our solution is simply his followed by leaving
this last letter alone. Our cost is the same as his.
(d) Starting with some input instance A = a
1
, . . . , a
n
) and B = b
1
, . . . , b
m
), what is the set of
instances that might get asked in the recursive back tracking algorithm of some friends friends
friend?
Answer: The set of subinstances will be a
1
, . . . , a
i
) , b
1
, . . . , b
j
)) [ i n, j m.
(e) What is the table indexed by, what is stored in it, and what order should it be lled in?
Answer: The table having one entry for each subinstance which can be indexed by i and j.
The tables will be cost[0..n, 0..m] and birdAdvice[0..n, 0..m]. Generally, the table is lled in
Smallest to largest, but the obvious order of looping for i = 0 to n and j = 0 to m respects
the dependencies between the instances as well.
(f) Give the code for the dynamic programming algorithm. You can use sentences in some places to
replace some blocks of code. (Do not include the code that constructs the solution.)
Answer:
algorithm Convert (a
1
, . . . , a
n
) , b
1
, . . . , b
m
)))
precond: An instance consists of two strings A and B.
postcond: optSol is an optimal way of converting A into B.
begin
table[1..n, 1..m] birdAdvice, cost
% Base Cases. (I am being a bit fancy here to save code.)
cost[1, 1] = 0
for i = 0 to n
cost[i, 1] =
end for
for j = 0 to m
cost[1, j] =
end for
a
0
= 0; b
0
= 0
% Loop over subinstances in the table.
for i = 0 to n
for j = 0 to m
% Fill in entry i, j)
if a
i
= b
j
then
birdAdvice[i, j] = 4 % leave alone
cost[i, j] = cost[i 1, j 1]
else
% Try possible bird answers.
% cases k = 1, 2, 3
% Get help from friend
cost
1
= cost[i 1, j] + 3
cost
2
= cost[i, j 1] + 4
cost
3
= cost[i 1, j 1] + 5
% end cases
% Take the best bird answer.
k
max
= a k [1, 2, 3] that maximizes cost
k

birdAdvice[i, j] = k
max
cost[i, j] = cost
kmax
48
end if
end for
end for
optSol = ConvertWithAdvice (a
1
, . . . , a
n
) , b
1
, . . . , b
m
)) , birdAdvice)
return optSol, cost[n, m])
end algorithm
(g) What is the running time to ll in the table?
Answer: As is usual, the running time is the number of bird answers times the number
of subinstances which is 3 n
2
= (n
2
).
(h) Given the table storing the cost of the optimal solutions and the table of birds answers, how do
you construct the solution for the instance A = a
1
, . . . , a
n
) and B = b
1
, . . . , b
m
).
Answer: See notes how use the birds advice to either iteratively or recursively follow one
path down the recursive back tracking algorithm.
53. Consider an arbitrary dynamic programming algorithm that given an instance returns one of the
possibly many optimal solutions.
(a) Suppose that in the bird and friends version of the dynamic programming algorithm, when given
an instance of size n, the bird is asked a question with K possible answers and then one friend is
asked to solve a subinstance that is never any bigger than size n 1. Briey describe what needs
to changed so that in addition to outputting one of the optimal solutions to the given instance,
the dynamic program also outputs the number of possible optimal solutions. Your change should
not signicantly change the running time.
Answer: If I want to count the number of fruit in a bowl, I can ask one friend to count for
me the number of red fruit and another the number of green fruit and another the number
of orange. My answer is the sum of these three. If all the orange fruit is rotten, then I might
not include the number of orange fruits in my sum. One complication is if there are some
fruits that are half red and half green. These might get double counted. I want to make sure
that my question colour disjointly partitions the fruit according to the answer given.
To be sure that we do not double count some optimal solutions we need that the set of
solutions consistent with one bird answer is disjoint from those consistent with another bird
answer. For example, if the birds answer tells us the last edge of the optimal path, then
those paths ending in the rst edge are clearly dierent than the paths ending in the second.
On the other hand, for the longest common subsequence problem, the birds answers were
the last letter of X is excluded and the last letter of Y is excluded. There are solutions
for which both are true. This causes a problem when counting solutions. Hence, we need
to change the dynamic programming algorithm so that each solution only has one valid bird
answer. For example, with the longest common subsequence problem, we could change the
birds answers to being either the last letter of X is excluded or the last letter of X is
included.
In the new dynamic programming algorithm, each friend, in addition to the cost of the optimal
solution for his subinstance and the birds advice, stores the number of optimal solutions for
the subinstance. This is computed as follow. Consider me to be one of these friends with one
of these SubInstances. I try each of the K possible birds answers. When trying k, I nd the
best solution to my instance from amongst its solutions that are consistent with this birds
answer k. I do this by asking a friend of mine to solve some subinstance. This friend tells me
the number of optimal solutions to this subinstance. Let num
k
be this number. Generally,
there is a one-to-one mapping between my friends optimal solutions and solutions to my
instance that are consistent with this birds answer k. Hence, I know that there are num
k
of
these. As before let optCost
k
be the cost of the best solution to my instance from amongst
its solutions that are consistent with this birds answer k. I compute the cost of my optimal
solution simply by optCost = max
k[K]
optCost
k
. There may be a number of birds answers
49
that lead to this same optimal cost optCost. Each of these lead to optimal solutions. We
were careful that the set of solutions consistent with one bird answer is disjoint from that
consistent with another bird answer. Hence, the total number of optimal solutions to my
instance is num =

k{k | optCost
k
=optCost}
num
k
.
(b) Let Num(n) denote the maximum number of possible optimal solutions that any dynamic program
as described above might output given an instance of size n. Give and solve the recurrence relation
for Num(n).
Answer: num =

k{k | optCost
k
=optCost}
num
k
. The maximum this can be is Num(n) =

k[1..K]
Num(n 1) = K Num(n 1) = K
n
.
(c) Suppose that there were this maximum number of solutions. What is the minimal number of bits
needed to describe each with a unique string? How would you describe a solution as a sequence
of birds answers? How does the length of this description compare to the minimum length?
Answer: If you have N objects, one way to distinguish them is to allocate to each a number
from one to N. This description requires log
2
(N) bits. It can be proved that no description
can be shorter than this. Given that there may be N = K
n
solutions, they need a description
of length at least log
2
(N) = log
2
(K
n
) = log
2
(K) n. One way of identifying a solution is
by listing the sequence of answers the bird provides in a 20-questions fashion to you, to your
friend, to his friend, and so on. Given that there are N

= K dierent bird answers, they

need a description of length at least log
2
(N

) = log
2
(K) = log
2
(K). If each subinstance is
one shorter than the instance, then they will have lengths n, n 1, n 2, . . . , 1 for a total
of n questions to the little bird. Therefore, it takes log
2
(K) n to list all the birds answers
k
1
, . . . , k
n
).
(d) Why can the program in polynomial time output the number of optimal solutions but not be able
to output all of these optimal solutions?
Answer: There may be K
n
of them. Hence, it takes exponential time to output them all.
However, the number of bits to represent this number is only log
2
(K
n
) = log
2
(K)n = (n).
Hence, this number can be outputted.
(e) Now suppose that in the bird and friends version of the dynamic programming algorithm, when
given an instance of size n, the bird is asked a question with K possible answers and then two
friend each are asked to solve a subinstance that is never any bigger than size
n
2
. (See Chains
of Matrix Multiplications). Again briey describe what needs to changed so that the dynamic
program also outputs the number of optimal solutions.
Answer: Given a birds answer k, I ask two friends to solve a subinstance and then take
these two solutions to produce my solutions. Let num
k,1
be the number of dierent optimal
solutions that the rst of my friends might give me and num
k,2
be the number that the second
might give. There are then num
k
= num
k,1
num
k,2
ways for me to combine one solution
from the rst and one solutions from the second to give me an optimal solution consistent
with this birds answer k. Generally, these are all distinct solutions. Again the number of
optimal solutions to my problem is num =

k{k | optCost
k
=optCost}
num
k
.
(f) Now let Num(n) denote the maximum number of optimal solutions that any dynamic program
as described in the last question might output given an instance of size n. Give the recurrence
relation for Num(n). In order to solve it, assume that Num(n) = b
n1
for some constant b and
solve for b.
Answer: num =

k{k | optCost
k
=optCost}
num
k,1
num
k,2
. The maximum this can be is
Num(n) =

k[1..K]
Num
_
n
2
_
Num
_
n
2
_
= K Num(
n
2
)
2
. Plugging in Num(n) = b
n1
(and Num
_
n
2
_
= b
n
2
1
) gives that
Num(n) = K Num
_
n
2
_
2
b
n1
= K
_
b
n
2
1

2
50
b
n1
= K b
n2
b = K.
Giving that Num(n) = K
n1
. It is interesting that is basically what we had before.
54. (22 marks) Dynamic Programming:
Consider an encoding of characters into binary strings in which dier-
ent characters may be encoded into strings of dierent length. We may
describe such an encoding by a table with two columns, where each char-
acter on the left is encoded by the bit string given on the right. See the
example on the right.
Example Decoding Table
Character Encoding
A 0
B 10
C 01
D 11
Using this table the encoding of ACDC is 0011101 (which comes from 0 01 11 01 , but the decoder
is not given the separation between the code-words). Notice that there may potentially be several
decodings of a given bit string. For instance, 001110 can be decoded either as AADB or as ACDA.
Your task is to devise an ecient algorithm which, given an encoded string a
1
, . . . , a
n
) 0, 1
n
and a decoding table, decodes the string into one of its possible string of characters. For example, if
a
1
, . . . , a
n
) = 001110 and the DecodingTable is as above, then optSol is either AADB or ACDA.
(a) (2 marks) Give all ways to decode 0100.
Answer: ABA = 0 10 0, and CAA = 01 0 0.
(b) (2 marks) We say that the question asked of the little bird must be small. What is the denition
here of small? Give an example of a big question. Give an example of a big question.
Answer: The number of dierent answers that she might give. What is the optimal solu-
tion? is big. What is the rst edge? is small.
(c) (2 marks) When we ask a (recursive) friend a question, what are the requirements on this question.
Answer: Meets the preconditions to the same problem and is smaller.
(d) (5 marks) Suppose that you have a trusting bird to whom you can ask a small question and you
have a trusting friend to whom you can ask a recursive question. Give a fast algorithm for this
problem. How do you construct your solution?
Answer: The algorithm is almost identical to Pretty Print. The question to the bird with
PP is How many words do I put on the last line? With Decoding, it is How many bits of
the input do I use in the code for the last letter? With an answer k, the you take the last k
bits of your input and look them up in the decoding table. If no letter corresponds to them,
then you tell the bird that she is wrong. If say letter C corresponds to them, then delete
these k bits from your input and give the rest to your friend. He decodes this. Our solution
is the friends decoded message concatenated with the birds letter C.
(e) (2 marks) Do we have a bird that we can trust? If so why can we trust her? If not how do we
change the algorithm to deal with this? What are we doing each step? How do we conclude?
Answer: We cannot trust the bird. Hence, we try each bird answer. For each, we nd the
best solution consistent with this birds answer. Then we take the best of these best.
(f) (2 marks) Do we have a friend that we can trust? If so why can we trust him? If not how do we
change the algorithm to deal with this?
(g) (2 marks) Why can you trust the friend to give the correct answer?
Answer: He is a version of ourselves, that can be given a smaller instance of the same
problem. We can trust him because strong induction tells us that we can.
(h) (2 marks) Starting with some input instance a
1
, a
2
, . . . , a
n
), what is the set of instances that
might get asked in the recursive back tracking algorithm of some friends friends friend . . .?
Answer: S = a
1
, a
2
, . . . , a
i
) [ i [1..n].
51
(i) (3 marks) In a general dynamic program, what is the table indexed by? In a fast dynamic program,
what does it store? Why do we not store the obvious thing?
Answer: Indexed by the subinstances. Stores the birds advice and the cost of the optimal
solution. We do not store the entire optimal solution because it likely has size (n) and
passing it from friend to friend can add an extra factor of (n) to the running time.
(j) (2 marks) Does a dynamic program recurse? Does it have friends? Explain.
Answer: It has a friend for each subinstance, but it does not recurse. When we ask a friend,
he looks his answer up in the table that he had previously computed.
(k) (6 marks) Give the code for the dynamic programming algorithm. You can use sentences in some
places to replace some blocks of code. (Do not include the code that constructs the solution.)
algorithm Decode (a
1
, . . . , a
n
) , DecodingTable)
precond: An instance consists of a coded message a and a DecodingTable.
postcond: optSol is a decoded messages.
For example, if a
1
, . . . , a
n
) = 001110 and the DecodingTable is as above, then optSol is
either AADB or ACDA.
The program also returns the cost of the solution, which is 1 if it is a valid decoding
and 0 if it is not.
begin
Answer:
table[0..n] birdAdvice, cost
% Base Cases.
birdAdvice[0] =
cost[0] = 1
% Loop over subinstances in the table.
for i = 1 to n
% Fill in entry i) for a
1
, a
2
, . . . , a
i
).
% Try possible bird answers, k = number of bits in the last letter.
for k = 1 to K K is the length of the longest code for a letter
if( a
ik+1
, a
ik+2
, . . . , a
i
) is the code for a letter C ) then
% Get help from friend
% subSol
k
= subSol[i m
k
] +C
cost
k
= cost[i m
k
] % i.e. ours is valid if our friends is valid.
else
cost
k
= 0 % Bird was wrong.
end if
end for
% Take the best bird answer.
k
max
= a k that maximizes cost
k

birdAdvice[i] = k
max
% subSol[i] = subSol
kmax
cost[i] = cost
kmax
end for
optSol = DecodeWithAdvice (a
1
, . . . , a
n
) , code, birdAdvice)
return optSol, cost[n])
end algorithm
(l) (2 marks) What is the running time to ll in the table?
Answer: As is usual, the running time is the number of bird answers times the number
of subinstances which is (Dn).
52
55. Consider the recurrence relation T(0) = T(1) = 2 and for n > 1 T(n) =

n1
i=1
T(i) T(i 1). We
consider the problem of computing T(n) from n.
(a) Show that if you implement this recursion directly in say the C programming language, that the
program would use exponentially, in n many arithmetic operations.
Answer: I have (n 1) 2 friends. Most of them have (n) friends. Most of them have
(n) friends. Hence, the total number of friends will be n
(n)
.
(b) Explain how to use dynamic programming that only uses O(n
2
) arithmetic operations for this
problem.
Answer: There are only n subinstances to solve, namely T(n

) for n

[0, n]. A dynamic

programming solves these in the order n

= 0..n so that nobody has to wait and stores the

solutions in a table so that they do not need to be recomputed. The straight code then loops
through each T(n

) and for each loops through i [1, n

1] for a total of (n
2
) time.
(c) Give an algorithm for this problem that only uses O(n) arithmetic operations.
Answer: T(n) = T(n1)T(n2)+

n2
i=1
T(i)T(i1) = T(n1)T(n2)+T(n1) =
T(n 1) (T(n 2) + 1). Computing each T(n

) using this algorithm from the solutions in

the table requires only constant time per n

.
(d) Which of these approximates T(n) the closest?
n
(n)
2
(n)
2
2
(n)
Answer: As said, T(n) T(n 1)
2
. Unwinding this gives,
T(n)
_
T(n 2)
2

2
T(n 2)
(2
2
)

_
T(n 3]
2

(2
2
)
T(n 3)
(2
3
)
... T(n i)
(2
i
)

T(0)
(2
n
)
2
2
(n)
56. Dynamic Programming: (See greedy version of this.) The box question in the last assignment question
proved that the three greedy algorithms given do not correctly choose which boxes to take. This
motivates trying dynamic programming. We will generalize the problem further by having values on
the boxes.
You are robbing a warehouse lled with sealed boxes B
i
for i [n]. Each box is specied by w
i
, c
i
, v
i
),
where w
i
is how heavy the box is, c
i
is its capacity in terms of how much weight can sit on top of it,
and v
i
is how much money the box is worth. You are able to put one stack of boxes on the truck.
A solution species the set S of boxes to take and the order to take them. A solution is valid if
i S,

j above i)
w
j
c
i
. The value of a solution is

iS
v
i
. The goal is to maximize this value.
Hint 1: You might want to change the precondition so that an instance also provides a value c which
is the maximum weight that can go onto the truck.
Hint 2: As motivated by the last question, you might as well assume that all the boxes are ordered
by w
i
+c
i
. Then for any subset S that you take, they are already in the correct order.
Hint 3: Instead of asking the little bird about the last box, ask her whether the rst box, i.e. the
one with the largest w
i
+c
i
, should be included in the solution.
Design a dynamic programming algorithm for this box stacking problem.
(a) For each bird answer, what subinstance do you ask the friend? How do you use this for your
answer?
Answer: Suppose that our instance species a capacity c of the truck and a set of remaining
boxes that could go on this truck. We sort these remaining boxes by w
i
+c
i
and ask the bird
whether the rst box in this sequence should go on the truck. If she says take this rst box
B
i
, then remove it from the set of boxes and put it on the truck. All the other boxes stay
unchanged. What changes is the capacity c of the truck. If w
i
> c, then the bird was wrong.
53
Otherwise, the truck now has remaining capacity c w
i
and box b
i
, which will be beneath
any remaining boxes, has capacity c
i
. Hence, the eective capacity is the minimum of these
two. The instance that we give our friend is that with truck capacity min(c w
i
, c
i
) and the
same boxes except for this rst one removed. Our answer is the same plus adding box B
i
and
its value v
i
back in. If she says no, simply delete box B
i
. Our answer is the same.
The recurrence relation is as follows. If w
i
c, then
Cost[c, i] = Max(Cost[min(c w
i
, c
i
), i + 1] +v
i
, Cost[c, i + 1])
If w
i
> c, then
Cost[c, i] = Cost[c, i + 1]
(b) What are the subinstances? What is the problem that must be solved for each of these subin-
stances? Which subinstance is our original instance?
Answer: For each i [n] and each c [c
max
], the subinstance c, i) asks which of the boxes
B
i
, B
i+1
, . . . , B
n
to take on a truck that can hold weight c. Here c
max
=

i
w
i
. Our instance
is n, c
max
) because we can consider all n boxes and the truck need not hold more than the
weight of all the boxes.
(c) Provide the pseudo code for your algorithm.
Answer:
algorithm StackBoxes (w
1
, c
1
, v
1
) , . . . , w
n
, c
n
, v
n
))
precond: n boxes. w
i
is how heavy the box is, c
i
is its capacity in terms of how much
weight can sit on top of it, and v
i
is how much money the box is worth.
postcond: optSol is an optimal way to stack some of the boxes so as to maximize the
total value of the boxes taken. optCost is its price.
begin
c
max
=

i
w
i
Sort boxes by w
i
+c
i
% Table: optSol[c, i] would store an optimal solution given a truck with capacity c and when
considering only the last i boxes,
but actually we store only the birds advice for the subinstance and the cost of its solution.
table[0..c
max
, 1..n + 1] birdAdvice, cost
% Base Cases: The base cases are when the number of boxes is zero.
For each, the solution is the empty truck with cost zero.
loop c = 0..c
max
% optSol[c, n + 1] =
cost[c, n + 1] = 0
birdAdvice[c, n + 1] =
end loop
% General Cases: Loop over subinstances in the table.
loop i = n to 1
loop c = 0 to c
max
% Solve instance c, i) and ll in the table.
% The bird and Friend Alg: The bird tells us either (1) exclude the i
th
item from
the truck or (2) include it. Either way, we remove this last object, but in case (2)
we decrease the capacity of the truck to min(c
i
, c w
i
). Then we ask the friend
for an optimal packing of the resulting subinstance. He gives us (1) optSol[c, i+1]
or (2) optSol[min(c
i
, c w
i
), i +1] which he had stored in the table. If the bird
had said we were to include the i
th
item, then we add this item to the friends
solution. This gives us optSol
k
which is a best stacking for c, i) from amongst
those consistent with the birds answer.
54
% Try each possible bird answers.
% cases k = 1, 2 where 1=exclude 2=include
% optSol
1
= optSol[c, i + 1]
cost
1
= cost[c, i + 1]
if(c w
i
0) then
% optSol
2
= optSol[min(c
i
, c w
i
), i + 1] i
cost
2
= cost[min(c
i
, c w
i
), i + 1] +v
i
else
% Bird was wrong
% optSol
2
=?
cost
2
=
end if
% end cases
% Having the best, optSol
k
, for each birds answer k,
we keep the best of these best.
k
max
= a k that maximizes cost
k

% optSol[c, i] = optSol
kmax
cost[c, i] = cost
kmax
birdAdvice[c, i] = k
max
end for
end for
optSol = StackBoxesWithAdvice (w
1
, c
1
, v
1
) , . . . , w
n
, c
n
, v
n
) , birdAdvice)
return optSol, cost[c
max
, 1])
end algorithm
(d) What is its running time?
Answer: # subinstances # bird answers = n c
max
2.
(e) What time complexity of this algorithm?
Answer: Exponential, because c
max
may be exponential in the number of bits to encode the
instance.
57. (30 marks) Dynamic Programming
You are given a sequence s of characters and your goal is to nd a longest subsequence p of s that
is a palindrome. A palindrome is a string which is the same as its reverse. For example, if s =
PROFESSOR, the optimal solution is p = ROSSOR, because ROSSOR is the longest string that is
both a subsequence of PROFESSOR (PROFESSOR), and a palindrome (the reverse of p is p itself).
Your goal is to write a dynamic programming algorithm to solve this problem. (Caution: do not try
to use the LCS algorithm on s and its reverse. It does not work.)
In the following, we will let s
i
denote the i
th
character of s, and s
ij
denote the (consecutive) substring
of s starting at character i and ending at character j, including all characters in between. For example,
if s = PROFESSOR, then s
5
=E and s
4,7
= FESS.
(a) If (the bird) were to tell you that the last character of s is not in the optimal solution p, how
would you use this information to solve the problem.
Answer: I would give my friend s with this last character removed. My answer would be the
same as his.
(b) If (the bird) were to tell you that both the rst and the last characters of s are in the optimal
solution p, how would you use this information to solve the problem.
Answer: If these characters are dierent, then I would tell her that she was wrong. If they
are the same, I would give my friend s with both of them removed. My answer would be the
same as his with the rst and last added back.
(c) What is a third answer that (the bird) might give so that all possiblities are covered?
55
Answer: The rst character of s is not in the optimal solution p.
(d) (6 marks) Consider the rst and last characters of the input string s, and whether they are part
of a longest palindromic subsequence p of s. What are the three possibilities?
Answer:
i. s
1
= s
n
and both are part of a longest palindromic subsequence of s.
ii. s
1
,= s
n
, and there is a longest palindromic subsequence of s that does not include s
1
.
iii. s
1
,= s
n
, and there is a longest palindromic subsequence of s that does not include s
n
.
(e) (3 marks) How would you describe the set of subinstances that must be solved in order to solve
the original instance? If the original instance is of length n, how many of these subinstances are
there?
Answer: The set of s
ij
, 1 i j n. There are
n

i=1
(ni +1) =
n

i=1
i =
1
2
n(n+1) of these.
(f) (8 marks) Let A(i, j) be the length of a longest palindromic subsequence of s
ij
. Use the three
possibilities identied in Part (a) to dene a recurrence relation for A(i, j). Be sure to include
the base cases.
Answer:
A(i, j) =
_

_
0 i > j
1 i = j
2 +A(i + 1, j 1) i < j and s
i
= s
j
maxA(i + 1, j), A(i, j 1) otherwise
(g) (2 marks) In what order must the table for A(i, j) be lled so that the answers to subinstances
are always available when needed?
Answer: Only elements on or above the main diagonal must be lled. These must be entered
so that elements below and to the left of the current element are available when the current
element is processed. One valid method is to ll the table diagonally, starting with j = i, and
progressing through j = i + 1, j = i + 2...j = i + n 1. Another method, slightly easier to
code, is to ll the upper triangle row-wise, starting from the bottom row.
(h) (4 marks) Fill in the table A(i, j) for the input instance s =PROFESSOR. (Pencil recommended.)
ij P R O F E S S O R
P
R
O
F
E
S
S
O
R
Answer:
56
ij P R O F E S S O R
P 1 1 1 1 1 1 2 4 6
R 0 1 1 1 1 1 2 4 6
O 0 0 1 1 1 1 2 4 4
F 0 0 0 1 1 1 2 2 2
E 0 0 0 0 1 1 2 2 2
S 0 0 0 0 0 1 2 2 2
S 0 0 0 0 0 0 1 1 1
O 0 0 0 0 0 0 0 1 1
R 0 0 0 0 0 0 0 0 1
(i) Provide the pseudocode for a dynamic programming algorithm that lls in table A(i, j).
algorithm Palindrome(s)
precond: s is a string of n characters
postcond: A(i, j) is a table of the length of the longest palindromic subsequence of s
ij
.
begin
for i = 2 to n
A(i, i 1) = 0
end
for i = 1 to n
A(i, i) = 1
end
for k = 1 to n 1
for i = 1 to n k
j=i+k
if s
i
= s
j
then A(i, j) = A(i + 1, j 1) + 2
else A(i, j) = max(A(i + 1, j), A(i, j 1))
end
end
end algorithm
(j) What is the running time of your algorithm?
Answer: (n
2
)
(k) (7 marks) Provide the pseudocode for a recursive algorithm that recovers an actual longest palin-
dromic subsequence of an input string s, given s and the lled-in table A(i, j) for s.
Answer: To print out one possible longest palindrome given A we call PrintOpt(A, 1, n),
where PrintOpt is:
PrintOpt(A, i, j)
if i > j then return end if
if i = j then
print s
i
return
end if
if s
i
= s
j
then
print s
i
PrintOpt(i+1,j-1)
print s
j
else if A(i + 1, j) A(i, j 1) then
PrintOpt(i+1,j)
else
PrintOpt(i,j-1)
end if
57
58. Dynamic programming
The SubsetSum problem is dened as follows: given positive integers a
1
..a
n
, and a positive integer t,
nd a subset S of (1 . . . n) such that
iS
a
i
= t, if it exists.
(a) Design a dynamic programming algorithm to solve the SubsetSum decision problem. In other
words, your algorithm will report whether a solution exists, but not what it is. You may use the
steps and language set out by James, those set out by Je, or some combination. Explain in point
form English the key table (array) used, the recurrence relation used to ll it in (as in recursive
backtracking, for those in Jes class), and the order in which it is lled. Also provide concise
pseudocode for your algorithm.
Answer:
Array:
A[i, ] is a 2D array over i [0, ..., n], [0, ..., t], specifying the maximum

jS
a
j
such
that

jS
a
j
and j ij S. An S such that
iS
a
i
= t exists i A[n, t] = t.
Recurrence:
A[0, ] = A[i, 0] = 0 [0, ..., t], i [0, ..., n].
A[i, ] = A[i 1, ] if a
i
>
= maxA[i 1, ], a
i
+A[i 1, a
i
] if a
i

Pseudocode:
A[0, ] = A[i, 0] = 0 [0, ..., t], i [0, ..., n]
for i = 0 : n
for = 0 : t
if a
i

A[i, ] = maxA[i 1, ], a
i
+A[i 1, a
i
]
else
A[i, ] = A[i 1, ]
end
end
end
(b) Which problem studied in class is SubsetSum the most similar to? How?
Answer: Knapsack: the a
i
map to the weights of the items, t maps to the capacity of the
knapsack. Instead of maximizing value, here our goal is to get as close to t as possible, which
corresponds to lling the knapsack as close to full as possible.
(c) In one or two sentences explain why the main dynamic programming algorithm does not construct
the actual solution and how the solution is constructed at the end.
Answer: The dynamic programming algorithm constructs a table of sums. The value in
A(n, t) tells us whether a sum totalling t exists, but it does not directly tell us what combi-
nation of the a
i
total that sum.
Just as for the knapsack problem, we can traverse the table to recover the subset. We start
by scanning the last column of the table from the (n, t) entry. A change from entry (i, t) to
entry (i 1, t) indicates that a
i
is included in the solution, and a
i
is thus added to S. We
then shift our scan point left in the table by a
i
and resume our scan from (i 1, t a
i
). This
process iterates until we hit a 0 entry.
Answer:
PrintOpt(i, t)
if i = 0 return
58
if A[i, t] = A[i 1, t]
Printopt (i 1, t)
Else
Printopt(i 1, t a
i
)
Print a
i
(d) Running Time:
i. What is the running time of the main dynamic programming algorithm as a function of n
and t? Why?
Answer: Initialization: (n) + (t).
Construction of A : (nt).
Total Running Time: (nt).
ii. What is the running time of the main dynamic programming algorithm as a function of the
size of the input? Why?
Answer:
Input size is proportional to n+log t. Thus running time is exponential in the input size.
iii. What is the running time of the algorithm that constructs the solution as a function of n and
t? Why?
Answer: Recovery of S : (n), since we traverse A one row at a time, from row n to (at
worst) row 0.
Chapter: 19: Reductions and NP-Completeness
Section: 19.1: Satisability Is At Least As Hard As Any Opti-
mization Problem
Section: 19.2: Steps to Prove NP-Completeness
Section: 19.3: Example: 3-Colouring is NP-Complete
Section: 19.4: An Algorithm for Bipartite Matching using the
Network Flow Algorithm
59. (15 marks) Reductions: The optimization problem P
alg
is specied by dening its instances I
alg
, its
solutions S
alg
for each such instance, and the cost cost(S
alg
) of each such solution. Given an instance
I
alg
the goal is to nd an optimal solution S
alg
. Similarly, the goal of the optimization problem P
oracle
,
given an instance I
oracle
, is to nd an optimal solution S
oracle
. In a reduction, you design a fast
algorithm Alg
alg
for P
alg
using an assumed fast algorithm Alg
oracle
for P
oracle
. In the three main
reductions done this term, the problem pairs [P
alg
to P
oracle
] were [Boy & Girls Marriage to Network
Flows], [Software to Min Cut], and [Circuit Satisability to 3-Colouring].
(a) (4 marks) Outline the basic code for Alg
alg
solving P
alg
using the supposed algorithm Alg
oracle
supposedly solving P
oracle
as a subroutine.
Answer:
algorithm Alg
alg
(I
alg
)
precond: I
alg
is an instance of P
alg
.
postcond: Determine whether I
alg
has a solution S
alg
and if so returns it.
begin
I
oracle
= InstanceMap(I
alg
)
ans
oracle
, S
oracle
) = Alg
oracle
(I
oracle
)
if( ans
oracle
= Y es) then
ans
alg
= Y es
S
alg
= SolutionMap(S
oracle
)
else
59
ans
alg
= No
S
alg
= nil
end if
return(ans
alg
, S
alg
))
end algorithm
(b) (3 marks) What steps do you have to take to prove that this reduction is correct?
Answer: We have to prove that Alg
alg
works if Alg
oracle
works. For this we prove
i. Given an instance I
alg
, InstanceMap(I
alg
) maps this to a valid instance I
oracle
.
ii. If S
oracle
is a solution for I
oracle
than S
alg
= SolutionMap(S
oracle
) is a solution for I
alg
whose cost is just as good.
iii. If S
alg
is a solution for I
alg
than S
oracle
= ReverseSolutionMap(S
alg
) is a solution for
I
oracle
whose cost is just as good.
(c) (4 marks) Give two purposes of reductions.
i.
ii.
Answer:
i. Designing new algorithms
ii. Arguing that a problem is hard or easy.
iii. identifying equivalence classes of problems.
(d) (2 marks) What does it mean for a problem to be in NP (Non-Deterministic Polynomial Time)?
Answer: Given an instance and a solution, it is easy to test its validity.
(e) (2 marks) What does it mean for a problem to be NP-Complete.
Answer: It is as hard as any problem in NP. (and is in NP)
60. Reductions
(a) There are two opposite purposes for reductions.
As an example of the rst purpose, we designed an agorithm for the Elephant problem (Problem
A) using an algorithm for the Longest Weighted Path Problem (Problem B).
As an example of the second purpose, we designed an algorithm for the Circuit Sat Problem
(Problem A) using an algorithm for the 3-Colouring Problem (Problem B).
State and contrast these two purposes. Be sure to be clear about what this reduction says about
Problem A and about Problem B.
Answer: The rst provided a new algorithm for problem A using a known algorithm for
problem B. The second argued that there is likely not a poly-time algorithm problem B
because there is likely not one for problem A.
(b) What are the formal requirements for a problem to be NP-Complete?
Answer: Given an instance and a potential solution, there is a poly time algorithm that
checks if it is a valid solution
61. (8 marks) Reductions and NP-Completeness
(a) (2 marks) In one sentence, state what a reduction is.
Answer: Using a supposed algorithm for one problem as a subroutine for an algorithm for
another problem.
(b) (2 marks) Give two purposes of reductions.
i.
ii.
60
Answer:
i. Designing new algorithms
ii. Arguing that a problem is hard or easy.
iii. identifying equivalence classes of problems.
(c) (2 marks) What does it mean for a problem to be in NP (Non-Deterministic Polynomial Time)?
Answer: Given an instance and a solution, it is easy to test its validity.
(d) (2 marks) What does it mean for a problem to be NP-Complete.
Answer: It is as hard as any problem in NP. (and is in NP)
62. Greenhouses and plants
A botanical garden is planning to build n greenhouses, each of which is intended to represent a dierent
climate zone. They have a list of m kinds of plants that they would like to grow there. Not every kind
of plant can grow in every greenhouse, though usually one kind can grow in more than one climate; so
associated with every greenhouse i [1 . . . n] is a list C
i
of plant varieties it can grow. In addition to
that, every greenhouse can host no more than 20 varieties of plants.
(a) Given the n, m and C
i
for 1 i n, specify an algorithm to determine the maximal number of
dierent varieties of plants that can be grown in these greenhouses (total over all greenhouses).
Hint: design a ow network that represents this problem.
Answer: This problem is similar to a bipartite matching problem. We need to construct a ow
network representing a matching of plants to greenhouses. The ow network T = (G, s, t, c)
for this problem will consist of the following:
The vertices of G will be s, t for source and sink, vertices
g
1
, . . . , g
n
representing greenhouses and vertices v
1
, . . . , v
m
representing varieties of plants.
Edges (s, g
i
) of capacity 20 for 1 i n. This will con-
strain every greenhouse to have at most 20 varieties of
plants.
Varieties
t s 20
1
1
1
1
1
1
1
1
1
1
1
1
1
1
Greenhouses
20
20
v
3
v
1
v
2
v
2
g
1
g
4
m
v
n
g
Edges (g
i
, v
j
) of capacity 1, 1 i n, 1 j m for pairs (i, j) such that j C
i
. These
edges will match varieties of plants with greenhouses where they can grow. Since we wish
to maximize the number of varieties of plants, we will not allow more than one plant of
the same variety to grow in a greenhouse.
Edges (v
j
, t) of capacity 1, where 1 j m. Restricting the capacity to 1 ensures that
every variety will be grown in at most one greenhouse.
To nd the maximal number of varieties of plants that can be grown in these greenhouses,
we run the Ford-Fulkerson algorithm on this network. The value of the resulting ow is the
maximal number of varieties that can be grown. Every greenhouse hosts no more than 20
varieties of plants due to the capacity constraint on (s, g
i
) edges, and these varieties can be
grown there because of the choice of (g
i
, v
j
) edges; therefore, this set of varieties can feasibly
61
be grown. To show that it is maximal we appeal to the fact that Ford-Fulkerson always
produces the maximal ow. Suppose that there is a better assignment of varieties to plants,
allowing for more varieties to be grown. We argue that every pair (i, j) where variety j is
assigned to be grown in the greenhouse i corresponds to an augmenting path in the network;
the path (s, g
i
, v
j
, t). Since no variety is chosen more than once, and no greenhouse hosts more
than 20 varieties, each pair (i, j) gives rise to an augmenting path satisfying the constraints.
Therefore, there would be a ow on the network with value greater than the value of the ow
produced by Ford-Fulkerson; a contradiction. Thus, the value of the ow on this network is
equal to the maximum number of varieties of plants that can be grown.
62
(b) How can you produce a list of chosen varieties for each greenhouse given the maximum ow on
your ow network? Is it uniquely determined by the original network?
Answer: Take the resulting ow on the network, and consider a cut between (s, g
1
, . . . , g
n
)
and (v
1
, . . . , v
m
, t). Since every cut has the same ow through it, and by the integrality
theorem if the capacities are integers then there is an integer-valued ow, there are exactly
[f[ edges in that cut that have ow of 1, and the rest have ow of 0. Now the list of chosen
varieties for a greenhouse i is the list of all j such there is an edge (g
i
, v
j
) and it has ow of
1.
The ow is not uniquely determined by the network, although its value is; the ow depends on
the order in which augmenting paths are chosen. For example, suppose there is one greenhouse
and 21 varieties of plants that can all grow in it. Then the resulting network will have ow
of 20, and 20 out of these 21 varieties will be chosen, but it is up to the implementation of
Ford-Fulkerson to determine which subset of 20 varieties will be chosen.
End of Chapter Questions
63. Reductions
(a) What are the formal requirements for a problem to be in the class Non-Deterministic Polynomial
Time?
Answer: Given an instance and a potential solution, there is a poly time algorithm that
checks if it is a valid solution
(b) What does it mean to say that Problem A reduces to Problem B? What does it say about their
relative diculties?
Answer: There exists a polynomial time algorithm to map an instance of Problem A to an
instance of Problem B. Problem A can be no harder than Problem B.
64. Wrestlers
There is a group of wrestlers that must be separated into some number of teams for a competition.
There are rivalries between some pairs of wrestlers. The goal is to split the wrestlers into teams so that
there are no rivalries between any two members of the same team (or report that it is impossible).
(a) One of the keys to understanding algorithms is to be able to identify when two problems that
look supercially dierent are the same at their core. The wrestling problem is in fact identical
to another problem you have seen in the very recent past. What is this problem? Explain how
you could transform the wrestler problem to this other problem to obtain a solution.
Answer: This is the 2-colouring problem. Map wrestlers to nodes and rivalries to edges.
The 2 colours correspond to the 2 teams.
(b) Suppose you are required to split the teams into exactly k groups. For what values of k will this
be feasible? Please justify your answer.
Answer: Since this is just 2-colouring, the problem is feasible for k = 2 and infeasible
(NP-Complete) for k 3.
(c) Suppose that you have an algorithm to split the wrestlers into two teams that minimizes the
within-team rivalries. Design an algorithm to set up the one-on-one matches for the competition.
Each match pits a member of the rst team against a member of the second team. Each wrestler
can ght in only one match. Your goal is to maximize the number of matches between rivals.
Describe your algorithm in point-form English. Draw a simple gure to illustrate the computation.
Answer: Construct the ow network below and run the Ford-Fulkerson algorithm to compute
a max ow network. Pair o wrestlers connected by non-zero ow, and pair o the remaining
wrestlers arbitrarily.
63
65. Do question 1 in Micahs problem set 4 from 2003. For 1b, try a bird-friend algorithm. This is a Cook
reduction in that it calls the decision algorithm many times. You dont need to worry about the 12
steps.
www.cs.yorku.ca/ je/courses/6121/micah.ass/ps4.ps
Answer:
(a) Suppose that P = NP. Hence, there is some poly-time algorithm Alg
SAT
for SAT. Consider
some problem P
harder
NP. Our goal is to prove that P
harder
is NP-Complete. We already
know that P
harder
NP and that SAT is NP-Complete. Hence, it is sucient to prove that
SAT P
harder
, namely that if we have a poly-time alg for P
harder
, then we have one for
SAT. We this is true. Our poly-time algorithm for SAT, simply ignores our alg for P
harder
and run Alg
SAT
.
(b) Given an instance S
1
, . . . , S
n
, t), we ask the little bird whether S
n
should be included in
the solution or not. If she says yes, we give the friend the instance S
1
, . . . , S
n1
, t S
n
)
and return the friends answer with S
n
included. If she says no, we give the friend the
instance S
1
, . . . , S
n1
, t) and return the friends answer unchanged. The trick is to know
in poly-time what the bird will answer. This is done by asking the poly-time algorithm for
the decision problem to solve the instance S
1
, . . . , S
n1
, t S
n
). If it says that there is a
solution for this instance, then we assume the bird says to include S
n
.
66. Reductions: The purpose of this question is to learn to appreciate why there are so many steps when
doing a reduction.
0) P
oracle
NP: Prove that problem P
oracle
is in NP.
5) InstanceMap: Dene a polynomial time mapping I
oracle
= InstanceMap(I
alg
).
6) SolutionMap: Dene a polynomial time algorithm mapping each potential solution S
oracle
for
I
oracle
to a potential solution S
alg
= SolutionMap(S
oracle
) for I
alg
.
7) Valid to Valid: Prove that if S
oracle
is a valid solution for I
oracle
, then S
alg
=
SolutionMap(S
oracle
) is a valid solution for I
alg
.
8) ReverseSolutionMap: Dene a polynomial time algorithm mapping each potential solution S

alg
for I
alg
to a potential solution S

oracle
= ReverseSolutionMap(S

alg
) for I
oracle
.
9) Reverse Valid to Valid: Prove that if S

alg
is a valid solution for I
alg
, then S

oracle
=
ReverseSolutionMap(S

alg
) is a valid solution for I
oracle
.
Consider the problem Block whose instance s, k) is a 01-string s and an integer k. Such an instance
is a yes instance if it has a solution consisting of a block s
i
, s
i+1
, . . . s
j
of all ones whose length is k.
(a) (2 marks) Is this Block problem in NP? If so, what would this mean?
Answer: It can easily be solved in poly-time and hence is automatically in NP. If a fairy god
mother wanted to help she could tell you where the block is, but this is not necessary. This
is not very exciting.
(b) (2 marks) Is this Block problem is NP-Complete? What would it mean?
Answer: It can be done in poly time. Hence, if it was NP-complete then all NP problems
could be done in poly time. This is unlikely.
(c) (2 marks) Your home work is to prove that Block is NP-complete by proving that 3-SAT
ploy
Block. You are to work in pairs. Your partner took on step 5 as follows. P
alg
is 3-SAT. As
an example, I
3-SAT
= (x or y or z) AND . . . AND (u or v or w) is an instance of 3-SAT. Given
such an instance, he maps it to an instance I
Block
= s, k) where s = 000 . . . 0 is the string
consisting of n zeros and k is 5. Besides a few hints which he ignored, he noted that he did in
fact meet the formal requirements of step 5. Being a bit of a know it all, your partner thought
steps 8 and 9 are silly and hence decided to skip them. He left you the task of doing steps 6 and
7. Carefully, review what these steps are and complete them if you can. Careful.
64
Answer: No problem. The instance I
Block
has no valid solutions because it is a no instance.
Because there are none, it is easy to map every valid solution of it to a valid solution of
I
3-SAT
.
(d) Because your previous partner dropped out of school, you got a new one. Having learned, he took
a new approach. He decided to map an instance I
3-SAT
of 3-SAT to an instance I
Block
= s, k)
where s = 111 . . . 1 is the string consisting of n 5 ones and k is 5. He went on to do steps 6
and 7 as follows. Given a solution s
i
, s
i+1
, . . . s
j
to I
Block
, he instructed how to construct a valid
solution to I
3-SAT
as follows. For each of the m clauses (x or y or z) of I
3-SAT
choose one of
the variables. If, like x, it is not negated in the clause then set the variable to true. If, like y,
it is negated, then set it to false. Either way the clause is satised. Doing this for each clause,
satises each clause. Hence, the assignment obtained is a valid assignment of the variables. This
new partner learned (from the guy who dropped out) the importance of doing steps 8 and 9 so
he left them for you to do.
i. (2 marks) Carefully, review what these steps are and complete them if you can. Careful.
Answer: No problem. The instance I
Block
has many valid solutions. One of then is the
substring s
1
, s
2
, . . . , s
5
. Any valid solutions of I
3-SAT
(if there are any) can be mapped
to it.
ii. (2 marks) Are you happy with your new partner? Did he follow steps 5, 6, and 7 correctly?
Explain.
Answer: He did 5 ne. 6 and 7 he did in one paragraph, which was a mistake. He should
have fully dened the assignment to the variables in step 6, before trying to prove it valid
in step 7. As is, he may have set some variable x to be true in order to satisfy one clause
but later set the same variable to false in order to satisfy another clause.
67. NP-Completeness
(a) The problem Clause-SAT is given a set of clauses where each clause is the OR of a set of literals
and each literal is either a variable or its negation. The goal is know whether you can set each
variable to true or false so that each clause is satised.
The course notes (along with an exercise on 3-SAT), prove that the problems (circuit) SAT, 3-
Colouring, Course Scheduling, Independent Set, and 3-SAT are NP-Complete. In a few sentences,
explain how you know that Clause-SAT is NP-Complete.
Answer: Clause-SAT is more general that 3-SAT. Hence, if you can solve any instance of
Clause-SAT then you can denitely solve any of 3-SAT. Also being in NP, it follows that
Clause-SAT is NP-Complete.
(b) The problem Card is dened by the following puzzle.
You are given a box and a collection of cards as indicated in the g-
ure. The box and each card has r rows and two columns of positions
in the same places. The box contains Xs in these positions. In each
card at each of these positions, there is either a hole punched out or
there is not. Each card must be placed in the box either face up or
ipped over left to right. It has a notch at its top which must be
at the top of the box. Consider some row in some card. Suppose
its left position has a hole but its right position does not. Putting
the card in face up covers the right X in the box of this row but not
the left. Putting the card in ipped over covers the left but not the
right. The goal is to cover each and every X in the box. A solution
species for each card whether to ip the card or not. A solution is
valid if it covers all the Xs.
X X
X X
X X
X X
X X
X X
X X
X X
X X
Box One side The other side
Prove that Card is NP-Complete by reducing it to Clause-SAT. Be sure to think about each of
the 12 steps. As a huge hint, I do step 5 for you. You are to write up steps 0, 6, and 7. For each
of these have a clear paragraph with pictures.
65
0) P
card
NP:
5) InstanceMap: Given an instance I
ClauseSAT
to Clause-SAT consisting of a set of clauses,
we construct as follows an instance I
card
= InstanceMap(I
ClauseSAT
) for the card game
consisting of a set of cards. Our box and cards will have one row for each clause. We will have
a card for each variable x. For each clause c, this card for x will have its left hole position
for row c not punched out if x appears positively in c and will have its right hole position for
c not punched out if x appears negatively in c. It will have all other hole positions punched
out. We will have one additional card that has all of its left hole positions punched out and
none of its right hole positions punched out.
6) SolutionMap:
7) Valid to Valid:
Answer:
0) P
card
NP: Given an instance consisting of a set of cards and a solution consisting of
an orientation of each card, it is easy to determine if every hole position is blocked.
6) SolutionMap: Given a solution S
card
for the instance I
card
=
InstanceMap(I
ClauseSAT
) consisting of a notipped/ipped orientation of each
card, we construct as follows a solution S
ClauseSAT
for instance I
ClauseSAT
consisting
of a true/false setting of each variable. First, if in the solution S
card
, the additional card
is ipped then we can ip the entire deck of cards over without changing whether all the
hole positions are covered. Hence, without loss of generality assume that in S
card
, the
additional card is not ipped. Then for each variable x, set it to true i its associated
card is notipped in S
card
.
7) Valid to Valid: Assume that S
card
is a valid solution, i.e. every hole position is covered.
Our goal is to prove that S
ClauseSAT
is a valid solution, i.e. every clause is satised.
Consider some clause c. Because the additional card is not ipped in S
card
and every
hole position is covered, we know that there is some variable card x covering the left hole
position for clause c. If this card is not ipped then its left hole position for c must not
be punched. In this case, x is set to true in S
ClauseSAT
and x appears positively in
c. Hence, this clause is satised. Alternatively, if this card is ipped then its right hole
position for c must not be punched. In this case, x is set to false in S
ClauseSAT
and x
appears negatively in c. Again, this clause is satised. Hence, each clause in S
ClauseSAT
is satised.
(c) Suppose you work for an airplane manufacturing company. Given any detailed specication of a
plane, right down to every curve and nut and bolt, your boss has a way of determining whether
it is a great plane or not. However, he has tried thousands of dierent specications and none of
them turn out to be great. He is completely frustrated.
Meanwhile back at home, your son didnt nish high school, is socially awkward, and is living in
your basement at 31. You blame it on the fact that he is completely addicted to playing video
games. You feel this is only one step better to being a crack addict. One game he plays all the
time is the card ipping puzzle described above. He has a magical ability to instantly put any set
of cards into the box so that all the Xs are covered.
You are are only able to do things described in the course notes (or hinted at in exercises). How
do you get your son a job at work so that he quickly becomes a billionaire?
Answer: Following your boss method, you write a JAVA program that takes as input the
description of plane and outputs whether or not it is great. (Given a xed number of bits
n to describe the input, it is not too hard to automatically compile this program into an
AND-OR-NOT circuit that does the same thing. The notes describe a way to convert this
circuit into a graph to be 3-Colored. The exercise in the notes hints at how to convert this
into an instance of 3-SAT, which is itself an instance of Clause-SAT. Above we describe how
to convert this into an instance of the Card game. You get your son to magically solve it. You
convert this solution into a solution for the 3-Sat instance, which you convert into a solution
66
of the graph coloring instance, which you covert into a solution of the circuit problem and
a valid input of the JAVA program. This you translate into a description of a great plane,
which you give your boss. You and your son go on to solving all of the worlds optimization
problems.
68. NP Complete
3-SAT
poly
3-Clique
3-SAT
poly
CourseScheduling
(a) Are these true? Why?
Answer: The rst is not because 3-Clique can be solved in (n
3
) time and hence is not likely
harder than 3-SAT which is believed to require exponential time.
The second we proved is true in class.
(b) In one sentence explain what needs to be done to prove it.
Answer: Given an oracle (a poly time algorithm) for CourseScheduling design an oracle for
3-SAT.
(c) How are these useful to you when working as an algorithm designer?
Answer: This second statement proves that CourseScheduling very likely requires expo-
nential time, because if it could be solved quickly then so could 3-SAT, which is believed to
require exponential time.
69. Let P
general
and P
restricted
be the same computational problem except that P
restricted
is dened only
on a subset of the instances on which P
general
is dened. For example, 3-SAT is the same problem as
SAT except that each clause has at most three literals. Clique is given a graph G and an integer k
and asks whether there is a clique in G of size k. 3-Clique is the same with k xed to 3.
(a) Compare and discuss the meaning of P
general

poly
P
restricted
vs P
restricted

poly
P
general
. What
are the intuitions as to why they may be true and why they may not be true. Which is easier to
prove?
(b) Which of SAT
poly
3-SAT and 3-SAT
poly
SAT are true. Give one sentence how you know.
(c) Which of Clique
poly
3-Clique and 3-Clique
poly
Clique are true. Give one sentence how you
know.
(d) Do steps 5-9 to prove that 3-SAT
poly
SAT. Hint: This REALLY is easy.
Answer:
(a) Clearly a restricted problem is easier because there are few instances that might be the worst
case. P
restricted

poly
P
general
is always true. P
general

poly
P
restricted
will not be true if the
instances considered by the restricted problem are the easy ones and not the worst case ones.
(b) As said, 3-SAT
poly
SAT is true because the rst is a restriction of the second (See proof
below). Surprisingly, SAT
poly
3-SAT is also true. One has to show that there are su-
ciently dicult instances that have 3 literals per clause. The reduction shows that being able
to solve the 3 literal case is enough to be able to solve all of NP.
(c) Again 3-Clique
poly
Clique is trivially true. Clique
poly
3-Clique is false (unless P = NP),
because Clique is NP-complete and 3-Clique can be solved in O(n
3
) time.
(d) 3-SAT
poly
SAT.
5) InstanceMap: Given an instance I
3-SAT
to 3-SAT, InstanceMap(I
3-SAT
) simply maps
to the same instance because this is also an instance of the more general SAT.
6&8) SolutionMaps: Given that the two instances I
3-SAT
and I
SAT
are the same, they
clearly have the same set of solution. We simply map solutions to themselves.
67
7&9) Valid to Valid: Being the same solution to the same instance clearly they are both
valid or both not valid.
Part IV: Appendix
Chapter: 22: Existential and Universal Quantiers
70. (Quantiers) Let Works(P, A, I) be dened to be true if algorithmA halts and correctly solves problem
P on input instance I.
Let P = Halting be the Halting problem which takes a Java program I as input and says yes if I
halts on the empty string. Otherwise, P must say

no

.
Let P = Sorting be the sorting problem which takes a list of numbers I as input and sorts them.
Let A
insertionsort
be the sorting algorithm which we learned in class.
Let A
yes
be the algorithm that on input I ignores the input and simply halts and says yes.
Similarly, let A
no
be the algorithm that simply says no.
Let A

be the algorithm that on input I ignores the input and simply runs for ever.
For each part, explain the meaning of what you are doing and why you dont do it another way.
(a) Express in rst order logic that Sorting is computable.
Answer: A, I, Works(Sorting, A, I). We know that there at least one algorithm, eg.
A = mergesort, that works for every input instance I.
(b) Express in rst order logic that Halting is not computable. Explain why you do it this way.
Answer: A, I, Works(Halting, A, I) We know that opposite statement is true. Every
algorithm fails to work for at least one input instance I.
(c) Express in rst order logic that there are uncomputable problems.
Answer: P, A, I, Works(P, A, I)
(d) What does the following mean and either prove or disprove it: I, A, Works(Halting, A, I).
(Not simply by saying the same in words For all I, exists A, Works(Halting, A, I))
Answer: It says that every input has some algorithm that happens to output the right
answer. It is true. Consider an arbitrary instance I. If on instance I, Halting happens to
say yes, then let A be the algorithm that simply halts and says yes. Otherwise, let A be
the algorithm that simply halts and says no. Either way A works for this instance I.
(e) What does the following mean and either prove or disprove it A, P, I, Works(P, A, I).
Answer: It says that every algorithm correctly solves some problem. This is not true
because some algorithm do not halt on some input instances. We prove the complement
A, P, I, Works(P, A, I) as follows. Let A be an algorithm that runs for ever on some
instance I

. Let P be an arbitrary problem. Let I be an instance I

on which A does not

halt. Note Works(P, A, I) is not true.
Chapter: 23: The Time (and Space) Complexity of an Algo-
rithm
Chapter: 24: Logarithms and Exponentials
Chapter: 25: Asymptotic Notation
Section: 25.1: Steps to Classify a Function
71. (4 marks) Suppose f(n) = ab
cn
n
d
log
e
(n). What values can the real numbers a, b, c, d, and e take on
and have f(n) be considered in the course notes to have a polynomial growth rate? Give an example.
Answer: a > 0, b
c
= 1, d > 0, and e anything. For example, f(n) = n
2
is polynomial.
68
Section: 25.2: More about Asymptotic Notation
72. The formal denition of f(n) n
(1)
is c
1
, c
2
, n
0
, n n
0
, n
c1
f(n) n
c2
. Formally prove that
if f(n) and g(n) n
(1)
, then f(n) g(n) n
(1)
. What is this class of functions called?
Answer: These functions are polynomials.
It includes f(n) = n
2
logn because it is bounded between n
2
and n
3
.
Because f(n) and g(n) n
(1)
, we know that
c
f,1
, c
f,2
, n
f,0
, n n
f,0
, n
c
f,1
f(n) n
c
f,2
c
g,1
, c
g,2
, n
g,0
, n n
g,0
, n
c
g,1
g(n) n
c
g,2
Consider some such constants c
f,1
, c
f,2
, n
f,0
, c
g,1
, c
g,2
, and n
g,0
.
Our goal is to now prove f(n) g(n) n
(1)
, i.e.
c
1
, c
2
, n
0
, n n
0
, n
c1
f(n) g(n) n
c2
.
Let c
1
= c
f,1
+c
g,1
, c
1
= c
f,2
+c
g,2
, and n
0
= max(n
f,0
, n
g,0
).
Let n any value (given by an adversary) such that n n
0
= max(n
f,0
, n
g,0
).
Because n n
f,0
, we have that n
c
f,1
f(n) n
c
f,2
Because n n
g,0
, we have that n
cg,1
g(n) n
cg,2
Because these are all positive, we can multiply the above two lines together giving
n
c1
= n
c
f,1
n
c
g,1
f(n) g(n) n
c
f,2
n
c
g,2
= n
c2
.
73. We have seen three algorithms for the sorted matrix problem with running times T
1
(n, m) = n + m,
T
2
(n, m) = nlog(
m
n
) and T
3
(n, m) = nlog(m). We will be considering families of matrices. In a
particular family. the hight n can have any value and the width m = m(n) is some function of n. The
recursive T
2
(n, m) algorithm was a pain to write and implement. Hence, we only want to use it for
families of matrices where the running is really little oh of the running time of other two algorithms,
i.e. not Theta. Use O, o, , , and , to bound for which functions m(n) we will use this algorithm.
Start by trying dierent functions. Give extreme examples that work and others that do not work.
Answer: Express the width as m(n) = n c(n), where c(n) is some function yet to be bound.
This gives
T
2
(n, m) = nlog(
m
n
) = n log(c(n)) and
T
1
(n, m) = n +m = n (c(n) + 1).
If c(n) = (1) is bounded by a constant, then T
2
(n, m) = (T
1
(n, m)), meaning we should not
bother with the recursive program. For example, even m(n) = 1000n is too small. On the
other hand, if c(n) = (1) is not constant but grows with n, then c(n) + 1 = (log(c(n))) and
log(c(n)) = o(c(n) + 1), giving T
2
(n, m) = o(T
1
(n, m)), meaning we should use the recursive
program. For example, m(n) = nlog log n grows fast enough.
Now express the width as m(n) = n n
c(n)
, where c(n) is some function yet to be bound. This
gives
T
2
(n, m) = nlog(
m
n
) = nlog(n
c(n)
) = c(n)nlog(n) and
T
3
(n, m) = nlog(m) = nlog(n
1+c(n)
) = (1 +c(n))nlog(n).
If c(n) = (1), meaning that it is either constant or grows with n, then T
2
(n, m) = (T
3
(n, m)),
meaning we should not bother with the recursive program. For example, even m(n) = n

n and
m(n) = n
1.0001
are is too big. On the other hand, if c(n) = o(1) is not constant but shrinks with
n, then c(n) = o(1+c(n)) and hence T
2
(n, m) = o(T
3
(n, m)), meaning we should use the recursive
program. For example, m(n) = n n
(1/

log n)
= n 2

log n
and m(n) = nlog n grow faster then n
but not too fast.
Chapter: 26: Adding Made Easy Approximations
Section: 26.1: The Technique
Section: 26.2: Some Proofs for the Adding Made Easy Tech-
nique
69
Chapter: 27: Recurrence Relations
74. Asymptotic Analysis (20 marks)
(a) (3 marks) Briey describe the structure of a typical recursive algorithm that has running time
T(n) = a T(
n
b
) +f(n).
Answer: Work of f(n) in stack frame and a recursive calls, each of which receives an input
that is reduced by a factor of
1
b
.
(b) (6 marks) Depending on the values of a, b and c, what is the theta approximation of the recurrence
T(n) = a T(
n
b
) +n
c
? For each case, give a one sentence justication.
Answer: If
log a
log b
> c, then the time is dominated by the base cases and T(n) = (n
log a
log b
).
If they are equal, then all the levels have about the same time and T(n) = (n
c
log n).
If
log a
log b
< c, then the time is dominated by the top level and T(n) = (n
c
).
(c) (4 marks) Give values for a, b, and c such that T(n) = a T(
n
b
) + n
c
= (n
4
3
) and the time is
dominated by the base cases.
Answer: If a = 2
4
= 16, b = 2
3
= 8, and c = 0, then T(n) = 16T(
n
8
)+1 = (n
log a
log b
) = (n
4
3
).
(d) (4 marks) True or False? All logarithms are base 2. No justication is necessary.
i. 5n
2
log n O(n
2
)
Answer: False. It is a factor of log n too big.
ii. 4
8n
O(8
4n
)
Answer: False: 4
8n
= 2
16n
, but 8
4n
= 2
12n
.
iii. 2
10 log n
+ 100(log n)
11
O(n
10
)
Answer: True: 2
10 log n
= n
10
, 100(log n)
11
O(n
10
).
iv. 2n
2
log n + 3n
2
(n
3
)
Answer: False: 2n
2
log n + 3n
2
O(n
3
), but 2n
2
log n + 3n
2
/ (n
3
).
70
(e) (3 marks) For each of the following sums, give the -approximation. No justication is necessary.
i.
log n
i=1
i
2
Answer: ((last term) (number of terms)), since the sum is arithmetic-like = (log
2
n
log n) = (log
3
n)
ii.
n
i=1
i
2
i!
Answer: = ((last term)), since the terms in the sum grow faster than geometric-like
= (n
2
n!
))
iii.
n
i=1
2
i
3
Answer: = (1). 3 < 1, and hence the sum is bounded-tail.
75. Consider the following functions.
(a) f(n) = 5 n
5.5
+n
2.5
log
100
(n)
(b) f(n) = 5
3

n + 5
4

n
(c) f(n) = 5 3
4n
+n
100
(d) f(n) =
5
n
1.01
+
7
n
0.99
(e) f(n) = 5 +sin(n
2
)
Sort the functions by growth rate from fastest to slowest. While you are at it sort all the terms
(3 marks). Now answer the following questions for each function, starting with the fastest growing
function. Please report your answer for both questions for each function before moving on to the next
function.
Classication: (2 marks per function) Provide a -approximation, if possible classifying the
function into (1), 2
(n)
, log
(1)
(n), or n
(1)
. What is this class called?
Summation: (2 marks per function) Approximate the sum (

n
i=1
f(i)). Which result did you
use and how did you use it? What type of sum is it?
Answer:
c >> a >> b >> e >> d, i.e., 3
4n
>> n
5.5
>> n
2.5
log
100
(n) >>
3

n >>
4

n >> 5+sin(n
2
) >>
n
0.99
>> n
0.1.01
(a) Class: f(n) = 5 n
5.5
+ n
2.5
log
100
(n). The n
2.5
log
100
(n) is smaller than n
2.5001
which is
clearly smaller that n
5.5
. Hence, f(n) (n
5.5
) n
(1)
. This is a polynomial function.
Sum: b
a
= 1 and d = 5.5 > 1. Hence, the sum is arithmetic and half the terms are
approximately equal. Hence,

n
i=1
f(i) = (n f(n)) = (n
6.5
).
(b) Class: f(n) = 5
3

n + 5
4

n = 5 n
1
3
+ 5 n
1
4
. Because
1
3
>
1
4
, the rst term dominates.
Hence, f(n) (n
1
3
) n
(1)
is an polynomial function. Though it is grows slower than n,
it is n
d
for d > 0.
Sum: b
a
= 1 and d =
1
3
> 1. Hence, the sum is arithmetic and half the terms are
approximately equal. Hence,

n
i=1
f(i) = (n f(n)) = (n
1
1
3
).
(c) Class: f(n) = 5 3
4n
+n
100
is dominated by the exponential term not the polynomial term.
f(n) (3
4n
) 2
(n)
is an exponential function.
Sum:

n
i=1
f(i) =

n
i=1
(3
4i
) =

n
i=1
(81
i
). Since 81 > 1, the sum is geometric increas-
ing and dominated by the last term. Hence

n
i=1
f(i) = (f(n)) = (3
4n
).
(d) Class: f(n) = 5 1
8n
= 5 = (1) is constant.
Sum: See (f)
71
(e) Class: f(n) =
5
n
1.01
+
7
n
0.99
= 5 n
1.01
+7 n
0.99
. Because 0.99 > 1.01, the second term
dominates and f(n) (
1
n
0.99
). This decreases with n so is to small to even be considered
constant, (1).
Sum: b
a
= 1 and d = 0.99 > 1. Hence, the sum is still arithmetic and half the terms are
approximately equal. Hence,

n
i=1
f(i) = (n f(n)) = (n
0.001
).
Sum: (not asked for) If f(n) =
5
n
1.01
, then b
a
= 1 and d = 1.01 < 1. Hence, the sum is
bounded tail and the sum is dominated by the rst term. Hence,

n
i=1
f(i) = (1).
(f) Class: f(n) = 5 + sin(n
2
). The n
2
was a trick because the sine is always between -1 and 1.
Hence, f(n) (1). Though it changes with n, the change is not considered signicant and
hence is said to be constant.
Not asked for: This would be the same for f(n) = 1 if n is odd, 2 if n is even (1).
Not asked for: If f(n) = sin(n
2
), then f(n) would not be in (1) because things in (n)
should be positive for big n.
Sum: b
a
= 1 and d = 0 > 1. Hence, the sum is arithmetic and half the terms are
approximately equal. Hence,

n
i=1
f(i) = (n f(n)) = (n).
76. Consider the following functions.
(a) f(n) = 5 n
5
(b) f(n) = 5
3

n
(c) f(n) = 5 3
4n
(d) f(n) =
5
n
2
(e) f(n) = 1 if n is odd, 2 if n is even
Sort the functions by growth rate from fastest to slowest. Then for each, do each of the following
things.
(a) Give its Theta approximation. If possible classifying it into (1), 2
(n)
, log
(1)
(n), or n
(1)
.
What is this class called?
(b) Approximate the sum (

n
i=1
f(n)). Which result do you use and how do you use it? What type
of sum is it?
(c) Describe an algorithm with a natural structure (like those found in the Appendix) which takes as
input a string containing n bits and prints (f(n)) His. Zero marks will be given for
loop i = 1 . . . f(n) print(Hi)
Answer:
(a) Order: f(n) = 5 3
4n
(3
4n
) 2
(n)
is an exponential function.
Sum: As given in the table of Chapter 26.1, b
a
= 3
4
= 81 > 1. Hence, the sum is geometric
increasing and dominated by the last term. Hence (

n
i=1
f(n)) = (f(n)) = (3
4n
).
Code: Note f(n) = 3
4n
= 81
n
. The recurrence relation T(n) = 81T(n 1) is similar to that
in the next assignment and evaluates to (f(n)) = (81
n
1
). Code which recurses 81 times
on inputs of size n 1 and prints one Hi on each base case has this running time.
(b) Order: f(n) = 5 n
5
(n
5
) n
(1)
is an polynomial function.
Sum: b
a
= 1 and d = 5 > 1. Hence, the sum is arithmetic and half the terms are
approximately equal. Hence, (

n
i=1
f(n)) = (n f(n)) = (n
6
).
Code: Code consisting of ve nested loops, each iterating n times, with a print(Hi) state-
ment in the middle will print n
5
His.
(c) Order: f(n) = 5
3

n (n
1
3
) n
(1)
is an polynomial function. Though is grows slower
than n, it is n
d
for d > 0.
Sum: b
a
= 1 and d =
1
3
> 1. Hence, the sum is arithmetic and half the terms are
approximately equal. Hence, (

n
i=1
f(n)) = (n f(n)) = (n
1
1
3
).
72
Code: The recurrence relation T(n) = 3T(
n
27
) from the next assignment evaluates to
(f(n)) = (n
1
3
). Code which recurses 3 times on inputs of size
n
27
and prints one Hi
on each base case has this running time.
(d) Order: f(n) = 1 if n is odd, 2 if n is even (1). Though it changes with n, the change
is not considered signicant and hence is said to be constant.
Sum: b
a
= 1 and d = 0 > 1. Hence, the sum is arithmetic and half the terms are
approximately equal. Hence, (

n
i=1
f(n)) = (n f(n)) = (n).
Code: Code which prints one Hi and another if n is even, prints this many His.
(e) Order: f(n) =
5
n
2
(
1
n
2
) decreases with n so is to small to even be considered constant,
(1).
Sum: b
a
= 1 and d = 2 < 1. Hence, the sum is bounded tail and the sum is dominated
by the rst term. Hence, (

n
i=1
f(n)) = (1).
Code: It is impossible to print more than zero and less than one Hi.
77. Solve each of the following recurrence relations. Which result do you use and how do you use it?
(a) T(n) = 3T(
n
27
) +n
2
.
(b) T(n) = 3T(
n
27
).
(c) T(n) = 3T(
n
27
) +n
3
(d) T(n) = 3T(
n
27
) + 2
n
(e) T(n) = 3T(n 4).
Answer:
(a) T(n) = 3T(
n
27
) + n
2
: As in the rst table of chapter 27.1, a = 3, b = 27, and c = 2.
We have
log a
log b
=
log
3
3
log
3
27
=
1
3
< 2 = c. Hence, the time is dominated by the top level and
T(n) = (f(n)) = (n
2
).
(b) T(n) = 3T(
n
27
): Same as above but, f(n) = 0. This is smaller than 1 = n
0
. Hence, we will
say c 0. This gives
log a
log b
=
1
3
> c. Hence, the time is dominated by the base cases and
T(n) = (n
log a
log b
) = (n
1
3
).
(c) T(n) = 3T(n 4). As in the second table of chapter 27.1, a = 3 > 1 and b = 4. Hence, the
time is dominated by the base cases and T(n) = (f(n)) = (a
n
4
).
Alternatively, just unwrap the recursion: T(n) = 3T(n 4) = 3(3T((n 4) 4)) = ... =
3
i
T(n 4i)... The base case occurs when n 4i = 0 i = n/4 T(n) (3
n/4
).
78. Math
(a) (5 marks) When f(n) has a polynomial growth rate, what is the theta approximation of the sum

n
i=1
f(i)? Give a one sentence justication.
Answer: Half the terms are approximately the same giving that the sum is approximately
the number of terms times the last term, i.e.

n
i=1
f(i) = (n f(n)).
(b) (5 marks) Quickly describe the structure of an algorithm that has running time T(n) = a T(
n
b
) +
f(n).
Answer: I do work f(n) and have a friends each of which receives an input that is
1
b
the size
of my own.
(c) (5 marks) Give values for a, b, and c such that T(n) = a T(
n
b
) + n
c
= (n
4
3
) and the time is
dominated by the base cases. Show the math.
Answer: If a = 2
4
= 16, b = 2
3
= 8, and c = 0, then T(n) = 16T(
n
8
)+1 = (n
log a
log b
) = (n
4
3
).
73
Chapter: 28: A Formal Proof of Correctness
Part Questions from Grad Class
79. Short Answer
(a) (2 marks) Fast Fourier Transforms (FFT) can be used for signal processing and many things. The
version that we covered, what was the input and what was the output.
(b) (2 marks) As a TA, you have lots of experience marking assignments. You are given JAVA code
written by rst year students and you can always tell whether or not they halt. You are also given
math statements like x, y, x+y = 5 and you can always tell whether they are true or not. Your
teacher says something about the halting problem and the math truth both being undecidable,
but you are convinced he is wrong. Debate this one way or the other.
Answer: The TA may be smarter than the rst year students, but let me give him a JAVA
program or a math statement for him to understand. For every xed algorithm for either the
halting problem or the math truth problem, there is an input for which these programs either
give the wrong answer or fail to halt.
============================
In question not answer in notes.
Exercise 0.0.9 There is a collection of software packages S
1
, . . . , S
n
which you are considering buying.
These are partitioned into two groups. For those i N [n], the costs of buying it out ways the benets
and hence it eectively costs you a given amount b
i
0 to buy it. For those j P [n], the benets out
way the costs of buying it and hence it eectively costs you a given amount b
j
0 to not buy it. Some of
these packages rely on each other; if S
i
relies on S
j
, then you will incur an additional cost of a
i,j
0
if you buy S
i
but not S
j
. Provide a polytime algorithm to decide the subset S [n] of S
1
, . . . , S
n
that
you should buy. The cost of your solution is cost(S) =

iSN
b
i
+

jSP
b
i
+

iS,jS
a
i,j
. Hint:
Do not design a new algorithm but do a reduction to min cut similar to that done for matching the
boys and girls.
As said, this answer is not in the book. I will put it in the extended book for the professors.
Answer: Lets start with some informal discussion. Given an instance I
Software
, the nodes
of I
NetworkFlow
= InstanceMap(I
Software
) will be the software packages S
1
, . . . , S
n
and two
additional nodes s and t. Our solution S
Software
= SolutionMap(S
NetworkFlow
) must specify for
each package S
i
whether or not to buy it. It is hard to get that information from S
NetworkFlow
if we consider S
NetworkFlow
to be the max ow. However, if we consider S
NetworkFlow
to be
the min cut U, V ), then we could either buy the software/nodes in U s or we could buy the
software/nodes in V t. If we buy S
i
but not S
j
, then cost(S
Software
) acquires an additional
cost of a
i,j
. If i U and j V , then cost(S
NetworkFlow
) acquires an additional cost of c
i,j
.
Hence, it is reasonable to buy the software/nodes in U s and not those in V t.
Lets now be more formal. Given an instance I
Software
, the network I
NetworkFlow
=
InstanceMap(I
Software
) will have an edge i, j) with capacity c
i,j
= a
i,j
for each pair of
software packages S
i
and S
j
. For each i N, there will be an edge i, t) with capacity c
i,t
= b
i
.
For each j P, there will be an edge s, j) with capacity c
s,j
= b
j
.
Given valid cut S
NetworkFlow
= U, V ) for this network, The solution S
Software
=
SolutionMap(S
NetworkFlow
) is that S
Software
= U s, i.e. buy the software associated with
the nodes in U s. This is a 1-1 mapping, because given a solution S
Software
one can map back
to a S
NetworkFlow
= SolutionMap
1
(S
Software
), where U = S
Software
+s and V = S
Software
+t.
The nal step is to prove that the costs of these two solutions are the same, i.e. cost(S
Software
) =
cost(S
NetworkFlow
). By the denition of the cost of a cut U, V ), we have that
74
cost(S
NetworkFlow
) =

uU,vV
c
u,v
=

iUs,jV t
c
i,j
+

iUN
c
i,t
+

jV P
c
s,j
=

iS
Software
,jS
Software)
a
i,j
+

iS
Software
N
b
i,t
+

jS
Software
P)
b
s,j
= cost(S
Software
)
This completes the proof.
80. There is a collection of software packages S
1
, . . . , S
n
which you are considering buying. These are
partitioned into two groups. For those i N [n], the costs of buying it out ways the benets and
hence it eectively costs you a given amount b
i
0 to buy it. For those j P [n], the benets
out way the costs of buying it and hence it eectively costs you a given amount b
j
0 to not buy it.
Some of these packages rely on each other; if S
i
relies on S
j
, then you will incur an additional cost of
a
i,j
0 if you buy S
i
but not S
j
. The goal is to provide a polytime algorithm to decide the subset
S [n] of S
1
, . . . , S
n
that you should buy in order to minimize your total cost.
I will start with a major hints, otherwise the problem would be too hard. Hint: Do not design a new
algorithm but do a reduction to min cut similar to that done for matching the boys and girls. Formally
a reduction is done as follows.
Instance Map: Recall the algorithm for matching the boys and girls. Given an instance I
boys&girls
for
this problem, the algorithm constructed an instance I
NetworkFlow
= InstanceMap(I
boys&girls
) for
the network ow problem. Basically, the boys and girls in the instance I
boys&girls
become nodes
of the graph I
NetworkFlow
with a directed edge from the boy to the girl if they are willing to marry
each other. In addition there is a node s with an edge to each boy and a node t with an edge
from each girl. We will do a similar thing here.
In general, such an instance map is the rst step of a reduction. Here we must to dene a
polynomial time algorithm InstanceMap(I
Software
) that, given an instance I
Software
of problem
P
Software
, constructs an instance I
MinCut
of problemP
MinCut
that has similar sorts of solutions.
(Recall that an instance of the min cut problem is exactly the same as that to the network ow
problem.) The nodes of I
MinCut
will be the software packages S
1
, . . . , S
n
and two additional nodes
s and t. It will have an edge i, j) with capacity c
i,j
= a
i,j
for each pair of software packages
S
i
and S
j
. For each i N, there will be an edge i, t) with capacity c
i,t
= b
i
. For each j P,
there will be an edge s, j) with capacity c
s,j
= b
j
.
Solution Map: Again, recall the algorithm for matching the boys and girls. A solution S
NetworkFlow
to the network ow problem species the ow along each edge of the graph. This is used to
produce a solution S
boys&girls
for the boys and girls problem. Basically, a boy marries a girl if
there is ow between them.
In general, such a solution map is the second step of a reduction. Here we must dene a polynomial
time algorithm SolutionMap(S
MinCut
) giving a 1-1 mapping between any valid solution S
MinCut
for the instance I
MinCut
= InstanceMap(I
Software
) you just constructed and a solution S
Software
for instance I
Software
that was given as input.
Given valid cut S
MinCut
= U, V ) for this network, The solution S
Software
=
SolutionMap(S
MinCut
) is that S
Software
= U s, i.e. buy the software associated with the
nodes in U s. This is a 1-1 mapping, because given a solution S
Software
one can map back to
a S
MinCut
= SolutionMap
1
(S
Software
), where U = S
Software
+s and V = S
Software
+t.
a) Cost Map: The next step is to prove that the costs of these two solutions are the same, i.e.
cost(S
Software
) = cost(S
MinCut
). Prove this. (10 marks)
75
Answer: By the denition of the cost of a cut U, V ) is the sum of the capacities of the
edges leaving the cut, ignoring those edges coming back into the cut. This gives
cost(S
MinCut
) =

uU,vV
c
u,v
=

iUN
c
i,t
+

jV P
c
s,j
+

iUs,jV t
c
i,j
=

iS
Software
N
b
i
+

jS
Software
P)
b
j
+

iS
Software
,jS
Software)
a
i,j
= cost(S
Software
)
By the denition of the software problem, the cost of the solution S
Software
is the sum of the
costs b
i
of buying the costly packages S
i
(i.e. i in S
Software
and in N); the sum of the costs
b
j
of not buying the valuable packages S
j
(i.e. j not in S
Software
but in P); and the sum of
the costs a
i,j
incurred for buying S
i
but not S
j
(i.e. i in S
Software
and j not in S
Software
).
b) Code: Give the algorithm/code that solves this Software problem. You may use the Network Flows
algorithm in the text as a subroutine. (10 marks)
Answer: This provides an algorithm for the problem P
Software
. Given in instance
I
Software
, construct I
MinCut
= InstanceMap(I
Software
). Run the standard Network
Flow algorithm on I
MinCut
to obtain a solution S
MinCut
. Finally, construct S
Software
=
SolutionMap(S
MinCut
).
algorithm Alg
Software
(I
Software
)
precond: I
Software
is an instance of the Software problem.
postcond: Returns the optimal set S
Software
of packages to buy.
begin
I
MinCut
= InstanceMap(I
Software
)
% I
MinCut
is an instance of both MaxFlow and of MinCut.
S
MaxFlow
, S
MinCut
) = NetworkFlow(I
MinCut
)
% S
MinCut
is the optimal solution for I
MinCut
.
S
Software
= SolutionMap(S
MinCut
)
return(S
Software
)
end algorithm
c) Proof : Proof that if your program says that the minimum cost is c
min
, then there is a solution
for the instance that has this cost. (5 marks)
Answer: We claim that this algorithm gives an optimal solution for our instance I
Software
.
The solution S
Software
returned witness the fact that you can obtain a solution of value
c
min
= cost(S
Software
).
d) Proof : Proof that if your program says that the minimum cost is c
min
, then there is not a better
solution for the instance. (5 marks)
Answer: If there was another solution S

Software
that had a better cost cost(S

Software
), then
the inverse map S

MinCut
= SolutionMap
1
(S

Software
) would give a solution S

MinCut
to
I
MinCut
whose cost cost(S

MinCut
) = cost(S

Software
) is better than the cost cost(S
MinCut
) =
cost(S
Software
). This contradicts the claim that the standard Network Flow algorithm ob-
tained an optimal solution S
MinCut
for I
MinCut
.
e) Example: (20 marks) Solve the Software problem for the instance I
Software
consisting of four
software packages: S
1
, . . . , S
4
. S
1
and S
2
are in N and as such cost you b
1
= 10 and b
2
= 3 to
buy them. S
3
and S
4
are in P and as such cost you b
3
= 9 and b
4
= 5 to not buy them. The
additional costs a
i,j
for buying S
i
but not S
j
are a
1,2
= 5, a
3,1
= 4, a
3,2
= 4, a
4,1
= 7,
a
4,2
= 3, and a
4,3
= 2.
76
(a) (2 marks) Draw the input S
NetworkFlow
to network ow.
(b) (5 marks) Draw the max ow for this network and give its rate of ow.
(c) (5 marks) Draw the augmentation graph for this max ow.
(d) (5 marks) Draw the min cut for this network and give its value.
(e) (1 marks) Which software packages should you buy?
(f) (2 marks) What things cost you and what is the total cost?
Answer:
4
S
S
1 1
S
S
4
S
4
1
S
2
S
S
3
s
t
9
4
10
5
3
3
5
2
4
7
S
1
4
S
3/3
7/9
3/4
9/10
4/4
5/5
0/2
0/3
5/7
0/5
t
s
3
S
S
2
Max Flow
2
S
S
3
s
t
9
4
10
5
3
3
5
2
4
7
Min Cut Augmentation Graph
9
3
5
t
s
3
S
S
2
7
2
4
2
5
3
5
3
2
1
1
value of cut = 5+4+3=12
rate = 7+5=12
Input Network
You should by S
3
and S
2
. You pay 5 for not buying S
4
, 4 for buying S
3
but not S
1
, and 3
for buying S
2
.
77