MATHEMATICAL FOUNDATION FOR ALGORITHM

NGUYEN V.M. MAN Abstract. We propose a few specic techniques at graduate level used in algorithm analysis and design. These are aimed for graduates in Applied Mathematics at HCMUNS.

Contents List of Tables Introduction to MAA 1. Steps in Algorithm Analysis 1.1. Steps in Algorithm Analysis a framework 1.2. Classication of Algorithms 1.3. Asymptotic notation Big O notation 1.4. Complexity of Algorithm 2. Induction and Recurrence relations 2.1. Induction 2.2. Solving recurrence relations 2.3. Recursive schemes 2.4. Applications of recurrence relations in Analysis of Specic Algorithms 2.5. Generating functions 3. Recursive Algorithms 3.1. Introduction 3.2. Recursive Graph Algorithms 3.3. The discrete Fourier transform 4. Algorithm Design Techniques 4.1. Greedy algorithms 4.2. Divide and Conquer (Decomposition)
Date : January 21, 2008. Key words and phrases. linear algebra.
1

1 2 3 3 3 3 4 5 5 6 8 10 12 15 15 15 18 19 19 19

MATHEMATICAL FOUNDATION FOR ALGORITHM

4.3. Dynamic programming 4.4. Search in space of states Combinatorial search 4.5. Parallel computing types and characteristics 5. Algorithms in various Applications domains 5.1. Shortest paths in Weighted Graphs 5.2. In Numerical Computation 5.3. In Scheduling Problems 6. Conclusion References List of Tables 1 Step by step runing of Dijkstras algorithm

19 19 19 19 19 24 24 24 24

24

Evaluation method: Written assignment and Presentation (50 %) and Final Exam (50 %). The written assignment must be submitted before the Final Exam three weeks, and the presentation will be shown in the last few weeks of the course. Each group of two graduate students has 45 minutes to present their case study.

NGUYEN V.M. MAN

Introduction to MAA

The aims the course

This lecture reviews Mathematical background for Algorithm Analysis (MAA) with advanced concepts of asymtotic notation. This course intergrates techniques for solving recurrences, algorithm design techniques into graph problems, numerical problems, scheduling problems and search problems. The structure of the course. The course consists of three parts: Part 0: Review- Foundation Part I: Algorithm Complexity Part II: Induction and Recurrence relations- Recursive Algorithms Part III: Algorithm Design Techniques and Applications.

Part I: Algorithm Complexity Steps in Algorithm Analysis Specic algorithms Part II: Induction and Recurrence relations for Algorithm Analysis Induction and Recurrence relations Recursive Algorithms Part III: Algorithm Design Techniques and Applications Algorithm Design Techniques Algorithms in various Applications domains

MATHEMATICAL FOUNDATION FOR ALGORITHM

Part I: Algorithm Complexity Studying motivation. Algorithm Analysis plays a crucial role in large scale computations nowadays in every branch of science, technology, nance, econmics millitary domains. The methods to be studied in MAA help us not only to nd right solutions of concrete problems but also provide ecient ways to do so in reasonable (acceptable) times.

1. Steps in Algorithm Analysis 1.1. Steps in Algorithm Analysis a framework. Three basic steps of a suggested framework for analysing algorithm are: input data size: depends on the problem being studied compute operations of algorithm to separate analysis from implementation on specic computer mathematical analysis (most important): aim at nding avarage values as well as the worst case of each measure. Worst-case and avarage-case analysis. Worst-case running time is the longest running time for any input of size N . It is an upper bound on running time for any input. 1.2. Classication of Algorithms. There are several following classes: (1) C (constant) (2) log N (3) N (4) N log N (5) N 2 (6) N 3 (7) 2N 1.3. Asymptotic notation Big O notation. Asymptotic running times of algorithms are described by functions whose domains are the set N of naturals. 1.3.1. notation. For a given function g (n), we denote by (g (n)) = {f (n) : c1 , c2 R+ and n0 such that 0 c1 g (n) f (n) c2 g (n), n n0 }.

NGUYEN V.M. MAN

We write f (n) = (g (n)) or f (n) (g (n)) with the same sense, namely g (n) is an asymptotically tight bound for f (n). 1.3.2. Big O-notation. The -notation bounds a function from above and below. O-notation is used when we have only asymptotic upper bound. Denition 1. For a given function g (n), we denote by O(g (n)) = {f (n) : c R+ and n0 such that 0 f (n) cg (n), n n0 }. Writing f (n) = O(g (n)) means some constant multiple of g (n) is an asymptotic upper bound on f (n). 1.4. Complexity of Algorithm.

MATHEMATICAL FOUNDATION FOR ALGORITHM

Part II: Induction and Recurrence relations for Algorithm Analysis.

2. Induction and Recurrence relations 2.1. Induction. 2.1.1. Principles. When you prove the truthness of a proposition P (n) depending on natural numbers n 0, there are many forms of mathematical induction, such as the following: Ordinary induction: if P (n0 ) is true, and if [P (n) P (n + 1)] is true for all n n0 then P (n) is true for all n n0 Strong induction: if P (n0 ) is true, and if [P (n0 ) P (n0 + 1) . . . P (n) P (n + 1)] is true for all n n0 then P (n) is true for all n n0 Variation 1: if [P (0) P (1)] is true, and if [P (n) P (n + 2)] is true for all n 0, then P (n) is true for all n 0; Variation 2: if P (0) and P (1) are true, and if [P (n) P (n + 1)] P (n + 2) is true for all n 0, then P (n) is true for all n 0. 2.1.2. Examples. (1) For all n N then n3 n mod 3 (2) Let (Fk )k=0,1,2,... be Fibonacci numbers, that is they are determined by F0 = 0, F1 = 1, and Fk = Fk1 + Fk2 for all k 2.
2 2 2 In Fibonacci numbers Fn , all n N then F1 + F2 + . . . + Fn = Fn Fn+1 .

(3) Prove that for all integers n 1, we have

2n 2 Fi Fi1 = F2 n. i=1

2.1.3. Well-ordering. Axiom (Well-ordering principle). Every nonempty subset S N has a smallest element. Counter-example : Q+ has no smallest element. Remark 1. Well-ordering, ordinary induction and strong induction all amount to the same thing!

NGUYEN V.M. MAN

2.2. Solving recurrence relations. A recurrence relation is a formula that permits us to compute the members of a sequence one after another, starting with one or more given initial values. 2.2.1. Linear recurrence relations. A recurrence relation is linear of order k i it is of the form:
k

a(n) =
i=1

ci a(n i) = c1 a(n 2) + c2 a(n 2) + . . . ck a(n k );

where ci are constants. How to solve a linear homogeneous recurrence? Guess and Verify is the rst method to apply to nding explicit form of a recurrence. Guess a(n) = n , and substitute it into the recurrence, to obtain the characteristic equation :
k k

(2.1)

k =
i=1

ci ki k
i=1

ci ki = 0

Simple case: Distinct roots of the characteristic equation. If Equation 2.1 has k distinct roots 1 , 2 , . . . , k then the general form of a(n) is

n n a(n) = b1 n 1 + b2 2 + + bk k

ih which coecients bi are found due to initial conditions/values a(1), . . . , a(k ). Example 1. Fibonacci numbers are determined by F0 = 0, F1 = 1, and Fk = Fk1 + Fk2 for all k 2. Find the clear form of Fn , all n N. Hard case: Repeated roots of the characteristic equation. Theorem 2. If r is a root of the characteristic equation of a recurrence with multiplicity m, then all of rn , nrn , n2 rn , . . . , nm1 rn are solutions to the recurrence. Example 2. Find the clear form of the recurrence fn = 2fn1 fn2 with the boundary conditions f0 = 0 , f1 = 1 .

MATHEMATICAL FOUNDATION FOR ALGORITHM

2.2.2. Inhomogeneous recurrences. An inhomogeneous linear recurrence relation of order k has the form:
k

a(n) =
i=1

ci a(n i) + g (n); where ci are constants.

Read book for 3-steps solving of linear inhomogeneous recurrences! Step 1: replace g (n) by 0, nd the homogeneous recurrence. Step 2: restore g (n) and nd a partial solution without boundary conditions Step 3: add the homogeneous and partial solution together to form the general solution. Example 3. Find the clear form of the recurrence fn = 4fn1 + 3n with the boundary condition f1 = 1 .

NGUYEN V.M. MAN

2.3. Recursive schemes. 2.3.1. Introduction. Few basic concepts are: a/ Recursive objects: - recursive function, and recursive procedure P : both are dened through itself, formally stated as P = S (a, P ); and - recursive problem. b/ Recursive programming: consists of two parts: - bringing recursive function and/or problem back to non-recursive form: tough for problems having recursive nature; - really recursion. Example 4. Consider two distinct ways to dene f (n) = n!, if n is a positive integer: Nonrecursive: f (n) is the product of all the whole numbers from 1 to n, and Recursive: if n = 1 then f (n) = 1, otherwise f (n) = n.f (n 1). 2.3.2. Recursive forms. (1) Recursive function. Functions can be dened through themselves, F = F (x, F ). In particular, order k 1 recursive functions are Fn = F (A, Fn1 , , Fnk ); or simpler, we have linear recursive functions
k

Fn =
i=1

ci Fni .

For instance, Fn = n! = n Fn1 is an order 1 recursive function, but fn = fn1 + fn2 , f0 = 0, f1 = 1 : is an order 2 recursive function Usage: - determine k initial values, and - dene the recursive function at step n-th through k previous steps

MATHEMATICAL FOUNDATION FOR ALGORITHM

Example 5. Find the number Sn of length n binary sequences x such that x has an even number of 0s. Hint: Sn = Sn1 + 3n1 , S1 = 2. (2) Recursive procedure. P = P (T ; while B do P ), in which T is a group of assignments, and B a Boolean expression. Example 6 (Shamos problem). Given a vector X Rn , view A is a subvector of X i A consists of a bunch of consecutive elements in X . Find A such that the sum
i

A[i] is

max. Hint: 4 algorithms having complexities from O(n3 ) O(n2 ) O(nlogn) O(n). The general scheme or the overall structure of a recursive procedure. Algorithm 1 The overall structure of a recursive procedure Input: L, a list of variables. Output: f (u) where u L. function RecursiveCalculate( L) (1) if trivial case then do trivial thing (2) else do call RecursiveCalculate( L1 ), L1 consists of smaller values of the variables may be do a few more things end function

2.3.3. Elimination of recursions. Must know recursive formulas, and nd clear forms, using 2.2. Exercises. (1) Eliminate the following recursions. a) xn+1 = cxn b) xn+1 = bn+1 xn , (n 0, x0 = 1) where n 0, x0 R, {bi } is a known sequence

c) xn+1 = bn+1 xn + cn+1 , where n 0, x0 R, two sequences {bi }, {ci } are given d) xn+1 = 3xn + n e) xn+1 = 2xn xn1 (n 0, x0 = 0) (n 1, x0 = 1, x1 = 5)

(2) (Bounding a sequence from a recurrent inequality.) Suppose that a sequence {xi } satises: xn+1 xn + xn1 + n2 (n 1, x0 = 1 = x1 ).

Prove that: xn O(n ), that is there exist two constants K, such that, asymptotically xn K.n .

10

NGUYEN V.M. MAN

2.4. Applications of recurrence relations in Analysis of Specic Algorithms. 2.4.1. Selection Sort. Input : a sequence of elements L = [s1 , s2 , . . . , sn ] on which an ordering can be dened Output : A new sequence of the same elements L1 but now elements are sorted in an increasing order. Selection Sort Algorithm SS (L). We apply 6 steps as follows. (1) Initial test. Set n = Length(L). If n = 1, Stop: the sequence is sorted (2) Initialization. max index = 1 (3) Repeatition. for i = 2 to n do if si > smaxindex then max index = i enddo; # nd max element # suppose s1 is max

(4) Swap position. Move max element to the end of L, use Swap(sn , smaxindex ) (5) Update. L1 = L \ L[n] (6) Recursive call: SS (L1 ) Analysis. To measure computing time needed by the Algorithm SS , we count the number of comparisons bn to sort n 1 elements. Of course b1 = 0. Wish to nd a recursive formula for bn . We count the number of comparisons in each line and take their sum to be bn . Line 12: 0 Line 3: has n 1 comparisons Line 4: 0 Line 5: apply the Algorithm SS for list L1 having n 1 elements: need bn1 comparisons. Hence, the number of comparisons when running SS (L) is bn = n 1 + bn1 DIY to see: bn = O(n2 ). 2.4.2. Binary Search. Introduction Input : a sequence of elements L = [si , si+1 , . . . , sj ] is sorted in an non-decreasing ordering and a key value x0 Output : index k such that sk = x0 or k = 0 when x0 L

MATHEMATICAL FOUNDATION FOR ALGORITHM

11

Binary Search Ideas Divide and Conquer approach: divide up the list into two parts, then apply the search in smaller parts, or the key is the middle one! Binary Search Algorithm BS (L) We apply 6 steps as follows. (1) Initialization. l = i, r = j # x left and right indices # failure

(2) Valid Test. If l > r then k = 0, Stop

r (3) Find midpoint. k := l+ 2

(4) Key Test. If x0 = sk then Stop and return (5) Update indices and iput. If x0 < sk then j = k 1 Else i = k + 1 L := L[i..j ] (6) Recursive call: BS (L1 ) Binary Search Analysis # that means x0 > sk

# succeeded

# try searching in smaller part

Dene the worst case computing time of BS to be the worst number of times an the algorithm is called for a sequence of n items. DIY to see that: an = 1 + a n then prove that BS has complexity in the class O(lg n). 2 The end of Week 3: 4/1/2008.

12

NGUYEN V.M. MAN

2.5. Generating functions. 2.5.1. Introduction. Generating functions are important in handling stochastic processes involving integral-valued random variables. 2.5.2. Elementary results. Suppose we have a sequence of real numbers a0 , a1 , a2 , . . . Introducing the dummy variable x, we may dene a function

(2.2)

A(x) = a0 + a1 x + a2 x2 + =
j =0

aj xj .

If the series converges in some real interval x0 < x < x0 , the function A(x) is called the generating function of the sequence {aj }. Fact 1. If the sequence {aj } is bounded by some cosntant K , then A(x) converges at least for |x| < 1 [Prove it!] Fact 2. In case of the sequence {aj } represents probabilities, we introduce the restriction

aj 0 ,
j =0

aj = 1 .

The corresponding function A(x) is then a probability-generating function. We consider the (point) probability distribution and the tail probability of a random variable X , given respectively by Prob{X = j } = pj , then the usual distribution function is P { X j } = 1 qj . The probability-generating function now is P (x) =
j =0

P { X > j } = qj ,

pj xj = E(xj ),

E indicates the expectation operator.

Finding a generating function from a recurrence. Multiply both sides by xn . Example: Fibonacci sequence

MATHEMATICAL FOUNDATION FOR ALGORITHM

13

fn = fn1 + fn2 = F (x) = x + xF (x) + x2 F (x) Finding a recurrence from a generating function. Whenever you know F (x), we nd its power series P , the coecicents of P before xn are Fibonacci numbers. How? Just remember how to nd partial fractions expansion of F (x), in particular a basic expansion 1 = 1 + x + 2 x2 + 1 x In general, if G(x) is a generating function of a sequence (gn ) then

G(n) (0) = n!gn

2.5.3. A liitle Random Variable. We mentioned above a random variable X before giving the (point) probability distribution, determined by Prob{X = j } = pj . Actually a random variable is a mapping! More precisely, Denition 3. A random variable X is a function from a set - sample space S to the reals R. For any b R, the preimage A := X 1 (b) = {w : X (w) = b} S is an event, then we understand

Prob{X = b} = Prob(A) =
w A

Prob(w).

For nite set - sample space S then obviously Prob{X = b} = Prob(A) = For a pair of random variables X, Y : S R, we say Denition 4. X and Y are independent if for all x, y R, we have |A| . |S |

14

NGUYEN V.M. MAN

Prob({X = x} {Y = y }) = Prob{X = x} Prob{Y = y } or in terms of conditional probability

Prob({X = x}|{Y = y }) = Prob{X = x}. This denition can be quite extended to the so-called mutually independent of a nite number n random variables. The expectation operator denes the expected value of a random variable X as Denition 5. E(X ) =
xRange(X )

Prob{X = x} x

If we consider X is a function from a sample space S to the naturals N, then E(X ) =

i=0

Prob{X > i}.(W hy ?)

Theorem 6. We have two important results of expectation. (1) (Linearity) E(X + Y ) = E(X ) + E(Y ) for any pair of random variables X, Y (2) (Independence) E(X Y ) = E(X ) E(Y ) for any pair of independent random variables X, Y 2.5.4. Random Variable: Application 1 in Stock Price Dynamics. Any random variable X dened on a sample space S can be looked at the other way. Let F = P (S ), each random variable X : S R is an F -measurable function in the sense that X 1 is a function from B to F where B is the Borel -algebra on R. (The Borel -algebra B on R is the -algebra generated by the open intervals of the reals R). 2.5.5. Convolutions. 2.5.6. Compound distributions. 2.5.7. Partial fraction expansions. 2.5.8. Moment and cummulant generating functions. The end of Week 4: 7/1/2008.

MATHEMATICAL FOUNDATION FOR ALGORITHM

15

3. Recursive Algorithms WEEK 5: 14/1/2008 3.1. Introduction. Consider two distinct ways to dene f (n) = n!, if n is a positive integer: 1/ Nonrecursive: f (n) is the product of all the whole numbers from 1 to n, and 2/ Recursive: if n = 1 then f (n) = 1, otherwise f (n) = n.f (n 1). The basic machinery we employ in the second method in fact is mathematical induction. Based on this powerful technique, we observe few more hallmarks of a recursive procedure as follows: it calls itself with arguments usually smaller than before, although there are no visible loops in the recursive routine; the thought processes are helpful. The general scheme or the overall structure of a recursive procedure. Algorithm 2 The overall structure of a recursive procedure Input: L, a list of variables. Output: f (u) where u L. function RecursiveCalculate( L) (1) if trivial case then do trivial thing (2) else do call RecursiveCalculate( L1 ), L1 consists of smaller values of the variables may be do a few more things end function

In the next part, we will work out a number of important examples of Recursive Algorithms, to see how the recursive structure helps us to analyze the running time or complexity. 3.2. Recursive Graph Algorithms. Introduction. The major distinction between Graph Algorithms and Sorting Algorithms is that Graph Algorithms seem to need exponential amounts of computing time. 3.2.1. The maximum independent set problem. Given a graph G = (V, E ), an independent set of vertices of G is a subset S V of vertices no two of which are connected by an edge of G. In Figure 1, for instance, S = {1, 2, 6} is an independent set; and S0 = {1, 2, 3, 6} is a maximum independent set in term of cardinality.

16

NGUYEN V.M. MAN

(G)
Figure 1. An independent set of G The problem of nding the size (i.e., cardinality) of the largest independent set in a given graph is computationally very dicult. All algorithms known up to date require exponential amounts of time, in their worst cases, although no one has proved the nonexistence of fast algorithms. We will try a recursive approach that is easy to understand and to program, but still needs a long time to run on a large graph G

A recursive algorithm to compute the largest independent set of G. We denote the size of the largest independent set of G by maxset (G). To apply the recursive thinking, we have to split the input, being the vertices or the edges of G, some how, to get smaller inputs. Let x some vertex v V , then we can distinguish two kinds of independent sets of G: A/ those contain vertex v , and B/ those dont contain vertex v . Denote by N eib(v ) the neighbor vertices that are connected to v by an edge.

* An independent set S of type A must consists of v , and more importantly, together with an independent set SA of vertices from the graph GA := G \ {v } \ N eib(v ). * An independent set S of type B doesnt contain vertex v , now is simply an independent set SB in the smaller graph GB := G \ {v }. The two graphs GA and GB must really have smaller

MATHEMATICAL FOUNDATION FOR ALGORITHM

17

vertex sets than the vertex set V of G, therefore we could nd two numbers maxset (GA ) and maxset (GB ). Then, maxset (G) = max { maxset (GB ), 1 + maxset (GA ) }. A recursive procedure to compute maxset (G). Algorithm 3 Computing maxset (G) Input: G = (V, E ), a nite graph. Output: maxset (G) function CalculateMaxIndSet( G) (1) if E = then (2) maxset (G) := |V (G)| else (3i) choose some nonisolated vertex v of G (3ii) nA := CalculateM axIndSet(GA ) where GA := G \ {v } \ N eib(v ) (3iii) nB := CalculateM axIndSet(GB ) where GB := G \ {v } maxset (G) = max(nA + 1, nB ). end function

Example 7. Let G be a 5-cycle [1,2,3,4,5,1], and if we select v = 1, the rst vertex, then N eib(v ) = {2, 5}. Now the two graphs GA and GB are:

(G_A)

(G_B)
Figure 2. Two smaller subgraphs of G

Analysis of complexity. We think of the amount of time as a function f (n) of n := |V (G)|, the number of vertices. In fact, on computer, G = (V, E ) is represented by a square (0, 1)-matrix A of order n or by a (0, 1)-list of length
n(n1) . 2

In either case, it turns out that (n2 ) bits describe the graph, and we are going

to estimate the worst running time or complexity f (n) of the number of vertices.

CalculateMaxIndSet(.) in terms of

If F (G) denotes the total amount of computational labor that we do in order to nd maxset (G), then in nontrivial cases, that is E = , F (G) consists of at least two recursive calls on smaller graphs and one computation of the larger of two numbers. Summarizing all, obviously we see that:

18

NGUYEN V.M. MAN

(3.1)

F (G) cn2 + F (GA ) + F (GB ).

Let f (n) := max|V (G)|=n F (G), and maximizing 3.1 over all graphs of n vertices give:

(3.2)

f (n) cn2 + f (n 2) + f (n 1).

W HY ?

Theorem 7. Let a sequence xn satisfy xn+1 xn + xn1 + n2 . Then we have xn C.n for some constants C and asymtotically. Analysis Using this theorem we get that f (n) = O(1.61n ). 3.3. The discrete Fourier transform. Introduction. The Fourier transform is a method of converting from one representation of a polynomial to another. More precisely, it converts the sequence of coecients of the polynomial to the sequence of values of that polynomial at a certain set of points.

MATHEMATICAL FOUNDATION FOR ALGORITHM

19

Part III: Algorithm Design Techniques and Applications. WEEK 6: 21/1/2008

4. Algorithm Design Techniques 4.1. Greedy algorithms. Principle. 4.2. Divide and Conquer (Decomposition). 4.2.1. Principle. 4.2.2. Illustration: Largest and smallest elements in a sequence. See note from 1995, UNS 4.3. Dynamic programming. 4.3.1. Principle. 4.3.2. Illustration: Biological Sequence Alignment. See PM Hungs thesis 4.4. Search in space of states Combinatorial search. 4.4.1. Principle. Combinatorial search: Why and how? 4.4.2. Illustration: Buidling up a column of a balanced factorial design. See T Hiens thesis. 4.5. Parallel computing types and characteristics. 4.5.1. Principles. Data Parallel. Algorithm Parallel. 4.5.2. Illustration: Buidling up a column of a balanced factorial design with Data Parallel.

5. Algorithms in various Applications domains 5.1. Shortest paths in Weighted Graphs.

20

NGUYEN V.M. MAN

5.1.1. The goal. Let G = (V, E, W ) be a positive weighted graph with vertices V , edges E and with the weight mapping W : E R+ . Choose two arbitrary vertices a, z V and let P be the set of all paths P from a to z . Consider the following problem: Find a path P0 P such that W (P0 ) = min {W (P ) : P P} where W (P ) :=
eP

W (e).

5.1.2. Ideas to nd single source shortest path in simple graphs. Suppose explicitly that G is simple with vertices V = {a = v1 , v2 , v3 , . . . , vn }, with weights W (vi , vj ) R+ whenever (vi , vj ) E , and W (vi , vj ) = otherwise. Without loss of generality we x vertex a = v1 , and formulate an iterative algorithm, due to Edsger W. Dijkstra, to nd a shortest path P0 from vertex a = v1 to any vertex z = a. For any vertex v , let L(v ) be its label, measuring the shortest length from a to v . At any step, some have temporary labels and the rest have permanent labels. Of course, at the begining L(a) = 0 (xed or permanent label) and L(v ) = (temporary labels) for v = a. Denote by S V the set of xed label vertices, in the sense that S = {v V : shortest length from a to v = L(v ) < }. Dijkstras algorithm D(G, a, z ) to nd a shortest path P0 from a = v1 to any vertex z = a is based on two basic ideas: (1) Enlarge as much as possible the set S until S consists of z (2) Ensure that whenever we add a vertex u to S then its permanent label L(u) must give the shortest length from the source a to the considered vertex u. Ideas are extracted from the observation that: if P0 = [a, . . . , w, z ] is a shortest path from a to z then for any vertex w appearing earlier than z in P0 then: L(w) < L(z ). We implement ideas by an iterative algorithm, where at each step/iteration we

MATHEMATICAL FOUNDATION FOR ALGORITHM

21

1/ update label L(v ) of every vertex v S to possible smallest weight (up to that iteration): be done by employing the rule L(v ) = min {L(v ), L(w) + W (w, v )} for v S, w S (w, v ) E ; 2/ nd the vertex u S that has L(u) minimal, and enlarge S = S {u}: be done easily since |V \ S | is nite and get smaller after each iteration. We accept the lemma below, its proof can be found in any standard graph book. Lemma 8. If L(u) is the permanent label of vertex u then L(u) is the length of a shortest path from a = v1 to u. 5.1.3. Dijkstras algorithm computes shortest path P0 = [a, . . . , w, z ]. . Algorithm 4 Computing P0 = [a, . . . , w, z ] Input: G = (V, E, W, a, z ), a positive weighted edge, simple, connected graph with a xed source a, and the sink (target) vertex z Output: L(z ) the length of a shortest path and the path P0 = [a, . . . , w, z ] itself function Singlesource-shortest path( G) 1. Initialization (1i) Set L(a) = 0, and L(v ) = for v = a. (1ii) Set S = (the set of xed label vertices) 2. Checking whether we arrive at z ? if z S then stop; output L(z ). 3. Get next vertex (3i) choose vertex u V \ S together with the smallest value of L(u). (3ii) Set S = S {u} 4. Revise labels For each v S , and there is vertex u S such that (u, v ) E , (v connects u), set L(v ) = min {L(v ), L(u) + W (u, v )} Go back line 2. end function

5.1.4. Example. Convention: we circle a xed vertex u (the uncircled vertices have temporary labels) when the algorithm proceeds. Whenever circling a xed vertex v , that vertex has permanent label and does not change value L(v ) at all; we will also label it with the name of the vertex from which it was labeled. This operation allows us to track back the full path P0 to the source a from the sink when the algorithm stops. Hence, at Line 2, when z S , return L(z ) and use the names at labels of vertices to provide the explicit path P0 .

22

NGUYEN V.M. MAN

We change names of vertices v1 , v2 , . . . , v6 to {a, b, c, d, e, z } respectively.

a= v1

000000 111111 1111111111111 00000 00000000 000000 111111 0000011111111 11111 00000000 000000 111111 (G) 00000 11111 00000000 11111111 000000 111111 00000 11111 00000000 11111111 000000 111111 2 2 00000 11111 00000000 11111111 000000 111111 00000 11111 00000000 11111111 000000 111111 00000000 00000 11111 11111111 000000 111111 00000000 00000 11111 11111111 000000 111111 00000 11111 00000000 11111111 000000 111111 00000 11111 00000000 11111111 1 000000 111111 000000 111111 000000 111111 00000 11111 00000000 11111111 000000 111111 000000 111111 00000000 11111111 000000 111111 000000 111111 00000000 11111111 000000 111111 000000 111111 00000000 11111111 000000 111111 000000 111111 00000000 11111111 000000 111111 1 2 000000 111111 00000000 11111111 000000 111111 000000 111111 00000000 11111111 000000 111111 000000 111111 00000000 11111111 000000 111111 000000 111111 00000000 11111111 000000 111111 111111111 000000000 000000 111111 00000000 11111111 000000 00000000 11111111 111111
d=v4 4 e=v5
Figure 3. Shortest path in a weighted graph G

b= v2

c = v3

z=v6

Initialization. Set L(a) = 0, circle a, and set L(v ) = for v = a. Set S = [] (empty list): no circled vertex has been found yet. Iteration 1: Line 2: Checking z S . No, z still is not circled. Line 3: Get next vertex. We select u V \ S = V together with the smallest value of L(u): only a is the candidate. Update S = S {a} = [a], cirle a. Line 4: Revise labels of vertices v S connecting to the last xed vertex in S . We label the vertices b and d adjacent to a S , obtain L(b) = min {L(b), L(a) + W (a, b)} = min {, 0 + 2} = 2, L(d) = min {L(d), L(a) + W (a, d)} = min {, 0 + 1} = 1, Iteration 2: Line 2: Checking z S . No, z still is not circled. Line 3: Get next vertex. We select u V \ S (or equivalently u S ) having the smallest value L(u): d is the candidate since L(d) < L(b). Update S = S {d} = [a, d], cirle d Line 4: Revise labels of vertices v S that connect to the last xed vertex in S . We label the vertex e adjacent to d, obtain the new label assign (a, 2) to b assign (a, 1) to d

MATHEMATICAL FOUNDATION FOR ALGORITHM

23

L(e) = min {L(e), L(d) + W (d, e)} = min {, 1 + 4} = 5, Iteration 3: Line 2: Checking z S . No, z still is not circled.

assign (d, 5) to e

Line 3: Get next vertex. For all u S choose the one having label smallest: b is the right candidate since L(b) = 2 < 5 = L(e). Update S = S {b} = [a, d, b], cirle b. Line 4: Revise labels of v S that connect to the last xed vertex in S . The vertices c, e both are adjacent to b, so we obtain the new labels L(c) = min {L(c), L(b) + W (b, c)} = min {, 2 + 7} = 9, L(e) = min {L(e), L(b) + W (b, e)} = min {5, 2 + 1} = 3, assign (b, 9) to c assign (b, 3) to e

Note that e has been just updated its label from (d, 5) to (b, 3), a better improvement! Iteration 4: Line 2: Checking z S . No, z still is not circled. Line 3: Get next vertex. For u S having L(u) smallest, then u = e is the best candidate since L(e) = 3 < 9 = L(c). Update S = S {e} = [a, d, b, e], cirle e. Line 4: Revise labels of v S and connect to the last xed vertex in S . The vertex z now is adjacent to e, obtain the new label L(z ) = min {L(z ), L(e) + W (e, z )} = min {, 3 + 2} = 5, assign (e, 5) to z.

Update S = S {z }. Go back line 2, stop. The explicit path P0 = [a, b, e, z ] from a to z is listed from the inverse trail of labels: [z (e, 5) (b, 3) (a, 2)], the shotrest length is L(z ) = 5.

5.1.5. Matrix form of Dijkstras algorithm. When you understand the algorithm, you can nd the answer by using the following Table 5.1.5. At Step 5, c has been just updated its label from (b, 9) to (z, 7), a better improvement!. Note 1 : if you want to nd shortest paths from source a to every vertex in graph, then you proceed computing the above table until no vertex occurs in the nal column. If this is the case, then the nal row of the mentioned table provides all shortest path from a to every vertex in graph.

24

NGUYEN V.M. MAN

Step

Vertex to be circled a 1 a 0 2 d 0 3 b 0 4 e 0 5 (nal) z 0 6 (extra) c 0 Table 1. Step by step

b c 2 2 2 9 2 9 2 7 2 7 runing

d e z 1 1 5 1 3 1 3 5 1 3 5 1 3 5 of Dijkstras

Uncircled vertices b, d, c, e, z b, c, e, z c, e, z c, z c algorithm

5.1.6. Complexity analysis of Dijkstras algorithm. Theorem 9. Let G = (V, E, W ) be a positive weighted edge, simple, connected graph having n vertices. If f (n) is the number of times Algorithm 5.1.3 examines an edge of G in the worst case, then f (n) = O(n2 ). 5.2. In Numerical Computation. Multiplication of matrices 5.3. In Scheduling Problems. Optimal paths in directed graphs 6. Conclusion

References [1] Morris W. Hisch, Stephen Smale, Dierential Equations, Dynamical Systems and Linear Algebra, 1980 [2] Robert Sedgewick Algorithms, Princeton, 1990 [3] T. Cormen, C. Leiserson, R. Rivest Introduction to Algorithms, MIT, 1997

Copyright in 2008 by Lecturer Working group Faculty Institution Address Email: Nguyen V.M. Man, Ph.D. in Algebraic Statistics (TU Eindhoven) Department of Computational Sciences Computer Science and Engineering University of Technology of HCMC 268 Ly Thuong Kiet, Dist. 10, HCMC, Vietnam mnguyen@designcomputing.net mnguyen@cse.hcmut.edu.vn