Algorithm Analysis: Running Time Big O and omega ()

A.R. Hadaegh Dr. Ahmad R. Hadaegh

National University

Page

Introduction
An algorithm analysis of a program is a step by step procedure for accomplishing that program In order to learn about an algorithm, we need to analyze it This means we need to study the specification of the algorithm and draw conclusion about the implementation of that algorithm (the program) will perform in general

A.R. Hadaegh Dr. Ahmad R. Hadaegh

National University

Page

 The issues that should be considered in analyzing an algorithm are: The running time of a program as a function of its inputs The total or maximum memory space needed for program data The total size of the program code hether the program correctly computes the desired result

The complexity of the program! "or example, how easy it is to read, understand, and modify the program The robustness of the program! "or example, how well does it deal with unexpected or erroneous inputs
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 3

 In this course, we consider the running time of the algorithm! The main factors that effect the running time are the algorithm itself, input data, the computer system, etc! The performance of a computer is determined by The hardware The programming language used and The operating system To calculate the running time of a general #$$ program, we first need to define a few rules! In our rules , we are going to assume that the effect of hardware and software systems used in the machines are independent of the running time of our #$$ program

A.R. Hadaegh Dr. Ahmad R. Hadaegh

National University

Page

Rule 1:
The time re%uired to fetch an integer from memory is a constant t(fetch), The time re%uired to store an integer in memory is also a constant t(store) "or example the running time of x y is: t(!etch" # t(store" because we need to fetch y from memory store it into x &imilarly the running time of x 1 is also t(!etch" # t(store" because typically any constant is stored in the memory before it is fetched!
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 5

Rule 2
The time re%uired to perform elementary operations on integers, such as addition t($), subtraction t('), multiplication t((), di)ision t(*), and comparison t(cmp), are all constants!

"or +xample the running time of y x#1 is: 2t(!etch" # t(store" # t (#" because you need to fetch x and ,: then add them together: and place the result into y:
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University

-(t(fetch) t($) t(store)

Page $

Rule 3:
The time re%uired to call a function is a constant, t(call) And the time re%uired to return a function is a constant, t(return)

Rule 4:
The time re%uired to pass an integer argument to a function or procedure is the same as the time re%uired to store an integer in memory, t(store)

A.R. Hadaegh Dr. Ahmad R. Hadaegh

National University

Page

 "or example the running time of y !(x" is: t(!etch" # 2t(store" # t(call" # t(!(x"" .ecause you need To fetch the )alue of x: /ass x to the function and store it into parameter: #all the function f(x): 0un the function: &tore the returned result into y:

t (fetch) t (store) t (call) t (f(x)) t (store)

A.R. Hadaegh Dr. Ahmad R. Hadaegh

National University

Page

&

Rule 5:
The time re%uired for the address calculation implied by an array subscripting operation li1e a2i3 is a constant, t(2 3)! This time does not include the time to compute the subscript expression, nor does it include the time to access (fetch or store) the array element "or example, the running time of y a(i) is: 3t(!etch" # t(( )" # t(store" .ecause you need To fetch the )alue of i: To fetch the )alue of a: To find the address of a2i3: To fetch the )alue of a2i3: To store the )alue of a2i3 into y:
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University

t(fetch) t(fetch) t(2 3) t(fetch) t(store)

Page '

Rule $:
The time re%uired to calculate a fixed amount of storage from the heap using operator new is a constant, t(new) This time does not include any time re%uired for initialization of the storage (calling a constructor)! &imilarly, the time re%uired to return a fixed amount of storage to the heap using operator delete is a constant, t(delete)! This time does not include any time spent cleaning up the storage before it is returned to the heap (calling destructor)
"or example the running time of "or example the running time of int+ ,tr ne- int. delete ,tr. is: is: t(ne-" # t(store" t(!etch " # t(delete" .ecause you need .ecause you need To allocate a memory: t(new) To fetch the address from ptr : t(fetch) And to store its address into ptr: t(store) And delete specifies location: t(delete)
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 1*

,! int &um (int n) -! 4 5! int result 678 9! for (int i6,8 i:6n8 i6i$,) ;! result 6 result $ i8 <! return result =! >
/tatement Time 5 t(fetch)$ t(store) 9a 9b 9c ; < t(fetch) $ t(store) (-t(fetch)$t(cmp)) ( (n$,) (-t(fetch)$t($) $t(store)) (n (-t(fetch)$t($) $t(store)) (n t(fetch)$ t(return) 0ode result 6 7 i6, i:6n $$i 0esult $6i 0eturn result

Total
A.R. Hadaegh Dr. Ahmad R. Hadaegh

2<t(fetch) $-t(store) $ t(cmp) $ -t($)3(n $ 2;t(fetch) $ -t(store) $ t(cmp) $ t(return) 3

National University Page 11

,! int func (int a2 3, int n, int x) -! 4 5! int result 6 a2n38 9! for (int i6n',8 i?678 i6i',) ;! result 6result (x $ a2i38 <! return result =! >
/tatement Time

5 9a 9b 9c ; <

5t(fetch)$ t(2 3) $ t(store) -t(fetch) $ t(') $ t(store) (-t(fetch)$t(cmp)) ( (n$,) (-t(fetch)$t(') $t(store)) (n (;t(fetch)$t(2 3)$t($)$t(()$t(store)) (n t(fetch)$ t(return)

Total

2(@t(fetch) $-t(store) $ t(cmp) $t(23) $ t(() $ t(')3(n $ 2(At(fetch) $ -t(store) $ t(23) $ t(') $t(cmp) $ t (return) )3
National University Page 12

A.R. Hadaegh Dr. Ahmad R. Hadaegh

 Bsing constant times such as t(fetch), t(store), t(delete), t(new), t($), C, ect ma1es our running time accurate Dowe)er, in order to ma1e life simple, we can consider the approximate running time of any constant to be the same time as t(,)! "or example, the running time of y x#1 is 5 because it includes two EfetchesF and one EstoreF in which all are constants "or a loop there are two cases: If we 1now the exact number of iterations, the running time becomes constant t(,) If we do not 1now the exact number of iterations, the running time becomes t(n) where n is the number of iterations
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 13

,! int Geometric (int x, int n) -! 4 5! int sum 6 78 9! for (int i678 i:6n8 $$i) ;! 4 <! int prod 6 ,8 =! for (int H678 H:i8 $$H) A! prod 6 prod ( x8 @! sum 6 sum $ prod8 ,7! > ,,! return result ,-! >

/tatement 5 9a 9b 9c < =a =b =c A @ ,,

Time 5(n$-) 9(n$,) -(n$,) -(n$,) n 5 i67 n 9 i67 n 9 i67 9(n$,) -

i$, i i

Total

(,,*-)n- $ (9=*-)n $ -=

A.R. Hadaegh Dr. Ahmad R. Hadaegh

National University

Page

14

,! int /ower (int x, int n) -! 4 5! if (n6 67) 9! return ,8 ;! else if (nI- 6 6 7) ** n is e)en <! return /ower (x(x, n*-)8 =! else ** n is odd A! return x( /ower (x(x, n*-)8 @! >
/tatemen t 5 9 ; < A n * 5 ' ' ' n1* (n is e2en" 5 ' ; ,7 $ T( n*- ) ' n1* (n is odd" 5 ' ; ' ,- $ T( n*- )

Total
A.R. Hadaegh Dr. Ahmad R. Hadaegh

,A $ T( n*- )
National University

-7 $ T( n*- )
Page 15

T(n"

5 1&#T( n32 " 2*#T( n32 "

!or n * !or n1* and n is e2en !or n1 * and n is odd

&uppose n 6 -1 for some 1?7! Jb)iously -1 is an e)en number, we get T(24" 1& # T(2451" 1& # (1& # T(2452"" 1& # 1& # (1& # T(2453"" 677 677 1&4 # T(2454" 1&4 # T(2*" 1&4 # T(1"

A.R. Hadaegh Dr. Ahmad R. Hadaegh

National University

Page

1$

 &ince T(,) is add number, the running time of T(,) is: T(1" 2* # T(*" 2* # 5 25 Therefore,

T(24"

1&4 # 25

If n 6 -1, then log n 6 log-1 indicating that 1 6 log n Therefore,

T(n"

1&log n # 25

A.R. Hadaegh Dr. Ahmad R. Hadaegh

National University

Page

1%

Asym,totic 8otation
&uppose the running time of two algorithms A and . are TA(n) and T.(n), respecti)ely where n is the size of the problem Dow can we determine TA(n) is better than T.(n)K Jne way to do that is if we 1now the size n ahead of time for some n=no! Then we may say that algorithm A is performing better than algorithm . for n= no .ut this is a special case for n=no! Dow about n = n1, or n=n2K Is A better than . for other cases tooK Bnfortunately, this is not an easy answer! e cannot expect that the size of n to be 1nown ahead of time! .ut we may be able to say that under certain situations TA(n) is better than T.(n) for all n ?6 n,
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 1&

 To understand the running times of the algorithms we need to ma1e some definitions: 9e!inition: #onsider a function f(n) which is non'negati)e for all integers n?67! e say that f(n) is big oh of g(n) which we write (f(n) is !(g(n)) if there exists an integer no and a constant c ? 7 such that for all integers n ?6no, f(n) :6c g(n) :xam,le: &how that f(n) 6 An $ ,-A is J(n-) An $ ,-A :6 c n- (lets set c 6 ,) 7 :6 cn- 'An ',-A 7 :6 (n',<) (n$A) Thus we can say that for constant c 6, and n ?6 ,<, f(n) is A.R. Hadaegh National University Dr. Ahmad R. Hadaegh J(n )

Page

1'

f(n) g,(n)69ng-(n)6-n-

g5(n)6n-

977 f(n)6An$,-A -77

,7

,;

-7

A.R. Hadaegh Dr. Ahmad R. Hadaegh

National University

Page

2*

Theorem: If f,(n) is J(g,(n)) and f-(n) is J(g-(n)), then f,(n) $ f-(n) 6 J (max(g,(n), g-(n))) ;roo!: If f,(n) is J(g,(n)) then f,(n) :6 c,g,(n) for some c, and n ?6 n, If f-(n) is J(g-(n)) then f-(n) :6 c-g-(n) for some c- and n ?6 n Let no6 max(n,, n-) and co 6 -(max(c,, c-)), consider the sum f,(n) $ f-(n) for some n ?6 no f,(n) $ f-(n) :6 :6 :6 c,g,(n) $ c-g-(n) co(g,(n) $ g-(n) )*co (max (g,(n), g-(n))
Page 21

A.R. Hadaegh Dr. Ahmad R. Hadaegh

Therefore, f,(n) $ f-(n) is J (max(g,(n), g-(n)) )

National University

Theorem: If f,(n) is J(g,(n)) and f-(n) is J(g-(n)), then !1(n" + !2(n" O(g1(n"+g2(n" " ;roo!: If f,(n) is J(g,(n)) then f,(n) :6 c,g,(n) for some c, and n?6n, If f-(n) is J(g-(n)) then f-(n) :6 c-g-(n) for some c- and n?6n Let no6 max(n,, n-) and co 6 c,(c-, consider the product of f,(n)(f-(n) for some n?6no f,(n) ( f-(n) :6 :6 c,g,(n) ( c-g-(n) co(g,(n) ( g-(n) )

Therefore, f,(n) ( f-(n) is J (g,(n)(g-(n) )

A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 22

Theorem: If f(n) is J(g(n)) and g(n) is J(h(n)), then !(n" is O(h(n"" ;roo!: If f(n) is J(g(n)) then f(n) :6 c,g(n) for some c, and n?6n, If g(n) is J(h(n)) then g(n) :6 c-h(n) for some c- and n?6n Let no6 max(n,, n-) and co 6 c,(c-, then f(n) :6 :6 :6 c,g,(n) c,c-h(n) co h(n)

Therefore, f(n) is J (h(n))

A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 23

The names o! common <ig O ex,ressions

:x,ression J(,) J (log n) J(log- n) J (n) J (n(log n) J(n-) J(n5) J(-n) 8ame #onstant logarithmic log s%uared Linear nlogn Muadratic #ubic exponential

A.R. Hadaegh Dr. Ahmad R. Hadaegh

National University

Page

24

0on2entions !or -riting Big Oh :x,ression

#ertain con)entions ha)e e)ol)ed which concern how big oh expression normally written: "irst, it is common practice when writing big oh expression to drop all but the most significant items! Thus instead of J(n- $ nlogn $ n) we simply write J(n-) &econd, it is common practice to drop constant coefficients! Thus, instead of J(5n-), we write J(n-)! As a special case of this rule, if the function is a constant, instead of, say J(,7-9), we simply write J(,)

A.R. Hadaegh Dr. Ahmad R. Hadaegh

National University

Page

25

Asym,totic =o-er Bound ( "

9e!inition: #onsider a function f(n), which is non'negati)e for all integers n?67! e say that f(n) is omega of g(n) which we write (f(n) is (g(n))" if there exists an integer no and a constant c ? 7 such that for all integers n?6 no, f(n) ?6 c g(n) :xam,le: &how that f(n) 6 ;n- ' <9n $ -;< is (n-) ;n- ' <9n $ -;< ;n- ' <9n $ -;< ;n- ' <9n $ -;< Nn9n- ' <9n $ -;< 9(n'A)?6 cn- let c6, ?6 n?6 7 ?6 7 ?6 7

Let no 6 A, we can say that for c6, and no?6A f(n) is (n-)
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 2$

,;77 f(n)6;n-'<9n$-;< ,777

f(n)6-n-

f(n)6n;77

,7

,;

-7

-;

A.R. Hadaegh Dr. Ahmad R. Hadaegh

National University

Page

2%

Other de!initions
9e!inition: #onsider a function f(n) which is non'negati)e for all integers n?67! e say that f(n) is theta of g(n) which we write (f(n) is (g(n)) if and only if f(n) is J(g(n)) and f(n) is (g(n)) 9e!inition: #onsider a function f(n) which is non'negati)e for all integers n?67! e say that f(n) is little o of g(n) which we write (f(n) is o(g(n)) if and only if f(n) is J(g(n)) and f(n) is not (g(n)) Oow lets consider some of the pre)ious examples in terms of the big J notations:
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University Page 2&

,! int func (int a2 3, int n, int x) -! 4 5! int result 6 a2n38 9! for (int i6n',8 i?678 ''i) ;! result 6result (x $ a2i38 <! return result =! >
/tatement /im,le Time model 5 ; 9a 9b 9c ; < 9 5n $ 5 9n @n Big O

The total running time is: J(,<n $ ,9) 6 J(max(,<n, ,9)) 6 J(,<n) 6 J(n)

Total
A.R. Hadaegh Dr. Ahmad R. Hadaegh

,<n $ ,9

J(,) J(,) J(n) J(n) J(n) J(,) J(n)

National University

Page

2'

,! int /refix&ums (int a2 3, int n) -! 4 5! for (int H6n',8 i?678 ''H) 9! 4 ;! int sum 6 78 <! for (int i678 i:6H8 $$i) =! sum 6 sum $ a2i38 A! a2H3 6 sum8 @! > ,7! return result ,,! >

/tatement Big O 5a 5a 5c ; <a <b <c = @

Total
A.R. Hadaegh Dr. Ahmad R. Hadaegh National University

J(,) J(,)(J(n) J(,)(J(n) J(,)(J(n) J(,)(J(n) J(,)(J(n-) J(,)(J(n-) J(,)(J(n-) J(,)(J(n) J(n-)
Page 3*