Asymptotic

Asymptotic Notation,
Notation,
Review
Review of
of Functions
Functions &
&
Summations
Summations
By, Nikhilesh Walde
CSE Department
PIET

Thursday, November 0 Comp 122, Spring 2004

4, 2021
 Asymptotic Complexity
 Running time of an algorithm as a function of
input size n for large n.
 Expressed using only the highest-order term in
the expression for the exact running time.
 Instead of exact running time, say Q(n2).
 Describes behavior of function in the limit.
 Written using Asymptotic Notation.

ymp - 2 Comp 122

 Asymptotic Notation
 Q, O, W, o, w
 Defined for functions over the natural numbers.
 Ex: f(n) = Q(n2).
 Describes how f(n) grows in comparison to n2.
 Define a set of functions; in practice used to compare
two function sizes.
 The notations describe different rate-of-growth
relations between the defining function and the
defined set of functions.

ymp - 3 Comp 122

 -notation
For function g(n), we define (g(n)),
big-Theta of n, as the set:
(g(n)) = {f(n) :
 positive constants c1, c2, and n0,
such that n  n0,
we have 0  c1g(n)  f(n)  c2g(n)

}
Intuitively: Set of all functions that
have the same rate of growth as g(n).

g(n) is an asymptotically tight bound for f(n).

ymp - 4 Comp 122

 -notation
For function g(n), we define (g(n)),
big-Theta of n, as the set:
(g(n)) = {f(n) :
 positive constants c1, c2, and n0,
such that n  n0,
we have 0  c1g(n)  f(n)  c2g(n)

}
Technically, f(n)  (g(n)).
Older usage, f(n) = (g(n)).
I’ll accept either…

f(n) and g(n) are nonnegative, for large n.

ymp - 5 Comp 122
 Example
(g(n)) = {f(n) :  positive constants c1, c2, and n0,
such that n  n0, 0  c1g(n)  f(n)  c2g(n)}

 10n2 - 3n = Q(n2)
 What constants for n0, c1, and c2 will work?
 Make c1 a little smaller than the leading
coefficient, and c2 a little bigger.
 To compare orders of growth, look at the
leading term.
 Exercise: Prove that n2/2-3n= Q(n2)
ymp - 6 Comp 122
 O-notation
For function g(n), we define O(g(n)),
big-O of n, as the set:
O(g(n)) = {f(n) :
 positive constants c and n0,
such that n  n0,
we have 0  f(n)  cg(n) }
Intuitively: Set of all functions
whose rate of growth is the same as
or lower than that of g(n).
g(n) is an asymptotic upper bound for f(n).
f(n) = (g(n))  f(n) = O(g(n)).
(g(n))  O(g(n)).
ymp - 7 Comp 122
  -notation
For function g(n), we define (g(n)),
big-Omega of n, as the set:
(g(n)) = {f(n) :
 positive constants c and n0,
such that n  n0,
we have 0  cg(n)  f(n)}
Intuitively: Set of all functions
whose rate of growth is the same
as or higher than that of g(n).
g(n) is an asymptotic lower bound for f(n).
f(n) = (g(n))  f(n) = (g(n)).
(g(n))  (g(n)).
ymp - 8 Comp 122
 Relations Between Q, O, W

ymp - 9 Comp 122

 Relations Between Q, W, O
Theorem
Theorem :: ForFor any
any two
two functions
functions g(n)
g(n) and
and f(n),
f(n),
f(n) == (g(n))
f(n) (g(n)) iff
iff
f(n)
f(n) == O(g(n))
O(g(n)) and f(n) == (g(n)).
and f(n) (g(n)).

 I.e., (g(n)) = O(g(n)) Ç W(g(n))

 In practice, asymptotically tight bounds are

obtained from asymptotic upper and lower bounds.

ymp - 10 Comp 122

 Asymptotic Notation in Equations
 Can use asymptotic notation in equations to
replace expressions containing lower-order terms.
 For example,
4n3 + 3n2 + 2n + 1 = 4n3 + 3n2 + (n)
= 4n3 + (n2) = (n3). How to interpret?
 In equations, (f(n)) always stands for an
anonymous function g(n) Î (f(n))
 In the example above, (n2) stands for
3n2 + 2n + 1.

ymp - 11 Comp 122

 o-notation
For a given function g(n), the set little-o:
o(g(n)) = {f(n):  c > 0,  n0 > 0 such that
 n  n0, we have 0  f(n) < cg(n)}.
f(n) becomes insignificant relative to g(n) as n
approaches infinity:
lim [f(n) / g(n)] = 0
n

g(n) is an upper bound for f(n) that is not

asymptotically tight.
Observe the difference in this definition from previous
ones. Why?
ymp - 12 Comp 122
 w -notation
For a given function g(n), the set little-omega:

w(g(n)) = {f(n):  c > 0,  n > 0 such that

0
 n  n0, we have 0  cg(n) < f(n)}.
f(n) becomes arbitrarily large relative to g(n) as n
approaches infinity:
lim [f(n) / g(n)] = .
n

g(n) is a lower bound for f(n) that is not

asymptotically tight.

ymp - 13 Comp 122

 Longest Common Subsequence
(LCS)
 In biological application, to compare the DNA
of two or more different organism
 A strand of DNA consist of string of molecules

called bases
 A strand of DNA can be expressed over the
finite set { A,C,G,T}

Thursday, November 0 Comp 122, Spring 2004

4, 2021
 Longest Common Subsequence (LCS)
 Example:
 S1 = ACCGGTCGAGTGCGCGGAAGCCGGCCGAA
 S2 = GTCGTTCGGAATGCCGTTGCTCTGTAAA
 FIND S3, in which the bases in S3 appear in each
of S1 and S2.
 The bases must appear in the same order but not
necessarily consecutively.
 S3 = GTCGTCGGAAGCCGGCCGAA---- LCS

ymp - 15 Comp 122

 Longest Common Subsequence (LCS)
 Example :
 X = ( A, B, C, B, D, A, B)
 Y = ( B, D, C, A, B, A)
 Z1 = (B, C, A)
 Z2 = (A, B, A)
 Z3 = (B, C, B, A)

ymp - 16 Comp 122

 LCS_ LENGTH (X,Y)
 1 m length [X]
 2 n  length [Y]
 3 for i  1 to m
4 do c[i,0]  0
 5 for j  0 to n
6 do c[0,j]  0

ymp - 17 Comp 122

 LCS_ LENGTH (X,Y)
 7 for i  1 to m
 8 do for j  1 to n
 9 do if Xi = Yi
 10 then c [i, j]  c [i-1,j-1] + 1
 11 b[i, j]  “ Inclined Arrow”
 12 else if c [ i-1, j] >= c [ i, j-1]
 13 c [i, j]  c [i-1, j]
 14 b[i, j]  “ Upward Arrow”
 15 else c [i, j]  c [i, j-1]
 16 b[i, j]  “ Leftward Arrow”
 17 Return c and b

ymp - 18 Comp 122

 Print_LCS (b, X, i, j)
 1 if i = 0 or j = 0
 2 then return
 3 if b[i, j] = “ Inclined arrow”
 4 then Print_LCS(b, X, i-1, j-1)
 5 Print Xi
 6 elseif b[i, j] = “ upward Arrow”
7 then Print_LCS(b, X, i-1, j)
 8 else Print_LCS(b, X, i, j-1)

ymp - 19 Comp 122

 Matrix Chain Multiplication
 Given a sequence (chain) < A1,A2,A3….An>
 We wish to find the product of n matrices.
 Use a standard algorithm for multiplying pair
of matrices as a subroutine once we have
parenthesized them.
 For e.g. <A1, A2, A3, A4>
 The Product can be fully parenthesized in five
different ways:

ymp - 20 Comp 122

 Matrix Chain Multiplication
 (A1 (A2(A3A4)))
 (A1 ((A2A3 )A4))
 ((A1A2)(A3A4))
 ((A1 (A2A3))A4)
 ((A1A2)A3) A4)
 Matrix multiplication is associative
 All Parenthesization yield the same product.

ymp - 21 Comp 122

 Matrix_Multiply (A,B)
 1 if column [A] <> row [B]
2 then error “ multiplication cannot perform”
 3 else for i  1 to row [A]
4 do for j  1 to column [B]
5 do C[i , j]  0
6 for k  1 to row [B]
7 do C[i,j]  C[i,j]+ A[i,k]* B[k,j]
8 Return C

ymp - 22 Comp 122

 Matrix Chain Multiplication
 If A[p,q] and B[q,r] then resultant matrix C[p,r]
 The time to compute C is dominated by the
number of scalar multiplication in Line 7
 Which is p x q x r.
 Different cost is incurred by different
parenthesization of a matrix product.
 For e.g. A1 = 10 x 100, A2 = 100 x 5,
A3 = 5 x 50
 ((A1.A2).A3) and (A1.(A2.A3))

ymp - 23 Comp 122

 Matrix Chain Multiplication
 Matrix Dimension
 A1 30 x 35
 A2 35 x 15
 A3 15 x 5
 A4 5 x 10
 A5 10 x 20
 A6 20 x 25

ymp - 24 Comp 122

 Matrix_Chain_Order (p)
 1 n  length[p] – 1
 2 for i  1 to n
3 do m[i, i]  0
 4 for £  2 to n // £ is the chain length.
5 do for i  1 to n - £ +1
6 do j  i + £ -1
7 m[i, j]  infinity

ymp - 25 Comp 122

 Matrix_Chain_Order (p)
8 for k  i to j – 1
9 do q  m[i,k] + m[k+1,j] + Pi-1 PkPj
 10 if q < m[i, j]
 11 then m[i, j]  q
 12 s[i, j]  k
 13 return m and s

ymp - 26 Comp 122

 Print_Optimal _Parens(s,i,j)
1 if i = j
2 then print “A”
3 else Print “ ( ”
4 Print_Optimal_Parens (s,i,s[i,j])
5 Print_Optimal_Parens (s,s[i,j]+1, j)
6 Print “ )”

 Output = ((A1(A2A3))((A4A5)A6)).

ymp - 27 Comp 122

 Assembly Line Scheduling
 Dynamic programming solves a manufacturing
problem
 The Colonel Motors Corporation produces
automobiles in a factory that has two assembly
lines
 An automobile chassis enters each assembly
line, has parts added to it at a number of
stations, and a finished auto exits at the end of
the line

ymp - 28 Comp 122

 Assembly Line Scheduling

ymp - 29 Comp 122

 Assembly Line Scheduling
 Each assembly line has n stations, numbered
j = 1, 2, ..., n. We denote the jth station on line i
(where i is 1 or 2) by Si,j.
 The jth station on line 1 (S1,j) performs the
same function as the jth station on line 2 (S2,j).
 The stations were built at different times and
with different technologies, however, so that the
time required at each station varies, even
between stations at the same position on the two
different lines
ymp - 30 Comp 122
 Assembly Line Scheduling
 Chassis enters station 1 of one of the assembly
lines, and it progresses from each station to the
next.
 Entry time ei for the chassis to enter assembly
line i and an exit time xi for the completed auto
to exit assembly line i.
 Find the fastest way through a factory
 After going through the jth station on a line, the
chassis goes on to the (j + 1)st station on either
line.
ymp - 31 Comp 122
 Assembly Line Scheduling
 There is no transfer cost if it stays on the same
line, but it takes time ti,j to transfer to the other
line after station Si,j.
 The problem is to determine which stations to
choose from line 1 and which to choose from
line 2 in order to minimize the total time
through the factory for one auto.
 Generally, It is used in rush Order.

ymp - 32 Comp 122

 Assembly Line Scheduling

ymp - 33 Comp 122

 Assembly Line Scheduling

ymp - 34 Comp 122

 Assembly Line Scheduling

ymp - 35 Comp 122

 FASTEST-WAY(a, t, e, x, n)
1 f1[1] ← e1 + a1,1
2 f2[1] ←e2 + a2,1
3 for j ← 2 to n
4 do if f1[j - 1] + a1,j ≤ f2[j - 1] + t2,j-1 + a1,j
5 then f1[j] ← f1[j - 1] + a1, j
6 l1[j] ← 1
7 else f1[j] ← f2[j - 1] + t2,j-1 + a1,j
8 l1[j] ← 2

ymp - 36 Comp 122

 FASTEST-WAY(a, t, e, x, n)
9 if f2[j - 1] + a2,j ≤ f1[j - 1] + t1,j-1 + a2,j
 10 then f2[j] ← f2[j - 1] + a2,j
 11 l2[j] ← 2
 12 else f2[j] ∞ f1[j - 1] + t1,j-1 + a2,j
 13 l2[j] ← 1
 14 if f1[n] + x1 ≤ f2[n] + x2
 15 then f* = f1[n] + x1
 16 l* = 1
 17 else f* = f2[n] + x2
 18 l* = 2
ymp - 37 Comp 122
 PRINT-STATIONS(l, n)
1 i ← l*
2 print "line " i ", station " n
3 for j ← n downto 2
4 do i ← li[j]
5 print "line " i ", station " j - 1

ymp - 38 Comp 122

 Assembly Line Scheduling
 PRINT-STATIONS would produce the output
 line 1, station 6
 line 2, station 5
 line 2, station 4
 line 1, station 3
 line 2, station 2
 line 1, station 1

ymp - 39 Comp 122

 Huffman codes
 Huffman codes are a widely used and very
effective technique for compressing data.
 Savings of 20% to 90% memory, depending on
the characteristics of the data being compressed.
 We consider the data to be a sequence of
characters.
 Huffman's greedy algorithm uses a table of the
frequencies of occurrence of the characters to
build up an optimal way of representing each
character as a binary string.
ymp - 40 Comp 122
 Huffman codes
 Suppose we have a 100,000-character data file
that we wish to store compactly.
 We observe that the characters in the file occur
with the frequencies given by Figure.
 That is, only six different characters appear, and
the character a occurs 45,000 times.
 If each character is assigned a 3-bit Fix Length
codeword,the file can be encoded in 300,000
bits.
 Using the variable-length code shown, the file
ymp - 41 Comp 122
 Huffman codes
 a b c d e f
 Frequency (in thousands) 45 13 12 16 9 5
 Fixed-length codeword 000 001 010 011 100 101
 Variable-length codeword 0 101 100 111 1101 1100

ymp - 42 Comp 122

 Fixed-length codeword

ymp - 43 Comp 122

 Variable-length codeword

ymp - 44 Comp 122

 Huffman codes

ymp - 45 Comp 122