Design and Analysis of

Computer Algorithms
Spring 08
Chuan-Ming Liu
Computer Science & Information Engineering
NTUT, Taiwan
1
Mobile Computing & Software Engineering Lab

Instructor
Chuan-Ming Liu ()
Office: Room 502-3
Computer Science and Information Engineering
National Taipei University of Technology
TAIWAN
Phone: (02) 2771-2171 ext. 4251
Email: cmliu@csie.ntut.edu.tw
Office Hours: Thu 10:10-12:00 and Fri 10:10-12:00
2

Mobile Computing & Software Engineering Lab

Text Books
Introduction to Algorithms, 2nd edition. T. Cormen, C.
Leiserson, R. Rivest, and C. Stein. McGraw-Hill, 2001.

Supplementary Texts
1. Computer Algorithms /C++, 2nd Edition. E. Horowitz, S. Sahni, and S.
Rajasekaran, W.H. Freeman and Company, 1998.
2. Computer Algorithms, 3rd edition. S. Baase and A. V. Gelder. AddisonWesley, 2000.
3. Algorithms in C++, 3rd. R. Sedgewick. Addison-Wesley, 1998
4. Algorithms, R. Johnsonbaugh and M. Schaefer, Pearson Prentice Hall, 2004.
5. Algorithm Design, Jon Kleinberg and va Tardos, Pearson/Addison-Wesley,
2006.
6. Algorithms, Sanjoy Dasgupta, Christos Papadimitriou, Umesh Vazirani, McGrawHill, 2008.

Mobile Computing & Software Engineering Lab

Course Outline

Introduction
Asymptotic analysis
Recurrence Relations
Data Structures
Average and worst analysis
Lower Bounds
Dynamic Programming
Graph algorithms
Search and Traversal Techniques
NP-Completeness
Approximation Algorithms
New Trends in Algorithms

Mobile Computing & Software Engineering Lab

Course Work

Assignments (30%): 6-8 homework sets

Report (15%, by reading a research paper)
Midterm (25%): 2hr exam
Final exam (30%): 2-3hr exam

Mobile Computing & Software Engineering Lab

Course Policy (1)

No late homework is acceptable.
For a regrade please contact me for the question
within 10 days from the date when the quiz or exam
was officially returned. No regrading after this period.
Cheating directly affects the reputation of the
Department and the University and lowers the morale
of other students. Cheating in homework and exam
will not be tolerated. An automatic grade of 0 will be
assigned to any student caught cheating. Presenting
another person's work as your own constitutes
cheating. Everything you turn in must be your own
doing.
6

Mobile Computing & Software Engineering Lab

Course Policy (2)

The following activities are specifically forbidden on
all graded course work:
Theft or possession of another student's solution or partial
solution in any form (electronic, handwritten, or printed).
Giving a solution or partial solution to another student,
even with the explicit understanding that it will not be
copied.
Working together to develop a single solution and then
turning in copies of that solution (or modifications) under
multiple names.
7

Mobile Computing & Software Engineering Lab

First Thing to Do
Please visit the course web site
http://www.cc.ntut.edu.tw/~cmliu/Alg/NTUT_
Alg_S08g/index.htm
Send an email to me using the email
address:cmliu@csie.ntut.edu.tw. I will make a
mailing list for this course. All the
announcements will be broadcast via this
mailing list.
8

Mobile Computing & Software Engineering Lab

Introduction
Chuan-Ming Liu
Computer Science & Information Engineering
National Taipei University of Technology
TAIWAN
9

Mobile Computing & Software Engineering Lab

Outline

Algorithms definition and applications

Algorithm Specification
Performance Analysis
Randomized Algorithms

10

Mobile Computing & Software Engineering Lab

What is an Algorithm?
Definition
An algorithm is a finite set of instructions that, if
followed, accomplishes a particular task. All the
algorithms must satisfy the following criteria:
Input
Output
Definiteness
Effectiveness
Finiteness
11

Mobile Computing & Software Engineering Lab

What is an Algorithm ?
Definiteness: each instruction is clear and
unambiguous
Effectiveness: each instruction is
executable; in other words, feasibility
Finiteness: the algorithm terminates after
a finite number of steps.

12

Mobile Computing & Software Engineering Lab

What is an Algorithm ?
Input

Definiteness

Output

Effectiveness
Finiteness

Computational Procedures
13

Mobile Computing & Software Engineering Lab

Procedures vs. Algorithms

Termination or not
One example for procedure is OS
Program, a way to express an
algorithm

14

Mobile Computing & Software Engineering Lab

Applications of Algorithms
Practical applications of algorithms are
ubiquitous, for example,
Human Genome Project
Internet
E-commerce (security issues)
Resource Management in Manufacturing
and other Commercial Setting

15

Mobile Computing & Software Engineering Lab

Short History (1)

The word algorithm derives from Al Kwarizmi
(about 790 - about 840), an Islamic
mathematician.
He laid out the basic methods for adding,
multiplying, and dividing numbers, as well as
extracting square roots and calculating digits
of .

16

Mobile Computing & Software Engineering Lab

Al Kwarizmi

JOC/EFR July 1999

http://www-history.mcs.st-andrews.ac.uk/Mathematicians/Al-Khwarizmi.html
17

Mobile Computing & Software Engineering Lab

Short History (2)

The 13th century Italian mathematician
Leonardo Fibonacci developed Al Kwarizmis
work further and propagandizing it.
However, Fibonacci is most widely known for
his famous sequence of numbers
0, 1, 1, 2, 3, 5, 8, 13, 21, 34

18

Mobile Computing & Software Engineering Lab

Leonardo Fibonacci

Corbis
19

Mobile Computing & Software Engineering Lab

Study on Algorithms

Devise: useful design techniques will learn

Correctness: learning how to prove
Analysis: why the devised algorithm is good
Testing: alternative way to show correctness

20

Mobile Computing & Software Engineering Lab

Describing Algorithms
Graphical view (flow charts)
Programming languages (C/C++, java, )
Pseudo-code representation

21

Mobile Computing & Software Engineering Lab

Pseudo-Code Conventions
Indentation as block structure
Loop and conditional constructs similar to
those in PASCAL, such as while, for, repeat,
if-then-else
Symbol
as the comment in a line
Using instead of = and allowing i j e
Variables local to the given procedure
22

Mobile Computing & Software Engineering Lab

Pseudo-Code Conventions
Array element accessed by A[i] and A[1..j] as
the subarray of A
NIL representing the pointer referring to no
object
Call by value procedures
Please refer to page 19 on the textbook.
23

Mobile Computing & Software Engineering Lab

Sorting Problem
Input: Given a sequence of n elements,
a1, a2, , an
Output: a permutation of these n elements in
non-decreasing order

24

Mobile Computing & Software Engineering Lab

Example
10 40 25 36

10 25 36 40

10 25 36 40

25

Mobile Computing & Software Engineering Lab

Insertion Sort Pseudo Code 1

26

Mobile Computing & Software Engineering Lab

Insertion Sort Pseudo Code 2

InsertionSort (a, n)
for j=2 to n
do
item = a[j];
i = j-1;
while ((i >= 1) && (item < a[i]))
do
a[i+1] = a[i]; i--;
a[i+1] = item;

27

Mobile Computing & Software Engineering Lab

Algorithm Analysis (1)

Machine-independent
Storage requirement
Space complexity: maximum amount of memory to
complete an algorithm

Computing time
Time complexity: amount of time to complete an
algorithm

28

Mobile Computing & Software Engineering Lab

Algorithm Analysis (2)

What to measure? How to measure?
Input size: depends on the problem
Number of items for sorting
Total number of bits for multiplying two integers
Sometimes, more than one input

Running time in terms of the number of

primitive operations or steps executed.
29

Mobile Computing & Software Engineering Lab

Analysis of Insertion Sort

30

Mobile Computing & Software Engineering Lab

10

Analysis of Insertion Sort

The best-case time complexity is (n)
How come?

Time complexity for the worst-case is (n2)

Even though the time complexity is O(n2), the
insertion sort runs fast when n is small, usually
n<=16.

31

Mobile Computing & Software Engineering Lab

Designing Algorithms
Many techniques for designing algorithms and
we will discuss some of them in the class
Insertion sort uses an incremental approach
We now discuss divide-and-conquer
approach using merge sort as an example.

32

Mobile Computing & Software Engineering Lab

Divide-and-Conquer (1)
General Description
A problem having n inputs
Divide the problem into k sub-problems
Solve each sub-problem
Combine (merge) all the results of the subproblems to derive the solution of the problem

Usually, the sub-problems keeps the same type

as the original problem
Recursive procedure
takes place

33

Mobile Computing & Software Engineering Lab

11

Divide-and-Conquer (2)
Can be implemented by a recursive algorithm
The computing time in general is
g ( n)
small n

T ( n) =
T
n
T
n
L
T
n
f
n
(
)
+
(
)
+
+
(
)
+
(
)
otherwise
2
k
1

where g(n) is the time when n is small and f(n) is

the time for dividing and combination
34

Mobile Computing & Software Engineering Lab

Example Merge Sort

Sorting Problem:
Input: Given a sequence of n elements, a1, a2, , an
Output: a permutation of these n elements in nondecreasing order

Using divide-and-conquer strategy

Divide the input sequence into subsequences with
smaller size
Merge the results of the subsequences toward the
final result
35

Mobile Computing & Software Engineering Lab

Example
1
2
3
4
5
6
7
8
9 10
310 285 179 652 351 423 861 254 450 520
(1,5)

(1,3)

(1,2)
1

(4,5)

3
2

(6,10)

(6,8)

(6,7)
6

(9,10)

10

7
36

Mobile Computing & Software Engineering Lab

12

Example for Merge

179 285 310 351 652

254 423 450 520 861

179 254 285 310 351 423 450 520 652 861

Then, copy to a[]

37

Mobile Computing & Software Engineering Lab

Merge Sort

38

Mobile Computing & Software Engineering Lab

Merge

39

Mobile Computing & Software Engineering Lab

13

40

Mobile Computing & Software Engineering Lab

Time Analysis
The merge process needs O(n) time since it
scans each element in worst case

a
n =1

T (n) =
2T (n / 2) + cn n > 1
T(n) = O(n log n)

41

Mobile Computing & Software Engineering Lab

14