Académique Documents
Professionnel Documents
Culture Documents
c
.y
d
o
r
b
i
ly
a
m
o
Table of Contents
I
II
III
IV
V
VI
Resources
Sorting
Data Structures
Trees
Graphs
Analysis of Algorithms
d
o
r
b
i
ly
a
r
a
c
.y
m
o
I. Resources
Presentation Sequence
Bibliography
d
o
r
b
i
ly
a
r
a
c
.y
m
o
Bibliography
m
o
c
.y
r
b
i
ly
a
r
a
d
o
Bibliography
m
o
c
.y
r
a
r
b
i
ly
a
d
o
Bibliography
m
o
c
.y
d
o
r
b
i
ly
a
r
a
Bibliography
m
o
c
.y
r
b
i
ly
a
r
a
d
o
Bibliography
m
o
c
.y
r
a
r
b
i
ly
a
d
o
II. Sorting
m
o
c
.y
r
b
i
ly
a
r
a
d
o
9
31/3/00
Sorting Techniques
Selection Sort
Straight Selection Sort
Quadratic Selection Sort
r
b
i
ly
a
Insertion Sort
Straight (Linear) Insertion Sort
Binary Insertion Sort
Shell Sort
m
o
r
a
c
.y
Exchange Sort
Straight Exchange Sort (Bubble Sort)
Shaker Sort
Quick Sort
Radix Sort
d
o
Tree Sort
Binary Tree Sort
Heap Sort
Merge Sort
10
31/3/00
m
o
Table of Comparison of
Performance of Sorting
Techniques
r
a
c
.y
r
b
i
ly
a
d
o
11
31/3/00
Specification of
Generic Sort
generic
r
b
i
ly
a
m
o
c
.y
r
a
d
o
procedure Sort_G
(Table: in out Table_Type);
-- Sort in increasing order of "<".
12
31/3/00
m
o
r
a
c
.y
r
b
i
ly
a
d
o
begin -- Sort_Demo
Ada.Text_IO. Put_Line
("Before Sorting: " & My_String);
Sort_String (My_String);
Ada.Text_IO. Put_Line
("After Sorting: "& My_String);
end Sort_Demo;
13
31/3/00
Selection Sort
m
o
c
.y
Principle
Basic operation: Find the smallest element in
a sequence, and place this element at the
start of the sequence.
r
b
i
ly
a
Start
r
a
Small
End
sorted part
d
o
Basic Idea:
Find the index Small;
Exchange the values located at Start and
Small;
Advance Start.
14
31/3/00
Selection Sort
r
b
i
ly
a
390
45
45
d
o
45
45
m
o
c
.y
Example
r
a
205
182
45
235
205
182
390
235
182
205
390
235
182
205
390
235
182
205
235
390
15
31/3/00
Selection Sort
m
o
c
.y
r
b
i
ly
a
r
a
d
o
16
31/3/00
Selection Sort
m
o
c
.y
Complexity
We will neglect the index operations. We will
therefore count only operations on the
elements of the sequence.
r
b
i
ly
a
r
a
d
o
17
31/3/00
Selection Sort
r
b
i
ly
a
m
o
c
.y
Assessment
The effort is independent from the initial
arrangement.
r
a
d
o
18
31/3/00
m
o
Insertion Sort
c
.y
Principle
Basic operation: Insert an element in a sorted
sequence keeping the sequence sorted.
r
b
i
ly
a
Array
r
a
sorted part
d
o
Linked List
...
sorted part
...
19
31/3/00
Insertion Sort
r
b
i
ly
a
205
45
45
d
o
45
45
m
o
c
.y
r
a
45
390
235
182
205
390
235
182
205
390
235
182
205
235
390
182
182
205
235
390
20
31/3/00
Insertion Sort
205
235
390
c
.y
205
235
390
182
205
235
390
390
205
235
235
390
205
205
235
390
182
205
235
390
m
o
r
b
i
ly
a
45
45
45
d
o
45
45
45
r
a
182
21
31/3/00
Insertion Sort
m
o
c
.y
r
b
i
ly
a
r
a
d
o
22
31/3/00
Insertion Sort
r
b
i
ly
a
m
o
c
.y
Complexity
n is the length of the sequence
The exterior loop is executed n-1 times.
r
a
Interior loop:
Best case: 0
Worst case: 1+2+...+(n-1) = (1/2)*n*(n-1)
On average: One must walk through half of
the list before finding the location where to
insert the element: (1/4)*n*(n-1)
d
o
Comparisons Exchanges
Best Case
n-1
2*(n-1)
Average
(1/4)*n*(n-1)
(1/4)*n*(n-1) + 2*(n-1)
(1/2)*n*(n-1) + 2*(n-1)
23
31/3/00
m
o
Bubble Sort,
or Straight Exchange Sort
c
.y
r
a
Principle
Basic Operation: Walk through the sequence
and exchange adjacent elements if not in
order.
r
b
i
ly
a
J J+1
sorted part
d
o
Basic idea:
walk through the unsorted part from the
end;
exchange adjacent elements if not in
order;
increase the sorted part, decrease the
unsorted part by one element.
24
31/3/00
Bubble Sort
r
b
i
ly
a
390
390
390
d
o
390
45
m
o
c
.y
r
a
205
182
45
235
205
182
45
235
205
45
182
235
45
205
182
235
390
205
182
235
25
31/3/00
Bubble Sort
r
b
i
ly
a
45
45
45
d
o
45
m
o
c
.y
r
a
390
205
182
235
390
205
182
235
390
182
205
235
182
390
205
235
26
31/3/00
Bubble Sort
r
b
i
ly
a
45
45
45
d
o
m
o
c
.y
r
a
182
390
205
235
182
390
205
235
182
205
390
235
45
182
205
390
235
45
182
205
235
390
27
31/3/00
Bubble Sort
r
a
r
b
i
ly
a
m
o
c
.y
No Sentinel
d
o
end loop;
end Sort_G;
28
31/3/00
Bubble Sort
r
a
r
b
i
ly
a
m
o
c
.y
With Sentinel
d
o
29
31/3/00
Bubble Sort
m
o
c
.y
Complexity
r
a
r
b
i
ly
a
d
o
30
31/3/00
Quick Sort
Principle
The Algorithm is recursive.
m
o
r
a
c
.y
d
o
r
b
i
ly
a
31
31/3/00
Quick Sort
m
o
c
.y
40 15 30 25 60 10 75 45 65 35 50 20 70
r
b
i
ly
a
r
a
40 15 30 25 20 10 75 45 65 35 50 60 70
40 15 30 25 20 10 35 45 65 75 50 60 70
d
o
35 15 30 25 20 10 40 45 65 75 50 60 70
Table 9: Partitioning
32
31/3/00
m
o
c
.y
r
b
i
ly
a
r
a
d
o
procedure Partition
(Table: in out Table_Type;
Pivot_Index: out Index_Type) is separate;
33
31/3/00
m
o
r
b
i
ly
a
r
a
c
.y
d
o
end if;
end Sort_G;
34
31/3/00
m
o
r
b
i
ly
a
c
.y
r
a
d
o
35
31/3/00
m
o
c
.y
r
b
i
ly
a
r
a
d
o
36
31/3/00
Quick Sort
r
b
i
ly
a
m
o
c
.y
Complexity
Worst case: The sequence is already
ordered.
Consequence: The partition is always
degenerated.
r
a
Storage space:
The procedure calls itself n-1 times, and the
requirement for storage is therefore
proportional to n. This is unacceptable.
Solution: Choose for the pivot the median of
the first, last and middle element in the table.
Place the median value at the first position of
the table and use the algorithm as shown
(Median-of-Three Partitioning).
d
o
Execution time:
The execution of Partition for a sequence of
length k needs k comparisons. Execution
time is therefore proportional to n2.
37
31/3/00
Quick Sort
m
o
c
.y
Complexity
Best case: The sequence is always divided
exactly at its mid-position.
r
b
i
ly
a
r
a
Suppose n = 2m
Quicksort for a sequence of size 2m calls itself
twice with a sequence of size 2m-1.
Storage space:
S2m = S2m-1 + 1
(the maximum for a recursive descent)
therefore:
S2m
m
and hence Sn = O(logn)
d
o
Time behavior:
C2m = 2C2m-1 + 2m
(The 2m elements must be compared
with the pivot)
therefore:
C2m
2m(m+1) and hence Cn = O(nlogn)
38
31/3/00
Quick Sort
m
o
c
.y
Complexity
r
a
r
b
i
ly
a
d
o
c0 = 1
n
1
ck 1 + cn k
c n = n 1 + -n k=1
39
31/3/00
Quick Sort
m
o
c
.y
Remarks
r
a
Parameter passing:
Beware of passing the Table parameter of
Sort_G by copy!
r
b
i
ly
a
Solution in Ada:
Write local procedures which use the index
bounds of the table as parameters, and
therefore work on the global variable Table.
d
o
40
31/3/00
c
.y
r
b
i
ly
a
m
o
r
a
d
o
41
31/3/00
Data Structures
m
o
c
.y
Stack (Pile)
Queue (Queue, File d'attente)
Deque (Double-Entry Queue,
Queue double entre)
Priority Queue (Queue de priorit)
Set (Ensemble)
Bag (Multiset, Multi-ensemble)
Vector (Vecteur)
Matrix (Matrice)
String (Chane)
(Linked) List (Liste chane)
Linear List (Liste linaire)
Circular List (Liste circulaire)
Doubly-linked List (Liste doublement
chane)
Ring (Anneau)
r
b
i
ly
a
r
a
d
o
42
31/3/00
Data Structures
m
o
c
.y
Tree (Arbre)
Ordered Tree (Arbre ordonn)
(children are ordered)
2-Tree (Arbre d'ordre 2)
(every node has 0 or 2 children)
Trie (from retrieval)
(also called "Lexicographic Search
Tree")
(a trie of order m is empty or is a
sequence of m tries)
Binary Tree (Arbre binaire)
Binary Search Tree (Arbre de recherche)
AVL-Tree (Arbre quilibr)
Heap (Tas)
Multiway Search Tree
B-Tree
r
b
i
ly
a
r
a
d
o
43
31/3/00
Data Structures
m
o
c
.y
Graph (Graphe)
Directed Graph (Graphe orient)
Undirected Graph (Graphe non orient)
Weighted Graph (Graphe valu)
DAG (Directed Acyclic Graph, Graphe
orient acyclique)
Map (Mappe, Table associative)
Hash Table (Table de hachage)
File (Fichier)
Sequential File (Fichier sequentiel)
Direct Access File (Fichier accs direct)
Indexed File (Fichier index, fichier en
accs par cl)
Indexed-Sequential File (ISAM) (Fichier
index tri)
r
b
i
ly
a
r
a
d
o
44
31/3/00
Representation of a
Data Structure
m
o
c
.y
r
a
r
b
i
ly
a
d
o
Logical Structure
45
31/3/00
Subset
m
o
c
.y
Representation
r
a
r
b
i
ly
a
d
o
46
31/3/00
Subset
r
a
r
b
i
ly
a
d
o
m
o
c
.y
Representation
c null
null null
null null
47
31/3/00
Subset
m
o
c
.y
Logic Properties
r
a
r
b
i
ly
a
d
o
48
31/3/00
Logical Structure or
Representation
r
a
c
.y
r
b
i
ly
a
m
o
d
o
49
31/3/00
m
o
c
.y
Linear list
r
b
i
ly
a
Circular list
r
a
null
Doubly-linked list
d
o
size
end
null
50
31/3/00
m
o
Representations by Lists
c
.y
Stack
top
r
b
i
ly
a
r
a
null
null
d
o
size
end
null
51
31/3/00
m
o
c
.y
Definition
r
a
r
b
i
ly
a
d
o
52
31/3/00
r
a
r
b
i
ly
a
m
o
c
.y
d
o
53
31/3/00
m
o
c
.y
Realization in Ada
r
a
r
b
i
ly
a
d
o
54
31/3/00
r
a
Constructors:
Create, build, and initialize an object.
r
b
i
ly
a
m
o
c
.y
Kinds of Operations
Selectors:
Retrieve information about the state of an
object.
Modifiers:
Alter the state of an object.
d
o
Destructors:
Destroy an object.
55
31/3/00
r
a
r
b
i
ly
a
m
o
c
.y
-- constructor
-- constructor
-- iterator
-- selector
-- constructor
-- selector
d
o
56
31/3/00
d
o
38
4927
11
315
5
2352
11
325
r
a
r
b
i
ly
a
8
7
6
5
4
3
2
1
m
o
c
.y
Example: Stack
Top
57
31/3/00
m
o
c
.y
r
b
i
ly
a
"
d
o
"
"
r
a
58
31/3/00
m
o
c
.y
Primitive Operation
r
b
i
ly
a
r
a
First Definition
An operation is said to be primitive if it cannot
be decomposed.
Example
d
o
procedure Pop
(S: in out Stack; E: out Element);
59
31/3/00
m
o
c
.y
Primitive Operation
r
a
Second Definition
An operation is said to be primitive if it cannot
be implemented efficiently without access to
the internal representation of the data
structure.
r
b
i
ly
a
Example
It is possible to compute the size of a stack by
popping off all its element and then
reconstructing it. Such an approach is highly
inefficient.
d
o
60
31/3/00
m
o
c
.y
r
a
Definition
A set of primitive operations is sufficient if it
covers the usual usages of the data structure.
r
b
i
ly
a
Example
A stack with a Push operation but lacking a
Pop operation is of limited value.
Is a stack without an iterator usable?
d
o
61
31/3/00
m
o
c
.y
r
a
Definition
A complete set of operations is a set of
primitive operations including a sufficient set
of operations and covering all possible
usages of the data structure; otherwise
stated, a complete set is a "reasonable"
extension of a sufficient set of operations.
r
b
i
ly
a
d
o
Example
Push, Pop, Top, Size and Iterate form a
complete set of operations for a stack.
62
31/3/00
m
o
c
.y
r
a
generic
Max: Natural := 100;
type Item_Type is private;
package Stack_Class_G is
type Stack_Type is limited private;
procedure Push (Stack: in out Stack_Type;
Item: in Item_Type);
procedure Pop (Stack: in out Stack_Type);
function Top (Stack: Stack_Type)
return Item_Type;
generic
with procedure Action
(Item: in out Item_Type);
procedure Iterate (Stack: in Stack_Type);
Empty_Error: exception;
r
b
i
ly
a
d
o
Full_Error: exception;
-- raised when an item is pushed on a full stack.
63
31/3/00
r
a
private
type Table_Type is array (1..Max)
of Item_Type;
type Stack_Type is record
Table: Table_Type;
Top: Integer range 0..Max := 0;
end record
end Stack_Class_G;
r
b
i
ly
a
m
o
c
.y
d
o
64
31/3/00
m
o
c
.y
r
a
r
b
i
ly
a
d
o
65
31/3/00
IV. Trees
Kinds of trees
Binary tree
Traversal of a binary tree
Search tree
Expression tree
Polish forms
Strictly binary tree
Almost complete binary tree
Heap
r
b
i
ly
a
m
o
r
a
c
.y
d
o
66
31/3/00
Kinds of Trees
m
o
c
.y
Tree (Arbre)
Ordered Tree (Arbre ordonn)
(children are ordered)
2-Tree (Arbre d'ordre 2)
(every node has 0 or 2 children)
Trie (from retrieval)
(also called "Lexicographic Search
Tree")
(a trie of order m is empty or is a
sequence of m tries)
Binary Tree (Arbre binaire)
Binary Search Tree (Arbre de recherche)
AVL-Tree (Arbre quilibr)
Heap (Tas)
Multiway Search Tree
B-Tree
r
b
i
ly
a
r
a
d
o
67
31/3/00
Binary Tree
m
o
c
.y
r
b
i
ly
a
r
a
d
o
68
31/3/00
Binary Tree
E is a finite set
(i) E is empty
r
b
i
ly
a
or
(ii)
E, Eg, Ed,
r Eg, r Ed,
Eg Ed = , E = {r}
$
c
.y
r
a
d
o
m
o
Eg
Ed
69
31/3/00
Binary Tree
A
B
r
b
i
ly
a
d
o
c
.y
r
a
C
m
o
I
70
31/3/00
m
o
c
.y
r
b
i
ly
a
r
a
d
o
71
31/3/00
m
o
r
b
i
ly
a
d
o
r
a
Preorder:
Inorder:
Postorder:
By level:
c
.y
I
ABDGCEHIF
DGBAHEICF
GDBHIEFCA
ABCDEFGHI
72
31/3/00
m
o
r
b
i
ly
a
d
o
I
c
.y
r
a
D
Preorder:
Inorder:
Postorder:
By level:
73
31/3/00
Search Tree
m
o
c
.y
r
b
i
ly
a
r
a
"
g (n),
key (n)
"
n
Ed (n)
key (n )
2
d
o
74
31/3/00
Search Tree
14
r
b
i
ly
a
4
3
d
o
5
Inorder:
c
.y
r
a
15
m
o
18
20
16
17
3 4 5 7 9 14 15 16 17 18 20
Application: Sorting
Input: 14, 15, 4, 9, 7, 18, 3, 5, 16, 20, 17
Processing: Build the tree
Result: Traverse in inorder
Application: Searching
Algorithms and Data Structures
1995-2000 Alfred Strohmeier, EPFL
75
31/3/00
Expression Tree
m
o
c
.y
r
b
i
ly
a
+
a
r
a
d
o
(i) a+b*c
b
(ii) (a+b)*c
log
!
x
(iii) log x
Algorithms and Data Structures
1995-2000 Alfred Strohmeier, EPFL
n
(iv) n!
76
31/3/00
Expression Tree
-
r
b
i
ly
a
m
o
c
.y
r
a
d
o
77
31/3/00
m
o
Polish Forms
(Notations polonaises)
r
a
c
.y
r
b
i
ly
a
d
o
* -
78
31/3/00
m
o
Expression Tree
:=
r
b
i
ly
a
+
r
a
d
o
b
c
.y
0.5
a
79
31/3/00
Other Trees
m
o
c
.y
r
b
i
ly
a
r
a
(i)
k+1;
(ii) If a node in the tree has a right
descendant at the level k+1, then all its left
descendants that are leaves are also at the
level k+1.
d
o
Heap (Tas)
A heap is an almost complete binary
(i)
tree.
(ii) The contents of a node is always
smaller or equal to that of the parent node.
80
31/3/00
V. Graphs
m
o
c
.y
Definitions
Oriented Graph (example and definitions)
Undirected Graph (example and definitions)
Representations
Adjacency Matrix
Adjacency Sets
Linked Lists
Contiguous Lists (matrices)
"Combination"
Abstract Data Type
List of Algorithms
Traversal
Shortest path
Representation of a weighted graph
Dijkstras Algorithm
Principle of dynamic programming
r
b
i
ly
a
r
a
d
o
81
31/3/00
Graphs
m
o
c
.y
Definitions
r
a
r
b
i
ly
a
d
o
{{
}|
}.
82
31/3/00
Graphs
m
o
c
.y
Definitions
r
a
r
b
i
ly
a
d
o
83
31/3/00
Directed Graphs
a
b
r
b
i
ly
a
c
r
a
d
m
o
c
.y
Example
d
o
V = {a, b, c, d}
E = {(a, a), (a, c), (c, d), (d, c)}
(a, a) is a self-loop (boucle)
multiple parallel arcs are prohibited (E is a
set!)
84
31/3/00
Directed Graphs
a
b
r
b
i
ly
a
c
r
a
d
m
o
c
.y
Example
d
o
85
31/3/00
Directed Graphs
a
b
r
b
i
ly
a
c
r
a
d
m
o
c
.y
Example
d
o
86
31/3/00
Directed Graphs
m
o
c
.y
Definitions
1.1. If (v, w) E then v is a predecessor
(prdcesseur) of w, and w is a successor
(successeur) of v.
r
b
i
ly
a
r
a
d
o
87
31/3/00
Directed Graphs
m
o
c
.y
Definitions
1.6. Two vertices are strongly connected
(fortement connexes) if there are paths
connecting each one to the other.
A digraph is strongly connected if all its
vertices are strongly connected.
r
b
i
ly
a
r
a
d
o
88
31/3/00
Undirected Graphs
a
b
r
b
i
ly
a
c
m
o
c
.y
Example
r
a
d
o
V = {a, b, c, d, e}
E = {{a, c}, {a, d}, {c, d}, {d, e}}
self-loops (boucle) are prohibited.
multiple parallel edges are prohibited (E is
a set!).
89
31/3/00
Undirected Graphs
m
o
c
.y
Definitions
r
a
r
b
i
ly
a
d
o
90
31/3/00
Undirected Graphs
m
o
c
.y
Definitions
r
a
r
b
i
ly
a
d
o
91
31/3/00
Graphs
r
b
i
ly
a
1
r
a
m
o
c
.y
Representations
d
o
Adjacency matrix
Adjacency sets (or lists)
Linked lists
Contiguous lists (matrices)
"Combinations"
92
31/3/00
Adjacency Matrix
1
r
b
i
ly
a
4
d
o
1 2 3 4
1
T T
2
T T
3
4 T T T
c
.y
r
a
2
m
o
a (i, j) = T
<=>
(i, j) is an arc
93
31/3/00
Adjacency Matrix
c
.y
r
b
i
ly
a
m
o
r
a
type Matrix_Type is
array (Vertex_Type range <>,
Vertex_Type range <>) of Boolean;
d
o
94
31/3/00
Adjacency Sets
1
r
b
i
ly
a
4
d
o
1 2, 3
2 3, 4
3
4 1, 2, 3
c
.y
r
a
2
m
o
95
31/3/00
Adjacency Sets
c
.y
r
b
i
ly
a
m
o
r
a
d
o
96
31/3/00
Linked Lists
graph
arc (1, 2)
vertex 1
r
b
i
ly
a
arc (2, 3)
vertex 2
null
d
o
vertex 3
vertex 4
arc (4, 1)
m
o
c
.y
arc (1, 3)
r
a
null
arc (2, 4)
arc (4, 2)
null
arc (4, 3)
null
null
97
31/3/00
Linked Lists
type Vertex_Type;
type Edge_Type;
type Vertex_Access_Type is
access Vertex_Type;
type Edge_Access_Type is
access Edge_Type;
r
b
i
ly
a
m
o
c
.y
r
a
d
o
98
31/3/00
m
o
Number
2
2
0
3
-
List
23
34
- 12
- - - -
r
b
i
ly
a
r
a
c
.y
3
-
d
o
99
31/3/00
"Combination"
vertex 1
vertex 2
vertex 3
r
b
i
ly
a
null
vertex 4
vertex 5
null
vertex 6
null
vertex 7
null
c
.y
3 null
r
a
4
m
o
null
null
d
o
100
31/3/00
Graphs
r
a
generic
type Vertex_Value_Type is private;
type Edge_Value_Type is private;
package Graph_G is
type Graph_Type is limited private;
type Vertex_Type is private;
type Edge_Type is private;
r
b
i
ly
a
m
o
c
.y
d
o
procedure Set
(Vertex: in out Vertex_Type;
Value: in Vertex_Value_Type);
function Value
(Vertex: Vertex_Type)
return Vertex_Value_Type;
...
101
31/3/00
Graphs
r
a
procedure Add
(Vertex: in out Vertex_Type;
Graph: in out Graph_Type);
procedure Remove
(Vertex: in out Vertex_Type;
Graph: in out Graph_Type);
procedure Add
(Edge: in out Edge_Type;
Graph: in out Graph_Type;
Source,
Destination: in Vertex_Type);
procedure Remove
(Edge: in out Edge_Type;
Graph: in out Graph_Type);
r
b
i
ly
a
m
o
c
.y
d
o
102
31/3/00
Graphs
r
a
function Is_Empty
(Graph: Graph_Type)
return Boolean;
function Number_of_Vertices
(Graph: Graph_Type)
return Natural;
function Source
(Edge: Edge_Type)
return Vertex_Type;
function Destination
(Edge: Edge_Type)
return Vertex_Type;
r
b
i
ly
a
m
o
c
.y
d
o
103
31/3/00
Graphs
r
a
generic
with procedure Process
(Vertex: in Vertex_Type;
Continue: in out Boolean);
procedure Visit_Vertices
(Graph: in Graph_Type);
r
b
i
ly
a
m
o
c
.y
generic
with procedure Process
(Edge: in Edge_Type;
Continue: in out Boolean);
procedure Visit_Edges
(Graph: in Graph_Type);
d
o
104
31/3/00
Graphs
r
a
generic
with procedure Process
(Edge: in Edge_Type;
Continue: in out Boolean);
procedure Visit_Adj_Edges
(Vertex: in Vertex_Type
[; Graph: in Graph_Type]);
...
end Graph_G;
r
b
i
ly
a
m
o
c
.y
d
o
105
31/3/00
Graph Algorithms
Depth-first search
Breadth-first search
Connectivity problems
Minimum Spanning Trees
Path-finding problems
Shortest path
Topological sorting
Transitive Closure
The Newtwork Flow problem
(Ford-Fulkerson)
Matching
Stable marriage problem
Travelling Salesperson problem
Planarity problem
Graph isomorphism problem
r
b
i
ly
a
m
o
r
a
c
.y
d
o
106
31/3/00
Graph Traversal
r
b
i
ly
a
B
d
o
c
.y
r
a
m
o
107
31/3/00
Depth-First Search
1 A
r
b
i
ly
a
2 B
3 E
d
o
4 G
5 C
m
o
c
.y
r
a
9 D
6 F
7 H
8 I
(A, B, E, G, C, F, H, I, D)
108
31/3/00
Breadth-First Search
1 A
r
b
i
ly
a
2 B
5 E
d
o
7 G
3 C
m
o
c
.y
r
a
4 D
6 F
8 H
9 I
(A, B, C, D, E, F, G, H, I)
109
31/3/00
Depth-First Search
m
o
c
.y
r
b
i
ly
a
r
a
d
o
110
31/3/00
Depth-First Search
-- pseudo-Ada
r
a
c
.y
generic
with procedure Visit (Vertex: in Vertex_Type);
procedure Depth_First (Graph: in Graph_Type);
r
b
i
ly
a
m
o
d
o
111
31/3/00
Depth-First Search
c
.y
separate (Depth_First)
procedure Traverse (Vertex: in Vertex_Type) is
begin
Visited (Vertex) := True;
Visit (Vertex);
for all W adjacent to Vertex loop
if not Visited (W) then
Traverse (W);
end if;
end loop;
end Traverse;
r
b
i
ly
a
m
o
r
a
d
o
112
31/3/00
Breadth-First Search
m
o
c
.y
r
b
i
ly
a
r
a
d
o
113
31/3/00
Breadth-First Search
c
.y
package Queue is
new Queue_G
(Element_Type => Vertex_Type);
r
b
i
ly
a
r
a
type Queue_of_Vertices is
new Queue.Queue_Type;
m
o
generic
with procedure Visit
(Vertex: in Vertex_Type);
procedure Breadth_First
(Graph: in Graph_Type);
d
o
114
31/3/00
Breadth-First Search
r
b
i
ly
a
m
o
c
.y
r
a
d
o
115
31/3/00
Shortest Path
m
o
c
.y
r
a
Statement 1:
Given a vertex Start and a vertex Target, find
the shortest path from Start to Target.
r
b
i
ly
a
Statement 2:
Given a vertex Start, find the shortest paths
from Start to all other vertices.
d
o
116
31/3/00
Representation of a
Weighted Graph
m
o
r
a
c
.y
r
b
i
ly
a
d
o
117
31/3/00
Dijkstras Algorithm
Start: starting vertex
r
b
i
ly
a
m
o
c
.y
Principle
r
a
d
o
118
31/3/00
Dijkstras Algorithm
1. Initialization
Q := {Start} d(Start) := 0;
S :=
r
b
i
ly
a
m
o
c
.y
Principle
r
a
2. Loop
2.1. Extract from Q the vertex C having the
smallest distance:
d (C) = min (d (V); V Q)
2.2. Add C to S (see Justification)
2.3. Add the vertices adjacent to C to Q, and
update their distances:
For every W adjacent to C:
if W Q: d(w) := d(C) + weight (C, W)
if W Q: d(w) :=
min (d(W),d(C) + weight (C, W))
d
o
3. Stop condition
Q is empty
119
31/3/00
Dijkstras Algorithm
r
a
13
Start
r
b
i
ly
a
d
o
m
o
c
.y
Example
120
31/3/00
Dijkstras Algorithm
m
o
c
.y
Example
Initialization
Q := {A}, S :=
d
First Loop (process A)
S := {A}, Q := {B, C, D}
d (B) := 13, d (C) := 4, d (D) := 7
Second Loop (process C)
S := {A, C}, Q := {B, D, E}
d (B) = 13, d(D) = 7, d (E) := 11
because d (E) := d (C) + weight (C, E)
Third Loop (process D)
S := {A, C, D}, Q := {B, E}
d (B) = 13, d (E) := 9, because
d (E) :=
min (previous value,d(D) + weight (D, E))
Fourth Loop (process E)
S := {A, C, D, E}, Q := {B}
d (B) := 10
Fifth and Last Loop (process B)
S := {A, B, C, D, E}, Q :=
,
r
a
(A) := 0
r
b
i
ly
a
d
o
121
31/3/00
m
o
Other Example
Start
A
2
6
r
b
i
ly
a
E
4
10
c
.y
r
a
5
d
o
122
31/3/00
Dijkstras Algorithm
m
o
c
.y
Justification
r
a
r
b
i
ly
a
d
o
Start
123
31/3/00
Dijkstras Algorithm
m
o
c
.y
r
a
Precondition:
Weight (V, W) = if there is no arc from V to W.
r
b
i
ly
a
Q: Priority_Queue_Type;
C: Vertex_Type;
d
o
124
31/3/00
Dijkstras Algorithm
m
o
c
.y
r
a
Precondition:
Weight (V, W) = if there is no arc between
V and W; and Weight (V, W) = 0 if V = W.
r
b
i
ly
a
S: Set_of_Vertices;
Start, C: Vertex_Type;
Min_Dist: Weight_Type;
Found: Boolean;
d
o
125
31/3/00
Dijkstras Algorithm
Found := True;
while Found loop
r
a
r
b
i
ly
a
Found := False;
Min_Dist = ;
m
o
c
.y
d
o
126
31/3/00
m
o
r
a
c
.y
Representation of a path
For each vertex on the path, store its
predecessor (on the path).
r
b
i
ly
a
d
o
127
31/3/00
m
o
Dynamic Programming
c
.y
Principle
r
a
r
b
i
ly
a
d
o
Start
I1
I2
End
128
31/3/00
m
o
c
.y
Classification of algorithms
Selection criteria
Complexity
Big O notation
Fundamental recurrence relations
Design of algorithms
Incremental algorithms
Greedy algorithms
Divide and conquer algorithms
Dynamic programming
Knapsack problem
Computability and complexity
Undecidable problems
Exponential time problems
Polynomial time problems
NP-complete problems
Satisfiability problem
r
b
i
ly
a
r
a
d
o
129
31/3/00
Algorithms
m
o
c
.y
Sorting
Searching
Sequential Searching, Binary Search,
Tree Search, Hashing, Radix Searching
String Processing
String Searching
Knuth-Morris-Pratt, Boyer-Moore,
Robin-Karp
Pattern Matching
Parsing (Top-Down, Bottom-Up,
Compilers)
Compression
Huffman Code
Cryptology
Image Processing
r
b
i
ly
a
r
a
d
o
130
31/3/00
Algorithms
Geometric Algorithms
Intersections
Convexity
Jordan Sorting
Closest-Point Problems
Curve Fitting
Mathematical Algorithms
Random Numbers
Polynomial Arithmetic
Matrix Arithmetic
Gaussian Elimination
Integration
Fast Fourier Transform
Linear Programming
Graph Algorithms
r
b
i
ly
a
d
o
m
o
r
a
c
.y
131
31/3/00
Selection Criteria
m
o
c
.y
r
a
r
b
i
ly
a
d
o
132
31/3/00
Complexity
r
b
i
ly
a
m
o
c
.y
r
a
d
o
Average-case analysis:
complexity for "average" problems.
133
31/3/00
Big O Notation
f, g: N+
r
b
i
ly
a
R+
m
o
c
.y
Definition
The big O notation defines equivalence
classes of real functions defined on the
natural numbers.
r
a
N, $
"
d
o
134
31/3/00
m
o
Big O Notation
c
.y
Calculus
1. Transitivity (transitivit)
If f is O(g) and g is O(h), then f is O(h).
r
b
i
ly
a
r
a
d
o
"
4. Product (produit)
If f1 is O(g1) and f2 is O(g2),
then f1f2 is O(g1g2).
135
31/3/00
Big O Notation
r
a
r
b
i
ly
a
m
o
c
.y
Example
Sum
O(2n3 + 5n2 + 3) = O(max (2n3, 5n2 + 3))
= O(2n3)
Scaling:
O(2n3)
d
o
= O(n3)
Transitivity:
O(2n3 + 5n2 + 3) = O(2n3)
and
O(2n3)
= O(n3)
therefore
O(2n3 + 5n2 + 3) = O(n3)
136
31/3/00
Big O Notation
r
a
O(1)
constant complexity
O(logn) logarithmic complexity
O(n)
linear complexity
O(nlogn)"nlogn" complexity
O(n2) quadratic complexity
O(n3) cubic complexity
O(nm) polynomial complexity
O(2n) exponential complexity
r
b
i
ly
a
m
o
c
.y
Typical Cases
d
o
137
31/3/00
m
o
Fundamental Recurrence
Relations
r
a
c
.y
r
b
i
ly
a
C1 = 1
Cn = Cn-1 + n, for n >= 2
therefore
Cn = Cn-2 + (n-1) + n
...
= 1 + 2 +... + n
= (1/2)*n*(n+1)
d
o
138
31/3/00
m
o
Fundamental Recurrence
Relations
r
a
c
.y
r
b
i
ly
a
C1= 0
Cn= Cn/2 + 1 n
Approximation with n = 2m
C2m = C2m-1 + 1
= C2m-2 + 2
...
= C2o + m
=m
d
o
139
31/3/00
m
o
Fundamental Recurrence
Relations
r
a
c
.y
r
b
i
ly
a
C1 = 1
Cn = Cn/2 + n n
Approximation with n = 2m
C2m = C2m-1 + 2m
= C2m-2 + 2m-1 + 2m
= 1 + 21 + 22 +... + 2m
= 2m+1 - 1
d
o
hence
Cn = 2n - 1
Example: ??
140
31/3/00
m
o
Fundamental Recurrence
Relations
c
.y
r
a
r
b
i
ly
a
traverse n elements
Cn/2 + Cn/2 : each half
Approximation: n = 2m
d
o
= 2C
T
2
2
---------------m
m1 + 2
m1
2
-------------------------- +
m1
1 = m+1
C m = 2 m+1
2
hence:
Cn n log n
Example: Quick sort
(
141
31/3/00
m
o
Algorithm Design
(Conception dalgorithmes)
c
.y
r
a
r
b
i
ly
a
d
o
142
31/3/00
Design Strategies
m
o
c
.y
Incremental algorithms
(incremental algorithms)
Insertion sort, linear search.
Greedy algorithms
(algorithmes gloutons)
Selection sort, shortest path by Dijkstra.
Divide-and-conquer algorithms
(algorithmes "diviser pour rgner")
Quick sort, binary search, convex hull.
Dynamic programing
(programmation dynamique)
Search with backtracking (recherche avec
rebroussement)
Pruning (lagage)
"Branch and bound"
Approximation
Heuristics (algorithmes heuristiques)
r
b
i
ly
a
r
a
d
o
143
31/3/00
m
o
Incremental Algorithms
c
.y
r
a
begin
R := some evident value;
while P empty loop
Select X in P;
Delete X in P;
Modify R based on X;
end loop;
end Solve;
r
b
i
ly
a
d
o
144
31/3/00
m
o
Incremental Algorithms
of the First Kind
r
a
c
.y
r
b
i
ly
a
d
o
145
31/3/00
m
o
Greedy Algorithms or
Incremental Algorithms of
the Second Kind
c
.y
r
a
r
b
i
ly
a
d
o
146
31/3/00
Divide-and-Conquer
Algorithms
r
a
c
.y
r
b
i
ly
a
m
o
d
o
147
31/3/00
Divide-and-Conquer
Algorithms
m
o
r
a
c
.y
r
b
i
ly
a
d
o
148
31/3/00
Convex Hull
(Enveloppe convexe)
r
b
i
ly
a
m
o
c
.y
r
a
d
o
Red + Blue = ??
Combine: Seems hard!
Algorithms and Data Structures
1995-2000 Alfred Strohmeier, EPFL
149
31/3/00
Convex Hull
(Enveloppe convexe)
A
max
r
b
i
ly
a
L
min
d
o
m
o
c
.y
r
a
Divide:
Find the points with the largest and
smallest Y coordinates, called A and B.
Allocate points to L or R depending on
which side of the line joining A and B, left
or right, they are.
150
31/3/00
Convex Hull
(Enveloppe convexe)
A
max
r
b
i
ly
a
L
min
d
o
m
o
c
.y
r
a
Solve L and R
151
31/3/00
Convex Hull
(Enveloppe convexe)
A
max
r
b
i
ly
a
L
min
d
o
m
o
c
.y
r
a
Combine:
Connect both A and B to the "right"
vertices of the convex hulls of L and R.
152
31/3/00
m
o
Dynamic Programming
c
.y
Principle of divide-and-conquer:
In order to solve a large problem, it is divided
into smaller problems which can be solved
independently one from each other.
r
b
i
ly
a
r
a
Dynamic programming
When one does not know exactly which
subproblems to solve, one solves them all,
and one stores the results for using them
later on for solving larger problems.
d
o
153
31/3/00
m
o
c
.y
r
b
i
ly
a
Name
Size
Value
A
3
4
B
4
5
C
7
10
r
a
D
8
11
E
9
13
Problem
Pack goods of the highest total value in the
knapsack, up to its capacity.
d
o
154
31/3/00
m
o
Obj
Best
4 4 4 8 8 8 12 12 12 16
A A A A A A A A A A
Obj
Best
4 5 5 8 9 10 12 13 14 16
A B B A B B A B B A
Obj
Best
r
b
i
ly
a
d
o
Obj
Best
r
a
Obj
Best
c
.y
9 10 11 12
4 5 5 8 10 10 12 14 15 16
A B B A C B A C C A
4 5 5 8 10 11 12 14 15 16
A B B A C D A C C A
4 5 5 8 10 11 13 14 15 17
A B B A C D E C C E
155
31/3/00
m
o
c
.y
r
b
i
ly
a
r
a
d
o
156
31/3/00
m
o
r
b
i
ly
a
r
a
c
.y
d
o
end loop;
end loop;
157
31/3/00
m
o
r
b
i
ly
a
Computable
PolynomialTime
c
.y
r
a
Undecidable/
Unsolvable
Intractable
where?
NP-Complete
d
o
Computability:
Whether or not it is possible to solve a
problem on a machine.
Machine:
Turing Machine
158
31/3/00
m
o
Undecidable Problems,
Unsolvable Problems
c
.y
r
a
r
b
i
ly
a
Examples:
Halting problem
Trisect an arbitrary angle with a compass
and a straight edge.
Intractable Problems
d
o
159
31/3/00
m
o
Polynomial-Time Problems
c
.y
Size N of a problem:
Number of bits used to encode the input,
using a "reasonable" encoding scheme.
r
b
i
ly
a
r
a
Efficiency of an algorithm:
Is a function of the problem size.
Deterministic algorithm/machine:
At any time, whatever the algorithm/
machine is doing, there is only one thing
that it could do next.
d
o
P:
The set of problems that can be solved by
deterministic algorithms in polynomial
time.
160
31/3/00
m
o
Non-Deterministic
Polynomial-Time Problems
r
a
c
.y
Non-determinism:
When an algorithm is faced with a choice
of several options, it has the power to
"guess" the right one.
r
b
i
ly
a
Non-deterministic algorithm/machine:
To solve the problem, "guess" the solution,
then verify that the solution is correct.
d
o
NP:
The set of problems that can be solved by
non-deterministic algorithms in polynomial
time.
161
31/3/00
m
o
Non-Deterministic
Polynomial-Time Problems
P
NP
r
b
i
ly
a
r
a
c
.y
d
o
162
31/3/00
m
o
NP-Complete Problems
c
.y
r
b
i
ly
a
r
a
Otherwise stated:
There is no known polynomial-time
algorithm.
It has not been proven that the problem is
intractable.
It is easy to check that a given solution is
valid.
d
o
163
31/3/00
Satisfiability Problem
m
o
c
.y
r
a
*
r
b
i
ly
a
d
o
164
31/3/00
m
o
NP-Complete Problems
c
.y
Satisfiability
Is a Boolean expression satisfiable?
r
a
Hamilton circuit
Does a (un)directed graph have a
Hamilton circuit (cycle), i.e. a circuit (cycle)
containing every vertex.
r
b
i
ly
a
d
o
Colorability
Is an undirected graph k-colorable? (no two
adjacent vertices are assigned the same
color)
165
31/3/00
m
o
NP-Complete Problems
c
.y
Knapsack problem
Fill a knapsack with goodies of best value.
Given integers i1, i2,..., in and k, is there a
subsequence that sums exactly k?
r
b
i
ly
a
r
a
Multiprocessor scheduling
Given a deadline and a set of tasks of
varying length to be performed on two
identical processors, can the tasks be
arranged so that the deadline is met?
d
o
166
31/3/00
m
o
r
a
c
.y
r
b
i
ly
a
d
o
3. "Approximation"
The problem is changed. The algorithm does
not find the best solution, but a solution
guaranteed to be close to the best (e.g. value
95% of best value)
167
31/3/00