Lecture 1

Preliminaries
COMP 523: Advanced Algorithmic Techniques
Lecturer: Dariusz Kowalski

Lecture 1: Preliminaries

Announcements
Textbook: J. Kleinberg, E. Tardos. Algorithm Design.
Addison-Wesley, 2005
Lectures: Wednesday 9-11, Friday 15-16 in 3.10
Tutorials: Thursday 11-12 in 1.01 Ashton Bldg
Eleni Akrida (akridel@hotmail com)

Office hours (room 3.11, Ashton Bldg):

Wednesday 12-13
Friday 14-15

Info and contact:

http://www.csc.liv.ac.uk/~darek/comp523.html
darek@liv.ac.uk

Lecture 1: Preliminaries

Assessment and feedback

Assessment: Exam (75%) and 2 tasks (25%
total)
Feedback:
Presentation and discussion of solutions after
assignment deadline
In case of specific questions, individual
appointments

Lecture 1: Preliminaries

Algorithms - what does it mean?

Algorithm - a list/structure of instructions, which are carried out
in a fixed order
to find the answer to a question
to calculate

Al-Khwarizmi (Muhammed Ibn Musa) Arabian

mathematician who introduced
the decimal
positional number system to
the Western world, the
author of Algebra
This module: techniques to design algorithms for problems
concerning data commonly processed by computers (written
in hard discs, propagated in networks, entered by user or
external device, etc.)

Lecture 1: Preliminaries

Types of algorithms
Constructive vs. Non-constructive
Discrete vs. Numerical
Deterministic vs. non-deterministic (e.g.,
Randomized, Quantum)
Sequential vs. Concurrent vs. DNA vs.
Exact vs. Approximate
and many others
Lecture 1: Preliminaries

Example
Problem: Find if a given word occurs in a given
text
Input: Text and word represented as lists
Output: Word found or not
o

Lecture 1: Preliminaries

Example - algorithm
Nave solution (exhaustive search):
Intuition:
Check letter after letter, starting from the beginning of the lists, if the
corresponding letters are equal
If some corresponding letters are not equal, re-start comparing from the second
letter of the text (and the first letter of the word)
Etc. from the 3rd letter, 4th letter, until the end of text or successful comparison

Implementation:
Initiate three pointers:
blue_pointer and black_pointer at the beginning of blue_list
yellow_pointer at the beginning of yellow_list

Initiate Boolean variable successful into false

Lecture 1: Preliminaries

Nave solution cont.

Repeat

stop := false
Move blue_pointer to the next element
Set black_pointer to blue_pointer
Move yellow_pointer to the first element of yellow_list
Repeat
Move black_pointer and yellow_pointer to the next elements in
corresponding lists
If values pointed by black_pointer and yellow_pointer are different
then stop := true

until yellow_pointer or black_pointer is at the end of its list or stop

If yellow_pointer is at the end of yellow_list and not stop then
successful := true

until successful or blue_pointer is at the end of blue_list

Output: successful
Lecture 1: Preliminaries

Example - how it works

Repeat

stop := false,
move blue_pointer to the next element, set black_pointer to blue_pointer,
move yellow_pointer to the first element of yellow_list
Repeat
Move black_pointer and yellow_pointer to the next elements in corresponding lists
If values pointed by black_pointer and yellow_pointer are different then stop := true

until yellow_pointer or black_pointer is at the end of its list or stop

If yellow_pointer is at the end of yellow_list and not stop then successful := true

until successful or blue_pointer is at the end of blue_list

Lecture 1: Preliminaries

What we expect from algorithms?

Correctness: computes desired output
Termination: stops eventually (or with high probability)
Efficiency: with respect to
Performance measure

Time (or total number of computation steps)Analysis

Size of memory used
depends on
Number of messages sent
Model!

Methods of measuring

Worst-case
Average-case
Smoothed (a subset of possible inputs, specific distribution of inputs, etc.)
Competitive (comparing to the best solution for particular input)
Expected

Lecture 1: Preliminaries

10

Example - correctness
Successful is set to true iff after some
execution of the internal loop the
yellow_pointer is at the end of the
yellow_list
yellow_pointer is at the end of the
yellow_list iff it came through the
whole yellow_list without coming back
to the beginning
it happens if the values pointed by
black_pointer and yellow_pointer
have been the same during all checks of
internal loop in the current run (thus a
copy of the word exists in the text)

Repeat

stop := false,
move blue_pointer to the next element,
set black_pointer to blue_pointer,
move yellow_pointer to the first element
of yellow_list
Repeat
Move black_pointer and yellow_pointer
to the next elements in corresponding
lists
If values pointed by black_pointer and
yellow_pointer are different then stop :=
true

until yellow_pointer or black_pointer

is at the end of its list or stop
If yellow_pointer is at the end of
yellow_list and not stop then
successful := true

until successful or blue_pointer is at the

end of blue_list
Lecture 1: Preliminaries

11

Example - termination
External Loop Invariant:
Repeat
stop := false,
At the beginning of the external loop,
move blue_pointer to the next element,
blue_pointer advances, and it has a finite
set black_pointer to blue_pointer,
number of advances to take before
move yellow_pointer to the first element
reaching the end of the text
of yellow_list
Repeat
Internal Loop Invariant:
Move black_pointer and yellow_pointer
to the next elements in corresponding
Each run of the internal loop finishes
lists
eventually;
If values pointed by black_pointer and
yellow_pointer are different then stop :=
It happens because yellow_pointer keeps
true
until yellow_pointer or black_pointer
advancing every iteration (first line of the
is at the end of its list or stop
loop), unless stop condition becomes true
If yellow_pointer is at the end of
or it reaches the end of the word (or
yellow_list and not stop then
successful := true
black_ pointer reaches the end of the text)
until successful or blue_pointer is at the
end of blue_list

Lecture 1: Preliminaries

12

Example - efficiency
Complexity measures: time, size of additional memory
Size of additional memory:
3 single pointers and 2 binary variables

Time complexity:
Worst-case: if the sequence is not included in the text then
time is (almost) proportional to the multiplication of sizes of
blue and yellow lists, e.g.,
text: aab repeated n times
word: aaa
1, 2 or 3 internal loop runs, 3n times, gives about 6n runs
Lecture 1: Preliminaries

13

Example - efficiency cont.

Time complexity (cont.):
Average case: text or sequence are randomly selected;
average short word should be find quickly in the
beginning of an average long text, in time proportional
to the squared length of the word (exercise)
Smoothed analysis: text must be randomly selected
from reasonable set of texts (usually complicated
analysis, depends on the family of texts - dictionary)

Lecture 1: Preliminaries

14

Efficiency - asymptotic notation

n - integer, size of data, f() - function over integers
Complexity O(f(n)) : there is a positive constant c such
that for every positive integer n complexity is at most
cf(n)
Complexity (f(n)) : there is a positive constant c such
that for every positive integer n complexity is at least
cf(n)
Complexity o(f(n)) : complexity divided by f(n) comes
to 0 with n going to infinity (strictly smaller than f(n))
Complexity (f(n)) : complexity divided by f(n) comes
to with n going to infinity (strictly bigger than f(n))
Lecture 1: Preliminaries

15

Example - asymptotic complexity

Worst case time:
If size of text is n and size of word is n/2 :

O((n/2)(n/2)) = O(n2)
If size of text is m and size of sequence is k :

O((m-k)k)

Lecture 1: Preliminaries

16

Examples
5n3+100 = O(n3) , 5n3+100 O(n2)
5n3+100 = (n3) , 5n3+100 (n4)
log n = o(na)
for any positive constant a
na = o(cn)
for any positive constants a,c
log (4n) = log n + log 4 = O(log n)
log (n4)= 4 log n = O(log n)
(4n)3 = 64n3 = O(n3)
(n4)3 = n12 = (n4)
(3n)4 = 81n = (3n)
Logarithms are to the base 2
Lecture 1: Preliminaries

17

Symmetric properties and tight bound

If f(n) = O(g(n)) then g(n) = (f(n)) and vice versa
If f(n) = o(g(n)) then g(n) = (f(n)) and vice versa
Definition:
If f(n) = O(g(n)) and f(n) = (g(n)) then
f(n) = (g(n)) and g(n) = (f(n))
Example:
9n2 + n +7 = (3n2) = (n2)
Lecture 1: Preliminaries

18

Transitive properties (order)

If f(n) = O(g(n)) and g(n) = O(h(n)) then
f(n) = O(h(n))
If f(n) = (g(n)) and g(n) = (h(n)) then
f(n) = (h(n))
If f(n) = (g(n)) and g(n) = (h(n)) then
f(n) = (h(n))

Lecture 1: Preliminaries

19

Sum and maximum

f1(n) + + fa(n) = (max(f1(n), , fa(n)))
for any constant positive integer a

Example:
3n2 + n +7 = (3n2) = (n2)
If the range of the summation index is not
constant:
in i = n(n+1)/2 = (n2)
(max{1, , n}) = (n)
Lecture 1: Preliminaries

20

Logarithmic time O(log n)

Reduce input to any fraction of it in
constant time
Example: searching if a given element x is
in a sorted array

Lecture 1: Preliminaries

21

Linear time O(n)

It is sufficient to look at each element of the
input constant number of times
Example: finding maximum in an (unsorted)
array

Lecture 1: Preliminaries

22

Time O(n log n)

Split the input into two pieces of the similar
size and merge both solutions in linear time
Example: sorting an array (split into halves,
sort each part separately, merge sorted parts
in linear time)

Lecture 1: Preliminaries

23

Quadratic time O(n2)

It is enough to consider pairs of elements
Examples:
finding the closest pair of points located in a
plane
sorting by comparison of subsequent elements
(insertion sort)

Lecture 1: Preliminaries

24

Polynomial time O(na)

Algorithm has many nested loops
Example: are there any two disjoint sets
among given family of n sets, each of at
most n elements (time O(n3))

Lecture 1: Preliminaries

25

Exponential time O(cn)

Algorithm considers many subsets of the
input
Example: exhaustive search to find the
largest clique contained in a given graph
(time O(2n))

Lecture 1: Preliminaries

26

Beyond exponential time:

n
O(n!), O(n ),
Algorithm searching in a large space
Example: search performed in the set of all
permutations (time O(n!))

Lecture 1: Preliminaries

27

Graphs
Set of nodes |V| = n
Set of edges |E| = m
Undirected edges/graph: pairs of nodes {v,w}
Directed edges/graph: pairs of nodes (v,w)

Set of neighbors of v : set of nodes connected by an edge

with v (directed: in-neighbors, out-neighbors)
Degree of a node: number of its neighbors
Path: a sequence of (non-repeating) nodes such that every
two consecutive nodes constitute an edge
Length of a path: number of nodes minus 1
Distance between two nodes: the length of the shortest
path between these nodes
Diameter: the longest distance in the graph
Lecture 1: Preliminaries

28

Lines, cycles, trees, cliques

Line

Cycle

Tree
Clique

Lecture 1: Preliminaries

29

Conclusions
Algorithm: list/structure of instructions
Algorithmic methods for problems arising
from computer and communication
applications
Guarantee correctness, termination and
efficiency
Model is important!

Asymptotic analysis of efficiency

Lecture 1: Preliminaries

30

Textbook and exercises

READING:
Chapter 2, up to section 2.4
OBLIGATORY EXERCISES:
Exercises 1,2,3 page 67
Solved exercises 1,2 pages 65,66
OPTIONAL:
Exercises 3,4,5,6 pages 67,68,69
Exercises 7,8 pages 69,70
Lecture 1: Preliminaries

31