Types of Algorithms

2
Algorithm classification

Algorithms that use a similar problem-solving approach

can be grouped together

Well talk about a classification scheme for algorithms

This classification scheme is neither exhaustive nor

disjoint

The purpose is not to be able to classify an algorithm as

one type or another, but to highlight the various ays in
hich a problem can be attacked
!
A short list of categories

Algorithm types e ill consider include"

#imple recursive algorithms

$acktracking algorithms

%ivide and con&uer algorithms

%ynamic programming algorithms

'reedy algorithms

$ranch and bound algorithms

$rute force algorithms

(andomi)ed algorithms
*
#imple recursive algorithms +

A simple recursive algorithm"

#olves the base cases directly

(ecurs ith a simpler subproblem

%oes some extra ork to convert the solution to the simpler

subproblem into a solution to the given problem

+ call these ,simple- because several of the other

algorithm types are inherently recursive
.
/xample recursive algorithms

To count the number of elements in a list"

+f the list is empty, return )ero0 otherise,

#tep past the first element, and count the remaining elements
in the list

Add one to the result

To test if a value occurs in a list"

+f the list is empty, return false0 otherise,

+f the first thing in the list is the given value, return true0
otherise

#tep past the first element, and test hether the value occurs in
the remainder of the list
1
$acktracking algorithms

$acktracking algorithms are based on a depth-first

recursive search

A backtracking algorithm"

Tests to see if a solution has been found, and if so, returns it0
otherise

2or each choice that can be made at this point,

3ake that choice

(ecur

+f the recursion returns a solution, return it

+f no choices remain, return failure

4
/xample backtracking algorithm

To color a map ith no more than four colors"

color56ountry n7"

+f all countries have been colored 5n 8 number of countries7 return

success0 otherise,

2or each color c of four colors,

+f country n is not adjacent to a country that has been

colored c

6olor country n ith color c

recursively color country n9:

+f successful, return success

+f loop exits, return failure

;
%ivide and 6on&uer

A divide and con&uer algorithm consists of to parts"

%ivide the problem into smaller subproblems of the same

type, and solve these subproblems recursively

6ombine the solutions to the subproblems into a solution to

the original problem

Traditionally, an algorithm is only called ,divide and

con&uer- if it contains at least to recursive calls
<
/xamples

=uicksort"

>artition the array into to parts 5smaller numbers in one

part, larger numbers in the other part7

=uicksort each of the parts

?o additional ork is re&uired to combine the to sorted

parts

3ergesort"

6ut the array in half, and mergesort each half

6ombine the to sorted arrays into a single sorted array by

merging them
:@
$inary tree lookup

Aeres ho to look up something in a sorted binary tree"

6ompare the key to the value in the root

+f the to values are e&ual, report success

+f the key is less, search the left subtree

+f the key is greater, search the right subtree

This is not a divide and con&uer algorithm because,

although there are to recursive calls, only one is used
at each level of the recursion
::
2ibonacci numbers

To find the n
th
2ibonacci number"

+f n is )ero or one, return one0 otherise,

6ompute fibonacci(n-1) and fibonacci(n-2)

(eturn the sum of these to numbers

This is an expensive algorithm

+t re&uires O(fibonacci(n)) time

This is e&uivalent to exponential time, that is, O(2

n
)
:2
%ynamic programming algorithms

A dynamic programming algorithm remembers past

results and uses them to find ne results

%ynamic programming is generally used for

optimi)ation problems

3ultiple solutions exist, need to find the ,best- one

(e&uires ,optimal substructure- and ,overlapping

subproblems-

Bptimal substructure" Bptimal solution contains optimal solutions to

subproblems

Bverlapping subproblems" #olutions to subproblems can be stored and

reused in a bottom-up fashion

This differs from %ivide and 6on&uer, here

subproblems generally need not overlap
:!
2ibonacci numbers again

To find the n
th
2ibonacci number"

+f n is )ero or one, return one0 otherise,

6ompute, or look up in a table, fibonacci(n-1) and

fibonacci(n-2)

2ind the sum of these to numbers

#tore the result in a table and return it

#ince finding the n

th
2ibonacci number involves finding
all smaller 2ibonacci numbers, the second recursive call
has little ork to do

The table may be preserved and used again later

:*
'reedy algorithms

An optimi)ation problem is one in hich you ant to

find, not just a solution, but the best solution

A ,greedy algorithm- sometimes orks ell for

optimi)ation problems

A greedy algorithm orks in phases" At each phase"

Cou take the best you can get right no, ithout regard for
future conse&uences

Cou hope that by choosing a local optimum at each step, you

ill end up at a global optimum
:.
/xample" 6ounting money

#uppose you ant to count out a certain amount of

money, using the feest possible bills and coins

A greedy algorithm ould do this ould be"

At each step, take the largest possible bill or coin
that does not overshoot

/xample" To make D1E!<, you can choose"

a D. bill

a D: bill, to make D1

a 2.F coin, to make D1E2.

A :@F coin, to make D1E!.

four :F coins, to make D1E!<

2or G# money, the greedy algorithm alays gives

the optimum solution
:1
A failure of the greedy algorithm

+n some 5fictional7 monetary system, ,krons- come

in 1 kron, 7 kron, and 10 kron coins

Gsing a greedy algorithm to count out :. krons,

you ould get

A :@ kron piece

2ive : kron pieces, for a total of :. krons

This re&uires six coins

A better solution ould be to use to 4 kron pieces

and one : kron piece

This only re&uires three coins

The greedy algorithm results in a solution, but not

in an optimal solution
:4
$ranch and bound algorithms

$ranch and bound algorithms are generally used for

optimi)ation problems

As the algorithm progresses, a tree of subproblems is formed

The original problem is considered the ,root problem-

A method is used to construct an upper and loer bound for a

given problem

At each node, apply the bounding methods

+f the bounds match, it is deemed a feasible solution to that particular

subproblem

+f bounds do not match, partition the problem represented by that

node, and make the to subproblems into children nodes

6ontinue, using the best knon feasible solution to trim

sections of the tree, until all nodes have been solved or
trimmed
:;
/xample branch and bound algorithm

Traveling salesman problem" A salesman has to visit

each of n cities 5at least7 once each, and ants to
minimi)e total distance traveled

6onsider the root problem to be the problem of finding the

shortest route through a set of cities visiting each city once

#plit the node into to child problems"

#hortest route visiting city A first

#hortest route not visiting city A first

6ontinue subdividing similarly as the tree gros

:<
$rute force algorithm

A brute force algorithm simply tries all possibilities

until a satisfactory solution is found

#uch an algorithm can be"

Bptimi)ing" 2ind the best solutionE This may re&uire finding all
solutions, or if a value for the best solution is knon, it may stop
hen any best solution is found

/xample" 2inding the best path for a traveling salesman

#atisficing" #top as soon as a solution is found that is good enough

/xample" 2inding a traveling salesman path that is ithin :@H

of optimal
2@
+mproving brute force algorithms

Bften, brute force algorithms re&uire exponential time

Iarious heuristics and optimizations can be used

Aeuristic" A ,rule of thumb- that helps you decide hich

possibilities to look at first

Bptimi)ation" +n this case, a ay to eliminate certain

possibilities ithout fully exploring them
2:
(andomi)ed algorithms

A randomi)ed algorithm uses a random number at

least once during the computation to make a decision

/xample" +n =uicksort, using a random number to choose a

pivot

/xample" Trying to factor a large number by choosing

random numbers as possible divisors
22
The /nd