Theory of Algorithms: Brute Force

Michelle Kuttel and Sonia Berman

(mkuttel | sonia@cs.uct.ac.za)

Objectives
To introduce the brute-force mind set To show a variety of brute-force solutions:
Brute-Force String Matching Polynomial Evaluation Closest-Pair Convex-Hull Exhaustive Search

To discuss the strengths and weaknesses of a brute-force strategy

Brute Force
A straightforward approach usually directly based on problem statement and definitions Motto: Just do it! Crude but often effective Examples already encountered:
Computing an (a > 0, n a nonnegative integer) by multiplying a together n times Computation of n! using recursion Multiplying two n by n matrices Selection sort

Brute-Force Sorting
What is the most straight-forward approach to sorting?
selection sort?
(n-1)n/2 O(n2) comparisons however, number of key swaps is n-1 O(n)

Bubble sort?
(n-1)n/2 - O(n2) comparisons number of key swaps depends on input Worst case?
4

Problem:

Brute Force String Matching

Find a substring in some text that matches a pattern Pattern: a string of m characters to search for Text: a (long) string of n characters to search in 1. Align pattern at beginning of text 2. Moving left to right, compare each character of pattern to the corresponding character in text UNTIL All characters are found to match (successful search); or A mismatch is detected 3. WHILE pattern is not found and the text is not yet exhausted, realign pattern one position to the right and repeat step 2.

Brute-Force String Matching

Example:
Pattern: Text: Trace: AKA ABRAKADABRA AKA AKA AKA AKA

Number of Comparisons:
In the worst case, m comparisons before shifting, for each of n-m+1 tries

Efficiency: !(nm)

Problem:

Brute Force Polynomial Evaluation

Find the value of polynomial p(x) = anxn + an-1xn-1 + + a1x1 + a0 at a point x = x0

Algorithm:

Efficiency? Improvements?

Problem:

Brute Force Polynomial Evaluation

Find the value of polynomial p(x) = anxn + an-1xn-1 + + a1x1 + a0 at a point x = x0

Algorithm:
p " 0.0 for i " n down to 0 do ! power " 1 ! for j " 1 to i do ! ! power " power * x ! p " p + a[i] * power return p

Efficiency? Improvements?

Brute Force Closest Pair

Brute Force Closest Pair

Problem:
Find the two points that are closest together in a set of n 2-D points P1 = (x1, y1), , Pn = (xn, yn) Using Cartesian coordinates and Euclidean distance

Brute Force Closest Pair

Problem:
Find the two points that are closest together in a set of n 2-D points P1 = (x1, y1), , Pn = (xn, yn) Using Cartesian coordinates and Euclidean distance

Algorithm:

Brute Force Closest Pair

Problem:
Find the two points that are closest together in a set of n 2-D points P1 = (x1, y1), , Pn = (xn, yn) Using Cartesian coordinates and Euclidean distance

Algorithm:

Brute Force Closest Pair

Problem:
Find the two points that are closest together in a set of n 2-D points P1 = (x1, y1), , Pn = (xn, yn) Using Cartesian coordinates and Euclidean distance

Algorithm:

Brute Force Closest Pair

Problem:
Find the two points that are closest together in a set of n 2-D points P1 = (x1, y1), , Pn = (xn, yn) Using Cartesian coordinates and Euclidean distance

Algorithm:

Brute Force Closest Pair

Problem:
Find the two points that are closest together in a set of n 2-D points P1 = (x1, y1), , Pn = (xn, yn) Using Cartesian coordinates and Euclidean distance

Algorithm:

Brute Force Closest Pair

Problem:
Find the two points that are closest together in a set of n 2-D points P1 = (x1, y1), , Pn = (xn, yn) Using Cartesian coordinates and Euclidean distance

Algorithm:

Brute Force Closest Pair

Problem:
Find the two points that are closest together in a set of n 2-D points P1 = (x1, y1), , Pn = (xn, yn) Using Cartesian coordinates and Euclidean distance

Algorithm:

Efficiency: !(n2)

Brute Force Closest Pair

Problem:
Find the two points that are closest together in a set of n 2-D points P1 = (x1, y1), , Pn = (xn, yn) Using Cartesian coordinates and Euclidean distance

Algorithm:
dmin " " for i " 1 to n-1 do ! for j " i+1 to n do ! ! d " sqrt((xi - xj)2 + (yi - yj)2) " " if d < dmin ! ! ! dmin " d; index1 " i; index2 " j return index1, index2

Efficiency: !(n2)

The Convex Hull Problem

Problem:
Find the convex hull enclosing n 2-D points Convex Hull: If S is a set of points then the Convex Hull of S is the smallest convex set containing S Convex Set: A set of points in the plane is convex if for any two points P and Q, the line segment joining P and Q belongs to the set

NonConvex

Convex

The Convex Hull Problem

Problem:
Find the convex hull enclosing n 2-D points Convex Hull: If S is a set of points then the Convex Hull of S is the smallest convex set containing S Convex Set: A set of points in the plane is convex if for any two points P and Q, the line segment joining P and Q belongs to the set

NonConvex

Convex

The Convex Hull Problem

Problem:
Find the convex hull enclosing n 2-D points Convex Hull: If S is a set of points then the Convex Hull of S is the smallest convex set containing S Convex Set: A set of points in the plane is convex if for any two points P and Q, the line segment joining P and Q belongs to the set

NonConvex

Convex

The Convex Hull Problem

Problem:
Find the convex hull enclosing n 2-D points Convex Hull: If S is a set of points then the Convex Hull of S is the smallest convex set containing S Convex Set: A set of points in the plane is convex if for any two points P and Q, the line segment joining P and Q belongs to the set

NonConvex

Convex

The Convex Hull Problem

Problem:
Find the convex hull enclosing n 2-D points Convex Hull: If S is a set of points then the Convex Hull of S is the smallest convex set containing S Convex Set: A set of points in the plane is convex if for any two points P and Q, the line segment joining P and Q belongs to the set

NonConvex

Convex

Brute Force Convex Hull P

4

P2 P3 P1 P5 P6 P7

P9

P8

Algorithm:

For each pair of points p1 and p2 Determine whether all other points lie to the same side of the straight line through p1 and p2

Efficiency:
for n(n-1)/2 point pairs, check sidedness of (n-2) others O(n3)

Pros and Cons of Brute Force

! Strengths:
Wide applicability Simplicity Yields reasonable algorithms for some important problems and standard algorithms for simple computational tasks A good yardstick for better algorithms Sometimes doing better is not worth the bother

" Weaknesses:
Rarely produces efficient algorithms Some brute force algorithms are infeasibly slow Not as creative as some other design techniques

Exhaustive Search
Definition:
A brute force solution to the search for an element with a special property Usually among combinatorial objects such a permutations or subsets suggests generating each and every element of the problems domain

Method:
1. Construct a way of listing all potential solutions to the problem in a systematic manner
All solutions are eventually listed No solution is repeated

2. Evaluate solutions one by one (disqualifying infeasible ones) keeping track of the best one found so far 3. When search ends, announce the winner

Travelling Salesman Problem

Problem:
Given n cities with known distances between each pair Find the shortest tour that passes through all the cities exactly once before returning to the starting city

Alternatively:
Find shortest Hamiltonian Circuit in a weighted connected graph

Example:
8

b
3 4

Travelling Salesman by Exhaustive Search

Tour a#b#c#d#a a#b#d#c#a a#c#b#d#a a#c#d#b#a a#d#b#c#a a#d#c#b#a Improvements: Cost . 2+3+7+5 = 17 2+4+7+8 = 21 8+3+4+5 = 20 8+7+4+2 = 21 5+4+3+8 = 20 5+7+3+2 = 17

Start and end at one particular city Remove tours that differ only in direction

Efficiency:
(n-1)!/2 = O(n!)

Knapsack Problem
Problem: Given n items
weights: w1 , w2 wn values: v1 , v2 vn a knapsack of capacity W Find the most valuable subset of the items that fit into the knapsack Item Weight Value Example (W = 16): 1 2 3 4 2kg 5kg 10kg 5kg R200 R300 R500 R100

Knapsack by Exhaustive Search

Subset {1} {2} {3} {4} {1,2} {1,3} {1,4} {2,3} Total Wgt 2 kg 5 kg 10 kg 5 kg 7 kg 12 kg 7 kg 15 kg Total Value R200 R300 R500 R100 R500 R700 R300 R800 Subset {2,4} {3,4} {1,2,3} {1,2,4} {1,3,4} {2,3,4} {1,2,3,4} Total Wgt 10 kg 15 kg 17 kg 12 kg 17 kg 20 kg 22 kg Total Value R400 R600 n/a R600 n/a n/a n/a

Efficiency: !(2n)

Final Comments on Exhaustive Search

Exhaustive search algorithms run in a realistic amount of time only on very small instances In many cases there are much better alternatives! In some cases exhaustive search (or variation) is the only known solution and parallel solutions can speed it up!