1

Data Structure & Algorithms

 2

Introduction
 3
Course Objectives

 Reinforce the concept that costs and benefits exist for every
data structure.

 Learn the commonly used data structures.

These form a programmer's basic data structure “toolkit.”

 Understand how to measure the cost of a data structure or

program.
These techniques also allow you to judge the merits of new data
structures that you or others might invent.
 4
Course Prerequisites

 The knowledge of C++ language is expected as strong background.

 Students should have taken the following courses in prior semesters.

 Programming Fundamentals
 Discrete Structures

Lahore Garrison University

 5
Analysis of Algorithms

 What are the key factors to consider while developing/creating software

 Why performance analysis?

 Speed==Fun?

 If a software feature can not cope with the scale of tasks users need to perform –

it is as good as dead.
Difference between complexity and
computation time

Computation time: The interval of solving a problem based on the embedded system
architecture. (in the field of computer sciences)

Complexity: The art of handling a problem based on the algorithm designed to solve a case.
 7

Suppose we have two algorithms, how can we tell which one is better?

We could implement both algorithms, run them both

 Expensive and error prone

Preferably, we should analyze them mathematically

 Algorithm analysis
Performance vs complexity

1. Performance:
how much time/memory/disk/... is actually used when a program
is run. This depends on the machine, compiler, etc. as well as the code.

2. Complexity:
how do the resource requirements of a program or algorithm scale, i.e.,
what happens as the size of the problem being solved gets larger?
Big O Notation

Big O notation (with a capital letter O, not a zero), is a symbol used in complexity theory,
computer science, and mathematics to describe the asymptotic behavior of functions.
Basically, it tells you how fast a function grows or declines
Functions defined in big o notation

O(1) constant
O(log(n)) logarithmic
O((log(n))c) polylogarithmic (same as O(log(n)) )
O(n) linear
O(n2) quadratic
O(nc) polynomial
O(cn) exponential
Understanding big o

Efficiency covers lots of resources, including:

1. CPU (time) usage (The most important)
2. Memory usage
3. Disk usage
4. Network usage
More about performance

The time required by a function/procedure is proportional to the number of "basic

operations" that it performs, like;

1. one arithmetic operation (e.g., +, *).

2. one assignment (e.g. x := 0)
3. one test (e.g., x = 0)
4. one read (of a primitive type: integer, float, character, Boolean)
5. one write (of a primitive type: integer, float, character, Boolean)
Regarding computing

We express complexity using big-O notation.

For a problem of size N:

A constant-time algorithm is "order 1": O(1)

A linear-time algorithm is "order N": O(N)
A quadratic-time algorithm is "order N squared": O(N2)
Infinite Time algorithm is “Order infinity”: O(inf)
Finding complexity

Generally, we have 6 cases

1. Statements
2. If else
3. Loop
4. Nested loop
5. Function call
Statement

The total time is found by adding the times for all

statement 1; statements:
statement 2;
... total time = time(statement 1) + time(statement 2) + ...
statement k;
+ time(statement k)

If each statement is "simple" (only involves basic

operations) then the time for each statement is constant
and the total time is also constant: O(1).
 16
Statements
 If Else

if (cond) then Here, either block 1 will execute, or block 2 will execute.
block 1 (statements) Therefore, the worst-case time is the slower of the two
else possibilities:
block 2 (statements)
end if; max(time(block 1), time(block 2))

If block 1 takes O(1) and block 2 takes O(N), the if-then-

else statement would be O(N)
 LOOP

The loop executes N times, so the sequence

for I in 1 .. N loop of statements also executes N times.
sequence of statements
end loop If we assume the statements are O(1), the
total time for the for loop is N * O(1), which
is O(N) overall.
 19
Loop(Linear Time)
 20
Loop(Linear Time)
 Nested LOOP

for I in 1 .. N loop
for Jin 1 .. M loop
The statements in the inner loop execute a
sequence of statements
total of N * M times. Thus, the complexity is
end loop;
O(N * M).
end loop;
 22
Nested Loop(Quadratic Time)
23
Function Calls

The behavior of function is same as statement if called once

Its behavior is statement in loop if it is called in loop

Its behavior is more like nested loop if it is called inside loop and it has an
characteristic loop inside as well
 25
Assignment#1

 What is the worst-case complexity of the each of the following code

fragments?
Two loops in a row:
for (i = 0; i < N; i++)
{ sequence of statements }
for (j = 0; j < M; j++)
{ sequence of statements }
How would the complexity change if the second loop went to N instead of
M?
 26
Assignment

 A nested loop followed by a non-nested loop:

for (i = 0; i < N; i++)
{ for (j = 0; j < N; j++)
{ sequence of statements } }
for (k = 0; k < N; k++)
{ sequence of statements }

 A nested loop in which the number of times the inner loop executes depends
on the value of the outer loop index:
for (i = 0; i < N; i++)
{ for (j = N; j > i; j--)
{ sequence of statements }
 27
Arrays
 2
Abstract Data Type

 There can be minimum four different

implementations of the List data structures:

 Using arrays
 Singly linked list
 Doubly linked list
 Circularly linked list.
 3
Abstract Data Type

 The interface to the List stayed the same, i.e., add(),

get(), next(), start(), remove() etc.

 The list is thus an abstract data type; we use it

without being concerned with how it is
implemented.
 5
Stacks

 Stacks in real life: stack of books, stack of plates

 Add new items at the top
 Remove an item at the top
 Stack data structure similar to real life: collection of
elements arranged in a linear order.
 Can only access element at the top
 6
Stack Operations

 Push(X) – insert X as the top element of the stack

 Pop() – remove the top element of the stack and
return it.
 Top() – return the top element without removing it
from the stack.
 7
Stack Operations

top 1
top 7 7
top 5 5 5
top 2 2 2 2
push(2) push(5) push(7) push(1)

top 21
top 7 7 top 7
5 5 5 top 5
2 2 2 2 top 2
1 pop() push(21) 21 pop() 7 pop() 5 pop()
 8
Stack Operation

 The last element to go into the stack is the first to

come out: LIFO – Last In First Out.
 What happens if we call pop() and there is no
element?
 Have IsEmpty() boolean function that returns true if
stack is empty, false otherwise.
 9
Stack Implementation: Array

 Worst case for insertion and deletion from an array

when insert and delete from the beginning: shift
elements to the left.
 Best case for insert and delete is at the end of the
array – no need to shift any elements.
 Implement push() and pop() by inserting and
deleting at the end of an array.
 10
Stack using an Array

top 1
2 5 7 1
7
5 0 1 2 3 4
2 top = 3
 11
Stack using an Array

 In case of an array, it is possible that the array may

“fill-up” if we push enough elements.
 Have a boolean function IsFull() which returns true is
stack (array) is full, false otherwise.
 We would call this function before calling push(x).
 12
Stack Operations with Array

int pop()
{
return A[current--];
}

void push(int x)
{
A[++current] = x;
}
 38
Stack Operations with Array

int top()
{
return A[current];
}
int IsEmpty()
{
return ( current == -1 );
}
int IsFull()
{
return ( current == size-1);
}

 A quick examination shows that all five operations take constant time.
 39
Stack Using Linked List

 We can avoid the size limitation of a stack

implemented with an array by using a linked list to
hold the stack elements.
 As with array, however, we need to decide where
to insert elements in the list and where to delete
them so that push and pop will run the fastest.
 40
Q&A

 Please feel free to ask any thing related to topic covered in these slides.

Lahore Garrison University

 41
Reference Material

 Text Book:
Nell Dale : C++ Plus Data Structures, Latest Edition, Jones & Bartlett
 Reference Text Books:
Data Structure using C/C++ by Mark Alan.
Yadidyah Langsam, Moshe J. Augenstein, Aaron M. Tenenbaum: Data
Structures using C and C++.
 Handouts:
Dr. Sohail Aslam: Data Structures Handouts.

Lahore Garrison University