Todays Topic:

Craig A. Shallahamer
OraPub, Inc.
craig@orapub.com
Quantifying
Oracle
Performance
Todays Topic:
Craig A. Shallahamer
OraPub, Inc.
craig@orapub.com
Scientifically
Evaluating the
Anticipated Effects of
Alternative
Performance
Solutions
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 1 of 24
2010 OraPub, Inc. Quantifying Performance
Want to dig deeper?
Training from OraPub
Oracle Performance Firefighting (I)
Adv Oracle Performance Analysis (II)
Oracle Forecasting & Predictive Analysis
OraPub 1-Day 2010 Perf Seminar
Books
Oracle Performance Firefighting (C. Shallahamer)
Forecasting Oracle Performance (C. Shallahamer)
Products
OraPub Stress Identifier now available.
Papers/Tools at www.orapub.com
M Solver web-site.
Scientifically Evaluating Alternative Performance Solutions
Introduction To Oracle Performance Optimization
Exploring Oracle Exploring Oracle s [Buffer Cache, Library Cache] s [Buffer Cache, Library Cache]
Craigs Blog at blogspot.com
Philadelphia
Oct 18-
Costa Mesa, CA
Nov 15-
Arizona/ Oct 4
Florida/ Oct 7
2010 OraPub, Inc. Quantifying Performance
OraPub presentations at OOW.
Sunday, 3pm
Resolving the Free Buffer Waits Event
S318617, Moscone West, L2, room 2005
Monday, 11am
Optimizing Internal Serialization Control
Unconference, Hotel Parc 55, Lombard room
Wednesday, 1pm
Evaluating the Anticipated Effect of Performance Sols.
Unconference, Hotel Parc 55, Lombard room
Thursday, 1:30pm
Quantifying Oracle Database System Performance
S315724, Moscone South, room 302
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 2 of 24
2010 OraPub, Inc. Quantifying Performance
OraPub is about Oracle performance.
OraPub is all about Oracle performance OraPub is all about Oracle performance
management; monitoring, firefighting, management; monitoring, firefighting,
quantitative and predictive analysis. quantitative and predictive analysis.
Web site started in 1995 and the company was Web site started in 1995 and the company was
founded in 1998 by Craig Shallahamer. founded in 1998 by Craig Shallahamer.
OraPub has always been about disseminating OraPub has always been about disseminating
Oracle database centric technical information. Oracle database centric technical information.
Consulting, training, books, papers, and Consulting, training, books, papers, and
products are now being offered. products are now being offered.
We have been on-site in 24 countries and our We have been on-site in 24 countries and our
resources have been received in probably resources have been received in probably
every country where there are DBAs. every country where there are DBAs.
Free Tools Free Tools
Free Papers Free Papers
Books Books
Products Products
Consulting Consulting
Training Training
Resources Resources
2010 OraPub, Inc. Quantifying Performance
Short resume.
Studies economics, mathematics, and computer science at Studies economics, mathematics, and computer science at
university in California, US. university in California, US.
Started working with Oracle technology in 1989 as a Forms Started working with Oracle technology in 1989 as a Forms
2.3 developer on Oracle version 5. 2.3 developer on Oracle version 5.
S Soon after started performance firefighting...daily! oon after started performance firefighting...daily!
Co-found both Oracle Co-found both Oracle s Core Technology and System s Core Technology and System
Performance Groups. Performance Groups.
Left Oracle to start OraPub, Inc. in 1998. Left Oracle to start OraPub, Inc. in 1998.
Authored 24 technical papers and worked in 24 countries. Authored 24 technical papers and worked in 24 countries.
Authors and also teaches his classes Authors and also teaches his classes Oracle Performance Oracle Performance
Firefighting Firefighting, , Adv Oracle Performance Analysis Adv Oracle Performance Analysis, and , and Oracle Oracle
Forecasting & Predictive Analysis Forecasting & Predictive Analysis. .
Authored the books, Authored the books, Forecasting Oracle Performance Forecasting Oracle Performance, and his , and his
new book, new book, Oracle Performance Firefighting Oracle Performance Firefighting. .
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 3 of 24
2010 OraPub, Inc. Quantifying Performance
If you want my firefighting book
Retail $49.95
Amazon $69.95
OraPub normal $39.95
OOW disc. $29.96
Cash in hand $30.00
2010 OraPub, Inc. Quantifying Performance
I think we can do better.
We get away with a lot and in our gut we know we can do
better.
Why cant completely honest questions be so seemingly
difficult to answer?
How do we objectively compare our proposed solutions?
How much of an impact can we expect?
Will users feel the impact?
What will the resulting utilization be?
How long with this SQL statement take?
Why is it so difficult to explain our solutions?
Answer: Analysis and communication methods.
Today I want to introduce you to how this type of analysis
and communication can be done.
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 4 of 24
2010 OraPub, Inc. Quantifying Performance
Our overall objective.
An introduction to how one
can objectively and
scientifically evaluate and
compare the anticipated
effect of alternative
performance solutions.
2010 OraPub, Inc. Quantifying Performance
In summary, here is the flow.
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 5 of 24
2010 OraPub, Inc. Quantifying Performance
First, plot a performance situation.
2010 OraPub, Inc. Quantifying Performance
Second combine respected disciplines.
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 6 of 24
2010 OraPub, Inc. Quantifying Performance
Third, objectively evaluate.
2010 OraPub, Inc. Quantifying Performance
Keep it as simple as possible.
Detail how to move
from red to blue
Provide just enough information to get your point
across and satisfy your audience.
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 7 of 24
2010 OraPub, Inc. Quantifying Performance
Step by step, how this is done.
Perform an Oracle response time analysis (ORTA) resulting in:
The bottleneck clearly identified,
Quantified total CPU consumption, and
Quantified total non-idle queue time.
Pick a good workload metric; one that correlates well with the
bottleneck.
Gather the total workload.
Calculate the key and classic performance parameters:
Arrival rate (work/time)
CPU service time (time/work)
Non-idle queue time (time/work)
Plot single point.
Create response time graph by combining the performance
situation with classic performance mathematics.
Objectively and scientifically compare alternative solutions!
2010 OraPub, Inc. Quantifying Performance
The sample interval.
All quantitative performance analysis
requires at least one sample interval
(either actual or inferred).
An interval requires a beginning and
ending performance situation snapshot.
The sample interval contains a single unit
of time, for example seconds or hours.
T
0
T
1
Snap 6754
Time 0930
Snap 6755
Time 1030
3600s
Section main data source: http://filebank.orapub.com/perf_stats/AWR_PROD25.html
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 8 of 24
2010 OraPub, Inc. Quantifying Performance
Lets get more Oracle specific.
Non-Idle
Wait Time
CPU Time
Other
IO
Srvr Prc
BG Prc
Time related to all the processed
work over a
specific period of time.
2010 OraPub, Inc. Quantifying Performance
The response time curve.
Graph created using
OraPubs RT Graph
Template.
Time
related to
completing
a single
unit of
work.
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 9 of 24
2010 OraPub, Inc. Quantifying Performance
Oracle RTA numeric classification.
While not visually as dazzling, numerically
classifying response time provides many benefits,
such as:
The entire performance situation is quantified providing
inputs into advanced analysis.
The construction naturally results in diagnosis.
The performance problem is highlighted.
Provides strong clues as to the operating system situation.
Makes before and after analysis simple.
Lends itself towards systematic diagnosis.
Allows for intelligent classification of time.
Can be used for casual and introductory work as well as
detailed and advanced analysis.
2010 OraPub, Inc. Quantifying Performance
A simple Oracle RTA template.
Notice the
response time
components are
ready for entry
based upon an
Oracle system.
Hence the term,
Oracle Response
Time Analysis or
ORTA for short.
Source: http://filebank.orapub.com/perf_stats/RTA_Template.pdf

Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 10 of 24
2010 OraPub, Inc. Quantifying Performance
Different example created using
OraPubs FF Diagnostic Template.
2010 OraPub, Inc. Quantifying Performance
RT numeric classification.
Different example created using
OraPubs FF Diagnostic Template.
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 11 of 24
2010 OraPub, Inc. Quantifying Performance
The workload and arrival rate.
The total work performed during a sample interval can
be represented a number of ways; 87,788,193 logical
IOs, 1,796,277 physical IOs, etc.
The arrival rate is the total workload over the sample
interval. Therefore, both a unit of work and a unit of
time are required. For example, LIOs/sec or PIOs/sec.
As the arrival rate increases, the system becomes
busier and at some point performance will degrade.
Suppose over a one hour interval, 87,788,193 LIOs
and 148,439 PIOs occurred. The workload per
second, that is, the arrival rate is:
87,788,193 LIOs / 3600 sec = 24,385.6 LIOs/sec
1,796,277 PIOs / 3600 sec = 499.0 PIOs/sec
2010 OraPub, Inc. Quantifying Performance
Response time statistics.
T
0
T
1
Snap 6754
Time 0930
Snap 6755
Time 1030
3600s
1,796,277 PIOs
87,788,193 LIOs
87,788,193 LIOs / 3600 sec = 24,385.6 LIOs/sec
Arrival rates
1,796,277 PIOs / 3600 sec = 499.0 PIOs/sec
Total LIO workload
Total PIO workload
Over the interval
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 12 of 24
2010 OraPub, Inc. Quantifying Performance
Servicing a unit of work.
Each unit of work that is serviced takes/requires/consumes a
specific amount of time some resource. For example 1.50 ms
of CPU.
For Oracle performance analysis, CPU consumption works
wonderfully as the unit of time. For example, over an 3600
second sample interval, Oracle may have consumed 2307
seconds of CPU.
Service time is the CPU time required to process the work
performed during the sample interval.
For example, if during the 3600 sec interval 2307 seconds of
CPU was consumed and 1,796,277 PIOs occurred, the
service time would be 2307 sec / 1,796,277 PIOs = 0.00128
sec/PIO = 1.28 ms/PIO.
In theory service time is constant, but in practice may
increase as the arrival rate increases.
2010 OraPub, Inc. Quantifying Performance
Response time statistics.
T
0
T
1
Snap 6754
Time 0930
Snap 6755
Time 1030
3600s
1,796,277 PIOs
87,788,193 LIOs
Arrival rates
Total LIO workload
Total PIO workload
Total CPU consumed 2307 CPU secs
2307 sec / 87,788,193 LIO = 0.0263 ms/LIO
Service times
2307 sec / 1,796,277 PIO = 1.28 ms/PIO
Over the interval
87,788,193 LIOs / 3600 sec = 24,385.6 LIOs/sec
1,796,277 PIOs / 3600 sec = 499.0 PIOs/sec
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 13 of 24
2010 OraPub, Inc. Quantifying Performance
Waiting to be serviced, queuing.
As the arrival rate increases, processes will have to wait/queue longer to
be serviced. At some point, the queue time becomes so significant,
performance becomes intolerable.
Common units of work are the same as with service time and the
workload; physical IOs, logical IOs, and block changes.
For Oracle performance analysis, Oracle wait interface non-idle time
works wonderfully as the unit of time. For example, over an 3600 second
sample interval, Oracle processes may have waited 1867 seconds.
Queue time is the wait time related to processing work during the sample
interval.
For example, if during the 3600 sec interval 1867 seconds of wait time
occurred and 1,796,277 PIOs where processed, the queue time would be
1867 sec / 1,796,277 PIOs = 0.00104 sec/PIO = 1.04 ms/PIO.
Oracle wait time can be classified in a number of different ways to aid in
diagnosis, analysis, and resolution.
2010 OraPub, Inc. Quantifying Performance
Response time statistics.
T
0
T
1
Snap 6754
Time 0930
Snap 6755
Time 1030
3600s
1,796,277 PIOs
87,788,193 LIOs
Arrival rates
Total LIO workload
Total PIO workload
Total CPU consumed 2307 CPU secs
Service times
Total wait time 1867 secs
1867 sec / 87,788,193 LIO = 0.0213 ms/LIO
Queue times
1867 sec / 1,796,277 PIO = 1.04 ms/PIO
Over the interval
87,788,193 LIOs / 3600 sec = 24,385.6 LIOs/sec
1,796,277 PIOs / 3600 sec = 499.0 PIOs/sec
2307 sec / 87,788,193 LIO = 0.0263 ms/LIO
2307 sec / 1,796,277 PIO = 1.28 ms/PIO
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 14 of 24
2010 OraPub, Inc. Quantifying Performance
R
t
= S
t
+ Q
t
Response time is a pain indicator; the higher the response
time the more the pain.
Response time is non-linear and is related to service time,
the number of servers (e.g., CPUs), workload, arrival rate,
utilization, and queue time.
As the workload increases, queue time increases which
increases response time. This assumes service time is
constant, which is not always the case.
A single performance sample interval creates a single point
on a response time curve.
Combining sample performance interval data with response
time math produces a representative response time graph,
providing a number of advanced analysis benefits.
2010 OraPub, Inc. Quantifying Performance
Response time statistics.
T
1
3600s
1,796,277 PIOs
87,788,193 LIOs
Arrival rates
Total LIO workload
Total PIO workload
Total CPU consumed 2307 CPU secs
Service times
Total wait time 1867 secs
Queue times
Over the interval
Total response time 4174 secs
0.026 + 0.021 = 0.0476 ms/LIO
Response times
1.28 + 1.04 = 2.32 ms/PIO
87,788,193 LIOs / 3600 sec = 24,385.6 LIOs/sec
1,796,277 PIOs / 3600 sec = 499.0 PIOs/sec
1867 sec / 87,788,193 LIO = 0.0213 ms/LIO
1867 sec / 1,796,277 PIO = 1.04 ms/PIO
2307 sec / 87,788,193 LIO = 0.0263 ms/LIO
2307 sec / 1,796,277 PIO = 1.28 ms/PIO
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 15 of 24
2010 OraPub, Inc. Quantifying Performance
First analyze performance situation.
2010 OraPub, Inc. Quantifying Performance
First analyze performance situation.
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 16 of 24
2010 OraPub, Inc. Quantifying Performance
Second, combine respected disciplines.
2010 OraPub, Inc. Quantifying Performance
Third, objectively evaluate.
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 17 of 24
2010 OraPub, Inc. Quantifying Performance
OraPubs comparison spreadsheet.
Use to
compare the
baseline with
two
alternatives.
Download
free from
OraPub.com
2010 OraPub, Inc. Quantifying Performance
Using OraPubs on-line M-Solver.
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 18 of 24
2010 OraPub, Inc. Quantifying Performance
2/3
2010 OraPub, Inc. Quantifying Performance
3/3a
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 19 of 24
2010 OraPub, Inc. Quantifying Performance
3/3b
2010 OraPub, Inc. Quantifying Performance
Reality is not so pretty.
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 20 of 24
2010 OraPub, Inc. Quantifying Performance
Reality is not so pretty.
2010 OraPub, Inc. Quantifying Performance
Keep it as simple as possible.
Detail how to move
from red to blue
Provide just enough information to meet get your
point across and satisfy your audience.
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 21 of 24
2010 OraPub, Inc. Quantifying Performance
In summarystep by step.
Perform an Oracle response time analysis (ORTA).
Pick a good workload metric.
Gather the total workload.
Calculate the key and classic performance
parameters.
Plot single point.
Create response time curve by combining
performance situation with classic performance
mathematics.
Objectively and scientifically compare alternative
solutions!
2010 OraPub, Inc. Quantifying Performance
Want to dig deeper?
Training from OraPub
Oracle Performance Firefighting (I)
Adv Oracle Performance Analysis (II)
Oracle Forecasting & Predictive Analysis
OraPub 1-Day 2010 Perf Seminar
Books
Oracle Performance Firefighting (C. Shallahamer)
Forecasting Oracle Performance (C. Shallahamer)
Products
OraPub Stress Identifier now available.
Papers/Tools at www.orapub.com
M Solver web-site.
Scientifically Evaluating Alternative Performance Solutions
Introduction To Oracle Performance Optimization
Exploring Oracle Exploring Oracle s [Buffer Cache, Library Cache] s [Buffer Cache, Library Cache]
Craigs Blog at blogspot.com
Philadelphia
Oct 18-
Costa Mesa, CA
Nov 15-
Arizona/ Oct 4
Florida/ Oct 7
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 22 of 24
2010 OraPub, Inc. Quantifying Performance
Visit the IOUG Booth This Week
Located in the User Group Pavilion - Moscone West,
2
nd
Floor
Learn why over 23,000 have joined IOUG and what it
can do for you
Chat with the IOUG Board of Directors
Hear about new regional IOUG BI user communities
Find out how to submit an abstract for COLLABORATE
11 IOUG Forum
Enter for a chance to win a COLLABORATE 11
registration
2010 OraPub, Inc. Quantifying Performance
Thank
You!
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 23 of 24
Todays Topic:
Craig A. Shallahamer
OraPub, Inc.
craig@orapub.com
Scientifically
Evaluating the
Anticipated Effects of
Alternative
Performance
Solutions
Craig Shallahamer's 2010 OracleWorld Presentation (c)2010 OraPub, Inc.
(c)2010 OraPub, Inc. Page 24 of 24