AWR Report Advisor (AWRA)

New Hire ;-)

Eri Soares, PMP

Juan Sepulveda

Oracle Support Engineer

Oracle Database Performance Team

March 2018

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |

Project General Information
• Project Name: AWR Advisor (AWRA) Automation – Level 3/4
• Resources:
– Eri Soares ( Developer / Architect )
– Juan Sepulveda (Architect / Database Performance Manager)
– Ahmed Aburob (Dadatabase Performance Automation Leader)
– Jenny Jackson (Database Automation Leader)
– Suyog Pathak (Database Performance Manager)
– Steve Franchi (Sponsor / Dbperf Senior Director / Support Automation Head)

• Actual Project Start: 10-Aug-17

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 2
AWRA Project [Over]Achievements
• Give it a simplified way to improve rules over time (continuous
improvement) ✔

• Provide support to engineers to develop themselves in analyzing AWR

Reports ✔

• Create a tool, flexible and transparent, not hard coded, to allow update,
inclusion and deletion of AWR Analysis Flow from end-to-end ✔

• AWRA is a unique Framework ✔

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 3
TIME TO RESOLUTION

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 4
 Dbperf Team
AWR Advisor (AWRA) & Team and AWRA
Excellence in
Continuous AWR Reports
Improvement & AWRA Web
Standardization 101000100110100101010101000101
Portal
0011010010101010100010101010010
1010001001101001010101010001010101
AWRA and AWR
001010101010001010101001010
Report Training
1010010100010101010010101000101010100
AWRA
Improvements &
Enhancements

AWRA BETA
Release Version

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 5

What is not realized (Main Points)
• Parse issue using Python library called ElementTree and broken AWR
Reports
–We need to replace it by Beautiful Soup
• Inclusion of AWR Report New Exadata related sections
• ASH Report parse for 12c on
• Organize recent code inclusions
–Some feature were coded to work under unnecessary limitation (Save
time)
• AWRA Documentation
• AWRA training

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 7
What is not realized (Main Points)
• Finish HTML version for AWRA Report Analysis
• Semi-auto integration with BugTag database
• Auto-identification of DQA Analysis Accuracy and coverage of Wait Events
• Auto-Validation of XML Files rules errors
• English Rationale output analysis
• AWR Report compare (Good/Bad)
–It impacts on some AWR Report user cases and Analysis results
• More tests
• Investigations for some DQA Classes/Sub Classes

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 8
Miscellaneous
• More than 5000 Lines of configuration on the XML File
– 8 DQA Wait Classes
– 56 DQA Wait Sub Classes
– 238 Wait Events
– 56 Investigation Template Names (Not all has checks and/or action plans yet)
– 101 Checks (Sanity Checks and investigation Checks)
– 32 Special Checks for SQLs
– 3 Wait Event sensibility
– 113 Action plans
• Compatible to 10.1 – 12.2
• 8860 Lines of Python code
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 9
Database Quick Analyzer Wait Classes
(DQA Wait Classes Analysis)

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 10
Wait Classes – AS IS

Pre-Defined Wait
Classes

1:1
Wait Event Wait Event Class

 It doesn’t provide enough information to determine where the

issue is coming
 Too generic
 Not taking into account the complexity of the correlations
among the wait events

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 11
Wait Classes – AS IS

Pre-Defined Wait
Classes

1:1
Wait Event Wait Event Class

Which additional information Wait Event Class in this model provides to

help in the investigation?

It’s very limited.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 12
DQA Wait Classes – TO BE *NEW THINKING!*

Root cause driven

Wait Classes

*:*
Wait Event Wait Event Class

 More specific issue driven wait classes

 Helps to narrow down the issue
 Allows to guide the investigation to the next step along with
the Initial Checks.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 13
DQA Wait Classes Analysis In Practice ….

Enq: tx – index
Log file sync Write complete waits
contention

1. Inserts on
1. Too much index (right 1. DBWR
commits side) settings

2. IO Write 2. IO Write 2. IO Write

This is what the DQA Wait Classes Analysis does.

We are going to kick in the investigation template IO_WRITE

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 14
DQA Wait Classes Analysis In Practice ….

Cursor: pin S wait on

Latch free Latch: redo allocation
X

1. xxxxxxx 1. xxxxxxx 1. xxxxxxx

2. CPU 2. CPU 2. CPU

Startvation Startvation Startvation

That’s going to trigger Investigation Template DBCPU

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 15
 • All flexible
• Allows new investigations
from scratch
• Powered by XML

ELT
DQA ANALYSIS INVESTIGATION CHECKS
(Extract/Load/Tranform) ACTION PLANS
INITIAL CHECKS - Root cause driven (Investigation
- Parse it - Detailed Action Plans
- Sanity checks - Group wait event Templates) RATIONALE
- Load it in Memory - Targeting either solving
- Focused on making analysis - Get the most out of
- Patternize it across the AWR Report the issue OR gathering
sure if any major issue in Single wait event NEXT PHASE
RDBMS version more relevant
CPU, MEM, IO is found analysis - Across SME
- Creates new relevant information on it
- Across SME(RDBMS) - Hot bugs checks
metrics

AWRA FRAMEWORK

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 16
Architecture Model – Investigation Item Vs. Investigation
Template
Example
Library
SGA Resizes cache
Invalidations

CPU and so
Parse
on
Mutexes

Investigation templates will be either 100% configured in the XML File Or

mixed with hard code.
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 17
AWRA Investigation Template / Documentation Example
IO_READ

From Non
From Users
Users

Indirect Direct
RMAN DATAPUMP
Reads Reads

Buffer Segments Action Plans Action Plans

HitRatio Outliers

Segments SQLs
Outliers Outliers

SQLs AWRA do not necessary take only one

Action Plans path. AWRA is not a regular decision
Outliers
tree, it takes all significant paths.
Action Plans

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 18
AWRA Action Plans Example
OSWATCHER

WINDOWS EXADATA LINUX/UNIX

Action Plans Action Plans Action Plans

SQLTXTRACT

SQLT SQLT
INSTALL GETREPORT

Action Plans Action Plans

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 19
Bug Matches – Only for bugs with wait event in symptoms…

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 20
 Issue
Today’s Limitations
Database Session/Job SQL Tuning
1. Do not use Good/BAD Approach
1. In case of customer issue is taking a relevant part of Database time (Almost not Database Performance)
2. There is a possibility to mitigate some cases by changing XML File as long as you are trained on AWRA and AWR Reports

2. Not Database Performance

1. If the issue is at session/job level it will be as helpful as the information in the AWR Report
2. This is not for SQL Tuning issues neither Database hung issues

3. Oracle RAC
1. Only analyze 1 AWR Report at a time which is not the best way

4. Bulk AWR Report Analysis

1. Because of the new HTML Report which can be removed

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 21
IT IS DEMO TIME!!
1. Parse Error
2. Small XML File (Small Brain)
3. Big XML File (Smart Brain)
4. Report Output Analysis
5. RAC Node 1

START DEMO

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 22
AWR ADVISOR (AWRA) – ALL ABOUT IT DOC ID
2370590.1

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 23
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 26