Oracle Exalytics BestPractices

Oracle Exalytics and TimesTen Best Practices
Mark Rittman, Technical Director, Rittman Mead

Oracle Openworld 2012, San Francisco, September 2012
T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
Saturday, 29 September 12
Mark Rittman
Mark Rittman, Co-Founder of Rittman Mead
Oracle ACE Director, specialising in Oracle BI&DW
14 Years Experience with Oracle Technology
Regular columnist for Oracle Magazine
Author of two Oracle Press Oracle BI books
Oracle Business Intelligence Developers Guide
Oracle Exalytics Revealed
Writer for Rittman Mead Blog :
http://www.rittmanmead.com/blog
Email : mark.rittman@rittmanmead.com
Twitter : @markrittman
About Rittman Mead

Oracle BI and DW gold partner
World leading specialist partner for technical excellence, solutions delivery and
innovation in Oracle BI
Approximately 30 consultants worldwide
All expert in Oracle BI and DW
UK based
Offices in US, Europe (Belgium) and India
Skills in broad range of supporting Oracle tools:
OBIEE
OBIA
ODIEE
Essbase, Oracle OLAP
GoldenGate
Exadata
What is Oracle Exalytics?

Hardware Element
Sun Fire X4470 M2 server
1TB RAM, 40 Cores, 3.6TB HDD
Software Element
OBIEE 11.1.1.6 with Exalytics Enhancements
Oracle Essbase 11.1.2 with Exalytics Enhancements
Oracle TimesTen 11.2.2.2 for Exalytics
New in v1.1 : Support for Oracle Endeca
Information Discovery, Golden Gate, ODI
Runs on 64-bit Oracle Linux
(Exalogic distribution)
OBIEE and Essbase are licensed as
Oracle BI Foundation
Exalytics features can only be used in
conjunction with Exalytics hardware
Rittman Mead and Oracle Exalytics

Rittman Mead are the first Oracle Partner in the UK to buy an Exalytics box
Ordered February 2012 through Arrow (reseller), delivered end-April 2012
Hosted in a secure datacenter in London Docklands, access via VPN
To be used for customer PoCs, training, testing and possible cloud hosting
Installation and Delivery Process

Installed by Oracle ACS over two days
Day 1 involved unpacking, racking, connecting to network etc
Day 2 was for installing the software (only ships with Oracle Linux)
Oracle WebLogic
Oracle BI EE 11g
Oracle TimesTen for Exalytics
Post-install config from ACS for
Creating initial TimesTen DB
Configuring BI EE for Exalytics
Configuring BI Server > TimesTen connection
Post-install config from Rittman Mead included:
Configuring RPD for Usage Tracking &
Summary Statistics
Connecting RPD to TimesTen DB
Exalytics as the Exa-Machine for BI

Runs the BI layer on a high-performance, multi-core, 1TB server
In-memory cache used to accelerate the BI part of the stack
If Exadata addresses 80% of the query performance,
Oracle BI
Exalytics addresses the remaining 20%
Consistent response times for queries
In-memory caching of aggregates
40 cores for high concurrency
In-Memory DB/Cache
Re-engineered BI and OLAP software
that assumes 40 cores and 1TB RAM
ERP/Apps
DW
Also Supports Essbase, and Endeca Information Discovery

In-Memory Essbase for planning, budgeting and sales analysis-style OLAP applications
Endeca Information Discovery for search/analytic applications against diverse data
Oracle
Exalytics
In-Memory
Machine
Information Discovery
Unified
Search
Faceted
Navigation
Interactive
Exploration
Unified
Indexing
Text
Analysis
Essbase Planning Engine

Data
Mashup
In-Memory Cache
Lock
Manager
Smart Storage
Manager
Enables High-Density Analysis of Big Data

BI tier is rarely the bottleneck, but it can be if very dense visualizations are used
Sparklines, grid of charts etc
Exalytics 40 cores and 1TB RAM make higher density presentation viable
Single query sent to the database
Exalytics breaks data up to create microcharts
Also helps support high numbers of concurrent users (100+)
Exalytics Architecture
Developer tools used for defining
and maintaining aggregate definitions
Client tools for OBIEE repository,
TimesTen and Oracle Database
Developer Workstation
Oracle BI Administration Tool
Summary Advisor Wizard
nqcmd
TimesTen Client
SQL Developer
Oracle Database 11g Client
Exalytics layer contains OBIEE,

TimesTen and Essbase
(not pictured)
Database tier contains FMW
database repository, including
usage tracking and summary statistics
Oracle Business Intelligence

WLS Admin
Server
WLS Managed
Server
Oracle BI
System Comp.
BI Server
Aggregate
Schema
Database for RCU Schemas

Enhanced Usage Tracking
Summary Statistics
Exalytics Under the Covers - How Does it Work?

Exalytics brings together different technologies, which are still standalone products in
their own right
To harmonise and optimise their use within Exalytics, it utilises the following techniques:
In-Memory Adaptive Data Mart
In-Memory Intelligent Result Cache
In-Memory Cubes
Some of these are genuine "secret sauce"
New functionality and algorithms
You can only get them through licensing Exalytics
Others are descriptions of DW/BI strategies, or existing product functionality, extended
to take advantage of the capacity for processing in memory that Exalytics has
Commonly-used aggregates are copied into Oracle TimesTen for Exalytics

Past query patterns are analyzed and suitable aggregates recommended
Oracle BI Server then uses these aggregates to make queries run faster
Aggregates change over time in
response to changes in query patterns
TimesTen
BI Server
Tools are provided for managing
and populating these aggregates
Aggregates
Data Warehouse
Detail-level
Data
Exalytics
In-Memory Adaptive Data Mart
OBIEE has an advanced result cache, which can support:

Queries matching previous ones
Queries matching a subset of the results of a cached query resultset
Queries aggregating the results of a cached query resultset
Stored in cache files on the OBIEE server filesystem
With Exalytics, the 1TB of RAM typically means
that these files are held in memory
RAMdisk can also be created for
BI Server & Presentation Server
TimesTen
BI Server
working area files
In-Memory
Result
Cache
Exalytics
In-Memory Intelligent Results Cache

New version of TimesTen specifically for Exalytics (and only available with Exalytics)
Support for analytic functions
Perform all the processing at source
Combine with being in-memory = should be very fast
Column compression
Whitepaper cites 5x
Given the hardware capacity, we could seriously contemplate loading the whole Data
Warehouse into memory
Opens up lots of interesting design potential
We can load aggregates into TimesTen, leave base data at source, and use OBIEEs
Vertical Federation capability to seamlessly report across both
All hidden from the end-user, all they will know is that their reports run fast!
TimesTen and OBIEE Architecture

Oracle BI Server communicates with TimesTen through TimesTen Client
Summary Advisor, and nqcmd use Oracle BI Server to access TimesTen
Typical single TimesTen database per Exalyics machine
Max TimesTen database size around 300MB
- Due to need to set aside equal
Summary
Oracle BI
Advisor
Server
Temp size for the Perm size selected
Clustered Exalytics boxes can be daisy-chained
ODBC
together using InfinBand connections
For HA scenarios, does not increase
available RAM
TimesTen
Memory-Resident
Summary advisor scripts write to both TimesTen
Database
databases, replicating aggregates
TimesTen databases can be wired together
for failover/HA purposes
nqcmd
Checkpoint
Files
Log
Files
Summary Advisor for Aggregate Recommendation & Creation

Utility within Oracle BI Administrator tool that recommends aggregates
Bases recommendations on usage tracking and summary statistics data
Captured based on past activity
Runs an iterative algorithm that searches,
each iteration, for the best aggregate
Recommendations Based on Enhanced Usage Tracking Data

Historically, usage tracking has been tracked using S_NQ_ACCT
Holds basic usage tracking statistics + logical SQL query
Now supplemented by S_NQ_DB_ACCT
Extra usage tracking information, includes physical SQL
Exalytics Summary Advisor uses S_NQ_SUMMARY_ADVISOR
Contains summary statistics, execution time etc
Gathered at same time as usage tracking when
Exalytics is enabed
Contents can be derived from usage tracking if needed
Oracle Exalytics Best Practices

Installation, Configuration for Multiple Environments, Management
Choosing What Goes in the Adaptive Data Mart
Where Does Exalytics Add Value?
Measuring Exalytics Performance and Metrics
Maintaining and Refreshing the Adaptive Data Mart
Configuring Multiple Environments (Consolidation)

Exalytics main value proposition is around performance
In-memory aggregates speeding up query performance
However many customers wish to use Exalytics for consolidation
Run multiple environments (test, dev, prod) on a single server
Consolidate multiple applications on a single server
Desire is usually down to cost (multiple physical servers, per-processor licensing)
Single Exalytics Server
Exalytics Svr 1
Exalytics Svr 2
Exalytics Svr 3
MWHOME1
MWHOME1
MWHOME2
MWHOME3
MWHOME1
MWHOME1
DEV
DEV
DEV
DEV
TEST
vs.
PROD
Common Consolidation Approach - Multiple Installs

Up until recently, customers wishing to consolidate usually performed multiple
Middleware Home installs
OBIEE element usually works fine (installer detects existing ports use, configures OK)
However TimesTen element may not work as expected with multiple installs
Checkpointing clashes between TT installs, single TT install vs. multiple installs etc
No resource caging between MWHome environments
Approach is therefore not recommended for consolidation use-cases
MWHOME1
MWHOME2
MWHOME3
DEV
DEV
DEV
Best Practice for Consolidation : Oracle Virtual Machine

Exalytics is now certified for use with
Oracle Virtual Machine, Oracles
hard-partitioning virtualization platform
OVM is installed first, on Exalytics bare
metal, with OVM Repository on external
DB server (as with BIPLATFORM, MDS)
Exalytics VM templates then used to create
required environments (dev, test etc)
Also allows for licensing in smaller
per-processor license blocks (5, 10, 15 etc)
OVM provides for resource caging,
isolating environments from each other

OVM Guest 1
OVM Guest 2
OVM Guest 3
MWHOME1
MWHOME1
MWHOME1
DEV
DEV
DEV
OVM Hypervisor (Oracle VM Server)

Oracle VM
Repository
Oracle VM
Manager
High Availability using Clustered Exalytics Servers

Multiple Exalytics servers can be clustered for HA and performance (load balancing)
Requires hardware load balancer above clustered servers
Linked together via InfiniBand (one for clustering, one for connection to Exadata)
Note: does not double available RAM
- aggs are replicated in each TT database
Note that Summary Advisor / nqcmd
does not replicate agg tables manual replication required
HA is between TT instances, and OBIEE
managed server + system components
Management Best Practices

Exalytics is primarily an OBIEE system, and usual best practices apply
May want to consider cloning tool (CloneZilla etc) for cloning clean server build
New : Oracle Enterprise Manager 12cR2 and BI Management Pack
OEM 12cR2 prerequisite for sub-capacity licensing using OVM
OEM 12cR2 Additional Capabilities vs. Fusion Middleware Control

Ability to report on DAC runs for BI Apps,
plus BI configuration settings etc
Usage tracking reporting of dashboard usage,
analysis run-times etc
Manage Essbase and other BI / DB /
Middleware components
Long-term persistence of metrics, SLAs, alerts
Middleware cloning/
management
What Should the TimesTen Adaptive Data Mart be Used For?

Standard approach is to store aggregates in the TimesTen datamart
Aggregated by the source DB, aggregates then cached in TT database
Other approaches could be used, however
Store whole detail-level dataset in the TT database
Store just recent detail-level data in TT, and use OBIEEs fragmentation feature
Use Essbase to hold the aggregates instead
Use Essbase to hold detail-level + aggregates
Where does Exalytics add value?
What should go in the TimesTen database?
Potential TimesTen Use Case : In-Memory Database Cache

Automatically replicate hot transactional data from Oracle into TimesTen
Use OBIEE fragmentation to enable automatic navigation between sources
Aggregation performed by both TimesTen, and by source DB (as appropriate)
Cache Group is
defined on TimesTen
Transactional data
Transactional data
DATE = TODAY
DATE = TODAY
In-Memory DataBase
Cache
LTS Fragmentation
in the RPD
specifies the data
cutoff
DATE < TODAY
No change to Oracle
source server, except
minor schema config
BI Server
OBIEE chooses the

correct source
based on date
range
Conclusions and Best Practices Around Detail/Aggregate Storage

Best use of OBIEE + TT is how the product was intended to be used
Use TimesTen for storing (caching) pre-created aggregates
Aggregates created by source DB (Oracle, SQL Server etc)
Aggregation not performed by TimesTen (as it would be if it held detail-level data)
IMDB interesting approach, but relies on TT performing aggregation (slow)
Same goes for holding whole detail-level source in TimesTen
Also, bear in mind that majority of performance benefit may come from re-hosting
OBIEE application on the Exalytics hardware
Particularly with Exadata sources, source DB may even be fastest host for aggs
Test for your data, do not pre-suppose outcome
Testing Exalytics
Need to quantify the performance benefit of various Exalytics deployment options
Take a baseline, and then repeat the same test for incremental changes
Several test framework options
Manual
Homebrew - Logical SQL + nqcmd
Performance test tool - JMeter, OATS, etc
Logical
SQL
Extract Logical
SQL, run with
nqcmd
TimesTen
nqcmd
BI Server
Run dashboards
manually, and
time with a
stopwatch
Presentation
Services
JMeter
Fully automated
web user
performance
test tool
Plan your testing

Tempting to go in all guns blazing
40 cores!! 1TB memory!! w00t!11!
Decide on the aim of the test. For example:
Validate a configuration option
Determine the response time for a dashboard
Test the performance under concurrent load
For configuration option testing, emphasis should be on absolute repeatability
If testing to determine system capacity, make sure the workload model is representative
Setting up JMeter
JMeter can run on Windows or *nix
No installation, just unzip
Record and customize test script
Execute from JMeter running from suitable location
Consider network, and host server size
Option 1
Run locally on
Exalytics
Option 2
Option 3
BI Server
BI Server
BI Server
TimesTen
TimesTen
TimesTen
JMeter
JMeter
Virtual Machine
JMeter
Run within a
VM on
Exalytics
Run remotely,
across
appropriate
network link to
Exalytics
Analyzing the test output

Whats the response time on a dashboard?
How long did

the dashboard
take to run?
How does the response time

change if there are concurrent
users?
Response Time
vs
Number of users
Other metrics
Will the system support xxxx users? is
a common question
JMeter is good at simulating multiple
users
Be careful with your workload model
though
For system capacity in particular, monitor
OS and OBIEE performance metrics
Consider EM12cR2 for long-term metrics
OBI metrics
OS metrics
Refreshing Aggregates Within the TimesTen Data Mart

Aggregates are built in TimesTen to support sub-second response times
Summary Advisor tool suggests aggregate dimensionality and grain, generates script
BI Servers Aggregate Persistence executes script:
1. Create TimesTen aggregate table
2. Populate TimesTen aggregate
3. Update RPD online with new aggregate metadata
Handles supporting dimensions too
Base data
Aggregates
OBIEE Aggregate
Persistence
Aggregate tables
loaded from base
data
Aggregate tables
created
TimesTen
RPD
RPD updated with

new aggregate
mappings
Aggregate Persistence limitations

To refresh an aggregate, it is deleted and rebuilt from scratch
The RPD is edited directly on the BI Server each time an aggregate is created or rebuilt
Build failures can be difficult to debug
If a rebuild fails it can leave the RPD in an inconsistent state with TimesTen
This is that the
error the user
sees
After a failed build, the aggregates

are still in the RPD, but no longer
exist in TimesTen
nqquery.log suggests possible

errors but no clear root cause
Alternatives to using Aggregate Persistence

However aggregates are refreshed, they must be included
in the RPD
Can be done manually
Aggregate Persistence is useful for this
Complete refresh of aggregate data
Extract the SQL that OBIEE generates in Aggregate
Persistence, run this through ODI
Write bespoke aggregate refresh code in ODI
Incremental refresh using GoldenGate and ODI
Instead of rebuilding aggregates in their entirety each
time, only update the part of the aggregate that has
changed on the base data
Incremental refresh of aggregates using GoldenGate and ODI

Changes to base data are replicated by GoldenGate onto Staging database
ODI Interfaces apply deltas to the TimesTen aggregates
Base data
(any supported DB)
GoldenGate
GoldenGate
Aggregates
CDC staging data
ODI
TimesTen
Oracle
GoldenGate
TimesTen
JDBC client
ODI
agent
Considerations if using Alternatives to Aggregate Persistence

Exalytics Out of the Box is designed around using Aggregate Persistence
Fair assumption that future developments and optimizations will be around this
model
If rate of change in aggregate requirements is high, then manual RPD changes can be
time-consuming
Hybrid approach - use Summary Advisor and Aggregate Persistence as an accelerator
for the initial RPD and Aggregate build, and then handle refreshes manually
There is no one answer -- It Depends!
Do you have time & skills available to write bespoke code?
Rate of change in your aggregate requirements
Existing deployment landscape and technologies available
Risk of missing out on Exalytics optimizations if too far from OOTB deployment?
Summary
Exalytics is a powerful platform for hosting OBIEE, optimized for a BI workload
New supported options for hosting multiple environments on a single server
Oracle VM now permitted for multiple environments and sub-capacity licensing
TimesTen is best used as a cache store for aggregate tables
Typically, use source DBs for initial aggregation and storage of detail-level data
Use JMeter or other metrics capture tools to measure performance before and after
Manage Exalytics (+ OVM) using EM12cR2 Cloud Control
Options to use ODI + GoldenGate as alternatives to Summary Advisor for agg refreshes
More Information
Thank you for attending this presentation
More information can be found at http://www.rittmanmead.com
Contact us at info@rittmanmead.com or mark.rittman@rittmanmead.com
Look out for our book, Oracle Business Intelligence Developers Guide due Q3 2012
Follow-us on Twitter (@rittmanmead) or Facebook (facebook.com/rittmanmead)
Oracle Exalytics and TimesTen Best Practices

Mark Rittman, Technical Director, Rittman Mead
Oracle Openworld 2012, San Francisco, September 2012

Oracle Exalytics BestPractices

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Oracle Exalytics BestPractices

Transféré par

Droits d'auteur :

Formats disponibles

Oracle Exalytics and TimesTen Best Practices

Mark Rittman, Technical Director, Rittman Mead

About Rittman Mead

What is Oracle Exalytics?

Rittman Mead and Oracle Exalytics

Installation and Delivery Process

Exalytics as the Exa-Machine for BI

Also Supports Essbase, and Endeca Information Discovery

Essbase Planning Engine

Enables High-Density Analysis of Big Data

Exalytics layer contains OBIEE,

Oracle Business Intelligence

Oracle TimesTen for Exalytics

Database for RCU Schemas

Exalytics Under the Covers - How Does it Work?

Commonly-used aggregates are copied into Oracle TimesTen for Exalytics

In-Memory Adaptive Data Mart

OBIEE has an advanced result cache, which can support:

In-Memory Intelligent Results Cache

Oracle TimesTen for Exalytics

TimesTen and OBIEE Architecture

Summary Advisor for Aggregate Recommendation & Creation

Recommendations Based on Enhanced Usage Tracking Data

Oracle Exalytics Best Practices

Configuring Multiple Environments (Consolidation)

Common Consolidation Approach - Multiple Installs

Best Practice for Consolidation : Oracle Virtual Machine

Single Exalytics Server

OVM Hypervisor (Oracle VM Server)

High Availability using Clustered Exalytics Servers

Management Best Practices

OEM 12cR2 Additional Capabilities vs. Fusion Middleware Control

What Should the TimesTen Adaptive Data Mart be Used For?

Potential TimesTen Use Case : In-Memory Database Cache

DATE < TODAY

OBIEE chooses the

Conclusions and Best Practices Around Detail/Aggregate Storage

Plan your testing

Analyzing the test output

How long did

How does the response time

Refreshing Aggregates Within the TimesTen Data Mart

RPD updated with

Aggregate Persistence limitations

After a failed build, the aggregates

nqquery.log suggests possible

Alternatives to using Aggregate Persistence

Incremental refresh of aggregates using GoldenGate and ODI

(any supported DB)

CDC staging data

Considerations if using Alternatives to Aggregate Persistence

Oracle Exalytics and TimesTen Best Practices

Vous aimerez peut-être aussi