Vous êtes sur la page 1sur 49

Exadata V2 / Sun Oracle Database Machine and POC results for Market Risk Reporting

Presentation is split into 2 parts


Slides 3 26 highlight Market Risk POC results, just completed for another Tier 1 Investment Bank I wanted to provide a very relevant real life example of Oracle Exatadatas performance capabilities Slides 27 - 49 provide an overview of Oracles Exadata / Database Machine

Market Risk Reporting & Analysis


POC just completed for another Tier 1 Investment bank

Value Proposition and why Oracle


We make near real-time / on-demand risk and reporting a reality This demonstrably improves risk management and governance This increases trading revenues, reduces trade losses and reduces operational costs We are not market risk specialists We do not have a pre-built market risk platform We do have game changing technology and experience of building risk platforms

Why are firms doing this now?


Most investment banks are unable to view risk on-demand or in near real-time

Especially by trade, by instrument, by book, by counter-party, across asset classes or bank wide
Why? Trading systems and risk systems are typically siloed Historically, risk has been calculated overnight Recent market events driving need for intra-day risk capabilities More advanced risk calculations e.g. gamma risk for credit derivatives has taken too long to be viable so quants ran simpler risk models not any more! Oracle Sun Exadata now offers Complex Risk Reporting and management of large data volumes in Near Real Time. Oracle Coherence is still the solution for Real Time Data Access needs

Tier 1 Market Risk POC - Test Configuration

HP Oracle Database Machine (V1 Hardware) Exadata V2 Software Oracle 11g Release 2 8 Oracle Database servers
64 Intel processor cores Oracle Enterprise Linux

14 Exadata Storage Servers


75.6TB raw storage

InfiniBand switches

Test Scenarios
Tests based on actual Market Risk data. Tests carried out on risk data volumes from 3 million (current) to 80 Billion (future) rows. Load tests based on loading from 3 million (current) to 2 Billion (Future) rows four times a day (8B total/day). Query tests based on actual SQL generated from frontend reporting systems. Additional query tests run using Materialised Views and OLAP cubes for comparison. The tests were carried out to provide a comparison between an Oracle Exadata solution and an existing Oracle Data Warehouse solution

Test Results
Loading: Current customer solution load takes 6 minutes (3M rows). Tested load on Exadata takes only 4 minutes to load 2 Billion rows

1000 times greater load capacity (Could be improved with tuning).


Data loaded in a compressed form (about 7-1 compression) saving large amounts of storage space and improving performance. Queries: All query tests initially carried out against base tables (no indexes, materialised views or other tuning). The current system has reached its performance limit Exadata scaled to data volumes over 25,000 times greater than the current system (and still had lots of room for greater expansion).

Test Results
Query Performance Enhancements:
Current system uses over 250 Materialised Views for performance reasons. It currently takes 25 minutes to create these views (against 3 million rows). It takes only 15 minutes to create all the views in Exadata (against 2 Billion rows). One OLAP Cube can be used to replace over 250 Materialised Views (and can provide many more dimensions of analysis!) It took only 10 minutes to create the single OLAP cube. Queries from the Cube have subsecond response times.

Test Results
A generic Materialised view (can be used in lots of different queries) reduced the query time for one of the sample queries from 1 Min 8 Seconds to 17 seconds and took only 20 seconds to create. A targeted Materialised View (specific to a number of queries including the example above) reduced the query time to sub-second and took less than 2 seconds to create (from the generic materialised view). A single OLAP cube took only 20 minutes to create, replaced all 250+ materialised views and provided many more dimensions of analysis than the MVs it replaced. The same query run against the OLAP cube was also subsecond

Sample Test Results


One of the sample queries currently takes 10 Mins 18 Secs to run against the 3M rows of data. The tested Exadata solution took only 2 mins 3 Secs to run against 80B rows of data!
Sample Query
02:18.2 02:01.0 01:43.7 01:26.4 01:09.1 00:51.8 00:34.6 00:17.3 00:00.0 3,000,000 25,000,000 200,000,000 1,600,000,000 13,000,000,000 80,000,000,000

One of the sample queries currently takes 1 Mins 45 Secs to run against the 3M rows of data. The tested Exadata solution took only 47.5 Secs to run against 80B rows of data
00:51.8 00:34.6 00:17.3 00:00.0 3,000,000 25,000,000 200,000,000 1,600,000,000 13,000,000,000 80,000,000,000

Sample Query

Future Testing
The Sun Oracle Database Machine is even faster (more than twice as fast).
Faster Processors - Intel Nehalem at both the Database and Storage tiers Faster I/O Infiniband Quad Data Rate cards (21 GBytes/Second) Flash Storage (5TB) for data caching Automatic Storage Indexes

Market Risk Platform High Level Target Architecture for Tier 1 bank
Oracle Components
Trading Apps Risk Engines Counterparty Data Reporting Analytics Modelling
MDM FX BI

Rates

Intra Day Data

Historical Data
Analytics Modelling

Credit

Risk Analytics powered by Oracle Business Intelligence


Data Integration Ad-hoc Analysis Interactive Dashboards Essbase Analytics Reporting & Publishing Proactive Detection and Alerts Disconnected & Mobile Analytics MS Office & Outlook Integration

Common Enterprise Information Model


Integrated Security, User Management, Personalization Multidimensional Calculation and Integration Engine Intelligent Request Generation and Optimized Data Access Services

OLTP & ODS Systems

Data Warehouse Data Mart

Essbase

SAP, Oracle PeopleSoft, Siebel, Custom Apps

Files Excel XML

Business Process

Daily Summary Example: Alerts

Alerts to show where immediate action is required

Daily Summary Example: Risk Measures

Dashboard display of measures and trends for the day

Daily Summary Example: Shortcuts

Shortcut links to reports

Daily Summary Example: Spatial

Data availability

Daily Summary Example: Summary

Daily Summary Example: Drill to Detail

Risk Reporting Example: Key Risks

Risk Reporting Example: RCSA Dashboard

Risk Reporting Example: CRO Dashboard

Risk Reporting Example: LOB Dashboard

Risk Reporting Example: Market Risk

Comprehensive Heterogeneous Data Integration Platform


Best-in-class Platform for Major Data Integration Requirements

Exadata V2 Sun Oracle Database Machine

Sun Oracle Database Machine


Grid is the architecture of the future Highest performance, lowest cost, fault tolerant, scalable on demand

Oracle Database Server Grid


8 compute servers 64 Intel Cores 576 GB DRAM

Exadata Storage Server Grid


14 storage servers 100 TB raw SAS disk storage or 336 TB raw SATA disk storage

InfiniBand Network
40 Gb/sec unified server and storage network Fault Tolerant

5TB flash storage!

Sun Oracle Database Machine Extreme Performance

Oracle Database Server Grid


Millions of transactions per minute Tens of millions of queries per minute Billions of rows per minute

Exadata Storage Server Grid


21 GB/sec disk bandwidth 50 GB/sec flash bandwidth 1 million I/Os per second

InfiniBand Network
880 Gb/sec aggregate throughput

Semiconductor Cache Hierarchy


Massive throughput and IOs through innovative Cache Hierarchy Database DRAM Cache
400GB raw capacity Up to 4TB compressed user data 100 GB/sec

Exadata Smart Flash Cache


5TB raw capacity Up to 50TB compressed user data 50 GB/sec raw scan 1 million IO/sec

Exadata disks
100TB or 336TB raw Up to 500TB compressed user data 21 GB/sec scan 50K IO/sec

Sun Oracle Database Machine Hardware Improvements


Same architecture as Exadata V1 Database Machine
Same number and type of Servers, CPUs, Disks

Latest Technologies
Faster
80% Faster CPUs 100% Faster Networking 50% Faster Disk Throughput 200% Faster Memory Xeon 5500 Nehalem 40 Gb InfiniBand 6 Gb SAS Links DDR3 DRAM 600 GB SAS Disks 2 TB SATA Disks 72 GB per DB Node

New

Bigger

33% More SAS Disk Capacity 100% More SATA Disk Capacity 125% More Memory 100% More Ethernet Connectivity

4 Ethernet links per DB Node

Plus Flash Storage!

Scale Performance and Capacity

Scalable
Scales to 8 rack database machine by just adding wires

Redundant and Fault Tolerant


Failure of any component is tolerated Data is mirrored across storage servers

More with external InfiniBand switches


Scales to hundreds of storage servers

Multi-petabyte databases

Drastically Simplified Deployments


Database Machine eliminates the complexity of deploying database systems
Months of configuration, troubleshooting, tuning

Database Machine is ready on day one


Pre-built, tested, standard, supportable configuration

Runs existing applications unchanged

Months to Days

Extreme performance out of the box

Best Machine for Data Warehousing

Best Data Warehouse Machine


Massively parallel high volume hardware to quickly process vast amounts of data
Exadata runs data intensive processing directly in storage

OLAP

Most complete analytic capabilities


OLAP, Statistics, Spatial, Data Mining, Real-time transactional ETL, Efficient point queries

ETL

Powerful warehouse specific optimizations


Flexible Partitioning, Bitmap Indexing, Join indexing, Materialized Views, Result Cache

Data Mining Dramatic new warehousing capabilities


New

Exadata Database Processing in Storage


Exadata storage servers implement data intensive processing in storage
Row filtering based on where predicate Column filtering Join filtering Incremental backup filtering Storage Indexing Scans on encrypted data Data Mining model scoring

New

10x reduction in data sent to DB servers is common


No application changes needed
Processing is automatic and transparent Even if cell or disk fails during a query

Simple Query Example


What were my sales yesterday? Optimizer Chooses Partitions and Indexes to Access

Exadata Storage Grid

Oracle Database Grid


Select sum(sales) where Date=24-Sept

Scan compressed blocks in partitions/indexes Retrieve sales amounts for Sept 24

SUM

10 TB scanned 1 GB returned to servers

Exadata Hybrid Columnar Compression


Data is grouped by column and then compressed Query Mode for data warehousing
Optimized for speed

New

10X compression typical


Scans improve proportionally

Archival Mode for infrequently accessed data


Optimized to reduce space 15X compression is typical Up to 50X for some data

Flash
60

Query Throughput

Query Throughput with Flash

Flash storage more than doubles scan throughput


50 GB/sec

50

Query Throughput
GB/sec Uncompressed Data

50

40

Flash
30

Combined with Hybrid Columnar Compression


Up to 50 TB of data fits in flash Queries on compressed data run up to 500 GB/sec

21
20

11.4
10

7.5

10

Disk
Database Machine

HITACHI
USP V

TERADATA
2550

NETEZZA SUN ORACLE


TwinFin 12

In-Memory Parallel Queries


QphH: 1 TB TPC-H
1,166,976 1,018,321

New

One Sun Oracle Database Machine rack


400GB of DRAM usable for caching

Exadata Hybrid Columnar Compression enables 4TB data in DRAM


315,842

Database release 11.2 introduces parallel query processing on DRAM cached data
Harnesses DRAM capacity of entire database cluster for queries Technology for world record benchmark

ParAccel

Exasol

Oracle

Faster than specialized inmemory warehouse databases

DRAM has 100x more bandwidth than Disk

Source: Transaction Processing Council, as of 9/14/2009: Oracle on HP Bladesystem c-Class 128P RAC, 1,166,976 QphH@1000GB, $5.42/QphH@1000GB, available 12/1/09. Exasol on PRIMERGY RX300 S4, 1,018,321 QphH@1000GB, $1.18/QphH@1000GB, available 08/01/08. ParAccel on SunFire X4100 315,842 QphH@1000GB, $4.57 /QphH@1000GB, available 10/29/07.

Exadata Storage Index


Transparent I/O Elimination with No Overhead Table
A B C D 1 3 5 5 Min B = 1 Max B =5

New

Index

Exadata Storage Indexes maintain summary information about table data in memory
Store MIN and MAX values of columns Typically one index entry for every MB of disk

8
3

Eliminates disk I/Os if MIN and MAX can never match where clause of a Min B = 3 query
Max B =8

Completely automatic and transparent

Select * from Table where B<2 - Only first set of rows can match

Benefits Multiply

10 TB of user data Requires 10 TB of IO

1 TB with compression

100 GB with partition pruning

Subsecond On Database Machine


20 GB with Storage Indexes 5 GB Smart Scan on Memory or Flash

Data is 10x Smaller, Scans are 2000x faster

DBFS - Scalable Shared File System


New
Database Machine comes with DBFS shared Linux file system
Shared storage for ETL staging, scripts, reports and other application files

Files stored as SecureFile LOBs in database tables stored in Exadata


Protected like any DB data mirroring, DataGuard, Flashback, etc.

5 to 7 GB/sec file system I/O throughput Load into database using External Tables

ETL Files in DBFS

ETL

More File Throughput than High-End NAS Filer

Best Machine for OLTP

The Disk Random I/O Bottleneck


300 I/O per Sec Disk drives hold vast amounts of data
But are limited to about 300 I/Os per second

Flash technology holds much less data


But can run tens of thousands of I/Os per second

Ideal Solution
Tens of Thousands of I/Os per Second
Keep most data on disk for low cost Transparently move hot data to flash Use flash cards instead of flash disks to avoid disk controller limitations Flash cards in Exadata storage

High bandwidth, low latency interconnect

Exadata Flash
Extreme Performance for Random I/O
New

Sun Oracle Database Machine has 5 TB of flash storage


4 high-performance flash cards in every Exadata Storage Server

Smart Flash Cache caches hot data


Not just simple LRU Knows when to avoid caching to avoid flushing cache Allows optimization by application table

Oracle is the First Flash Optimized Database

Exadata Smart Flash Cache


Extreme Performance Database Machine achieves: 20x more random I/Os
Over 1 million per second

2x faster sequential query I/O


50 GB/sec

10x better I/O response time


Sub-millisecond

Greatly Reduced Cost


5X More I/Os than 1000 Disk Enterprise Storage Array
10x fewer disks needed for I/O Lower Power

Maximum Availability Architecture


Real Application Clusters Active Data Guard

ASM

WAN
Secure Backup

Fast Recovery Area

Protection from
Server Failures Storage Failures Network Failures Site Failures

Real-time remote standby open for queries Human error correction


Database, table, row, transaction level

Online indexing and table redefinition Online patching and upgrades

Complete, Open, Integrated Security


Monitoring

Configuration Management
Access Control

Audit Vault

Total Recall

Database Vault
Encryption and Masking

Label Security

Advanced Security

Secure Backup

Data Masking