DB2 11.1 New Features Highlight

2016-04-18
DB2 V11.1 Technology Overview
2016 IBM Corporation
Safe Harbor Statement

Copyright IBM Corporation 2016. All rights reserved.
U.S. Government Users Restricted Rights - Use, duplication, or disclosure restricted by GSA ADP Schedule Contract
with IBM Corporation
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES
ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE
INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED AS IS WITHOUT WARRANTY OF ANY
KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON CURRENT THINKING
REGARDING TRENDS AND DIRECTIONS, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT
NOTICE. FUNCTION DESCRIBED HEREIN MY NEVER BE DELIVERED BY I BM. IBM SHALL NOT BE
RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS
PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS
INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS
FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY
AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE.
IBM, the IBM logo, ibm.com and DB2 are trademarks or registered trademarks of International Business
Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms
are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols
indicate U.S. registered or common law trademarks owned by IBM at the time this information was published.
Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM
trademarks is available on the Web at Copyright and trademark information at
www.ibm.com/legal/copytrade.shtml
DB2 Version 11.1 Overview

Themes and Highlights
Packaging and Pricing
Installation, Migration, and Platform Considerations
Manageability Enhancements
Enterprise Security
pureScale Enhancements
BLU on MPP
BLU Enhancements
SQL Functions and Syntax Compatibility
New DB2 Capabilities

Petabyte Scale In-Memory
Warehousing
Build or expand your operational warehouse

with petabyte scale BLU Technology
Broader Analytical Horizons
Leverage enhanced analytical functions

within BLU to broaden the workloads inside
your DB2 warehouse
Protect your Data
Enterprise class data encryption to ensure

the security for all your databases
Transact with Confidence
Manage your business critical systems with

the confidence that DB2 will scale to large
data and user volumes
Always there Business
Additional DB2 pureScale capabilities

ensures your Business Systems are always
up and available
DB2 Version 11.1 Highlights

Core Mission Critical Workloads :
Extending DB2 Leadership
Comprehensive Enterprise Security
Enterprise Encryption
Centralized Key Managers (KMIP)
Warehousing Workloads :
Most Consumable, Most Scalable
In-Memory Warehousing Platform
Massive Scale Warehousing at
In-Memory Performance
MPP BLU Scalability
PB scale in-memory warehousing
Availability 2nd only to DB2 for zOS

Simple Fast Deployment
Up and running in hours
Even Greater Availability
Zero data loss DR with HADR
More online management
More Platforms Supported
Power Linux (LE)
Virtualization for RDMA (x86)
Significant Core Database Advances

Very Large Database Performance
Higher user throughput
Simpler, Faster, More Online Upgrades
Faster, no need for offline backup

Streamlined HADR upgrade
DB2 Version 9.7 direct to 11.1
5
Next Gen In-Memory Performance,

Function & Workloads
Faster ELT/ETL performance
More Query Workloads Optimised
More Function supported
Generated Columns
RCAC
OLAP + BLU Perf
Enhanced Compatibility
Multi-Lingual SQL Advances
PostgresSQL
Support for European Languages
Codepage 819
DB2 Version 11.1 Announce and General Availability

DB2 Version 11.1 announced on April 12th
General Availability (eGA) on June 15th
Always a possibility that eGA dates can change
Announcement also includes

End of marketing for DB2 Version 10.5
End of marketing for DB2 Business Value Offerings
End of service for DB2 Version 9.7 and 10.1
New packaging and pricing
End of Marketing for DB2 Version 10.5 and Features

End of Marketing for DB2 Version 10.5 (September 30th, 2016)
All DB2 Editions and features/offerings
Performance Management Feature Version 10.5
Tools and DB2 Connect Version 10.5
End of Marketing for DB2 Business Value Offerings

IBM DB2 Encryption Offering
Provides data encryption at rest
Now included in all offerings without charge
Existing customers will drop S&S on feature when they migrate
IBM DB2 Business Application Continuity Offering

Provides the continuous availability of IBM DB2 pureScale technology, for
maintenance and failover capabilities
Existing customers get a license exchange to AWSE or AESE when they migrate
IBM DB2 BLU Acceleration In-Memory Offering

Provides columnar and dynamic in-memory acceleration
Existing customers get a license exchange to AWSE or AESE when they migrate
7
End of Service for DB2 Version 9.7 and 10.1

Announcing the End of Service for both DB2 Version 9.7 and 10.1 in
conjunction with the announcement of DB2 Version 11.1
Effective End of Service date of September 30th, 2017
DB2 Version 11.1 will support a direct upgrade from DB2 Version 9.7, 10.1,
and 10.5
Provides customers the ability to migrate either to:
DB2 Version 10.5 - been in the market for a longer time
DB2 Version 11.1 - provides a longer service period (at least 5 years)
Provide customers sufficient time to upgrade/migrate

18 month notice for End of Service of DB2 Version 9.7 and 10.1
Extended support contracts can be negotiated for those customers requiring
a longer time to migrate
End of Service date is not applicable to SAP customers with an ASL as they
have a different end of service period
Packaging: Workgroup Server Edition

DB2 Express Edition replaced by Workgroup Server Edition
A new, more competitive offering with additional capability
pureScale Standby Node Option
Table partitioning, Encryption
Multi-dimensional Clustering
Limited Federation (DB2 & Informix)
Exclusions
Metric
New Price
Workgroup
Express
PVU
100
158
68
AUSI
354
554
291
Virtual Server
14,100
Sockets
18,800 Only for existing customers
7520
A virtual server in prior releases was limited to 4GB of memory and 4

processor cores. The new limit is 128GB of memory and 8 processor
cores.
Data Partitioning
Prices are in US dollars and are subject to change.
SQL Warehouse (SQW)
BLU Acceleration, Compression, Materialized Query Tables
No MQ or CDC Replication
Simple Limits
Authorized User (AUSI) , PVU, and Virtual Server
16 Cores (8 for Virtual Server)
128 GB of memory with no database size limit
9
Packaging: Enterprise Server Edition

Enterprise Server Edition now contains additional capability
pureScale Standby Node Option
Table partitioning, Encryption
Multi-dimensional Clustering and Materialized Query Tables (MQT)
Limited Federation (DB2 & Informix)
Exclusions
Data Partitioning
SQL Warehouse (SQW)
BLU Acceleration, Compression
No MQ or CDC Replication
No Server or User Limits

Optional (Workgroup and Enterprise)
DB2 Performance Management Offering (WLM) with Data Server
Manager Enterprise Edition
Advanced Recovery Feature
10
Packaging: Advanced Workgroup and Enterprise Edition

Licensing
Advanced Enterprise Edition
PVU, AUSI, Terabyte license
Advanced Workgroup Edition

PVU, AUSI, Terabyte license
Restrictions: 16 cores, 128 GB memory, 4 sockets (Terabyte)
Restrictions
DB2 Connect included for using SQW tooling to access DB2 for z and
DB2 for i
InfoSphere Data Architect limited to 10 authorized users
Cognos Analytics limited to 5 authorized users
Optional Packages
Advanced Recovery Feature
11
DB2 Direct Editions

New Delivery Mechanism for DB2 licenses
New license metrics to facilitate hybrid cloud deployments
Acquire the product directly online (Passport Advantage)
Option to deploy either on-premises or on cloud
Two Versions depending on Requirements

DB2 Direct Standard Edition 11.1
Has all of the database features of DB2 Workgroup Server Edition
DB2 Direct Advanced Edition 11.1

Has all of the database features of DB2 Advanced Enterprise Server Edition
Newly introduced simplified license metric, the Virtual Processor Core

(VPC) sold as a monthly license charge
Predictable maintenance releases
12
Virtual Processor Core - VPC

Virtual Processor Core
VPC: Virtual Cores available to the operating system
1 VPC for every VPC available to a virtual Servers Operating System, or 1 VPC for
each physical core of a non-partitioned physical server
If the number of VPCs is greater than the physical cores, then you only need to
license the number of physical cores on the machine
Minimum of 2 VPCs per deployment (1 VPC for idle/warm standby)
Benefits to Customers
Simplifies Processor and Sub-capacity Licensing
Aligns to Cloud Hosted Solution Licensing standards across MSPs
Makes it easier for customers On-prem and in Cloud to stay in compliance
Metric
Standard
Advanced
VPC Monthly
135
354
Prices are in US dollars and are subject to change.
13
Federation Included in Packaging

Integrated support for homogeneous federation
Single install replacing any prior separate Infosphere Federation Server install
Support for upgrading from either a DB2 database product or Infosphere
Federation Server
Additional Wrappers in Advanced Editions

DB2, PureData System for Analytics (PDA), Oracle, Informix, dashDB,
SQLServer, BigSQL, SparkSQL, Hive, Impala, and other Big Data sources.
Application
14
INSTALLATION, MIGRATION, AND

PLATFORM CONSIDERATIONS
15
Operating Systems - Supported

New Operating System Support
Power Linux LE (Little Endian)
Red Hat Enterprise Linux (RHEL) 7.1+
SUSE Linux Enterprise Server (SLES) 12
Ubuntu 14.04 LTS
Supported Operating Systems

Intel 64-bit
Windows 7, 8.1, 10, Windows Server 2012 R2

Red Hat Enterprise Linux (RHEL) 6.7+, 7.1+
SUSE Linux Enterprise Server (SLES) 11SP4+, 12
CentOS 6.7, 7.1
Ubuntu 14.04 LTS
AIX Version 7.1 TL 3 SP5+

zLinux
Red Hat Enterprise Linux (RHEL) 7.1+
SUSE Linux Enterprise Server (SLES) 12
16
Operating Systems - Discontinued

In DB2 V11, the following operating systems (on any platform) are
no longer supported for Client or Server:
HP-UX
Solaris
Power Linux BE
Inspur K-UX
Migration
Customers on these platforms will continue to be supported until the end-ofservice date for DB2 V10.5 (last release that supports these platforms)
17
Operating Systems - Virtualization

IBM System z
IBM Processor Resource/System Manager
z/VM and z/KVM on IBM System z
IBM Power
IBM PowerVM and PowerKVM and IBM Workload Partitions on
IBM Power Systems
Linux X86-64 Platforms

Red Hat KVM
SUSE KVM
VMWare ESXi
Docker container support Linux only
Microsoft
Hyper-V
Microsoft Windows Azure on x86-64 Windows Platforms only
pureScale support on Power VM/KVM, VMWare, and KVM

18
Streamlined Upgrade Process

Upgrade directly from Version 9.7, 10.1 and 10.5 (3 releases back)
Ability to roll-forward through database version upgrades
Upgrading from DB2 Version 10.5 Fix Pack 7, or later
Users are no longer required to perform an offline backup of existing
databases before or after they upgrade
A recovery procedure involving roll-forward through database upgrade
now exists
Applies to all editions and configurations except Database Partitioning
Feature (DPF)
HADR environments can now be upgraded without the need to reinitialize the standby database after performing an upgrade on the
primary database
Applies to all editions except DB2 pureScale
DB2 Version 10.5 Fix Pack 7, or later
19
MANAGEABILITY AND TOOLING
20
INPLACE Table Reorganization

The manageability of large range partitioned tables has
been improved
A single partition of a partitioned table can now be reorganized with the
INPLACE option if:
the table has no nonpartitioned indexes
ON DATA PARTITION is specified
Only one data partition can be reorganized at a time

INPLACE table reorganization can be run only on tables that are at least
three pages in size
21
New options for the ADMIN_MOVE_TABLE

REPORT
The REPORT option can be used to monitor the progress of table moves
Calculates a set of values to monitor the progress of a single or multiple
table moves
Focus is the COPY and REPLAY phase of a running table move
To get values for all table moves, tabschema and tabname must be NULL or
the empty string
TERM
The TERM option can be used to terminate a table move in progress
Terminates a running or table move
TERM will force off the application running the table move, roll back all open
transactions and set the table move to a well defined operational status
From here, the table move can be cancelled or continued
22
Remote Storage Option for Utilities

Remote storage is now accessible from:
INGEST, LOAD, BACKUP, and RESTORE
Accessed through the use of storage access aliases
Supported Storage
IBM SoftLayer Object Storage
Amazon Simple Storage Service (S3)
23
db2relocatedb Options
The db2relocatedb includes a new option: -g
For generating scripts that can be used as input for database
relocation operations
For cloning a database with different database name and paths
To extract path information from snapshot image without requiring a
DB2 instance
Generates a configuration file and specifies the name of the file

containing the configuration information
24
DB_NAME=oldName,oldName
DB_PATH=oldPath,oldPath
INSTANCE=oldInst,oldInst
NODENUM=nodeNumber
LOG_DIR=oldDirPath,oldDirPath
CONT_PATH=oldContPath1,oldContPath1
CONT_PATH=oldContPath2,oldContPath2
...
STORAGE_PATH=oldStoragePath1,oldStoragePath1
STORAGE_PATH=oldStoragePath2,oldStoragePath2
...
FAILARCHIVE_PATH=oldDirPath
LOGARCHMETH1=oldDirPath
LOGARCHMETH2=oldDirPath
MIRRORLOG_PATH=oldDirPath
OVERFLOWLOG_PATH=oldDirPath
NO TABLESPACE Option for Backup Database

DB2 History File
Contains information about log file archive location, log file chain etc.
If you use snapshot backups you want to have current information about log
file location to perform point in time recovery (RECOVER command)
For RECOVER you need a current version of the history file
NO TABLESPACE backup allows you to create a current and consistent
backup of the history file in a convenient way
If you use COMPRESS option of the BACKUP command, the created backup
image is small
BACKUP with NO TABLESPACE option

A NO TABLESPACE backup does not contain tablespaces
A no tablespace backup is used to restore the history file by using the
HISTORY FILE option with the RESTORE DATABASE command
HISTORY FILE keyword is specified to restore only the history file from the
backup image
25
DB2 Support for the NX842 Accelerator

DB2 backup and log archive compression now support the NX842
hardware accelerator on POWER 7+ and POWER 8 processors
DB2 BACKUPs require the use of a specific NX842 library
backup database <dbname> compress comprlib libdb2nx842.a
Backups can be compressed by default with NX842

Registry variable DB2_BCKP_COMPRESSION has to be set to NX842
Use the following backup command format:
backup database <dbname> compress
Log archive compression is also supported

Update the database configuration parameter LOGARCHCOMPR1 or
LOGARCHCOMPR2 to NX842
update database configuration for <dbname>
using LOGARCHCOMPR1 NX842
Note: These two parameters can still take different values
26
DB2 Backup Compression Performance Results

Preliminary results from early system testing
About 50% DB2 backup size reduction compared to uncompressed
Factor 2x less CPU consumption compared to DB2 compression
Very significant reduction in CPU consumption
Very significant reduction in elapsed time
Maintains almost all of the compression storage benefits
100%
06:00
90%
Not
Compressed
05:00
80%
70%
04:00
50%
03:00
Minutes
60%
Backup to /dev/null
Software Compressed
H/W
Compressed
Size related to Uncompressed

Backup Runtime to /dev/null in minutes
Restore in minutes
40%
02:00
30%
20%
01:00
10%
0%
00:00
Uncompressed
Compressed
NX842 Compressed
Internal Tests at IBM Germany Research & Development

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will
experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage
configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
27
ENTERPRISE SECURITY
28
Encryption and Enterprise Key Management

Encrypted flows between HADR primary and secondary
Simplified integration via SSL/TLS
Initial support on Linux x86
V11.1 adds support for KMIP 1.1 complaint centralized

key managers
Validated on IBM's Security Key Lifecycle Manager (ISKLM)
DB2 Native
Encryption
KMIP 1.1
Local Keystore
File
DB2 V10 FP5
Simple Key Mgt : a local flat file used

for a specific DB2 instance
Centralized Key
Manager
DB2 V11.1
Hardware
Security Module
Technology
Preview
Enterprise Key Mgt : a centralized

key manager or HSM that can be used
across many databases, file systems
and other uses across an enterprise
Direct support for Hardware Security Modules (HSMs) (Preview)

Support to include SafeNet Luna & Thales nShield Connect+
29
PURESCALE ENHANCEMENTS
30
DB2 pureScale: Simplified Install and Deployment

Fast Up and Running
Up and running in hours compared to
competitive cluster databases
Install re-engineering includes:

Push-Button install for pureScale clusters
Socket complexity reduced by at least 40%
Smarter defaults, intuitive options, parallel & quick pre-deployment host validation
30-step native GPFS setup reduced to simple 4-step DB2 install process
Also easier conversion to GPFS replication post-deployment using db2cluster
Increased Resiliency for aborted/partial installations

Clean rollback for re-installation
Additional assistance via:

Simplified documentation
Enhanced pre-checking of storage, tiebreaker devices, network adapters,
firmware libraries
Intuitive and user-friendly errors & warnings
31
Linux Virtualization Enhancements

RDMA over Converged Ethernet (RoCE) support added in VMWare
RoCE SR-IOV for RHEL 7.2 only
Allows a single adapter to be shared across multiple patitions
Single-Root I/O Virtualization (SR-IOV)

Standard that enables one PCI Express (PCIe) adapter to be presented as
multiple separate logical devices (Virtual Functions) to virtual machines
Allow the virtual machines to run native RoCE and achieve near wire speed
performance.
Can be enabled on Mellanox ConnectX-3/ConnectX-3 Pro/Connect X-3 VPI
adapters for Ethernet
32
VMWare RoCE Adapter Sharing Example
VMWare
Partitions
Shared RoCE Adapters
RoCE
Network
33
HADR Support for SYNC and NEARSYNC Mode

Support for SYNC and NEARSYNC has been added to pureScale
This enhancement combines the continuous availability of DB2 pureScale
with the robust disaster recovery capabilities of HADR providing an integrated
zero data loss (i.e. RPO=0) disaster recovery solution
HADR peer window (hadr_peer_window) is not supported
HADR support with pureScale now includes:

SYNC, NEARSYNC, ASYNC and SUPERASYNC modes
Time delayed apply, Log spooling
Both non-forced (role switch) and forced (failover) takeovers
CF
CF
Primary Cluster
CF
34
CF
Standby DR Cluster
GDPC Support Enhancements
M1
M3
CFP
CFS
Site A
M2
M4
Site B
Workload fully balanced
DB2 V11 adds improved high availability for Geographically

dispersed DB2 pureScale clusters (GDPC) for both RoCE & TCP/IP
Multiple adapter ports per member and CF to support higher bandwidth and
improved redundancy at the adapter level
Dual switches can be configured at each site to eliminate the switch as a sitespecific single point of failure (i.e. 4-switch configuration)
Member 1
ro0 ro1
en2
Primary
CF
ro0 ro1
Member 3
en2
ro0 ro1
en2
Site 1
Storage
Switch 1
Switch 2
Peer 1
Peer 1
GPFS
Site 3
replication
Storage
Switch 3
Switch 4
Peer 2
Peer 2
Site 2
ro0 ro1
35
en2
Member 2
ro0 ro1
en2
Secondary
CF
ro0 ro1
en2
Member 4
Improved Performance for Highly Concurrent Workloads

Streamlined bufferpool latching protocol implemented in DB2 V11
Reduces contention which can develop on large systems with many threads
Particularly helpful with transactional workloads
Workload #1 - DB2 ESE

9000
8000
7000
6000
5000
4000
3000
2000
1000
0
tps
Workload #2 DB2 pureScale

14000
12000
1.58x
10000
2.1x
8000
6000
4000
2000
DB2 V10.5 fp5
5040
DB2 V11.1
7950
Workload 1 based on an industry

benchmark standard
POWER7 32c, 512 GB
tps
DB2 V10.5 fp5

5848
DB2 V11.1
12448
Workload 2 implements a warehouse-based

transactional order system
4 members, 2 CFs with 16c, 256 GB
36
Horizontal Scaling with DB2 pureScale on POWER Linux
Thousands of SQL statements/s
Scale-out Throughput DB2 pureScale on LE POWER Linux

140
Sockets
RDMA
133
120
100
105
82.7
80
70
60
40
50.8
26.8
72.8
38
20
0
1 member
2 members
3 members
4 members
80% read / 20% write OLTP workload

POWER8 4c/32t, 160 GB LBP
10 Gb RoCE RDMA Ethernet / 10 Gb TCP sockets
37
Improved Table TRUNCATE Performance in pureScale

More efficient processing of Global Bufferpool (GBP) pages
Speeds up truncate of permanent tables especially with large GBP sizes
Helps DROP TABLE and LOAD / IMPORT / INGEST with REPLACE option
Enables improved batch processing with these operations
Example
Workload with INGEST (blue) and TRUNCATE (green) of an unrelated table
DB2 v11.1 has much smaller impact on OLTP workload than DB2 10.5 fp5
Application throughput - DB2 v10.5 fp5
38
Application throughput DB2 v11.1
Unified Workload Balancing with pureScale

Version 11.1 extends the configuration options for member subsets
allowing the user to explicitly define alternate members in a subset
This allows users currently using client affinity to move their configuration to
using member subsets so that they can exploit the new benefits such as
dynamic server side reconfiguration
With member subsets, you can isolate application workloads to a

specific set of members
This provides greater flexibility and member-level workload management.
Applications that connect to a database alias that is associated with a
member subset balance their workload between the members in the subset
The members included in the subset can be modified dynamically, impacting
where the workload of applications assigned to the member subset runs
Using new member subset management routines, you can create, alter, and
drop member subset objects
Managing these member subset definitions, you can add or drop members to
a member subset, or enable or disable a member subset
39
DB2 pureScale Health Check

Unified health check tool for a DB2 pureScale cluster
Post-installation command:
db2cluster verify
Validations performed include, but are not limited to, the following:
40
Configuration settings in peer domain and GPFS cluster

Communications between members and CFs
Replication setting for each file system
Status of each disk in the file system
DB2 BLU IN A MPP ENVIRONMENT
41
BLU Acceleration: MPP Scale Out

Technology
Pervasive SMP & MPP Query Parallelism
Inter-partition query parallelism simultaneous with
intra-partition- parallelized, memory-optimized,
columnar, SIMD-enabled, BLU processing
Value Proposition
Improve Response Time
Query #1
Query #1
processing
Query #1
processing
Query #1
processing
1/3 data
1/3 data
1/3 data
Hash partition
(BLU
Acceleration)
Hash partition
(BLU
Acceleration)
Hash partition
(BLU
Acceleration)
All servers contribute to the processing of a query
Massively Scale Data

Significantly beyond current practical limits
Streamline BLU Adoption

Add BLU Acceleration to existing data warehouses
42
DB2 10.5 BLU

Capacity
DB2 V11.1 BLU

Capacity
10s of TB
1000s of TB
100s of Cores
1000s of Cores
Demonstrating BLU MPP Linear Scaling

DB2 Version 11.1 on an IBM Power Systems E850 Cluster
Scaling Hardware at
constant Data Volume
Scaling Hardware along

with Data Volume
120
1.92x
200
100
Queries Per Hour
Queries Per Hour
250
150
100
50
0
QpH
3 Node (10TB) - 6
MLNS
6 Node (10TB) - 12
MLNS
111
213
QpH
Held up!
80
60
40
20
0
QpH
3 Node (10TB) - 6 MLNS
6 Node (20TB) - 12
MLNS
113
109
Scaling was measured in two different ways

Doubling the hardware but keeping the database constant
Doubling the hardware and doubling the database size
Both tests used the BD Insights Heavy Analytics Internal Workload
43
Outstanding BLU MPP Performance

1.9x Higher Throughput with DB2 Version 11.1 vs. the Competition
Query Throughput BD Insights (4TB)
300
1.9x
Faster!!
250
Queries Per Hour
200
150
100
50
0
QpH
Competition on 3 Nodes of AWS

145
DB2 V11.1 on 3 Nodes of AWS

274
44
Performance Improvements
BLU Acceleration includes a number of performance improvements
in the columnar engine
Faster SQL MERGE processing
Nested Loop Join Support
Industry Leading Parallel Sort
Push-down of a number of OLAP functions into the BLU engine
45
Industry Leading Parallel Sort

Leverages the latest sort innovations from IBM TJ Watson Research
and DB2 Development
Enhancements can increase BLU Acceleration performance by as much as 13.9X
Query elapsed time(s)
BLU Sort+OLAP on SMP Environment

Row Sort
2000
BLU Sort
1500
1000
4.6 X faster
500
13.9 X faster
5.4 X faster
0
Sort+OLAP query 1
Sort+OLAP query 2
Sort+OLAP query 3
Configuration Details
On 4-socket Intel Xeon platform with 72 Cores and 742G RAM
1 TB TPC-DS database
Query scenarios involving multiple sort and OLAP operations
46
BLU Sort on MPP Environment

Enhancements can increase BLU Acceleration performance by as
Row sort vs BLU sort
much as 1.86X
Query elapsed time (s)
Row Sort
45
40
35
30
25
20
15
10
5
0
BLU Sort
1.86 X faster
1.48 X faster
1.51 X faster
Sort query 1
Sort query 2
Sort query 3
BLU MPP with 1 physical node/4-MLNs and 8 cores per MLN
4-socket, 40 Cores Intel Xeon platform and 1 TB RAM
Used Sort Target Workload 1 TB (an internal workload based on the TPC-DS
schema), with 10-Million rows to sort for each scenarios
Query scenarios involving single ORDER BY concentrating on sort operation
47
Demonstrating BLU Single Instance Improvement

DB2 V11.1 on Intel Haswell EP
Reasons for Improvement
Query Throughput BD
Insights (800GB)
1.36x
Queries Per Hour
1200
Native BLU
Evaluation
Native Sort
Native OLAP (usually combined with sort)
Enables query plans to remain as much as
possible within the columnar engine
Query Rewrite
Improvements
Find areas to improve degree determination

and improve parallel use
1000
800
600
400
200
0
QpH
DB2 V10.5 FP5

703.85
DB2 V11.1
955.82
Improved
SORTHEAP
Utilization
SORTHEAP used for building hash tables for

JOINs, GROUP BYs, and other runtime work
Efficient use allows for more concurrent intraquery and inter-query operations to co-exist.
2 socket, 36 core Intel Xeon E5-2699 v3 @ 2.3GHz
192GB RAM
BD Insights Internal Multiuser Workload 800GB
48
Optimized SQL Support for Columnar Tables

SQL OLAP improvements for deeper in-database analytics with
column-organized tables
Additional Oracle Compatibility Support
DB2 : a polyglot
database
Wide rows
Logical character support (CODEUNITS32)
DGTT support (except not logged on rollback preserve rows)

Parallel insert into not-logged DGTT from BLU source
IDENTITY and EXPRESSION generated columns

European Language support (Codepage 819)
NOT LOGGED INITIALLY support
Row and Column Access Control (RCAC)
ROWID Support
Faster SQL MERGE processing
Nested Loop Join Support
49
SQL Functions Optimized for Columnar Mode

String Functions
LPAD, RPAD
TO_CHAR
INITCAP
Numeric Functions
POWER, EXP, LOG10, LN
TO_NUMBER
MOD
SIN, COS, TAN, COT, ASIN, ACOS, ATAN
TRUNCATE
Date and Time Functions

TO_DATE
MONTHNAME, DAYNAME
Miscellaneous
COLLATION_KEY
50
OLAP Functions Support by BLU

OLAP functions supported by BLU:
RANK, DENSE_RANK, ROW_NUMBER
OLAP column functions supported by BLU:

AVG
COUNT, COUNT_BIG
MIN, MAX
SUM
FIRST_VALUE
RATIO_TO_REPORT
Note: Window aggregation group clause is limited to:

ROWS/RANGE BETWEEN UNBOUNDED PRECEDING AND
UNBOUNDED FOLLOWING
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
ROWS BETWEEN CURRENT ROW AND CURRENT ROW (not supported
for FIRST_VALUE)
51
Columnar Engine Native Sort + OLAP Support

No longer compensated on single instance DB2 V11.1
52
OLAP Query Elapsed Time (s)

(lower is better)
Elapsed Time in Seconds
with v1 as(
select i_category, i_brand, cc_name, d_year, d_moy, sm_type,
sum(cs_sales_price) sum_sales,
avg(sum(cs_sales_price)) over
(partition by i_category, i_brand, cc_name, d_year)
avg_monthly_sales,
rank() over
(partition by i_category, i_brand, cc_name
order by d_year, d_moy) rn
from BDINSIGHTS.item
, BDINSIGHTS.catalog_sales BDINSIGHTS.date_dim, BDINSIGHTS.call_center
, BDINSIGHTS.ship_mode
where cs_item_sk = i_item_sk and cs_sold_date_sk = d_date_sk
and cc_call_center_sk= cs_call_center_sk
and cs_ship_mode_sk = sm_ship_mode_sk
and d_year = 2000
group by i_category
, i_brand , cc_name , d_year , d_moy, sm_type),
v2 as(
select v1.i_category, v1.i_brand, v1.cc_name, v1.d_year, v1.d_moy
, v1.avg_monthly_sales, v1.sum_sales, v1.sm_type, v1_lag.sum_sales psum
, v1_lead.sum_sales nsum
from v1
, v1 v1_lag , v1 v1_lead
where v1.i_category = v1_lag.i_category
and v1.i_category = v1_lead.i_category and v1.i_brand = v1_lag.i_brand
and v1.i_brand = v1_lead.i_brand and v1. cc_name = v1_lag.cc_name
and v1. cc_name = v1_lead.cc_name and v1.rn = v1_lag.rn + 1
and v1.rn = v1_lead.rn - 1)
select *
from v2
where d_year = 2000
and avg_monthly_sales > 0
and case when avg_monthly_sales > 0
then abs(sum_sales - avg_monthly_sales) / avg_monthly_sales
else null end > 0.1
order by sum_sales - avg_monthly_sales
, cc_name
fetch first 100 rows only
25
20
4.3x Faster!!
15
10
5
0
OLAP Query
V10.5
V11.1
22.814
5.326
Columnar Engine Native Sort + OLAP Support

Access Plan Difference with Native Evaluator support
53
BLU Acceleration: Massive Gains for ELT & ISV Apps

BLU Declared Global Temporary Table (not-logged DGTT) Parallelism
16x Faster !
54
SQL ENHANCEMENTS
55
BINARY and VARBINARY data

BINARY and VARBINARY data types
Allow binary string data to be stored and manipulated without the overhead of
using a BLOB type
A binary string is a sequence of bytes that are used to store as pictures,
sound, or mixed media
BINARY and VARBINARY data types are compatible with each other and are
compatible with the BLOB data type
Binary string data types are not compatible with character string data types,
except those character strings that are defined as FOR BIT DATA
Support for BINARY and VARBINARY data types enhances compatibility with
other relational database management systems
BINARY can contain 1-254 bytes
VARBINARY can be up to 32672 bytes
56
New CREATE FUNCTION statement for aggregate UDFs

The new CREATE FUNCTION (aggregate interface) statement allows
you to create your own aggregate functions
An aggregate function returns a single value that is the result of an evaluation
of a set of like values, such as those in a column within a set of rows
Use your choice of programming language
Four sections within the function are defined based on the stage of the
aggregation process
57
INITIATE
ACCUMULATE
MERGE
FINALIZE
Support for the (+) Outer Join Operator

Support for the outer join operator enhances cross-vendor support
Queries can use the outer join operator (+) as alternative syntax
within predicates of a WHERE clause
The use of the Oracle compatibility mode is not required to enable
this feature
58
COLLATION_KEY Function
COLLATION_KEY
The COLLATION_KEY function returns a VARBINARY string that represents
the collation key of the expression argument, in the specified collation
This function is used in SQL for ordering and comparison operations
Format
COLLATION_KEY( expression, collation_name, [length] )
Arguments
Expression - An expression for which the collation key is determined
Collation_Name - An expression that specifies the collation to use when the
collation key is determined
Length - An expression that specifies the length attribute of the result in bytes
Example:
The following query orders employees by their surnames by using the
language-aware collation for German in code page 923
SELECT FIRSTNME, LASTNAME FROM EMPLOYEE
ORDER BY COLLATION_KEY (LASTNAME, 'SYSTEM_923_DE')
59
Extensions that enhance SQL compatibility

NTH_VALUE
The NTH_VALUE function returns the expression value for the nth-row row in
an OLAP window
CUME_DIST
The CUME_DIST function is a distribution function that returns a cumulative
distribution of a row within an OLAP window
PERCENT_RANK
The PERCENT_RANK function is a distribution function that returns a relative
percentile rank of a row within an OLAP window
OFFSET clause added to subselect

The offset-clause specifies the number of rows to skip before any rows
are retrieved
DISTINCT clause added to LISTAGG aggregate function

If DISTINCT is specified in the LISTAGG function, duplicate string-expression
values are eliminated
60
Compatibility features for Netezza Performance Server

Use the SQL_COMPAT global variable to activate the following
optional NPS compatibility features
db2set SQL_COMPAT='NPS'
Compatibility Features
Double-dot notation
You can use double-dot notation to specify a database object
TRANSLATE parameter syntax

TRANSLATE (char-string-exp, from-string-exp ,to-string-exp)
NPS Operator Symbols

The operators ^ and ** are both interpreted as the exponential operator, and the
operator # is interpreted as bitwise XOR
Grouping by SELECT clause columns

You can specify the ordinal position or exposed name of a SELECT clause column
when grouping the results of a query
Routines written in NZPLSQL

The NZPLSQL language can be used in addition to the SQL PL language.
61
New Functions, Data Types and Columnar Optimization

Date/Time
Date/Time
Statistics
Bit Manipulation
Data Types
Strings
DATE_PART
ADD_YEAR
COVARIANCE_SAMP
HASH
INT2
STRPOS
RANK
FIRST_VALUE
DATE_TRUNC
ADD_MONTHS
STDDEV_SAMP
HASH4
INT4
STRLEFT
DENSE_RANK
RATIO_TO_REPORT
AGE
ADD_DAYS
VARIANCE_SAMP
HASH8
INT8
STRRIGHT
ROW_NUMBER
EXP
LOCALTIMESTAMP
ADD_HOURS
CUME_DIST
TO_HEX
FLOAT4
REGEXP_COUNT
LPAD
LOG10
NOW Function
ADD_MINUTES
PERCENT_RANK
RAWTOHEX
FLOAT8
REGEXP_EXTRACT
RPAD
COLLATION_KEY
THIS_QUARTER
ADD_SECONDS
PERCENTILE_DISC
INT2AND
BPCHAR
REGEXP_INSTR
TO_CHAR
LN
THIS_WEEK
DAYOFMONTH
PERCENTILE_CONT
INT2OR
BINARY
REGEXP_LIKE
INITCAP
TO_NUMBER
THIS_YEAR
FIRST_DAY
MEDIAN
INT2XOR
VARBINARY
REGEXP_MATCH_COUNT
TO_DATE
MOD
THIS_MONTH
DAYS_TO_END_OF_MONTH
WIDTH_BUCKET
INT2NOT
LOG
REGEXP_REPLACE
MONTHNAME
SIN
NEXT_QUARTER
HOURS_BETWEEN
COVAR_POP
INT4AND
RANDOM
REGEXP_SUBSTR
DAYNAME
COS
NEXT_WEEK
MINUTES_BETWEEN
STDDEV_POP
INT4OR
BTRIM
POWER
TAN
NEXT_YEAR
SECONDS_BETWEEN
VAR_POP
INT4XOR
AVG
COT
NEXT_MONTH
DAYS_BETWEEN
VAR_SAMP
INT4NOT
COUNT
ASIN
NEXT_DAY
WEEKS_BETWEEN
INT8AND
COUNT_BIG
ACOS
INT8OR
MIN
ATAN
INT8XOR
MAX
TRUNCATE
INT8NOT
SUM
EXTRACT
62
OLAP Pushdown
OLAP Pushdown
Reference Information
DB2 Version 11.1 Information Center (Preview)
www.ibm.biz/db2v11docs
DB2 Version 11.1 beta program

Contact db2beta@ca.ibm.com
63

DB2 11.1 New Features Highlight

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

DB2 11.1 New Features Highlight

Transféré par

Droits d'auteur :

Formats disponibles

2016-04-18

DB2 V11.1 Technology Overview

2016 IBM Corporation

Safe Harbor Statement

2016 IBM Corporation

DB2 Version 11.1 Overview

2016 IBM Corporation

New DB2 Capabilities

Build or expand your operational warehouse

Broader Analytical Horizons

Leverage enhanced analytical functions

Protect your Data

Enterprise class data encryption to ensure

Transact with Confidence

Manage your business critical systems with

Always there Business

Additional DB2 pureScale capabilities

DB2 Version 11.1 Highlights

Availability 2nd only to DB2 for zOS

Significant Core Database Advances

Faster, no need for offline backup

Next Gen In-Memory Performance,

DB2 Version 11.1 Announce and General Availability

Announcement also includes

2016 IBM Corporation

End of Marketing for DB2 Version 10.5 and Features

End of Marketing for DB2 Business Value Offerings

IBM DB2 Business Application Continuity Offering

IBM DB2 BLU Acceleration In-Memory Offering

2016 IBM Corporation

End of Service for DB2 Version 9.7 and 10.1

Provide customers sufficient time to upgrade/migrate

2016 IBM Corporation

Packaging: Workgroup Server Edition

18,800 Only for existing customers

A virtual server in prior releases was limited to 4GB of memory and 4

2016 IBM Corporation

Packaging: Enterprise Server Edition

No Server or User Limits

2016 IBM Corporation

Packaging: Advanced Workgroup and Enterprise Edition

Advanced Workgroup Edition

2016 IBM Corporation

DB2 Direct Editions

Two Versions depending on Requirements

DB2 Direct Advanced Edition 11.1

Newly introduced simplified license metric, the Virtual Processor Core

2016 IBM Corporation

Virtual Processor Core - VPC

Prices are in US dollars and are subject to change.

2016 IBM Corporation

Federation Included in Packaging

Additional Wrappers in Advanced Editions

2016 IBM Corporation

INSTALLATION, MIGRATION, AND

2016 IBM Corporation

Operating Systems - Supported

Supported Operating Systems

Windows 7, 8.1, 10, Windows Server 2012 R2

AIX Version 7.1 TL 3 SP5+

2016 IBM Corporation

Operating Systems - Discontinued

2016 IBM Corporation

Operating Systems - Virtualization