DB2 Tech Talk PureData Systems Presentation PDF

Technical Tour of PureData Systems for Transactions and Operational Warehousing
Formerly
James Cho, Architect IBM PureData Systems November 15, 2012
2012 IBM Corporation
DB2 Tech Talk series host and todays presenter: Serge Rielau
SQL Architect, DB2 for Linux, UNIX, and Windows at IBM

Serges blog: www.sqltips4db2.com Todays technical presenter . . . James Cho Architect, PureData Systems A few details . 1. Access the presentation for this Tech Talk: bitly.com/ttfilenov 2. Next steps and troubleshooting guide: click Attachments in this webcast window
2
Need webcast troubleshooting help? Click attachments
Disclaimer
The information contained in this presentation is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided as is, without warranty of any kind, express or implied. In addition, this information is based on IBMs current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other documentation. Nothing contained in this presentation is intended to, or shall have the effect of:
Creating any warranty or representation from IBM (or its affiliates or its or their suppliers and/or licensors); or
Altering the terms and conditions of the applicable license agreement governing the use of IBM software.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
Agenda
IBM PureSystems Family Review PureData
Database Architecture
PureData for Transactions Product Offering New Features Simplified Experience Pure Console PureData for Operational Analytics Single Part number simplified building blocks
New Features and Enhancements

Simplified Experience Pure Console
IBMs family of expert integrated systems has expanded
Systems with integrated expertise and built for cloud
Built-in Expertise
Capturing and automating what experts do
Integration by Design
Deeply integrating and tuning hardware and software
Simplified Experience
Making every part of the IT lifecycle easier
5
PureSystems simplify each phase of the systems lifecycle
Streamlined:
Pre-integrated, pretested, ready to run Optimized configurations with high availability Single part to order
Accelerated:
System Up and running in hours Easily deploy new application and data services Rapidly replicate environments for development and test
Simplified:
Dynamically scale to meet changing demand Automatic failover and high availability Fully integrated management
Integrated:
Single support number System level updates and upgrades Simplified problem determination
Patterns of expertise
IBM PureSystems Family

How much flexibility, integration and workload optimization do you want out of the box?
Infrastructure
Integrated and optimized infrastructure with flexibility
Application Platform
Integrated and optimized application platform
Data Platform
Integrated and optimized data platform Delivers high performance data services to transactional and analytics applications
New PureSystem with models optimized exclusively for data workloads
New
Runs your choice of applications and middleware
Built on IBM middleware to accelerate deployment of your choice of applications

Delivering application platform services
Delivering IT infrastructure services
Different data workloads have different characteristics
System for Transactions
Database services that handle large volumes of transactions with high availability, scalability and integrity
System for Analytics

powered by Netezza technology
Data Warehouse services for complex analytics and reporting on data up to petabyte scale with minimal administration
System for Operational Analytics
Operational Warehouse services for continuous ingest of operational data, complex analytics, and a large volume of concurrent operational queries
Database Architectures
Single, Shared Storage, Shared Nothing
Optimized for OLTP
Optimized for Analytics
Single Database View Tran
Single Database View SQL 1
Tran 1
Tran 2
Tran 3
SQL 1 SQL 1 SQL 1
DB2
DB2
DB2
DB2
DB2
DB2
DB2
Log
Log
Log
Log
Log
Database
Shared Data Access
Part 1
Part 2
Part 3
Core DB2
Ideal for OLTP and data marts Pure Applications
9
DB2 pureScale Data Sharing

Ideal for active/active OLTP/ERP scale out PureData for Transactions
DB2 InfoSphere Warehouse (aka DPF)

Ideal for data warehousing with MPP scale out for near linear scalability PureData for Analytics PureData for Operational Analytics
Specialized data systems deliver greater performance and efficiency

E-commerce Customer Analysis Real Time Fraud Detection
Scalable Transactional Database Transaction Processing
Analytics Data Warehouse Reporting and Analytics Sequential reads & sequential data loads
Operational Data Warehouse Operational Analytics Random and sequential reads & data loads + continuous ingest Analytics split into many parts and narrow scope operations, all running in parallel Partitioned data access
Random reads & random updates

Many transactions with narrow data scope accessing the same database Shared access to all data
10
Analytics with broad data scope, split into many parts across data partitions to run in parallel
Partitioned data access
Todays Data Challenges Demand Transactional Performance and Efficiency
Data systems optimized for transactions

High performance and throughput Efficient scalability - for traditional and cloud environments Simplicity - across development and operations
11
Uninterrupted access to data with consistent performance
Traditional systems - build it yourself

Over several days/weeks:
1. Define High Availability topology 2. Configure HW/SW/Network 3. Set up storage pools 4. Install multiple operating systems 5. Install database instances 6. Set up primary and secondary management systems 7. Set up database members 8. Set up backup processes 9. Test, tune, reconfigure
PureData System - built-in expertise

6-node database cluster In minutes,
1. Just specify database, description and topology pattern
12
Automatic node recovery in seconds

Design point is to maximize availability during failure recovery processing
Log Log Log Log
When a database member fails, only in-flight data remains locked until member recovery completes In-flight = data being updated on the failed member at the time it failed
CF
Shared Data
CF
Database member failure Only data in-flight updates locked during recovery
% of Data Available
100
Target time to row availability <20 seconds
50
Time (~seconds)
13
Pattern Based Deployment Across Systems

HA Proxy HA Proxy Application Server
Deploy
Software application
Application Server
Caching Server
Caching Server
Database
Database
14
PureData System for Transactions

Full Rack Capabilities (Large)
384 processor cores 6.1 TB of memory (DRAM) 19.2 TB of flash (SSD) More cores to optimize transactional performance Allows more queries to execute entirely in-memory Allows placement of your most important transactional database objects on the fastest storage (48x400B) High performance storage for todays growing data demands (144x900GB) Provides sustained high performance transactional throughput Automatically migrates most important data to the fastest storage Allows you to store more data in less space while speeding queries through reduced I/O transfers RDMA (Remote Direct Memory Access) for low latency and near-linear scalability High speed redundant database connectivity speeds application performance
128 TB of disk (HDD) 1,500,000 IOPS Advanced Storage Tiering Advanced Adaptive Compression up to 10x High Speed Interconnect Dual 10 GB network
15
* Raw storage space
PureData System for Transactions 3 standard configurations to choose from

Upgrade Upgrade
Configurations
Small Rack
Medium Rack 1 12
Large Full Rack 2 24
Compute Chassis Blacktip ITEs (16 cores per ITE) Cores Memory V7000 + Exp User Capacity Raw SSD Storage (400 GB drives)
1 6
96 1.5 TB 2 18.6 TB 4.8 TB 32.4 TB
192 3.1 TB 4 37.2 TB 9.6 TB 64.0 TB
384 6.1 TB 8 74.4 TB 19.2 TB 128.0 TB
Raw HDD Storage (900 GB drives)
16
Pattern Based Database Deployment

Consolidate
Capture your expertise in patterns for consistent, reliable deployment of critical database services
more that 100 database servers to a single system for optimal resource efficiency and easier administration
Optimize
data compression with up to 10x storage space savings
Cluster Topology
Deploy
Vir HTTP tua Server l Operating Ap system pli an ce Metadata
Innovate
faster by deploying new databases in minutes
Vir Application tua Server l Operating Ap system pli an ce Metadata
Virt Application ual Server Ap Operating plia system nce Metadata
Accelerate
deployment of new database services in cloud environments using patterns of expertise
Deploy
Database
17
Consistent IBM PureSystem console to manage all resources and workloads Easy integration with data center monitoring tools and processes Role-based security and tasks Single point of contact for support System firmware and OS updates applied with no planned downtime
18
PureData login page
19
Welcome page System Console
20
Welcome page Workload Console
21
Workload Management Pure Data for Transactions
22
Database Cluster Topology: Deploy
23
Database Cluster Topology: View
24
Manage a pS instance (con.)
25
User management
26
Manage a pS instance - users
27
Manage a pS instance - user groups
28
Create a new DB pattern
29
Database patterns
30
Deploy a DB from a DB pattern
31
Databases
32
Manage a Database
33
Manage a Database - backup
34
Manage a Database - storage
35
Manage a Database storage (con.)
36
Manage a Database - users
37
Manage a Database user groups
38
System Maintenance Sparta Data System for Transactions
39
Fix Pack Download
40
Fix Pack management
41
Error event log
42
Diagnostic levels
43
A closer look at PureData System for Operational Analytics
44
IBM PureData System
Pattern based database deployment in minutes, not hours1
System for Transactions Handles more than 100 databases on 1 system2 10-100x faster than traditional custom systems4 20x greater concurrency and throughput for tactical queries than previous Netezza technology5 Continuous ingest of operation data Handles 1000+ concurrent operational queries3 Up to 10x storage savings with adaptive compression6
1. Based on IBM internal tests and system design for normal operation under expected typical workload. Individual results may vary. 2. Based on one large configuration 3. Based on IBM internal tests of prior generation system, and on system design for normal operations under expected typical workload. Individual results may vary. 4. Based on IBM customers' reported results. "Traditional custom systems" refers to systems that are not professionally pre-built, pre-tested and optimized. Individual results may vary. 5. Based on IBM internal performance benchmarking 6. Based on client testing is the DB2 10 Early Access Program
System for Analytics

powered by Netezza technology
System for Operational Analytics
45
IBM PureData System for Operational Analytics New Features

Embedded 7710 Single box solution scales down as well as up o The starting building block for 7700 scalable Consolidated GUI Console significantly improved operational experience o Single pane of glass o Alerts for all HW/SW components o Maintenance for all firmware/software cluster wide o Integrated OPM application management Enhanced availability fewer components, reduced outage time o Integrated Backup leveraging 900 GB HDD o Roving Standby, Hot swap SSD o 56x fewer managed resources; 1/3rd fewer relationships Enhanced next generation SSD and enclosure o Dense (1U) Dual Controller SSD enclosure o RAID-5 at high IOPS and bandwidth Higher Density simpler manufacturing, shipment, service o One HA group per rack: no FC cables cross racks o P730 2U server: the rack space
oDense SSD enclosure, Double Dense IO cards
Higher I/O Bandwidth high ingest capability, high scaling o 40 Gbps etherchannel interconnect o 64 Gbps FC storage bandwidth New Software Stack DB210 capabilities o AIX 7.1 + DB2 10
46
PureData for Operational Analytics Building Blocks

Capacity Add-on Data Racks
Server Nodes
IBM 8205-E6C IBM 8231-E2C
XL Foundation Rack
Cable ingress / egress Cable Ingress / Egress
Data Rack (s)

Cable Cable ingress Ingress // egress Egress Filler panels
Filler panels
Storage
IBM SSD 5888 IBM V7 2076-124
PDU
10G Enet SW 4 10G Enet SW 3 1G Enet SW 4 1G Enet SW 3 10G 10GEnet EnetSW SW2 10G Enet SW 1
SSD Enclosure 2
PDU
Foundation Server
740_2
IBM V7 2076-224
HMC 2
Keyboard/Monitor/KVM
Ethernet Switches
IBM 1G 1455-64C IBM 10G 1455-48E SAN switches SAN48B-5 HMC 7042-CR6
Document Drawer
Foundation Modules
PDU
1G Enet SW 2 1G Enet SW 1
HMC 1
Filler panels
SAN Switch 2 SAN Switch Switch Pwr Cord1Routing
Storage Expansion 2
PDU
Storage Controller 1 Storage Expansion 1 SSD Enclosure 1 Foundation Standby

740_1
Cable Ingress / Egress Cable ingress / egress
42U 41U 40U 39U 38U 37U 36U 35U 34U 33U 32U 31U 30U 29U 28U 27U 26U 25U 24U 23U 22U 21U 20U 19U 18U 17U 16U 15U 14U 13U 12U 11U 10U 9U 8U 7U 6U 5U 4U 3U 2U 1U
SSD Enclosure_4 Data Server

730_4
Storage Controller_6 Storage Expansion_6 Storage Controller_5 Storage Expansion_5 SSD Enclosure_3 Data Server
PDU
730_3
Storage Controller_4 Storage Expansion_4 Storage Controller_3 Storage Expansion_3

SAN Switch 2 SAN Switch Switch Pwr Cord 1 Routing
SSD Enclosure_2 Data Server

730_2
PDU
Storage Expansion_2 Storage Controller_1 Storage Expansion_1 SSD Enclosure_1 Data Standby
730_1
PDU
Storage Controller_2
Cable Cable Ingress ingress // Egress egress
42U 41U 40U 39U 38U 37U 36U 35U 34U 33U 32U 31U 30U 29U 28U 27U 26U 25U 24U 23U 22U 21U 20U 19U 18U 17U 16U 15U 14U 13U 12U 11U 10U 9U 8U 7U 6U 5U 4U 3U 2U 1U
Data Module 3
PDU
Data Module 2
Data Module 1
Roving Standby Module
Figures are not to scale

47
Operational Analytics Configuration Sizes

Modules Size XS System A1791-01 MTM 8279-A01 Foundation 1 Data 0 Primary Data (TB) 10.8 Backup Cold Data (TB) 21.6 Comments 0.5D
S
M
A1791-02
A1791-03
8279-A02
8279-A03
1
1
1
2
32.4
54.0
64.8
108.0
1.5D
2.5D 3.5 to 9.5D
A1791-04
8279-A04
3+
75.6
151.2
(add up to 2 full add-on racks AD3) 3.5 to 18.5D
XL
A1791-X4
8279-A05
1+ switches
3+
75.6
151.2
(add up to 5 full add-on racks AD3) 1D 2D 3D full rack
Add-on data 1 Add-on data 2 Add-on data 3
A1791-E1 A1791-E2d A1791-E3
8279-AD1 8279-AD2 8279-AD3
n/a n/a n/a
1 2 3
21.6 43.2 64.8
43.2 86.4 129.6
Capacity per data module (foundation module has half)

21.6 TB Primary Hot Active data 43.2 TB free space (for backups, cold data, etc.)
48
PureData System for Operational Analytics System Console
System Console
Management Unified Appliance UI Alerting via SNMP & Email
(Single System Software Status, Hardware)
Monitoring
Platform Monitoring
(Hardware and non workload specific)
Maintenance Wizard
(Launch point integration)
Maintenance
Provide Easy to Use, Common interfaces for management of Pure System Family
User Authentication
(For console roles/access only)
Workload Monitoring
OPM (SSO, Debranding, LIC)
License Acceptance
On initial interface access
49
Simplified maintenance with pre-integrated fixes
Single point of contact for support Automated updates for faster maintenance
All hardware firmware and OS software patches integrated and tested together at the factory
50
51
52
53
Enhanced Information Center

Information is organized according to topic area and operational lifecycle Overview, Planning, Getting Started, Operational Tasks, Advanced Tasks, Troubleshooting, Reference Each section has further detail, and links to relevant information Easily searchable within the Information Center Easily searchable from any search engine Ability for users to enter comments on topics and participate in discussions
54
Information Center Sample comment
55
HA tools
hareset Soft reset will bring the domain to a base state Clear failed, stuck, pending states - remove resource group requests across entire configuration Rebuild HA configuration completely drop the domain, create domain, create all resources/groups/relationships/equivs etc. hachkconfig Detect and repair resource model problems Detect OS configuration issues that impact HA and alert Can be run automatically or manually
56
High Availability Configuration

Nodes are grouped into HA groups consisting of 4 servers where 3 are active
Failover
Passive Node
Active Node
Each and every server can access the storage of all other servers in its group
One failure per HA group is supported. No performance hit when running in failover Failback is NOT required after failed node is repaired
Failover
Failover
Active Node
Active Node
Active Node
Active Node
Failover
Failover
Active Node
Failover
Passive Node
57
DB2 10 Key Technology

Improved efficiencies, performance and Manageability
Improved Cost Efficiencies Real-time Operational Analytics Improved Productivity and Manageability
Clients have experienced cases of 10x storage space savings via Adaptive Compression Multi-Temperature data management provides improved efficiencies and performance
Faster, more accurate decision making with realtime operational analytics Continuous Ingest of data Built-In Time Travel query enabling faster historical and trend analytical queries
Integrated and simplified system monitoring and maintenance Simplified support for multi-tenant operational warehouses Row and Column access controls to support multiple tenant operational warehouses
58
Multiple Instances of at least 3x Faster Query Performance*

Multi-core parallelism enhancements Performance improvements for:
Queries over star schemas

Queries with joins and sorts Queries with aggregation Hash joins Higher performance Up to 35% faster out-of-the-box performance Multiple instances of at least 3x faster when using new features *
IBM and Intel have collaborated over a decade to optimize DB2 performance with Intel Parallel Studio 2011, software development suite on Intel Xeon processors. We are excited to see a ~10x improvement in query processing performance using DB2 10 over the previous DB2 version, running on IBM System x3850 using Intel Xeon Processor E7. Customers can now realize dramatically greater performance boost at lower cost per query running IBM DB2 10 on servers powered by Intel Xeon processors. Pauline Nist, GM Software Strategies, Intels Datacenter & Connected Systems Group * Based on both external tests by partners, as well as internal tests of IBM DB2 9.7 FP3 vs. DB2 10.1 with new compression features on P6-550 systems with comparable specifications using data warehouse / decision support workloads, as of 29 Mar 2012. 59
Adaptive compression
Reduce Storage Requirements
Save time and cost
Database Compression: Clients have experienced cases of 10x storage space savings Backup Compression: Shrink backup and disaster recovery storage
Improved Performance Workloads run against compressed data result in performance improvements Significantly reduces I/O requirements to process queries
Advanced row compression technique with two compression dictionary levels: table-level page-level Improves compression as data changes
Reduce Costs
Less storage hardware, Smaller footprint reduced power consumption
60
Your Feedback is Important!

Access SmartSite to complete your session surveys Any web or mobile browser at iodsmartsite.com Any SmartSite kiosk onsite Each completed session survey increases your chance to win an Apple TV with daily drawing sponsored by Alliance Tech
61
DB2 Tech Talk: Next Steps Roadmap: PureData Systems

Reference Step One
Visit the PureData Systems Web page bit.ly/pure_data Read the materials
PureData Systems White Paper: The Big Data Zoo Taming the Beast ibm.co/TDl9sV Read the data sheet ibm.co/TDlhsh Watch IBM developerWorks for additional content at ibm.com/developerworks/data
Step Two
Call IBM to schedule a demo or learn more 1 800 966-9875 (U.S) 1-888-746-7426 (Canada) 1800-425-3333 (India) Or visit http://www.ibm.com/planetwide/ for contact information worldwide Information Management Tech forums on developerWorks ibm.co/ZEt7Yw Serge Rielau SQL Tips Blog: www.sqltips4db2.com
Step Three
Take the Test Drive

ibm.co/RVYaeI
Step Four
Connect with the community!

Information Management bit.ly/InfoMgmtCommunity Business Analytics bit.ly/AnalyticsCommunity Enterprise Content Management bit.ly/ECMCommunity IBM Champions ibm.com/champion Recognizing individuals who have made the most outstanding contributions to Information Management, Business Analytics, and Enterprise Content Management communities
62
Upcoming Tech Talks
Dont miss these in-depth DB2 10.1 feature talks! Row and Column Access Control (RCAC) in DB2 10 and InfoSphere Warehouse 10 Walid Rjaibi, IM Data Security Chief Architect December 12, 2012 Reg: bit.ly/tt2012dec
How to register : DB2 Tech Talks web site
Stay Tuned for the 2013 schedule!
Dates and topics subject to change and modification.

63
Questions
Listening in replay? Questions: www.sqltips4db2.com Click submit a question.
64
Thanks for attending!
Please rate the session Presentation download: bitly.com/ttfilenov Next Steps document: click the attachments button
65

DB2 Tech Talk PureData Systems Presentation PDF

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

DB2 Tech Talk PureData Systems Presentation PDF

Transféré par

Droits d'auteur :

Formats disponibles

Technical Tour of PureData Systems for Transactions and Operational Warehousing

James Cho, Architect IBM PureData Systems November 15, 2012

2012 IBM Corporation

SQL Architect, DB2 for Linux, UNIX, and Windows at IBM

Need webcast troubleshooting help? Click attachments

2012 IBM Corporation

Need webcast troubleshooting help? Click attachments

2012 IBM Corporation

New Features and Enhancements

Need webcast troubleshooting help? Click attachments

2012 IBM Corporation

IBMs family of expert integrated systems has expanded

Systems with integrated expertise and built for cloud

Need webcast troubleshooting help? Click attachments

2012 IBM Corporation

PureSystems simplify each phase of the systems lifecycle

Need webcast troubleshooting help? Click attachments

2012 IBM Corporation

IBM PureSystems Family

Runs your choice of applications and middleware

Built on IBM middleware to accelerate deployment of your choice of applications

Delivering IT infrastructure services

Need webcast troubleshooting help? Click attachments

2012 IBM Corporation

Different data workloads have different characteristics

System for Transactions

System for Analytics

System for Operational Analytics

Need webcast troubleshooting help? Click attachments

Single Database View Tran

Single Database View SQL 1

Shared Data Access

DB2 pureScale Data Sharing

DB2 InfoSphere Warehouse (aka DPF)

Need webcast troubleshooting help? Click attachments

Specialized data systems deliver greater performance and efficiency

Scalable Transactional Database Transaction Processing

Random reads & random updates

Need webcast troubleshooting help? Click attachments

2012 IBM Corporation

Todays Data Challenges Demand Transactional Performance and Efficiency

Data systems optimized for transactions

Need webcast troubleshooting help? Click attachments

2012 IBM Corporation

Traditional systems - build it yourself

PureData System - built-in expertise

Need webcast troubleshooting help? Click attachments

2012 IBM Corporation

Automatic node recovery in seconds

Target time to row availability <20 seconds

Need webcast troubleshooting help? Click attachments

2012 IBM Corporation

Pattern Based Deployment Across Systems

Need webcast troubleshooting help? Click attachments

2012 IBM Corporation

PureData System for Transactions

Need webcast troubleshooting help? Click attachments

* Raw storage space

PureData System for Transactions 3 standard configurations to choose from

Large Full Rack 2 24

96 1.5 TB 2 18.6 TB 4.8 TB 32.4 TB

192 3.1 TB 4 37.2 TB 9.6 TB 64.0 TB

384 6.1 TB 8 74.4 TB 19.2 TB 128.0 TB