Largest Table: 250GB

# of Tables > 1G: 400

# of Tablespaces: 500

Small Objects
1 Partition

Large Objects
(ODS / PSA)
12 Partitions

DB Server - P595 (Squadron) 64Way / 256 GB RAM

DB Partition 0

10

30

31

Basis Tables
Master Data

MidSize
Objects (Fact)
8 Partitions

Dimension
Tables

Fil e Sytems Space al l ocated for the 60 TB database

Total = 65 TB

21
27

21
04
20
85
19
82

2000

19
63

19
96
20
36
20
89
20
47
20
10
21
17
20
45
20
43
20
85
20
95
20
45
20
43
20
56
20
66
20
95
20
84
19
97

20
47
20
94
21
09
20
89
20
24
20
62
20
93
20
49

20
83
21
15

2500

Giga Byt es

1500

500

48
3

1000

0
0

10

11

12

13

14

15

16

17

18

19

20

21

22

DB2 Pa rtitions

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

NestlClone

DB2 DPF
Partitions
0-5
Single LPAR

Phase1
DB2
Partition 0
(1-5 dormant)

Phase2
DB2
Partition 0
(1-5
dormant)

DB2
Partitions
6-13

DB2
Partitions
14-21

DB2
Partitions 6 -13
DB2
Partitions 14 -21
DB2
Partitions 22 -29
DB2
Partitions 30 -37

DB2
Partitions
22-29

DB2
Partitions
30-37

6->33 Partitions
As Received from
Nestl: single DB
LPAR with 6 DB2
DPF partitons

Redistribution of data
over 33 DB2 DPF
partitions and 5 x p5
LPARs

Phase2
Scalability: DB2 distributed over 5
p5-p595s. Final phase is p5 virtualization:
shared processor pool

P8 P24

P8 P24

P9 P25

P9 P25

P16 P32

P16 P32

P17 P33

P20 P36

P20 P36

P8 P24

P17 P33

P8 P24

P16 P32

P21 P37

P21 P37

P16 P32

P9 P25

P9 P25

P17 P33

P17 P33

P6 P22

P6 P22

P7 P23

P7 P23

P10 P26

P10 P26

P11 P27

P11 P27

P6 P22
P7 P23

P6 P22

P10 P26

P7 P23

P11 P27
P12 P28

P12 P28

P13 P29

P13 P29

P12 P28

P12 P28

P13 P29

P13 P29

P14 P30

P14 P30

P15 P31

P15 P31

P14 P30

P14 P30

P15 P31

P15 P31

DA 6

DA 4

DA 7

DA 5

DA 3

P18 P34

P18 P34

P19 P35

P19 P35

P10 P26

P18 P34

P18 P34

P11 P27

P19 P35

P19 P35

DA 1

DA 2

DA 0

Target

P21 P37

Target

P20 P36

P21 P37

Source

DA 0

P20 P36

Source

DA 2

Phase2

Phase1
Redistribution
of DB2 data
from shared
to non-shared
lesystems

Redistibute
the DB2
nodes over
5 LPARs
and 4
storage
servers

Grow to
20TB

DS8300
7-20TB

Grow to 60
Terabyte

DS8300Turbo
20TB

Ea. 512 Disks (146GB 15K rpm) 54

TBs Capacity (production + FC)
4 x DS8300Turbo
60TB

T0042

T0041

T0040

T0039

T0038

T0037

T0036

T0035

T0034

T0033

T0032

CNODE12

T0031

T0030

CNODE11

T0029

T0028

T0027

CNODE10

T0026

T0025

CNODE9

T0024

T0023

T0022

CNODE8

T0021

T0020

CNODE7

T0019

T0018

CNODE6

T0017

T0016

T0015

T0014

T0013

T0012

T0011

T0010

T0009

T0008

T0007

T0006

T0005

12:11

11:51

11:31

11:11

10:51

10:31

10:11

9:51

9:31

9:11

8:51

8:31

8:11

7:51

7:31

7:11

6:51

6:31

6:11

5:51

5:31

I/ORate (IOPS)

12:11

11:51

11:31

11:11

10:51

10:31

10:11

9:51

9:31

9:11

8:51

8:31

8:11

7:51

7:31

7:11

6:51

6:31

6:11

5:51

5:31

I/ORate (IOPS)
1800

N14 Total I/ORate - Individual Data Volumes

1600

1400

1200

1000
800

600

400

200
0

Time

N37 Total I/ORate - Individual Data Volumes

1800

1600

1400

1200

1000

800

600

400

200

CPU by WLM classes sys3db1p 9/1/2006

CNODE13

60

50

40

30

20

10

AIX operating system

AIX operating system

DB2 UDB ESE

DB2 UDB ESE

Tivoli Data Protection
for SAP (DB2)

Online backup
tape restore
Log archiving

Tivoli Data Protection

for SAP (DB2)

Backup of
FlashCopy image

Tivoli Storage Manager

API Client

SHMEM

Log archiving (LAN)

Tivoli Storage Manager

LANfree backup/restore
API Client
SHMEM
TCP/IP (loopback) Tivoli Storage Agent
Tivoli Data Protection
for ACS

production server

TCP/IP

LTO3

Management and
Workow for
FlashCopy Backup

tape delegation for LANfree

Tivoli Data Protection

for ACS

DS8000

FlashCopy

Source

FC
Target

http:
Port 5988
CIM Agent

Tivoli Storage
Manager Server
Management and
Workow for
FlashCopy Backup

backup server

Interface to
FlashCopy
CopyServices

Restore per DB2 Partition

2 Tape Devices used

800,00
700,00

470 GB/h => 133 MB/sec.

600,00

GB/hour

500,00
400,00
300,00
200,00
100,00
0,00
0

6 7 8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
Partition Number

Rollforward Recovery per Partition

NODE0000
NODE0014
NODE0023
NODE0032

NODE0006
NODE0015
NODE0024
NODE0033

NODE0007
NODE0016
NODE0025
NODE0034

NODE0008
NODE0017
NODE0026
NODE0035

NODE0009
NODE0018
NODE0027
NODE0036

NODE0010
NODE0019
NODE0028
NODE0037

NODE0011
NODE0020
NODE0029

NODE0012
NODE0021
NODE0030

NODE0013
NODE0022
NODE0031

250

200

logles recovered [GB]

DB2 Partition 0

150

100

50

0
03:30:00

32 in
Parallel
04:30:00

05:30:00

06:30:00

time

Ref.
KPI-1

KPI-2

KPI-3.a
KPI-3.b

Description
Recovery of an entire days work
Restore database and roll forward
500GB of logs
Disaster recovery
Complete database restore from tape plus
roll forward of 2TB of logs
Daily backup to tape
Full database backup to tape
Daily backup to disk
Completed incremental FlashCopy
of production day

Requirement
<8hrs

<24hr

<12hr
<8hr

Achievement
2hr 40min
using IBM
FlashCopy
10hrs
Using LTO3
tapes
6hr 10min
using LTO3
30min
DS8300 Turbo

Customer
Basis

Phase 1

Phase 2
AIX 5.3
DB2 V9

AIX 5.3
DB2 V8

AIX 5.2
DB2 V8

7TB - 20 TB
20TB - 60 TB

Workload Distribution Ratio DB:APPS

7
6
5
Physical 4
CPUs 3
2
1
0

6
3,9

DB
APPS

2,25

Query Aggregate Load

KPI-A

KPI-D

AIX 5.3
micropartions
DB2 V9

P595 N 0

P595 N 1

P595 N 2

P595 N 3

P595 N 4

DB

DB

DB

DB

DB

Online

Online

Online

Online

Online

Upload

Upload

Upload

Upload

Backup

Backup

Backup

Backup

Aggregate

Extractor

SAP CI

Max CPU:20

DB

Online

Max CPU:48

Max 10

P595 N 1 to 4 : MAX CPU per P595 = 64

CE= 10

CE = 5

Upload

CE= 16
8

Backup

Total Virtual CPU = 86

Backup

CE= 7

Total CE = 38

Apps
DB
Linear (DB)

PhysCPUUtil per Component

250

219

PhyCPU Utilized

200

100
50

146

140

150

72

70
31

35

49

0
KPIA-20

KPID-20

KPID-60

KPIG-60

Load vs CPU Ratio for Scale (GHz basis)

6
Load.Sc ale

Increase Factor

5
4

Phyc .Sc ale

Linear (Load.Sc ale)

2
1

Linear (Phyc .Sc ale)

0
KPIA-20

KPID-20

KPID-60

KPIG-60

25

140
120

20
100

15

80

LOAD
AGGR
RTIME

60

10

40

5
20

KPI KPI KPI KPI KPI KPI KPI KPI KPI KPI KPI KPI KPI
A
A
A
D
D
D
D
D
D
D
D
D
G

10x TXTRATE

i b m .c o m

i b m -s a p .c o m

s a p .c o m

i b m .c o m