Académique Documents
Professionnel Documents
Culture Documents
isicc@de.ibm.com
Contents
1. Executive Summary............................................................................................. 2
2. Audience .............................................................................................................. 3
3. Backup concept for SAP HANA ........................................................................ 3
4. Supported HANA configuration ........................................................................ 3
5. Supported storage systems................................................................................ 3
6. Central Theme ..................................................................................................... 4
7. Naming convention ............................................................................................ 4
8. Setup of IBM Spectrum Virtualize / IBM Storwize family ................................. 6
9. Setup of LINUX systems ....................................................................................... 7
10. Preparation of SAP HANA .................................................................................. 8
11. Installation of IBM Spectrum Protect for ERP ................................................... 9
12. Preparation of FlashCopy backup script......................................................... 9
13. Backup best practices ..................................................................................... 12
14. FlashCopy restore .............................................................................................. 12
15. Recover the SAP HANA system ....................................................................... 13
16. Resources ........................................................................................................... 17
17. Notices ................................................................................................................ 17
18. Trademarks ......................................................................................................... 19
19. Sample Output .................................................................................................. 20
1. Executive Summary
Many customers seek solutions to fulfill the requirements of an industrialized
data center. The SAP HANA in-memory computing platform, along with
IBM Storage Systems, extends the possibilities for enterprises. These data
centers, including all SAP and non-SAP applications, must provide the following
fundamental characteristics:
Page - 2
2. Audience
This paper is intended as an architecture and configuration guide to setup IBM
Spectrum Virtualize (SVC or Storwize) and FlashCopy for SAP HANA TDI within
a SAN environment.
This document has been written for IT technical specialists and architects, with
advanced skill levels on SUSE Linux Enterprise Server, korn shell programming, and IBM Spectrum Virtualize (IBM SAN Volume Controller, SVC) or IBM
Storwize.
Page - 3
6. Central Theme
Assuming SAP is already installed, up, and running:
1. Setup ssh com between all HANA nodes; if not done already
2. Install Spectrum Protect for ERP (TSM for ERP/HANA)
3. Setup ssh com between the host the script is running on and Storwize
(SVC)
4. Define user friendly names (alias) for multipath
5. Create FlashCopy target Virtual Disks on Storwize (SVC)
6. Create SAP USERKEY for root on the first HANA node
7. Modify the parameters of the scripts to your needs
7. Naming convention
To easy setup and configuration, a strict use of naming convention is required.
The following syntax is used in this section:
<name> = <parameter>
SID
= ANA
If a name is serving as parameter then this is specified as
name
= %NAME%
SIDPATH = /usr/sap/%SID%
SIDPATH = /usr/sap/ANA
_ means this character cannot be changed, because it a fundamental requirement by e.g. the Linux OS.
_ means this is the name to use, and if desired, it can be changed consistently.
_ indicates an example of associated numbers
/hana/shared/global.ini
partition_#_data__lvmname
= %VG_DATA%-%LV_DATA%
# alias for partition_#_data__lvmname = PD
Page - 4
= %VG_LOG_%-%LV_LOG_%
= PL
=
=
=
=
VG_DATA_?PD?
VG_LOG_?PL?
LV_DATA_?VG_DATA?
LV_LOG_?VG_LOG_?
Please notice that the minus (hyphen) sign may not be part of the VG
or LV name, because it serves as tie sign for the multipath driver:
/dev/mapper/VGname-LVname
alias names in /etc/multipath.conf
(UUID_DATA) alias = %SID%_DATA_?VG_DATA?_#
(UUID_LOG_) alias = %SID%_LOG_?VG_LOG_?_#
Virtual Disk (VD) names on SVC
VD_DATA
VD_LOG_
VD_TARGET
= SAP_HANA_%SID%_DATA_?UUID_DATA?
= SAP_HANA_%SID%_LOG_?UUID_LOG_?
= %VD_DATA%_%TARGET_SETS%
Example:
Assuming the HANA system with SID:ANA is installed as scale out on
three nodes: host1, host2, host3.
Every DATA VG consists of two physical volumes (PV),
every LOG VG consists of one PV.
Four FlashCopy versions should be stored, therefor the parameter
TARGET_SETS has four entries.
TARGET_SETS=FC0 FC1 FC2 FC3
global.ini
partition_1_data__lvmname
partition_1_log__lvmname
partition_2_data__lvmname
partition_2_log__lvmname
partition_3_data__lvmname
partition_3_log__lvmname
=
=
=
=
=
=
VG_DATA_1-LV_DATA_1
VG_LOG_1-LV_LOG_1
VG_DATA_2-LV_DATA_2
VG_LOG_2-LV_LOG_2
VG_DATA_3-LV_DATA_3
VG_LOG_3-LV_LOG_3
consists
hosts LV
consists
hosts LV
host 2
VG_DATA_2 consists of PV ANA_DATA_2_1 & ANA_DATA_2_2
...
host 3
VG_DATA_3 consists of PV ANA_DATA_3_1 & ANA_DATA_3_2
....
VDISK names on SVC
SAP_HANA_ANA_DATA_1_1
SAP_HANA_ANA_DATA_1_1_FC0
SAP_HANA_ANA_DATA_1_1_FC1
SAP_HANA_ANA_DATA_1_1_FC2
SAP_HANA_ANA_DATA_1_1_FC3
SAP_HANA_ANA_DATA_1_2
SAP_HANA_ANA_DATA_1_2_FC0
SAP_HANA_ANA_DATA_1_2_FC1
SAP_HANA_ANA_DATA_1_2_FC2
SAP_HANA_ANA_DATA_1_2_FC3
SAP_HANA_ANA_LOG_1_1
SAP_HANA_ANA_DATA_2_1
...
= PV ANA_DATA_1_1
= PV ANA_DATA_1_2
= PV ANA_LOG__1_1
= PV ANA_DATA_2_1
Create the necessary Virtual Disks (VD) for DATA and LOG as thin provisioned volume svctask mkvdisk ...
Map these volumes to all hosts belonging to the HANA system, including stand-by nodes svctask mkvdiskhostmap force ...
Page - 6
Create the necessary Virtual Disks (VD) as target for DATA VD, also thin
provisioned. Do not map them to any host. svctask mkvdisk ...
Page - 7
{
360050768019382668800000000000208
ANA_DATA_1_1
{
360050768019382668800000000000209
ANA_DATA_1_2
{
36005076801938266880000000000020a
ANA_DATA_2_1
{
36005076801938266880000000000020b
ANA_DATA_2_2
{
36005076801938266880000000000020c
ANA_DATA_3_1
{
36005076801938266880000000000020d
ANA_DATA_3_2
{
360050768019382668800000000000231
LOG_1_1
{
360050768019382668800000000000232
LOG_2_1
{
360050768019382668800000000000233
LOG_3_1
dataalignment 1m /dev/mapper/DATA_1_1
dataalignment 1m /dev/mapper/DATA_1_2
VG_DATA_1 /dev/mapper/DATA_1_1 /dev/mapper/DATA_1_2
-l -i2 -I 256k 100%VG -n LV_DATA_1 VG_DATA_1
-b size=4096 -s size=4096 /dev/mapper/VG_DATA_1-LV_DATA_1
Setup the shared file system for /hana/shared. Typically, this is done via NFS.
Please notice that all hosts, including the stand-by host(s) needs to be included, and available after boot.
Setup ssh communication though keys for root between all HANA nodes. Also
enable ssh communication to itself, which means running on Host nodeA:
nodeA# ssh root@nodeA
Page - 8
Deployment
You can install the fc_backup.sh script on the HANA system, or on another
LINUX system, e.g. /hana/shared
If the script is installed on a remote host, then there is no need to place Storwize
into the same IP network with the HANA system; the Storwize can be placed in
a secondary network. Then the LINUX system hosting the scripts needs connection into both networks. Having two IP networks is a choice, but no requirement.
Page - 9
Parameterization
Before the backup script can be used, the parameterization must be adjusted.
Parameters must be given in the form:
<name>=<value>
Please note that there must not be any white space on either side of the =
operator. The name of parameters must not be changed. LINUX and this script
are case sensitive ! Many parameters refer to other parameters, for example in
path names or file names. In this case, the parameter referred to is denoted in
the form ${<name>}. As an example, the SAP System ID of the SAP HANA Database system is stored in a parameter named SID. The path at which the software executables of SAP HANA Database System with SAP System ID ANA are
installed is provided via a parameter SIDPATH:
SID=ANA
SIDPATH=/usr/sap/${SID}
All available parameters are described in the following sections. The first column of each parameter table denotes whether the parameter must be
changed:
means that the parameter must be adjusted to your installation
denotes a parameter that may be adjusted to you needs if so desired
means that typically this parameter refers to a default setting of SAP HANA
Database that is very unlikely to be changed in any database installation.
Default
LOG_LEVEL
BATCH
false
Page - 10
Description
Specifies the kind of messages to print:
0: no msg will be print, except errors
1: flow information
2: trace information
If set to true (lowercase) all message will
be saved in the log file, no output to
screen. Useful if the script is schedule e.g.
via cron.
--
INSTANCE
--
SIDPATH
/usr/sap/${SID}
INSTPATH
${SIDPATH}/HDB
${INSTANCE}
HANA_SHARED
/hana/shared
FC_HOME
/hana/shared
USER_STORE_KEY
--
LOG_FILE (s)
1)
NODE_LIST
--
Parameter
backup
restore
ID
Displays all FlashCopy backup and the associated HANA Snapshot backup ID (SVC CLI)
For housekeeping purpose. Deletes the entry for the SNAPSHOT ID
in the HANA data base. This ID can be queried with the option
query
Just test the parameterization
No other function will be performed
Parameter
Description
query
delete
test
Options
USER_STORE
batch
Page - 11
Description
KEY
###
all
EXTERNAL_BACKUP_ID
IBM_ANA:FC3
IBM_ANA:FC2
IBM_ANA:FC1
IBM_ANA:FC0
Status
successful
successful
successful
successful
Comment
IBM-SVC-Backup
IBM-SVC-Backup
IBM-SVC-Backup
IBM-SVC-Backup
fc_backup.sh -restore=FC2
Page - 12
Time Completed
2016-10-05_10:53:44
2016-10-05_10:51:38
2016-10-05_10:44:43
2016-10-05_10:44:04
Next >
Page - 13
Next >
List will be empty; it will take some time to get the snapshot listed:
Page - 14
Next >
Next >
Page - 15
Finish
Page - 16
16. Resources
IBM System Storage Architecture and Configuration Guide for SAP HANA TDI
https://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102347
IBM Tivoli Storage Manager for ERM, data protection for SAP HANA
http://www-01.ibm.com/support/knowledgecenter/SSZHVN_6.4.1/com.ibm.itsm.erp.doc/welcome.html
17. Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document
in other countries. Consult your local IBM representative for information on the
products and services currently available in your area. Any reference to an IBM
product, program, or service is not intended to state or imply that only that IBM
product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property
Page - 17
Page - 18
18. Trademarks
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of
International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first
occurrence in this information with the appropriate symbol ( or ), indicating
US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common
law trademarks in other countries. A current list of IBM trademarks is available
on the Web at http://www.ibm.com/legal/copytrade.shtml
The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:
FlashCopy, FlashSystem, IBM, IBM FlashSystem, Storwize, System Storage
The following terms are trademarks of other companies:
SAP, and the SAP logo, HANA are trademarks of the SAP AG, Germany, other
countries, or both.
Other company, product, or service names may be trademarks or service
marks of others.
Page - 19
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
Info_:
======================================================================
========= BACKUP STARTED =============================================
host:ph15na1 has mount point:/hana/data/ANA/mnt00001 and these VDISKs:M_DATA_1_2 M_DATA_1_2 -- PV:DATA_1_1 DATA_1_2
host:ph15na2 has mount point:/hana/data/ANA/mnt00002 and these VDISKs:M_DATA_2_1 M_DATA_2_2 -- PV:DATA_2_1 DATA_2_2
Using this FlashCopy target:FC3.
Creating HANA snapshot backup
BACKUP_ID in state PREPARED: 1475657616048
Creating SVC/Storwize FlashCopy backup
svctask mkfcconsistgrp -name IBM_ANA_FC3 -autodelete
svctask stopfcmap IBM_ANA_FC3_DATA_1_1
svctask stopfcmap IBM_ANA_FC3_DATA_1_2
svctask stopfcmap IBM_ANA_FC3_DATA_2_1
svctask stopfcmap IBM_ANA_FC3_DATA_2_2
svctask chfcmap -consistgrp IBM_ANA_FC3 -copyrate 0 -cleanrate 100 IBM_ANA_FC3_DATA_1_1
svctask chfcmap -consistgrp IBM_ANA_FC3 -copyrate 0 -cleanrate 100 IBM_ANA_FC3_DATA_1_2
svctask chfcmap -consistgrp IBM_ANA_FC3 -copyrate 0 -cleanrate 100 IBM_ANA_FC3_DATA_2_1
svctask chfcmap -consistgrp IBM_ANA_FC3 -copyrate 0 -cleanrate 100 IBM_ANA_FC3_DATA_2_2
syncing disks on ph15na1
syncing disks on ph15na2
Starting Consistency Group: IBM_ANA_FC3
Removed Consistency Group: IBM_ANA_FC3
New HANA BACKUP_ID: 1475657616048 with state SUCCESSFUL -- STORAGE_ID: IBM_ANA:FC3
========= BACKUP COMPLETED SUCCESSFULLY ==============================
Current number of SNAPSHOTs:5, MAX:4, going to delete ID:1475656961987
======================================================================
Trace:
Info_:
Info_:
Trace:
2016.10.05
2016.10.05
2016.10.05
2016.10.05
2016.10.05
2016.10.05
Trace:
Trace:
Trace:
Trace:
Trace:
Info_:
query_DATA_VDISK()
query_DATA_VDISK()
query_DATA_VDISK()
query_DATA_VDISK()
query_DATA_VDISK()
query_DATA_VDISK()
10:44:15
10:44:16
10:44:16
10:44:16
10:44:17
10:44:17
connect to host:ph15na1.
======================================================================
========= BACKUP STARTED =============================================
ssh -n root@ph15na1 mount | grep vg_data_1-lv_data_1 | sed s:on:: |
sed s:/dev/disk/by-id/dm-name-:: | sed s:/dev/mapper/:: | awk { print \$1 \" \" \$2 }
Host:ph15na1, MP=/hana/data/ANA/mnt00001, LV=vg_data_1-lv_data_1, VG=vg_data_1.
ID:DATA_1_1 -- UUID:600507640081811fe8000000000008f5
SVC VD:M_DATA_1_1
ID:DATA_1_2 -- UUID:600507640081811fe8000000000008f6
SVC VD:M_DATA_1_1
host:ph15na1 has mount point:/hana/data/ANA/mnt00001 and these VDISKs:M_DATA_1_2 M_DATA_1_2 -PV:DATA_1_1 DATA_1_2
Page - 21
# ./fc_backup.sh -Q
HANA_BACKUP_ID EXTERNAL_BACKUP_ID
1475657616048
IBM_ANA:FC3
1475657489216
IBM_ANA:FC2
1475657074028
IBM_ANA:FC1
1475657035659
IBM_ANA:FC0
Status
successful
successful
successful
successful
Comment
IBM-SVC-Backup
IBM-SVC-Backup
IBM-SVC-Backup
IBM-SVC-Backup
Time Completed
2016-10-05_10:53:44
2016-10-05_10:51:38
2016-10-05_10:44:43
2016-10-05_10:44:04
./fc_backup.sh -R FC2
2016.10.05 14:46:39 Info_: Starting SVC CLI FlashCopy restore with TARGET FC2.
2016.10.05 14:46:41 Info_: svctask mkfcconsistgrp -name IBM_ANA_FC2_R -autodelete
2016.10.05 14:46:42 Info_: svctask mkfcmap -source M_DATA_1_1_FC2 -target M_DATA_1_1 -name IBM_ANA_FC2_DATA_1_1_R
-consistgrp IBM_ANA_FC2_R -copyrate 100 -cleanrate 100 -autodelete
2016.10.05 14:46:44 Info_: svctask mkfcmap -source M_DATA_1_2_FC2 -target M_DATA_1_2 -name IBM_ANA_FC2_DATA_1_2_R
-consistgrp IBM_ANA_FC2_R -copyrate 100 -cleanrate 100 -autodelete
2016.10.05 14:46:46 Info_: svctask mkfcmap -source M_DATA_2_1_FC2 -target M_DATA_2_1 -name IBM_ANA_FC2_DATA_2_1_R
-consistgrp IBM_ANA_FC2_R -copyrate 100 -cleanrate 100 -autodelete
2016.10.05 14:46:48 Info_: svctask mkfcmap -source M_DATA_2_2_FC2 -target M_DATA_2_2 -name IBM_ANA_FC2_DATA_2_2_R
-consistgrp IBM_ANA_FC2_R -copyrate 100 -cleanrate 100 -autodelete
2016.10.05 14:46:50 Info_: ph15na1 umount /dev/mapper/vg_data_1-lv_data_1 ... successful
2016.10.05 14:46:51 Info_: ph15na2 umount /dev/mapper/vg_data_2-lv_data_2 ... successful
2016.10.05 14:46:52 Info_: svctask startfcconsistgrp -prep -restore IBM_ANA_FC2_R
2016.10.05 14:46:56 Info_: ph15na1 mount /dev/mapper/vg_data_1-lv_data_1 /hana/data/ANA/mnt00001 ... successful
2016.10.05 14:46:59 Info_: ph15na2 mount /dev/mapper/vg_data_2-lv_data_2 /hana/data/ANA/mnt00002 ... successful
2016.10.05 14:46:59 Info_: SVC CLI FlashCopy completed successful for TARGET FC2.
# ./fc_backup.sh -T
SID .......................................................................... ANA
INSTANCE ...................................................................... 00
HDBSQL ............ /usr/sap/ANA/HDB00/exe/hdbsql -U 001 -i 00 -j -a -x -n ph15na1
Status of HANA data base ................................................. ON line
TARGET_SETS ...................................................... FC0 FC1 FC2 FC3
host:ph15na1 has this mount point ........................ /hana/data/ANA/mnt00001
with these PV ....................................................... DATA_1_1
VD ..................................................... M_DATA_1_1
....................................................... DATA_1_2
..................................................... M_DATA_1_2
host:ph15na2 has this mount point ........................ /hana/data/ANA/mnt00002
with these PV ....................................................... DATA_2_1
VD ..................................................... M_DATA_2_1
....................................................... DATA_2_2
..................................................... M_DATA_2_2
Source VD has these targets
IBM_ANA_FC0_DATA_1_1 ==>
IBM_ANA_FC1_DATA_1_1 ==>
IBM_ANA_FC2_DATA_1_1 ==>
IBM_ANA_FC3_DATA_1_1 ==>
Page - 22
Page - 23