DTR Support

SAP HANA System Replication (HSR) on Systems
with Dynamic Tiering
SP 00 through SP 03
This document describes HANA system replication on systems with dynamic tiering.
SAP HANA documentation and SAP HANA dynamic tiering documentation are available on the SAP Help Portal.
What’s New in SAP HANA Dynamic Tiering includes changes for all supported releases.
DT_HSR_SUPPORT.Docx 1
+
Table of Contents
Changes in SAP HANA HSR Behavior..................................................................................................................................4
Multitier Replication .....................................................................................................................................................4
Replication Modes ........................................................................................................................................................4
System Replication Operation mode .............................................................................................................................4
System Replication Options ...........................................................................................................................................4
System Replication Alerts ..............................................................................................................................................4
Communication Network Separation .............................................................................................................................5
Security: System Replication Communication (Legacy SSL Mode) ..................................................................................5
Security: System Replication Communication (Default Authentication) .........................................................................5
Security: allowed_sender Parameter .............................................................................................................................6
Log Shipping..................................................................................................................................................................6
System Replication Alerts ..............................................................................................................................................6
Cluster Manager Hooks .................................................................................................................................................6
Initialize or Reregister Secondary System with Pre-existing and Compatible Persistence ...............................................6
Takeover .......................................................................................................................................................................7
Failback .........................................................................................................................................................................8
Upgrade ........................................................................................................................................................................8
Log Retention ................................................................................................................................................................8
Log Timestamp ..............................................................................................................................................................9
Point-in-time Recovery (PITR) Post Takeover Using Log Backups from Former and New Primary ...................................9
Out-of-Space Handling ..................................................................................................................................................9
Adding or Deleting Service on the Fly ..........................................................................................................................10
Active/Active Support .................................................................................................................................................11
Console Tool (hdbcons) ...............................................................................................................................................11
Hot Standby ................................................................................................................................................................11
Initialization of Secondary Site ....................................................................................................................................11
Replication of .ini File ..................................................................................................................................................12
Multitenant Database Container Support when Multiple Tenants Have Dynamic Tiering.............................................12
Enable Replication.......................................................................................................................................................12
Disable Replication ......................................................................................................................................................12
Tenant Copy ................................................................................................................................................................12
Multitenant Database Containers with Dynamic Tiering ..............................................................................................13
Retention Time Estimation ..........................................................................................................................................13
Configuration Parameters ...............................................................................................................................................13
Monitoring View and Statistics ........................................................................................................................................17
Secure Internal Communication ......................................................................................................................................22
Point-in-time Recovery Post Takeover.............................................................................................................................22
Backups in a Shared Location ......................................................................................................................................22
Backups Not in a Shared Location ................................................................................................................................22
Example: .....................................................................................................................................................................23
Version Information ........................................................................................................................................................26
Coding Samples...............................................................................................................................................................26
Copyright ........................................................................................................................................................................26
Changes in SAP HANA HSR Behavior
The following HANA behaviors change when you add the SAP HANA dynamic tiering to the landscape.
Unless otherwise noted, all changes introduced in earlier versions apply to SAP HANA dynamic tiering SP 03. For more
information about new and enhanced features, see What’s New in SAP HANA Dynamic Tiering at
https://help.sap.com/viewer/aba22114586b4be5b497610e1bd4cae2/latest. The release notes are cumulative, showing
changes in an historical context by support package.
Multitier Replication
• See “SAP HANA Multitier System Replication” in the SAP HANA Administration Guide.
• SP 00 and 01: With dynamic tiering in the landscape, supports only 2-tier HA replication.
• SP 02 and 03 (Changed): With dynamic tiering in the landscape, supports multitier replication with the following
restrictions:
 Replication from tier1 to tier2 can only have sync replication mode.
 Replication from tier2 to tier3 can only have async replication mode.
 Delta store (RLV) cannot be enabled on the primary system.
For existing (SP 01 or older) 2-tier HSR setups including dynamic tiering, you must first upgrade both sites
before adding the tertiary site.
Replication Modes
• See “Configure the Secondary System" and "Set Up System Replication" in the SAP HANA Administration Guide.
• SP 00 and 01: With dynamic tiering in the landscape, SAP HANA supports sync mode without the full_sync
option.
• SP 02 and 03: (Changed) With dynamic tiering in the landscape, 2-tier replication is supported with these
restrictions:
 sync replication mode is supported
 async replication mode is supported but delta store (RLV) cannot be enabled on primary
With dynamic tiering in the landscape, 3-tier is supported with the restrictions listed under Multitier Replication.
System Replication Operation mode

• See “Operation Modes for SAP HANA System Replication” in SAP HANA Administration Guide.
• SP 00 and 01: With dynamic tiering in the landscape, replication supports logreplay only.
• SP 02 and 03: (Changed) Supports logreplay and logreplay_readaccess. See Active/Active Support.
System Replication Options

• See Configuration Parameters for usage recommendations when your landscape includes dynamic tiering.
System Replication Alerts

• See “M_EVENTS System View” in SAP HANA SQL and System Views Reference, “System Replication Alerts” in SAP
HANA Administration Guide and Out of Space Handling.
• SP 00 and 01: For the dynamic tiering disconnect event, SAP HANA reports a master indexserver alert with the
description field showing the prefix ES.
• SP 02 and 03: (Changed) The master indexserver still reports dynamic tiering events, but now sets two
M_EVENTS monitoring view columns. SOURCE_HOST is set to the dynamic tiering host and SOURCE_PORT is set
to the dynamic tiering port. Two new alerts are supported: replay backlog and shipping backlog.
Communication Network Separation

• See “System Replication Scenario” in the SAP HANA Security Guide and “Connections for Distributed SAP HANA
Systems” and “Internal Host Name Resolution” in the SAP HANA Administration Guide.
• SP 00 and 01: Does not support private/internal IPs or listeninterface option.
• SP 02: (Changed) Although dynamic tiering does not support network separation for dynamic tiering
communication in general, it is supported for system replication.
• SP 03: (Changed) Although dynamic tiering does not support network separation for general communication,
dynamic tiering system replication supports network separation and private/internal IPs.
• Make sure that your firewalls allow the following connections required for dynamic tiering system replication:
 Secondary to primary on DT replication port
 Secondary to primary on DT SQL port
 Primary to secondary on DT SQL port
 Secondary to primary on DT SQL port
• If you have a tertiary system, your firewall must also allow:
 Tertiary to secondary on DT replication port
 Tertiary to secondary on DT SQL port
 Secondary to tertiary on DT SQL port
Security: System Replication Communication (Legacy Configuration of Secure Internal

Communication)
• See “Secure Internal Communication Between Sites in System Replication Scenarios" in the SAP HANA Security
Guide.
• SP 00 through 03: dynamic tiering does not support Legacy SSL mode.
Dynamic tiering only supports SystemPKI TLS/SSL for dynamic tiering system replication communication.
Note: The recommended SSL mode for system replication communication is SystemPKI.
If Legacy SSL mode is configured, dynamic tiering internal system replication communication is non-SSL.
Security: System Replication Communication (Authentication)

• See Secure Internal Communication.
• SP 00 and 01: the dynamic tiering service did not support authentication.
• SP 02 and 03: (New) SAP HANA systems with dynamic tiering support authentication for system replication
communication and use TLS/SSL for authentication.
Security: allowed_sender Parameter
• See "Host Name Resolution for System Replication" in the SAP HANA Administration Guide.
• If no separate network channel was configured for the SAP HANA system replication communication between the
involved sites, you can use the allowed_sender parameter to restrict communication between primary and
secondary to certain hosts.
• SP 00 and 01: the dynamic tiering service did not support allowed_sender.
• SP 02 and 03: (New) The dynamic tiering service now supports allowed_sender.
Log Shipping
• See “Data transferred to the Secondary System,” in SAP HANA Administration Guide.
• SP 00 through 03: Transactions that involve dynamic tiering may commit on the primary system, and generate
logs, while the secondary system initializes. Dynamic tiering must ship these logs, called missing logs, after
initialization.
Secondary dynamic tiering pulls missing logs first. Only when the last missing log has been fetched does the
primary start to ship active logs (those that ship synchronously with primary transactions). For this reason,
dynamic tiering may take a long time or may not reach ACTIVE replication status.
• If the dynamic tiering secondary system disconnects and reconnects to the primary, the secondary system must
again pull the missing logs before primary dynamic tiering ships active logs.
• SAP recommends that user or tools only do bulk transactions on the primary dynamic tiering system when the
secondary dynamic tiering service has replication status ACTIVE (and not when the replication status is SYNCING,
INITIALIZING or ERROR). The active secondary replication for dynamic tiering helps ensure that the missing log is
always small, and the secondary dynamic tiering can always catch up to the primary dynamic tiering.
System Replication Alerts

• See "System Replication Alerts" in the SAP HANA Administration Guide.
• SP 00 and 01: The disconnect event/alert generated by dynamic tiering is reported as a master indexserver alert
with the description field showing the prefix "ES."
• SP 02 and 03: (Changed) Instead of using the ES prefix, the M_EVENTS monitoring view now has two separate
columns, sourceHost and sourcePort. The master indexserver still creates events for dynamic tiering, but sets
the sourceHost column to the dynamic tiering host and the sourcePort column to the dynamic tiering port.
Cluster Manager
• See "Example HA/DR Provider Implementation" and "Multiple-Host System Concepts”" in the SAP HANA
Administration Guide.
• SP 00 through 03: If dynamic tiering is in your landscape, you cannot use the cluster manager.
Initialize or Reregister Secondary System with Pre-existing and Compatible Persistence

• See Upgrade, “Initializing the Secondary,” and “Auto-Failover for High Availability” in the SAP HANA Administration
Guide.
• SAP HANA does delta log (or data) based re-initialization.
• SP 00 and 01: Secondary dynamic tiering is reinitialized completely (full data shipping) on every secondary
registration. This includes various use cases like failback, reregister, test takeover, and near-zero downtime
upgrades. Restriction as such only applies to dynamic tiering service.
• SP 02 and 03: (Changed) Dynamic tiering supports delta log optimization when reregistering a secondary under
following conditions:
 There must be a compatible PITR log position on primary and secondary to perform delta initialization.
 PITR log files starting with the most compatible position up to the current PITR log position must be
present on both primary and secondary.
 Database files status should be same on both primary and secondary.
 No alter or drop database file on either primary or secondary from the most compatible position.
º No new DB files should be added on secondary post takeover or from the most compatible position.
If delta store (RLV) is enabled or disabled on either primary or secondary server from the most
compatible position, then delta optimization is not possible.
º New DB files can be added on primary from the most compatible position.
 Systems cannot be resynced using delta initialization if it is a result of failed full initialization.
 The landscape is not available for takeover (not replication safe) until initialization is complete for all
services.
Takeover
• See "Stop a System" and "Perform a Takeover" in the SAP HANA Administration Guide.
• SP 00 and 01: Takeover is only marked complete when dynamic tiering status is ACTIVE post restart so the entire
landscape is not available until dynamic tiering is back. Restriction applies to entire system.
• SP 02: (Changed) Asynchronous (optional) restart is available. By default, takeover waits until dynamic tiering
status is ACTIVE. Dynamic tiering is internally restarted on takeover (even in online takeover). Hot standby is not
supported for dynamic tiering.
 To configure whether takeover should wait for a dynamic tiering restart or not, use the option
global.ini/[system_replication] takeover_wait_until_esserver_restart on the secondary site before you
attempt takeover. Once takeover is done, reset the option if you do not need the subsequent takeovers
to follow this setting.
• SP 03: (Changed) During takeover, the dynamic tiering service takes a backup of its existing database logs from
the active log area of the secondary site (logs that were not backed up on the primary site prior to takeover). If
for any reason, the backup of existing logs on the dynamic tiering service fails, it results in the overall SAP HANA takeover
failing.
 A new ini setting has been introduced in the global.ini, [system_replication] section with the name
“takeover_esserver_without_log_backup” and the default value is false.
 If a takeover attempt fails and you see a message like the following in an esserver trace file, the backup
of the existing log area failed:
“Error backing up existing log area. Found ‘0’ logx in ‘xxx’ directory.”
 To skip attempting log backup during takeover, set takeover_esserver_without_log_backup to true on
the secondary site and retry takeover. Reset the option after takeover completes if you do not need
subsequent takeovers to follow this setting.
 Skipping log backup during takeover breaks the log continuity for the dynamic tiering service. If you skip
a log backup, you cannot perform point-in-time recovery of the new primary site using the data and log
backups taken on the old primary site before takeover. SAP recommends that you take a full data
backup of the primary site as soon as it is available after takeover. Use this new data backup to perform
any subsequent point-in time recoveries of the new primary site.
Failback
• See "Configuring the Secondary System" and "Performing a Failback" in the SAP HANA Administration Guide.
• SP 00 and 01: Secondary dynamic tiering will be reinitialized completely. Restriction, as such, only applies to
dynamic tiering service.
• SP 02 and 03: (Changed) Dynamic tiering supports delta log optimization when reregistering a secondary under
conditions. See Initialize or Reregister Secondary System with Pre-existing and Compatible Persistence.
 The landscape is not available for takeover (not replication safe) until initialization is complete for all
services.
Upgrade
• See “Updating SAP HANA Dynamic Tiering” in SAP HANA Dynamic Tiering Installation and Update Guide.
• SP 00 through SP 03: Software upgrade requires downtime when dynamic tiering is part of the landscape.
Dynamic tiering can handle optimized resync or delta initialization, but disallows near zero downtime upgrades
because both software versions may not be compatible to support system replication. SAP recommends that
you upgrade both sites together, then restart them.
To upgrade:
1. Stop transactions on primary system.
2. Wait until replication status on secondary site becomes ACTIVE for dynamic tiering.
3. Stop both sites.
4. Upgrade binary on both sites.
5. Restart both sites.
Log Retention
• See "Log Retention" in the SAP HANA Administration Guide.
• SP 00 through 02:
 When enable_log_retention is set to off (disabled), then there is no log retention for SAP HANA systems
and the backed-up log files are marked as FREE for immediate reuse.
 Dynamic tiering systems always do log retention regardless of this option.
 If enable_log_retention is set to on (enabled) and logshipping_max_retention_size is set to 0 and the
log DISK_FULL, then the primary SAP HANA system hangs and resumes execution once disk space is
available.
 In this scenario, dynamic tiering cannot write critical information during the commit or checkpoint, and
may fail. This behavior difference only applies to the dynamic tiering service. To resolve the issue, do
one of the following:
1. Add more space to the log volume
or
2. Delete backed-up log files for point-in-time recovery (.eslog files), but as a side effect, you may need
to reinitialize secondary dynamic tiering in case those files were not replicated on the secondary
system.
• SP 03: (New) Two new values for enable_log_retention are introduced for HANA but unsupported by SAP HANA
dynamic tiering. If you set force or force_on_takeover, then the behavior of log retention on dynamic tiering is
undefined.
Log Timestamp
• SP 00 and 01: Not applicable.
• SP 02 and 03: (New) A timestamp is added to the names of log (.eslog) files for SAP HANA dynamic tiering.
Point-in-time Recovery (PITR) Post Takeover Using Log Backups from Former and New Primary
• See Point-in-time Recovery Post Takeover.
• SP 00 through SP 01: For the dynamic tiering service, the PITR log file names are based on sequence numbers
that are identical on post-takeover primary and secondary systems. Be careful when performing PITR using
mixed-log backups; avoid accidental overwrites of PITR log files on the secondary side.
• SP 02 and 03: (Changed) All file names are unique (primary vs secondary), so no extra precaution is needed.
Out-of-Space Handling
See “Persistent Data Storage in the SAP HANA Database” in the SAP HANA Administration Guide.
• SP02 and 03: (New) Supports out-of-space handling when point-in-time log area space for dynamic tiering falls
below low threshold.
Dynamic tiering raises appropriate alerts to HANA for each of these events:
 DML commands are blocked
 DDL commands except dbspace modifications are rolled back
 Essential operations such as checkpoint, transactions in post commit, and dbspace modification DDLs go
through using threshold space.
 When threshold space is used up all operations are blocked until space is made available.
Low threshold size is user configurable by set option ES_LOG_THRESHOLD_SIZE, in MB.
The default option value is based on the dbspace size. For smaller databases under 1TB, the threshold is set to
10% of dbspace size, or 2 G, whichever is greater. For larger database sizes, it is set to 1% of dbspace size.
To turn off threshold processing, set the configurable threshold size to 1 M.
If you run out of PITR log space:
1. Check the configured option ES_LOG_THRESHOLD_SIZE. If that is set to a much larger value than the
typical PITR log volume, lower it to a value that can still accommodate the typical PITR log volume.
2. If the setup is for HSR, check the option logshipping_max_retention_size. The HANA default is 1 TB.
Setting it to a lower value can ensure that automatic log truncation can truncate some PITR log files and
free up space.
3. If the setup is not for HSR, check if log backup and manual truncation of PITR log files can be set up
regularly to free up space.
4. If the above steps still do not resolve the out-of-space condition:
º Add more space to the PITR log file system, or
º Free up space manually, or
º Move the PITR files to a different file system with more space, and create a soft link from the
original PITR log directory to the new file system directory.
Adding or Deleting Service on the Fly
See “CREATE EXTENDED STORAGE Statement [Dynamic Tiering]” or “DROP EXTENDED STORAGE Statement
[Dynamic Tiering]” in the SAP HANA SQL and System Views Reference.
• SP 00 and 01: The dynamic tiering service does not support CREATE or DROP EXTENDED STORAGE on the
primary system when system replication is enabled.
To work around this issue:
1. Stop the secondary system.
2. Run sr_unregister on the secondary if the secondary is already registered. Do not start secondary after
this.
3. Run sr_disable on primary system.
4. Create extended storage.
5. Take a full data backup of the primary system.
6. Run sr_enable on primary system.
7. Run sr_register on secondary system if it was unregistered in Step ii
8. Start the secondary system. Note: As a side effect of step iii, some SAP HANA services may need full
reinitialization.
• SP 02 and 03: [Changed] Supports addition and removal of dynamic tiering service on-the-fly in any given tenant
of an HSR primary system.
To create dynamic tiering service on a different host that is not in the landscapes yet for the already replicating
HANA systems:
1. Add a new host on the secondary site. Always add hosts on consumer sites (secondary and tertiary)
before the primary site. Give the host extended_storage_worker role and optionally specify a worker
group.
2. Add a new host on the primary site with extended_storage_worker role and optionally specify a worker
group.
3. Provision the host on the primary to one of the tenant databases on which you intend to create dynamic
tiering.
4. Create extended storage on the tenant database.
For example, create host2 on the secondary and host1 on the primary, with both in worker group group1.
Provision host1 to tenant database DB1.
In this example, SAP HANA:
º Maps the hosts appropriately based on their worker groups in the system replication topology
information.
º Provisions host2 automatically to tenant database DB1 on the secondary system.
º Starts replicating dynamic tiering volume from host1 on the primary to host2 on the secondary as
part of replication for tenant database DB1.
Dynamic tiering hosts added on both sites (primary and secondary) are mapped per their specified worker
groups. If worker groups are not specified while adding new dynamic tiering hosts, then the system uses its own
mechanism to map dynamic tiering hosts across primary and secondary systems.
To create the dynamic tiering service on a HANA worker host that is already mapped for system replication:
1. Add extended_storage_worker role to the desired host on the secondary system. (Always add the role
to secondary and tertiary sites before primary sites.)
2. Add extended_storage_worker role to corresponding mapped host on the primary system.
3. Provision the host on the primary system to one of the tenant databases where you plan to create
dynamic tiering, for example DB1.
4. Create extended storage on the tenant database on the primary system.
For example, add the extended_storage_worker role to host2 on the secondary and then to host1 on the
primary. Provision host1 to tenant database DB1, then create dynamic tiering on the tenant database DB1 on
host1.
SAP HANA provisions host2 automatically to tenant database DB1 on secondary and starts replicating dynamic
tiering volume from host1 on the primary to host2 on the secondary as part of replication for tenant database
DB1.
Multitarget System Replication

• See “Multitarget System Replication” in the SAP HANA Administration Guide.
• SP 00 through 02: Feature not supported.
• SP 03: (New) If dynamic tiering is part of the landscape, the site does not allow multitarget system replication.
Active/Active Support
• See “Active/Active (Read Enabled)” in the SAP HANA Administration Guide.
• SP 00 and 01: If dynamic tiering is part of the landscape, the site does not allow active/active support, even for
SAP HANA services.
• SP 02 and 03: (Changed) If active/active (read enabled) is used with dynamic tiering services, there is no read
access to dynamic tiering data on the secondary site. When dynamic tiering is part of landscape, active RO
access is allowed using operation mode logreplay_readaccess on other HANA services (IS) but without dynamic
tiering access. (A query that accesses data on secondary dynamic tiering returns an error.) When you query
M_PERSISTENCE_ENCRYPTION_STATUS monitoring view on the primary system, the persistence encryption
status of esserver volumes on the secondary system does not display.
Console Tool (hdbcons)

• See “M_SERVICE_REPLICATION System View” in the SAP HANA SQL and System Views Reference.
• SP 00 through 03: The dynamic tiering service does not support hdbcons. You must query replication statistics
for the dynamic tiering service from the monitoring view M_SERVICE_REPLICATION.
Hot Standby
• See “Performing a Takeover” in the SAP HANA System Administration Guide.
• SP 00 through 03: If dynamic tiering is part of the landscape, the site does not allow hot standby, even for SAP
HANA services. On takeover, dynamic tiering is internally restarted. Takeover is marked as complete only after
dynamic tiering is back. Takeover is only marked complete when dynamic tiering status is ACTIVE after restart,
so the entire landscape is unavailable until dynamic tiering is back. The option
takeover_wait_until_esserver_restart allows the landscape to be available before dynamic tiering restart
completes.
Initialization of Secondary Site

• See “Initialize the Secondary with Storage Copy from Primary” in the SAP HANA System Administration Guide.
• SP 00 through 03: The dynamic tiering service does not support initialization of secondary site with binary
storage copy.
Replication of .ini File
• See “Monitoring INI File Parameter Changes” in the SAP HANA Administration Guide.
• SP 00 and 01: The system parameter checker may not work for esserver.ini file.
• SP 02 and 03: (Changed) The system parameter checker works for esserver.ini.
Multitenant Database Container Support when Multiple Tenants Have Dynamic Tiering
• See “SAP HANA System Replication with Tenant Databases” in the SAP HANA Administration Guide.
• SP 00: Multitenant database containers don’t support dynamic tiering during HANA system replication.
• SP 01: SAP HANA generally supports HSR in presence of any number of tenants, but multitenant database
containers don’t support SAP HANA system replication when more than one tenant database has dynamic
tiering configured.
• SP 02 and 03: (Changed) HANA system replication works with multiple tenants configured with extended
storage, if each tenant has its own dynamic tiering service.
 You cannot co-deploy more than one dynamic tiering service on the same host.
 Dynamic tiering services across different tenant database do not need to be running for enabling system
replication on SAP HANA.
All existing dynamic tiering HSR functionality restrictions still hold true when HSR is configured on MDC setups
with dynamic tiering.
Enable Replication
• See “Creating Backups” in the SAP HANA Dynamic Tiering Administration Guide and “Setting Up SAP HANA System
Replication” in the SAP HANA Administration Guide.
• SP 00: Not applicable.
• SP 01: Enable replication fails on the entire landscape if the tenant having dynamic tiering is not running or a full
data backup has not been done on that tenant To work around this issue, start the tenant having dynamic
tiering, take a full data backup of this tenant and then attempt to enable replication on the system.
• SP 02 and 03: Enable replication works on the entire landscape if the tenant(s) running dynamic tiering are not running or
a full data backup has not yet been done on the tenant(s). Once a data backup of the tenant running dynamic tiering is
taken, secondary services of that tenant (including dynamic tiering) start replicating automatically.
Disable Replication
• See “Disabling SAP HANA System Replication” in the SAP HANA Administration Guide.
• SP 00 through 02: If system replication is disabled on the primary system, then log truncation is disabled. Logs
are not deleted from the log directory. To clean up the log area, you need to manually delete all logs except the
current log (log with the highest log sequence and sub-sequence).
• SP 03: If system replication is disabled on the primary system then log truncation is disabled. Logs in the existing log area of
dynamic tiering are deleted as part of ‘Disable Replication’.
Tenant Copy and Move

• See “Creating Backups” in the SAP HANA Dynamic Tiering Administration Guide and “SAP HANA System Replication
with Tenant Databases” and “Setting Up SAP HANA System Replication” in the SAP HANA Administration Guide.
• SP 00 through 03: If dynamic tiering is present on the HANA system replication primary system, the entire
system does not support tenant copy functionality.
Retention Time Estimation
• See “Monitoring SAP HANA System Replication with the SAP HANA Cockpit” in the SAP HANA Administration Guide.
• SP 02 and 03: [New] Dynamic tiering does not support the SAP HANA Cockpit SP 02 functionality that supplies
estimated log retention and estimated log full times.
Configuration Parameters
Parameters defined in the system_replication section of the global.ini file configure replication between the primary and
secondary system.
For descriptions and configuration change policies of these parameters, see "Frequently Used Configuration Parameters
in SAP HANA," attached to SAP Note 2036111 - Configuration parameters for the SAP HANA system.
With exceptions as noted, the parameters behave the same way in dynamic tiering and HANA systems.
Parameter System Description Dynamic Tiering Behavior
actual_mode Secondary Stores the replication mode specified in the SP 01: hdbnsutil –
sr_register command. See Replication Modes. sr_changeMode allows
only the value sync if
dynamic tiering is part of
the landscape.
SP02 and 03: Supports the
value sync for 2-tier
systems with a delta store
and the values sync and
async for 2-tier systems
without a delta store.
allowed_sender Primary Option change takes effect on restart of primary SP 01: Not supported.
SP 02 and 03: Same as
SAP HANA.
datashipping_parallel_ch Primary See "System Replication Configuration Parameters" Not supported.

annels in the SAP HANA System Administration Guide.
enable_full_sync Primary Change takes effect as soon you set parameter with It is strongly
“reconfigure clause.” See "System Replication recommended NOT to
Configuration Parameters" and "System Replication turn on ‘enable_full_sync’
Details" in the SAP HANA Administration Guide. option when dynamic
Note: Use the command hdbnsutil -sr_fullsync tiering is part of the
landscape as dynamic
tiering does not support
full sync mode and, thus,
there is no value added to
the entire system in terms
of data loss resiliency.
If you set this option
without the command
"hdbnsutil -sr_fullsync,"
all the HANA services run
in
SYNC_WITH_FULL_SYNC
mode while dynamic
tiering runs in SYNC mode.
Note: hdbnsutil
-sr_fullsync command
returns an error if you
turn ON full_sync mode
when dynamic tiering is
part of the landscape.
enable_log_retention Primary Values are ‘AUTO’, ‘ON’, and ‘OFF’. When SP 01 and 02: Dynamic
enable_log_retention is set to ‘OFF’ (disabled), tiering does not support
then there is no log retention for SAP HANA the setting 'OFF.'
systems and the backed-up log files are marked as When set to 'OFF' SAP
FREE for immediate reuse. HANA truncates logs as
Dynamic tiering systems always do log retention soon as they are backed
regardless of this option. up and ships them to the
secondary system.
For dynamic tiering, logs
are truncated periodically.
If you change the value,
you must restart dynamic
tiering (the esserver
service) for the new value
to take effect..
SP 03: Same as SAP HANA
– no restart required.
Dynamic tiering does not
support new log retention
mode 'Force' and
'ForceOnTakeover.'
enable_ssl Primary See "Legacy Configuration of Secure Internal Dynamic tiering does not
and Communication" in the SAP HANA Security Guide. support legacy
secondary Option change takes effect on the next restart configuration of secure
of the primary or secondary system. internal communication.
SP 01: Same as SAP HANA.
However dynamic tiering
only supports SystemPKI.
To use SSL for dynamic
tiering replication port
communication, you must
also set the following
parameter:
[communication]
ssl = systempki
SP 02 and 03: Dynamic
tiering supports TLS/SSL
based authentication
(same config as SAP
HANA) and is the default.
For end to end TLS/SSL
encryption, user needs to
turn ON "enable_ssl"
listeninterface Primary Option change takes effect on restart of primary SP 01: Not supported.
and system. See "Internal Host Name Resolution" in the Primary/Secondary
secondary SAP HANA System Administration Guide. dynamic tiering
disconnects replication
when a reconfigure
request sets this option to
“.internal.” The REPLICATI
ON_STATUS_DETAILS and
the DISCONNECT event for
dynamic tiering indicate
that dynamic tiering does
not support
“hostname_resolution.”
SAP HANA. Dynamic
tiering now supports
private interconnect.
logshipping_async_buffer Primary See "System Replication Configuration Parameters" SP 01: Not supported.
_size in the SAP HANA System Administration Guide. SP 02 and 03: Same as
SAP HANA.
logshipping_async_wait_ Primary See "System Replication Configuration Parameters" SP 01: Not supported.
on_buffer_full in the SAP HANA System Administration Guide. SP 02 and 03: Same as
SAP HANA.
logshipping_max_retenti Primary Option change takes effect as soon as it is set with SP 01 and 02: Requires a
on_size “reconfigure clause” See "System Replication dynamic tiering restart
Configuration Parameters" in the SAP HANA System for this option to take
Administration Guide. effect.
SP 03: Same as SAP HANA
- no longer needs a restart
logshipping_timeout Primary See "System Replication Configuration Parameters" Same as SAP HANA.
in the SAP HANA System Administration Guide.
operation_mode Secondary See "System Replication Configuration Parameters" SP 01: hdbnsutil –

in the SAP HANA System Administration Guide. sr_register allows only the
value “logreplay," the only
mode supported by
dynamic tiering.
SP 02 and SP 03:
Supports the values
“logreplay” and
“logreplay_readaccess.”
See Active/Active Support.
reconnect_time_interval Secondary See "System Replication Configuration Parameters" SP 00 – SP 03: Same as

in the SAP HANA System Administration Guide. SAP HANA.
system_replication_host Primary Option change takes effect on restart of primary. SP 01: Dynamic tiering
name_resolution and does not support
secondary hostname_resolution.
Primary/secondary
dynamic tiering
disconnects replication
when reconfigure request
sets this option to
“.internal." The REPLICATI
ON_STATUS_DETAILS and
the DISCONNECT event for
dynamic tiering indicate
that dynamic tiering does
not
support “hostname_resol
ution.”
SAPHANA. Dynamic tiering
now supports private
interconnect.
Monitoring View and Statistics
The proxy monitoring views supported by SAP HANA on the primary site for monitoring on the secondary site show no
information for the dynamic tiering service. SAP HANA supports no dynamic tiering-specific proxy monitoring views.
For details about replicated services, see “M_SERVICE_REPLICATION System View” in the SAP HANA SQL and System
Views Reference.
The following columns in M_SERVICE_REPLICATION System View do not apply to dynamic tiering systems:
• LAST_SAVEPOINT_VERSION
• LAST_SAVEPOINT_LOG_POSITION
• LAST_SAVEPOINT_START_TIME
• MAX_REPLAY_BACKLOG_SIZE
• MAX_REPLAY_BACKLOG_TIME
• REPLAY_BACKLOG_TIME
• SHIPPED_SAVEPOINT_VERSION
• SHIPPED_SAVEPOINT_LOG_POSITION
• SHIPPED_SAVEPOINT_START_TIME
• SHIPPED_DELTA_REPLICA_COUNT
• SHIPPED_DELTA_REPLICA_SIZE
• SHIPPED_DELTA_REPLICA_DURATION
• SHIPPED_LAST_DELTA_REPLICA_SIZE
• SHIPPED_LAST_DELTA_REPLICA_START_TIME
• SHIPPED_LAST_DELTA_REPLICA_END_TIME
Note these behavior differences for dynamic tiering:
• Dynamic tiering does not report an error message in the REPLICATION_STATUS_DETAILS column in the
M_SERVICE_REPLICATION system view on initialization failure.
• After REPLICATION_STATUS for dynamic tiering becomes ACTIVE, the SHIPPED_LOG_POSITION may remain
smaller than LAST_LOG_POSITION. You may ignore this, as dynamic tiering doesn't account for internal
transactions in SHIPPED_LOG_POSITION.
• In SP 00 and SP 01, Replay alert for a service depends on replay backlog that is generally computed based on
SHIPPED_LOG_POSITION and REPLAYED_LOG_POSITION. Dynamic tiering reports commit redolog offset for the
two positions, and they provide no measurement on the overall replay backlog for dynamic tiering, so the replay
alerts for dynamic tiering that are generated by the system are correct.
• SP 00 and 01: Dynamic tiering does not show progress in the REPLICATION_STATUS_DETAILS column when the
REPLICATION_STATUS is syncing.
• SP 02: Dynamic tiering shows progress in the REPLICATION_STATUS_DETAILS column when the
REPLICATION_STATUS is initializing or syncing. Dynamic tiering changed to support REPLAY_BACKLOG_SIZE.
The replay alerts are correctly generated because computation relies on the REPLAY_BACKLOG_SIZE statistic.
• SP 03: Same as above.
Monitoring statistics:
Column name Datatype Unit Description Dynamic
HOST VARCHAR(64) Host name Yes
PORT INTEGER Internal port Yes
VOLUME_ID INTEGER Persistence Volume ID Yes
SITE_ID INTEGER Generated ID of the primary site Yes
SITE_NAME NVARCHAR(64) Logical name for the primary site provided by the Yes
site administrator
SECONDARY_HOST VARCHAR(64) Secondary host name Yes
SECONDARY_PORT INTEGER Secondary internal port Yes
SECONDARY_SITE_ID INTEGER Generated ID of the secondary site Yes
SECONDARY_SITE_NAME NVARCHAR(64) Logical name for the secondary site provided by Yes
the site administrator
SECONDARY_ACTIVE_STATUS VARCHAR(16) Status of the secondary node (see Yes
ACTIVE_STATUS in M_SERVICES)
SECONDARY_CONNECT_TIME TIMESTAMP Timestamp the secondary connected to the Yes
primary. If there are reconnects from the
secondary side, this field contains the last
connect time.
SECONDARY_RECONNECT_COUNT INTEGER Number of reconnects from secondary side for Yes
this service
SECONDARY_FAILOVER_COUNT INTEGER Number of failovers for this service on Yes
secondary side.
SECONDARY_FULLY_RECOVERABLE VARCHAR(5) TRUE: No full data backup is needed after Yes
takeover on secondary. Backups created
on the primary + local log segments allow full
database recovery.
FALSE: Log segments needed for a full
database recovery are missing. After takeover,
a full data backup must be executed before full
recovery up to the most recent point in time can
be executed.
REPLICATION_MODE VARCHAR(16) Configured replication mode: Yes
SYNC: Synchronous replication with
acknowledgement when buffer has been written
to disk. Supported in limited configurations.
SYNCMEM: Not supported. Synchronous
replication with acknowledge when buffer
arrived in memory
ASYNC: Asynchronous replication. Supported
in limited configurations.
UNKNOWN: is set, if replication mode cannot
be determined (for example. if there are
communication errors when getting status
information from a service)
REPLICATION_STATUS VARCHAR(16) Current status of replication: Yes
UNKNOWN: secondary did not connect to
primary since last restart of the primary
INITIALIZING: initial data transfer occurs,
in this state, the secondary is not usable at all
SYNCING: secondary is syncing again (e.g.
after a temporary connection loss or restart of
the secondary)
ACTIVE: Initialization or sync with primary
is complete and secondary is continuously
replicating, if crash occurs, no data loss will
occur in SYNC mode
ERROR: error occurred on the connection
(details can be found in
REPLICATION_STATUS_DETAILS)
REPLICATION_STATUS_DETAILS VARCHAR(2048) Additional information for Yes
REPLICATION_STATUS, e.g. error text, if
status is ERROR
FULL_SYNC VARCHAR(16) Indicates if the service is currently operating in No
full sync. Note: Dynamic tiering does not
support full sync. The command hdbnsutil -
sr_fullsync returns an error if run on a
landscape with dynamic tiering. When full sync
is enabled in a running system
(global.ini/[system_replication]/enable_full_sync
), full sync might not be active immediately. This
prevents the system from blocking transactions
immediately when setting the parameter to true.
Instead, the process requires two steps. the
administrator must first enable full sync. Next,
the secondary is connected and becomes
ACTIVE, which activates full sync internally.
DISABLED: full sync is not configured at
all (global.ini/[system_replication]/enable_full_
sync = false)
ENABLED: full sync is configured, but it is
not yet active, so transactions do not block in
this state. To become active, the secondary
must connect and REPLICATION_STATUS
must be ACTIVE.
ACTIVE: full sync mode is configured and
active. If a connection of a connected
secondary is getting closed, transactions on the
primary side will block in this state.
If full sync is enabled when an active secondary
is currently connected, the FULL_SYNC will be
immediately set to ACTIVE.
LAST_LOG_POSITION BIGINT Last known log position on primary Yes
LAST_LOG_POSITION_TIME TIMESTAMP Timestamp of last known log position Yes
LAST_SAVEPOINT_VERSION INTEGER Last savepoint version on primary No
LAST_SAVEPOINT_LOG_POSITION BIGINT Log position of current savepoint No
LAST_SAVEPOINT_START_TIME TIMESTAMP Timestamp of current savepoint No
SHIPPED_LOG_POSITION BIGINT Last log position shipped to secondary Yes
SHIPPED_LOG_POSITION_TIME TIMESTAMP Timestamp of last log position being shipped to Yes
secondary
SHIPPED_LOG_BUFFERS_COUNT BIGINT Number of log buffers shipped to secondary Yes
SHIPPED_LOG_BUFFERS_SIZE BIGINT Bytes Size of all log buffers shipped to secondary Yes
SHIPPED_LOG_BUFFERS_DURATION BIGINT Micr Time spent shipping all the log buffers to the Yes
osec secondary.
onds
SHIPPED_SAVEPOINT_VERSION INTEGER Last savepoint version shipped to secondary No
SHIPPED_SAVEPOINT_LOG_POSITIO BIGINT Log position of last shipped savepoint No
N
SHIPPED_SAVEPOINT_START_TIME TIMESTAMP Timestamp of last shipped savepoint No
SHIPPED_FULL_REPLICA_COUNT BIGINT Number of full data replicas shipped to Yes
secondary
SHIPPED_FULL_REPLICA_SIZE BIGINT Bytes Total size of all full replicas shipped to Yes
secondary
SHIPPED_FULL_REPLICA_DURATION BIGINT Micr Duration for shipping all full data replica Yes
osec
onds
SHIPPED_LAST_FULL_REPLICA_SIZE BIGINT Bytes Size of last full data replica shipped to Yes
secondary
SHIPPED_LAST_FULL_REPLICA_STAR BIGINT Start time of last full data replica Yes
T_TIME
SHIPPED_LAST_FULL_REPLICA_END_ BIGINT End time of last full data replica Yes
TIME
SHIPPED_DELTA_REPLICA_COUNT BIGINT Number of delta data replicas shipped to No
secondary
SHIPPED_DELTA_REPLICA_SIZE BIGINT Bytes Total size of all delta data replicas shipped to No
secondary
SHIPPED_DELTA_REPLICA_DURATIO BIGINT Micr Duration for shipping of all delta data replicas No
N osec
onds
SHIPPED_LAST_DELTA_REPLICA_SIZ BIGINT Bytes No
E
SHIPPED_LAST_DELTA_REPLICA_STA BIGINT Start time of last data delta replica No
RT_TIME
SHIPPED_LAST_DELTA_REPLICA_EN BIGINT End time of last data delta replica No
D_TIME
ASYNC_BUFFER_FULL_COUNT BIGINT Number of times that the asynchronous Yes
replication buffer filled since the last service
restart (0 for replication modes sync/syncmem).
REPLAY_BACKLOG_SIZE BIGINT Bytes Current replication backlog in bytes, that Yes,
means, size of all log buffers, that have been starting
created on primary site, but not yet been sent to SP 02
the secondary site. Even in replication modes
sync/syncmem this column can have a value
different from 0. Here it represents the size
log buffers, that are in the local send queue
(max number of those buffers is the
number configured log buffers on primary site).
REPLAY_BACKLOG_TIME BIGINT Micr Current replication backlog in microseconds, in No
osec other words, the time difference between time
onds of the last sent log buffer and the current log
buffer. Even in replication modes
sync/syncmem, this column can have a value
different from 0, because log buffers are in still
in the send queue (max number of those
buffers is the number configured log buffers on
the primary site).
MAX_REPLAY_BACKLOG_SIZE Bytes Maximum replication backlog in bytes (max No
value of BACKLOG_SIZE since
system startup).
MAX_REPLAY_BACKLOG_TIME Micr Maximum replication backlog in microseconds No
osec (max value of BACKLOG_TIME since
onds system startup).
Secure Internal Communication
System PKI is the recommended SSL mode for system recovery communication. Dynamic tiering only supports System
PKI SSL for system replication. To set up System PKI for internal system replication for dynamic tiering, follow the steps
in "Secure Internal Communication Between Sites in System Replication Scenarios" in the SAP HANA Security Guide. Use
the default SSL crypto provider (commoncrypto) when dynamic tiering is configured.
Note: Dynamic tiering systems do not support legacy configuration for internal system replication communication. If
you use legacy configuration, the internal system communication between dynamic tiering servers on the primary and
secondary sites will not use TLS/SSL.
Point-in-time Recovery Post Takeover

These steps apply to SP 01 and 02 only.
Backups in a Shared Location

If the data backups and log backups are written to a shared location, mount this location on both primary and
secondary sites. After a takeover by the secondary, the primary site can still be write log backups until log backups
are stopped or disabled. Thus, log backups taken by the secondary site may be overwritten by log backups taken by
the primary, and, consequently, database recovery may not be possible.
For this reason, on dynamic tiering systems as on all SAP HANA systems, after takeover by the secondary site,
disable log backups on the primary site. See "Points to Note: System Replication" in the SAP HANA Administration
Guide.
Backups Not in a Shared Location

For point-in-time recovery after takeover, the log directory path should point to the secondary system's log backup
directory. If the .eslog files are missing in the secondary system's log backup directory, selectively copy the missing
.eslog files from the primary system's log directory. The names of .eslog files can match on the primary and
secondary systems after takeover, in case the primary system continues to run post-takeover. If a .eslog file is
already in the secondary system's log backup directory, do not copy the same file from the primary system. Never
overwrite the secondary system's .eslog files.
If backup archives exist at a shared location, and .eslog files needed for recovery are distributed across multiple
directories, then the secondary system's log directory takes precedence when you list the log directories in the
recovery command.
In summary, before point-in-time recovery on the secondary server after takeover:
1. Copy all the files from the log backup directory on the primary system to the log backup
directory on the secondary system except for the common *.eslog file present in the log
backup directory on the primary system and in the log backup directory on the secondary
system.
2. Copy all the files in the data backup directory on the primary system to the data backup
directory on the secondary system.
Example:
Here’s an example of how to prepare for point-in-time recovery when the backups are not in a shared location. In this
example, the primary system is Site1 and the secondary system is Site2.
1 Take a full backup called F1 on Site1.
On Site1, contents of data backup dir /usr/sap/M34/HDB34/backup/data
-rw-r----- 1 m34adm system1 159744 Oct 14 05:28 COMPLETE_DATA_BACKUP_databackup_0_1

-rw-r--r-- 1 m34adm system1 49657668 Oct 14 05:28
COMPLETE_DATA_BACKUP_databackup_1024_1_1476422901360.1
2 Take a log backup on Site1.
On Site1, contents of log backup dir /usr/sap/M34/HDB34/backup/log
-rw-r----- 1 m34adm system1 12288 Oct 14 05:28 log_backup_0_0_0_0.1476422917294
-rw-r--r-- 1 m34adm system1 15597568 Oct 14 05:44 log_backup_1024_000_000001_0001.eslog
-rw-r--r-- 1 m34adm system1 2524396 Oct 14 05:44 log_backup_1024_20161014_054421285.1
3 On Site1, execute:
hdbnsutil -sr_enable --name=primary
4 On Site2, perform initialization.
5 On Site1, take a log backup, creating the following new files in the log backup directory:
6 On Site1, run DML commands and take a log backup. These new files are created in the log directory:
7 On Site2, perform takeover.
8 On Site2 perform DMLs and take log backup. These new files are created in the log directory:
9 On Site2, perform DMLs and take log backup. These new files are created in the log backup directory:
10 The final contents of the log backup directory on Site1 are:
11 The final contents of the log backup directory on Site2 are:
12 Before you perform PITR on Site2:
a) On Site1, in the log backup directory, run: ls -lrt | grep *.eslog
b) On Site2, in the log backup directory, run: ls -lrt | grep *.eslog
c) Excluding the common *.eslog file found on both Site1 and Site2, copy all the remaining files from log backup
directory on Site1 to the log backup directory on Site2.
This step copies these files to Site2:
d) Copy all the files from the data backup directory on Site1 to the data backup directory on Site2.
This step copies these files to Site2:

-rw-r--r-- 1 m34adm system1 49657668 Oct 14 05:28
COMPLETE_DATA_BACKUP_databackup_1024_1_1476422901360.1
Perform point-in-time recovery on Site2.
Version Information
Date Revision Summary
30 April 2018 2.1 Corrected “Cluster Manager Hooks” section and
renamed it to “Cluster Manager”.
06 April 2018 2.0 Some restrictions removed for 2.0 SP 03. Added
takeover_esserver_without_log_backup setting for
global.ini.
26 July 2017 1.0 Supersedes SAP note 2356851.
Coding Samples
Any software coding and/or code lines / strings (“Code”) included in this documentation are only examples and are not
intended to be used in a productive system environment. The Code is only intended to better explain and visualize the
syntax and phrasing rules of certain coding. SAP does not warrant the correctness and completeness of the Code given
herein, and SAP shall not be liable for errors or damages caused by the usage of the Code, unless damages were caused
by SAP intentionally or by SAP’s gross negligence.
Copyright
©2017 SAP SE or an SAP affiliate company. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express
permission of SAP SE or an SAP affiliate company. The information contained herein may be changed without prior
notice.
Some software products marketed by SAP SE and its distributors contain proprietary software components of other
software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without
representation or warranty of any kind, and SAP or its affiliated companies shall not be liable for errors or omissions
with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that
are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein shall
be construed as constituting an additional warranty.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered
trademarks of SAP SE (or an SAP affiliate company) in Germany and other companies. All other product and service
names mentioned are the trademarks of their respective companies.
Please see http://www.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and

notices.

DTR Support

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

DTR Support

Transféré par

Droits d'auteur :

Formats disponibles

SAP HANA System Replication (HSR) on Systems

with Dynamic Tiering

System Replication Operation mode

System Replication Options

System Replication Alerts

Communication Network Separation

Security: System Replication Communication (Legacy Configuration of Secure Internal

Security: System Replication Communication (Authentication)

System Replication Alerts

Initialize or Reregister Secondary System with Pre-existing and Compatible Persistence

Multitarget System Replication

Console Tool (hdbcons)

Initialization of Secondary Site

Tenant Copy and Move

datashipping_parallel_ch Primary See "System Replication Configuration Parameters" Not supported.

operation_mode Secondary See "System Replication Configuration Parameters" SP 01: hdbnsutil –

reconnect_time_interval Secondary See "System Replication Configuration Parameters" SP 00 – SP 03: Same as

Point-in-time Recovery Post Takeover

Backups in a Shared Location

Backups Not in a Shared Location

-rw-r----- 1 m34adm system1 159744 Oct 14 05:28 COMPLETE_DATA_BACKUP_databackup_0_1

This step copies these files to Site2:

Please see http://www.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and

Vous aimerez peut-être aussi