Business Continuity: Remote Replication

Module 4.4

2006 EMC Corporation. All rights reserved.

Remote Replication
After completing this module, you will be able to: Explain Remote Replication Concepts
Synchronous/Asynchronous Connectivity Options

Discuss Host and Array based Remote Replication Technologies

Functionality Differences Considerations Selecting the appropriate technology

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 2

Remote Replication Concepts

Replica is available at a remote facility
Could be a few miles away or half way around the world Backup and Vaulting are not considered remote replication

Synchronous Replication
Replica is identical to source at all times Zero RPO

Asynchronous Replication
Replica is behind the source by a finite margin Small RPO

Connectivity
Network infrastructure over which data is transported from source site to remote site

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 3

Synchronous Replication
A write has to be secured on the remote replica and the source before it is acknowledged to the host

Disk

Ensures that the source and remote replica have identical data at all times
Write ordering is maintained at all times
Replica receives writes in exactly the same order as the source
Server
Data Write

1 4 3

Data Acknowledgement

Synchronous replication provides the lowest RPO and RTO

Goal is zero RPO RTO is as small as the time it takes to start application on the remote site
Disk

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 4

Synchronous Replication
Response Time Extension
Application response time will be extended due to synchronous replication
Data must be transmitted to remote site before write can be acknowledged Time to transmit will depend on distance and bandwidth
Writes MB/s Max

Bandwidth
To minimize impact on response time, sufficient bandwidth must be provided for at all times

Average

Time

Rarely deployed beyond 200 km

2006 EMC Corporation. All rights reserved. Business Continuity Remote Replication - 5

Asynchronous Replication
Write is acknowledged to host as soon as it is received by the source

Data is buffered and sent to remote

Some vendors maintain write ordering Other vendors do not maintain write ordering, but ensure that the replica will always be a consistent re-startable image
Server
Data Write

Disk

1 2 4

Data Acknowledgement

Finite RPO
Replica will be behind the Source by a finite amount Typically configurable
2006 EMC Corporation. All rights reserved.

Disk

Business Continuity Remote Replication - 6

Asynchronous Replication

Response Time unaffected Bandwidth

Need sufficient bandwidth on average
Writes MB/s Max

Buffers
Need sufficient buffers

Can be deployed over long distances

Average

Time

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 7

Remote Replication Technologies

Host based
Logical Volume Manager (LVM)
Synchronous/Asynchronous

Log Shipping

Storage Array based

Synchronous Asynchronous Disk Buffered - Consistent PITs
Combination of Local and Remote Replication

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 8

LVM Based Remote Replication

Duplicate Volume Groups at local and remote sites All writes to the source Volume Group are replicated to the remote Volume Group by the LVM
Synchronous or Asynchronous
Log
Physical Volume 1 Physical Volume 2 Physical Volume 3 Physical Volume 1

Log
Physical Volume 2 Physical Volume 3

Volume Group
Local Site

Network

Volume Group
Remote Site

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 9

LVM Based Remote Replication

In the event of a network failure
Writes are queued in the log file When the issue is resolved the queued writes are sent over to the remote The maximum size of the log file determines the length of outage that can be withstood

In the event of a failure at the source site, production operations can be transferred to the remote site

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 10

LVM Based Remote Replication

Advantages
Different storage arrays and RAID protection can be used at the source and remote sites Standard IP network can be used for replication Response time issue can be eliminated with asynchronous mode, with extended RPO

Disadvantages
Extended network outages require large log files CPU overhead on host
For maintaining and shipping log files

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 11

Host Based Log Shipping

Logs

IP Network

Original

Offered by most DB Vendors

Advantages
Minimal CPU overhead Low bandwidth Standby Database consistent to last applied log
Stand By
2006 EMC Corporation. All rights reserved. Business Continuity Remote Replication - 12

Logs

Array Based Remote Replication

Replication performed by the Array Operating Environment
Host CPU resources can be devoted to production operations instead of replication operations Arrays communicate with each other via dedicated channels
ESCON, Fibre Channel or Gigabit Ethernet

Replicas are on different arrays

Primarily used for DR purposes Can also be used for other BC operations
Production Array
Remote Array

Network Production Server Source

Distance

Replica

DR Server

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 13

Array Based Synchronous Replication

Network links

Source

Target

Write is received by the source array from host/server Write is transmitted by source array to the remote array

Remote array sends acknowledgement to the source array

Source array signals write complete to host/server

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 14

Array Based Asynchronous Replication

Network links

Source

Target

Write is received by the source array from host/server

Source array signals write complete to host/server Write is transmitted by source array to the remote array Remote array sends acknowledgement to the source array

No impact on response time Extended distances between arrays Lower bandwidth as compared to Synchronous
2006 EMC Corporation. All rights reserved. Business Continuity Remote Replication - 15

Array Based Asynchronous Replication

Ensuring Consistency
Maintain write ordering
Some vendors attach a time stamp and sequence number with each of the writes, then ship the writes to the remote array and apply the writes to the remote devices in the exact order based on the time stamp and sequence numbers Remote array applies the writes in the exact order they were received, just like synchronous

Dependent write consistency

Some vendors buffer the writes in the cache of the source array for a period of time (between 5 and 30 seconds) At the end of this time the current buffer is closed in a consistent manner and the buffer is switched, new writes are received in the new buffer The closed buffer is then transmitted to the remote array Remote replica will contain a consistent, re-startable image on the application
Business Continuity Remote Replication - 16

2006 EMC Corporation. All rights reserved.

Array based Disk Buffered Consistent PITs

Local and Remote replication technologies can be combined to create consistent PIT copies of data on remote arrays RPO usually in the order of hours Lower Bandwidth requirements

Extended distance solution

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 17

Extended Distance Consistent PIT

SOURCE
REMOTE

Source

Network Links

Local Replica

Local Replica

Remote Replica

Create a Consistent PIT Local Replica on Source Array Create a Remote Replica of this Local Replica Optionally create another replica of the Remote replica on the remote array if needed Repeatas automation, link bandwidth, change rate permit

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 18

Synchronous + Extended Distance Consistent PIT

SOURCE Sync Source BUNKER REMOTE
Network Links

Remote Replica
Local Replica

Local Replica

Network Links

Remote Replica

Synchronous replication between the Source and Bunker Site Create consistent PIT Local Replica at bunker Create Remote Replica of bunker Local Replica Optionally create additional Local Replica at Target site from the Remote Replica if needed Repeatas automation, link bandwidth, change rate permit
2006 EMC Corporation. All rights reserved. Business Continuity Remote Replication - 19

Remote Replicas Tracking Changes

Remote replicas can be used for BC Operations
Typically remote replication operations will be suspended when the remote replicas are used for BC Operations

During BC Operations changes will/could happen to both the source and remote replicas
Most remote replication technologies have the ability to track changes made to the source and remote replicas to allow for incremental re-synchronization Resuming remote replication operations will require resynchronization between the source and replica

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 20

Primary Site Failure Operations at Remote Site

Remote replicas are typically not available for use while the replication session is in progress In the event of a primary site failure the replicas have to be made accessible for use Create a local replica of the remote devices at the remote site Start operations at the Remote site
No remote protection while primary site issues are resolved

After issue resolution at Primary Site

Stop activities at remote site Restore latest data from remote devices to source Resume operations at Primary (Source) Site
2006 EMC Corporation. All rights reserved. Business Continuity Remote Replication - 21

Array Based Which Technology?

Synchronous
Is a must if zero RPO is required Need sufficient bandwidth at all times Application response time elongation will prevent extended distance solutions (rarely above 125 miles)

Asynchronous
Extended distance solutions with minimal RPO (order of minutes) No Response time elongation Generally requires lower Bandwidth than synchronous Must design with adequate cache/buffer or sidefile/logfile capacity

Disk Buffered Consistent PITs

Extended distance solution with RPO in the order of hours Generally lower bandwidth than synchronous or asynchronous
2006 EMC Corporation. All rights reserved. Business Continuity Remote Replication - 22

Storage Array Based Remote Replication

Network Options
Most vendors support ESCON or Fibre Channel adapters for remote replication
Can connect to any optical or IP networks with appropriate protocol converters for extended distances
DWDM SONET

IP Networks

Some Vendors have native Gigabit Ethernet adapters which allows the array to be connected directly to IP Networks without the need for protocol converters

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 23

Dense Wavelength Division Multiplexing (DWDM)

DWDM is a technology that puts data from different sources together on an optical fiber with each signal carried on its own separate light wavelength (commonly referred to as a lambda or ). Up to 32 protected and 64 unprotected separate wavelengths of data can be multiplexed into a light stream transmitted on a single optical fiber.
Optical Channels

ESCON Fibre Channel Gigabit Ethernet Optical

Electrical

Optical Lambda

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 24

Synchronous Optical Network (SONET)

SONET is Time Division Multiplexing (TDM) technology where traffic from multiple subscribers is multiplexed together and sent out onto the SONET ring as an optical signal Synchronous Digital Hierarchy (SDH) similar to SONET but is the European standard SONET/SDH, offers the ability to service multiple locations, its reliability/availability, automatic protection switching, and restoration
STM-1

OC3

OC48 OC48

SONET

STM-16 STM-16

SDH
2006 EMC Corporation. All rights reserved. Business Continuity Remote Replication - 25

Rated Bandwidth
Link Escon Fibre Channel Bandwidth Mb/s 200 1024 or 2048

Gigabit Ethernet
T1 T3

1024
1.5 45

E1
E3

2
34

OC1
OC3/STM1 OC12/STM4

51.8
155.5 622.08

OC48/STM16
2006 EMC Corporation. All rights reserved.

2488.0
Business Continuity Remote Replication - 26

Module Summary
Key points covered in this module: Remote Replication Concepts
Synchronous/Asynchronous Connectivity Options

Host and Array based Remote Replication Technologies

Functionality Differences
Considerations Selecting the appropriate technology

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 27

 Check Your Knowledge

What is a Remote Replica? What are the possible uses of Remote Replicas?

What is the difference between Synchronous and Asynchronous Replication?

Discuss one host based remote replication technology? Discuss one array based remote replication technology? What are differences in the bandwidth requirements between the array remote replication technologies discussed in this module?

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 28

Apply Your Knowledge

Upon completion of this topic, you will be able to: Enumerate EMCs Remote Replication Solutions for the Symmetrix and CLARiiON arrays Describe EMCs SRDF/Synchronous Replication Solution Describe EMCs MirrorView/A Replication Solution

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 29

EMC Remote Replication Solutions

EMC Symmetrix Arrays
EMC SRDF/Synchronous EMC SRDF/Asynchronous EMC SRDF/Automated Replication

EMC CLARiiON Arrays

EMC MirrorView/Synchronous EMC MirrorView/Asynchronous

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 30

EMC SRDF/Synchronous - Introduction

Array based Synchronous Remote Replication technology for EMC Symmetrix Storage Arrays
Facility for maintaining real-time physically separate mirrors of selected volumes

SRDF/Synchronous uses special Symmetrix devices

Source arrays have SRDF R1 devices Target arrays have SRDF R2 devices Data written to R1 devices are replicated to R2 devices

SRDF uses dedicated channels to send data from source to target array
ESCON, Fibre Channel or Gigabit Ethernet are supported

SRDF is available in both Open Systems and Mainframe environments

2006 EMC Corporation. All rights reserved. Business Continuity Remote Replication - 31

SRDF Source and Target Volumes

SRDF R1 and R2 Volumes can have any local RAID Protection
E.g. Volumes could have RAID-1 or RAID-5 protection

SRDF R2 volumes are in a Read Only state when remote replication is in effect
Changes cannot be made to the R2 volumes

SRDF R2 volumes are accessed under certain circumstances

Failover Invoked when the primary volumes become unavailable Split Invoked when the R2 volumes need to be concurrently accessed for BC operations

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 32

SRDF/Synchronous
1. Write received by Symmetrix containing Source volume
Source Host

2. Source Symmetrix sends write data to Target 3. Target Symmetrix sends acknowledgement to Source 4. Write complete sent to host

Target Host

Channel Director (CD)

Channel Director (CD)

2
Remote Link Director (RLD) Remote Link Director (RLD) Remote Link Director (RLD)

Channel Director (CD)

Channel Director (CD)

Global Cache Director

Global Cache Director Remote Link Director (RLD)

Disk Director (DD)

Disk Director (DD)

Disk Director (DD)

Disk Director (DD)

Symmetrix Containing Source (R1) Volumes

Symmetrix Containing Target (R2) Volumes

Application does not receive I/O acknowledgement until data is received and acknowledged by remote Symmetrix Write completion time is extended - No impact on Reads Most often used in campus solutions
2006 EMC Corporation. All rights reserved. Business Continuity Remote Replication - 33

SRDF Operations - Failover

Purpose Make Target Volumes Read Write Source Volume status is changed to Read Only SRDF Link is suspended

Before
RW RO RO RW

Source Volume

Target Volume

After

Source Volume

Target Volume

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 34

SRDF Operations - Failback

Makes target volume Read Only, resumes link, synchronize R2 to R1, and write enables source volume

Before
RO RW

Source Volume

Target Volume

RW sync
Source Volume

RO

After

Target Volume

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 35

SRDF Operations - Split

Enables read and write operations on both source and target volumes Suspends replication

Before
RW RO

Source Volume

Target Volume

RW

RW

After

Source Volume

Target Volume

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 36

SRDF Operations Establish/Restore

Establish - Resume SRDF operation retaining data from source and overwriting any changed data on target Restore - SRDF operation retaining data on target and overwriting any changed data on source

RW

RO

RW

RO

Source Volume

Target Volume

Source Volume

Target Volume

Establish
2006 EMC Corporation. All rights reserved.

Restore
Business Continuity Remote Replication - 37

EMC CLARiiON MirrorView/A Overview

Optional storage system software for remote replication on EMC CLARiiON arrays
No host cycles used for data replication

Provides a remote image for disaster recovery

Remote image updated periodically - asynchronously Remote image cannot be accessed by hosts while replication is active Snapshot of mirrored data can be host-accessible at remote site

Mirror topology (connecting primary array to secondary arrays)

Direct connect and switched FC topology supported WAN connectivity supported using specialized hardware
2006 EMC Corporation. All rights reserved. Business Continuity Remote Replication - 38

MirrorView/A Terms
Primary storage system
Holds the local image for a given mirror

Secondary storage system

Holds the local image for a given mirror

Bidirectional mirroring
A storage system can hold local and remote images

Mirror Synchronization
Process that copies data from local image to remote image

MirrorView Fractured state

Condition when a Secondary storage system is unreachable by the Primary storage system
2006 EMC Corporation. All rights reserved. Business Continuity Remote Replication - 39

MirrorView/A Configuration
MirrorView/A Setup
MirrorView/A software must be loaded on both Primary and Secondary storage system Remote LUN must be exactly the same size as local LUN Secondary LUN does not need to be the same RAID type as Primary Reserved LUN Pool space must be configured

Management via Navisphere Manager and CLI

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 40

MirrorView/A Initial Synchronization

Primary Image Secondary Image

A B C D E F B E
Host Tracking DeltaMap Transfer DeltaMap

A B C D E F

0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 1 1 1 1 1
MAP

Snapshot

RLP

MAP

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 41

MirrorView/A Update
Primary Image Secondary Image

A E A B C D E F
Host Tracking Transfer DeltaMap Transfer Tracking DeltaMap

A B C D E F E B

0 1 0 0 0 0 0 1 1 0 0 0 1 0 0 0
MAP

Snapshot

RLP

MAP

B E

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 42

MirrorView/A Promotion (Update Failure)

Primary Image Secondary Primary Image

E A B C D E F
Host

A B C D E F B
Promote Secondary

Transfer DeltaMap
Tracking DeltaMap

0 0 0 0 1 0 1 0 0 0 1 1 0
MAP

Snapshot

RLP

MAP

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 43

Consistency Groups
Group of secondary images treated as a unit Local LUNs must all be on the same CLARiiON

Remote LUNs must all be on the same CLARiiON

Operations happen on all LUNs at the same time
Ensures a restartable image group

2006 EMC Corporation. All rights reserved.

Business Continuity Remote Replication - 44