SAN Client Deployment - Best

Practices and Performance Metrics

Issue Date: 10th March 2010
Version number: 1.5
Revision Date: 07 July 2015
Applies to: NetBackup 7.0, 7.1, 7.5, 7.6, 7.6.1, and 7.7
Note: This is a living document and will be subject to periodic updates. Please
check the date and version number to ensure you are referencing the latest
version.

This document is provided for informational purposes only and is intended for distribution only by Symantec employees to selected
partners and customers. All warranties relating to the information in this document, either express or implied, are disclaimed to the
maximum extent allowed by law. The information in this document is subject to change without notice. Copyright 2010 Symantec
Corporation. All rights reserved. Symantec, the Symantec logo and NetBackup are trademarks or registered trademarks of Symantec
Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners.

Contents
1.0
1.1
1.2
1.3
1.4
1.5
1.6
2.0

INTRODUCTION .................................................................................................................. 1
ADDITIONAL READING .......................................................................................................... 1
PERFORMANCE PARAMETERS FOR CONFIGURING THE SAN CLIENT ....................................... 2
SAN INTERCONNECT ........................................................................................................... 2
SAN CLIENT SCALABILITY AND ZONING BEST PRACTICES....................................................... 2
MEDIA SERVER PROFILE ...................................................................................................... 3
CLIENT PROFILE .................................................................................................................. 4
BEST PRACTICES .............................................................................................................. 6

2.1
SEGREGATED FT MEDIA SERVERS ....................................................................................... 6
2.2
COMBINED FT/LAN MEDIA SERVERS ................................................................................... 7
2.3
MULTI-STREAMING BACKUP ................................................................................................. 8
2.3.1
Stream distribution ..................................................................................................... 8
2.3.2
Connection restrictions on the FT Media Server ....................................................... 8
2.3.3
Increasing the maximum target ports available to a SAN Client ............................... 9
2.3.4
Increasing the number of SAN clients per target port................................................ 9

TECH54778 SAN Client Best Practices and Performance Metrics

Page i

10th March 2010

1.0 Introduction
In todays 24/7 business environment, corporations are looking for ways to protect their data
without disrupting business operations. This eliminates the options of using the LAN during of
peak hours to backup and protect the data.
The NetBackup SAN Client feature takes advantage of Fiber Channel Storage Area Network
(SAN) connections to perform a fast backup of a client machine directly to a Media Server,
moving data transfers off the LAN and away from conflicting traffic. The SAN Client provides a
high performance data transfer for the client and offloads the LAN with the backup traffic.
The primary purpose of the SAN Client is to back up large applications and file systems faster
and more efficiently than is possible using LAN based backup or locally attached backup devices
(a SAN Media Server). However the SAN Client architecture does not allow the same degrees of
parallel backup operation as the alternative approaches and is therefore less suited to handling
large numbers of small backup jobs. As a general rule SAN Client should be deployed selectively
on servers where the data volumes are such that LAN backup within a reasonable time period is
impractical it should not be deployed with a very large number of clients with a wide range
of backup sizes and a small number of FT Media Servers target ports (grouped together) as this
may lead to excessive resource contention on the FT Media Servers when there is a wide range
of job sizes.
This document describes the NetBackup SAN Client feature and provides some information on
performance considerations and best practices when deploy the SAN Client. The document is
intended to provide backup administrators with the necessary information to help our customers
take full advantage of this advanced feature and maximize their ROI.

1.1 Additional Reading

The following documents provide more information on the subjects covered in this paper:

NetBackup SAN Client and Fibre Transport Guide:

http://www.symantec.com/docs/DOC5332

NetBackup Device Configuration Guide (OS specific configuration steps for SAN Clients):
http://www.symantec.com/docs/DOC5332

SAN Client and Fibre Transport troubleshooting tech note:

http://www.symantec.com/docs/TECH51454

TECH54778 SAN Client Best Practices and Performance Metrics

Page 1

10th March 2010

1.2 Performance Parameters for configuring the SAN Client

To implement the SAN Client, the user needs to deploy a Fiber Channel based SAN between the
client and the Media Server. The Server and Clients can each have multiple SAN ports zoned
together.
A Media Server that can receive data from a SAN Client is referred to as a Fiber Transport or FT
Media Server. The HBAs on the Media Server that receive the data are referred to as Fiber
Transport Targets and must be of a supported type (the HBAs used on the SAN Client itself can
be of any type).
If configured correctly the feature allows a client transfer rate exceeding two Terabytes of data in
an hour. Obtaining this type of performance requires using the right hardware not only for the
connectivity between the client and the Media Server but also for the front end client data and the
back end storage disks. The whole system must be configured and tuned to achieve the desired
performance requirement.
The rest of this document lists the profile of the appropriate hardware and how to configure the
system to deliver the optimum performance.

Figure 1 - SAN Client Components

This section provides some information about the type of hardware and the main parts that affect
the performance of the system.

1.3 SAN Interconnect

When configuring the SAN Client, the first step is to deploy a Fiber Channel SAN between the
client and the Media Server.
The architecture of the SAN should be such that the throughput between SAN Clients and Fiber
Transport Media Servers is maximized and the latency is minimized. For example, one important
requirement is a SAN interconnect design that has no points of congestion. One way of
achieving no congestion is to avoid using Inter Switch Links (ISL) and trunking. It is therefore
recommended the connection between SAN Clients and the Fiber Transport Servers be on the
same switch and switch blade wherever possible.
If you have to use ISLs, we recommend that you under subscribe them. At any time, the trunking
between switches should be able to handle the throughput required for your environment's
Quality of Service requirements.

1.4 San Client Scalability and Zoning Best Practices

The larger amount of client initiators with overlapping zones the more likely there will be a
widespread reconfiguration notice to all initiators. This can result in all initiators trying to scan all
visible Fibre Transport ports at the same time. Because of this potential, there is a technical
limitation to how many SAN client initiator ports should be used per Fibre Transport Media Server
target port.

TECH54778 SAN Client Best Practices and Performance Metrics

Page 2

10th March 2010

Symantec recommends a maximum of 30 SAN Client initiator ports zoned with a Fibre Transport
target port because of the effective queue depth of about 60 for a QLogic hardware/firmware
target port. Because the NetBackup Fibre Transport media server will simulate two "pseudo tape
devices" on each Fibre Transport target port, the number of commands arriving will be two times
the number of initiators. Keeping the number of SAN Clients per port at 30 or below prevents
each target port from having to queue more than 60 low-level fibre channel commands at any
point in time.
Limiting the number of initiator ports for each target port to 30 will prevent the queue from being
overflowed. With more than 30 clients per port there is risk that the Fibre Transport Media Server
will not respond to a SCSI inquiry. This could have undesirable results on the client operating
system during boot or device discovery.
While all inclusive zoning is supported for SAN Client configurations, it is not recommended. An
all inclusive zone runs a risk of a single miss-configured client or server affecting all the other
SAN Clients in the zone.

1.5 Media Server Profile

The FT Media Server is only supported on a limited number of platforms and selected Qlogic2xxx
cards and their OEM variants as Fiber Transport Targets on the Media Server. For SAN Clients
and for the backup storage devices on the FT Media Server we support QLogic, Emulex, and
most OEM Fiber Channel cards. The range of platforms and HBAs supported is constantly being
expanded. Refer to the Hardware Compatibility List (HCL) for more information on the
supported types of hardware:
http://www.netbackup.com/compatibility
Below is a list of some important parameters for tuning the system for high performance.
1) Use PCI-express 4/8 lane or PCI-x slots with a speed of 133MHz or higher. Avoid
oversubscribing your PCI-x and PCI-e buses. If possible use PCI-e slots to connect
the backend storage devices. If PCI-e slots are not available make certain that your
Media Server hardware has different dedicated PCI-x buses for the backend storage
devices and the Fiber Transport targets.
2) Do not use PCI cards on the same bus as the required QLogic FC HBAs. A slower
PCI card reduces the speed of the controlling bus and therefore all other cards in that
bus. Consequently, data transfer rates are reduced and performance is degraded.
3) Make sure the total aggregated IO throughput is not more than the total memory
bandwidth.
4) In general, we recommend that you dont use more than 4 HBA Fiber Transport ports
per Media Server, as 4 Fiber Transport ports provide a maximum actual transfer rate
of 600MB/second. However dont use more than 8 ports in any circumstance. Since
the backend storage must match or exceed the Fiber Transport Data Rate, the
bandwidth requirement on a system with 8 Fiber Transport ports would be 2.4
GB/second which is near the maximum I/O load on most systems. Up to 8 Fiber
Transport ports are possible; however, there are diminishing returns as the available
bandwidth is spread over the ports.
5) The QLA-2344 four-port FC adapter's usable aggregate performance is not
significantly greater than a two-port QLA-2342 when the QLA-2344 four-port FC
adapter is used in the same PCI-x slot for SAN Client target mode. The advantage
that a QLA-2344 HBA offers is the ability to spread its aggregate performance over
four ports instead of two.
The QLA-2344 HBA performs similarly to two QLA-2342 HBAs but uses one less PCI
slot if the following is true:

You use a direct-connection (rather than FC switches or bridges) between SAN

clients and a Fibre Transport (FT) media server.

TECH54778 SAN Client Best Practices and Performance Metrics

Page 3

10th March 2010

Only two ports are fully loaded with Fibre Transport traffic at the same time.

6) Use a system test tool (like dd) to make sure the backend disk can write as fast as
the expected FT transfer speed. For example, if you expect to transfer an aggregate
of 600 MB/s over 8 streams, verify that your disk can sustain at least 600 MB/s rate
using 8 streams with 256k blocks.
7) Finally, a multi-processor system is required. While the actual CPU utilization for
Fiber Transport streams is small, keeping the interrupt latency low is required to
maintain high transfer rates. This requires a minimum of two CPU. An x86_64
based system that has a constant 600 MB/s of Fiber Transport data will utilize
about 50% of a dual core 3 GHz processor. A very general rule of thumb would be
that there should be at least 5 MHz of CPU speed for every 1 MB/s of data transfer.
8)

Expect the performance of target mode 4 and 8 Gbit/s FC to be similar because the
latency of processing each transfer command at both the client and media server end
is a significant portion of the transport time. 8 Gbit/s FC target ports can only provide
a higher per port aggregate transfer rate when more client ports are concurrently in
use and you should not count on this occurring in practice.

1.6 Client profile

On the client side three parameters are to be considered:
1) Operating System The SAN Client is supported on the following operating systems:
Solaris, Windows, AIX, HP-UX and Linux. Please refer to the release notes and
HCLs for details of the operating system versions supported. Note that the SAN
Client is generally supported on the same operating system versions as a Media
Server, not a LAN Client.
2) CPU - As with the Media Servers, the rule we use is 1MB/s for every 5 MHz of CPU
cycle. This means that the CPU use for your application server may be reduced by
up to 5GHz during a particularly heavy I/O load.
3) HBA - If the data is stored on a SAN attached disk, do not use the same HBA for the
disk and backup SANs. This allows you to better tune your environment including
ISL links and HBA parameters. While the same HBA can be successfully used for
both SANs, this practice is discouraged from a performance standpoint. Sharing an
initiator introduces a potential performance bottleneck issue.
Additionally, it
complicates any SAN instability issues. SAN reorganizations and fault recoveries
tend to spread through all zones (including overlapping zones) that contain an end
point affected by the failure or reorganization.
4) Disk Speed - The speed at which the client can read the data is a fundamental factor.
Again you can use dd to test the disk performance and tune it. Keep in mind that the
Fiber Transport uses 256k block sizes, so use this factor when profiling the
performance of your disk arrays.
5) Data Streams - The minimum optimal number of data streams between a SAN Client
and a Fiber Transport Media server is two streams per available Fiber Transport
target port. For example, a SAN Client backup to a Media Server with 4 available
ports should utilize 8 data streams, writing 1 stream to each LUN available between
the Media Server and the SAN Client. Writing more data streams will create
contention for the LUNs when the data sources are capable of supplying over half of
the maximum data rate for the port, and using less streams of data will mean that the
ports on the Media Server are not fully utilized.

6) The SAN Client is not fully qualified to support IPv6.

In common with all types of backup operation care should be taken when planning backup
policies and schedules to avoid situations in which large numbers of small backup jobs
monopolize too many resources on all the FT Media Servers and delay the start of large backup

TECH54778 SAN Client Best Practices and Performance Metrics

Page 4

10th March 2010

jobs such that one or more of them fail to start early enough to allow them to complete within the
backup window.

TECH54778 SAN Client Best Practices and Performance Metrics

Page 5

10th March 2010

2.0 Best practices

One question that usually comes to mind when designing and implementing SAN Client in a
backup environment is should I mix regular clients with SAN Clients on the same Media Servers
or should I segregate them. The answer to this question depends on your environment and
performance requirements.

2.1 Segregated FT Media Servers

In this scenario the FT Media Servers are dedicated for SAN Client backup only. This case is
recommended when you have different tiers of backend storage. The high end, high
performance disk storages are connected to the FT Media Servers. This provides enough traffic
to optimize the usage of the backend storage.
Another benefit of this architecture is that its easy to distribute the clients to the different Media
Servers. As they all have the same speed, an equal distribution of the clients to the Media
Servers is usually enough.
In a dedicated Fiber Transport backup pool, you must monitor the environment so that you
always know that the load on the Media Servers is at an acceptable level. If the load is such that
the Media Servers can't handle all the backup requests from SAN Clients in an acceptable
amount of time, the pool will have to be grown, and likewise you may consider shrinking the pool
of dedicated Media Servers if you find that the servers sit idle or underutilize the available Disk
I/O.
The advantage of a segregated FT Media Server pool is that you are better able to guarantee the
class of service necessary is available to SAN Clients since you don't have to allow for LAN
based transfers. Disadvantages are that more Media Servers are required overall and/or the pool
of Media Servers available to the SAN Clients is smaller than it would be in a mixed environment.
This means that the failure of a single Media Server will have a larger impact on the overall SAN
Client backup window than it would in a shared environment.

Figure 2 - Segregated FT Servers

TECH54778 SAN Client Best Practices and Performance Metrics

Page 6

10th March 2010

2.2 Combined FT/LAN Media Servers

In this scenario the same FT Media Servers are used to backup both SAN and LAN based
clients. Using Media Servers in this way addresses the principle disadvantage of the segregated
model by making more Media Servers available for both FT and LAN backup, thus reducing the
number of Media Servers required and/or increasing the number available for SAN and LAN
backups. This is particularly useful in environments where some client platforms are not
supported with the SAN Media Server.
However it should be noted that LAN jobs utilize more CPU per MB of data transferred and the
LAN protocol interrupt load can adversely affect interrupt latency and reduce FT data rates.
Additional CPU's may or may not help depending on the affinity of the interrupt processing for
one CPU on some operating systems.

Figure 3 - Combined FT/LAN Media Servers

A major advantage to this model is that it utilizes a larger group of servers with lower performance
requirements for a cost savings on a per-server basis. Since the pool of FT servers is more
distributed, it is expected that each server will have a smaller data transfer load while still
maintaining the same overall throughput performance due to the cumulative effect of having more
active Media Servers.
The downside to this model is that it makes it harder to guarantee a higher Quality of Service to
individual large application servers. An individual application server may support 600 MB/s of
backup I/O, but if the selected Media Server is already writing many streams of LAN based
backups from multiple clients there will be a performance impact to both the SAN Clients and
LAN Clients.

TECH54778 SAN Client Best Practices and Performance Metrics

Page 7

10th March 2010

2.3 Multi-Streaming backup

The typical maximum practical transfer rate for a single 2 gigabit Fiber Channel port is around
160 MB/sec and this is the maximum transfer rate you can expect to see between the SAN Client
and FT Media Server over a single port.
The read speed of a single backup stream from the disk system on the client may be significantly
less than this maximum figure, resulting in apparently poor performance. However it is possible
to run multiple backup streams over the same port between the client and server and thus better
backup performance may be achieved by using multi-streamed backups.
The performance of individual backup streams will vary depending on the read speed of the
source data. This can be established (on UNIX and Linux clients) by running a performance test
using a dd command of the form:
time dd if=<path> of=/dev/zero bs=256k count=8192
Specifying the path to be backed up should be specified in place of <path>. This will return a
value for the elapsed backup time which, in turn, will enable you to calculate the read speed of
that stream. Once you have a feel for the read speed of each stream you should be able to
configure a policy with sufficient parallel streams to achieve the maximum throughput on the Fiber
Channel port.
Note that in some cases multiple streams may contend with each other for disk controller
resources resulting in further performance degradation when streams are run in parallel and
further tuning may be required.
If transfer rates greater than 160 MB/s are required it is possible to use multiple HBA Ports for the
client and the server in conjunction with multiple streams. The ratio of client ports to server ports
depends on the speed of the HBA cards in the client. For example, if you have to transfer 300
Megabytes per second from a SAN Client to an FT Media Server, you will need to have 2 x two
gigabit client ports and the Media Server will require 2 x two gigabit target ports.

2.3.1 Stream distribution

When multiple streams are used from a client with more than 1 HBA port, the jobs are distributed
in a round robin fashion with one stream being assigned to each port in turn. E.g. if 4 jobs are run
using 2 ports then job1 will use port 1, job 2 will use port 2, job 3 will share port 1 with job 1 and
job 4 will share port 2 with job 2. This means that as job streams complete and new job streams
do not take their place the ports will be under-utilized in the same way that tape drives using
multiplexing may be under utilized as the number of concurrent jobs decreases.

2.3.2 Connection restrictions on the FT Media Server

On important limiting factor to bear in mind when running multiple streams to multiple ports on a
the same FT Media Server is that the maximum number of connections allowed to a single FT
Media Server is 16. This means that the total number of concurrent backup streams across all
the available ports on the FT Media Server cannot exceed 16. This figure is increased to 32
connections in NetBackup 7.0, 7.1, 7.5, 7.6, and 7.6.1. Beginning with the 7.7 release, the
following are the maximum concurrent values:
On a Linux FT media server host:

40
Symantec recommends that you use 32 or fewer
connections concurrently on Linux.
On Linux hosts, you can increase that maximum
by setting a NetBackup touch file,
NUMBER_DATA_BUFFERS_FT.

For NetBackup Appliance model 5230 and later:

40

For NetBackup Appliance model 5330 and later:

40

TECH54778 SAN Client Best Practices and Performance Metrics

Page 8

10th March 2010

On a Solaris FT media server host:

64

It is also important to remember that each port used by a specific client impacts the performance
of other SAN Clients utilizing the same Media Server. If two SAN Clients utilize the same Media
Server with only four target ports, and each client claims all four of those ports, the clients will
contend for those ports. The effect of this contention is that the client with a lower latency will
'win' and transfer data faster than the higher latency client, sometimes as much as twice as fast.
If each client had only claimed two ports, they would have been able to share the available ports
and not run into contention issues.
The key to tuning is therefore to limit the number of ports used by a SAN Client while maximizing
the number of streams on each port but ensuring that the maximum number of connections
across all the available ports does not exceed the maximum.

2.3.3 Increasing the maximum target ports available to a SAN Client

By default, SAN Clients supports a maximum of two Fiber Transport ports at any given time; this
allows FT Media Servers to balance an I/O load fairly among multiple SAN Clients.
If a particular client requires a higher Quality of Services than is provided by two Fiber Channel
ports then it is possible to increase the number of ports per server the client is allowed to use by
executing the following command.
nbftconfig -changeclient -np 4 -C <clientName>
The above command changes the number of ports for the specific client <clientName>. To
change it for all clients the setconfig option is used:
nbftconfig setconfig np 4
Note that setting np 4 will have no effect on a client that only has a 2 port HBA.

2.3.4 Increasing the number of SAN clients per target port

By default, one Fibre Transport port can only be used by up to two different SAN Clients at any
given time; this prevents oversubscribing a FT Media sever port to multiple clients.
In environments with a greater number of clients per Media Server target port, this value can be
increased. This may reduce the Quality of Service and should be done only if necessary. This
value is set on a Master server and applies to all the Fibre Transport Media Servers in that
domain. The value can be configured by executing the following command
nbftconfig setconfig ncp 4
The recommended value is less than or equal to 4.

TECH54778 SAN Client Best Practices and Performance Metrics

Page 9

10th March 2010