Vous êtes sur la page 1sur 45

Symmetrix VMAX3 Internals: Essentials

Storage Provisioning For Open Systems

This module discusses how VMAX storage is presented to an open systems host using the
feature auto-provisioning groups. It starts by reviewing connectivity fundamentals and
prerequisites that must be configured on the VMAX.

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

While this module is about the tactical aspects of storage provisioning, it is important to
understand that provisioning is much more than device creation, zoning, mapping, and
masking. Storage provisioning must be a part of an overall planning, implementation, and
monitoring process rather than a standalone activity.
It is easy to think of storage capacity in terms of MB, GB, TB, but it is critical to think of
capacity in terms of IO/sec and throughput at expected response time. Too many systems
today have stranded capacity in terms of GB/TB where the system cannot meet the required
performance service level agreements before the full capacity of the array is utilized. One of
the goals of VMAX3 is to change the discussion to bring Service Level Objectives (SLO) to the
forefront of provisioning and to make planning, fulfillment, and monitoring a part of an
integrated process.

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Storage provisioning should include an understanding of the full IO stack as illustrated in the
diagram. Starting from the top, the host initiates a read or write operation. The IO operation
is passed by the application to the IO handlers in the operating system and the Logical
Volume Management layer. Path management software, such as PowerPath, is between the
Host Bus Adapters and the upper layers of the IO stack. Path management software is a part
of most open systems environments and performs two important functions: Load balancing
and path failover.
While it is possible to connect a HBA directly to a frontend director port, most environments
will be connected through a Storage Area Network, or SAN, that consists of one or more
interconnected switches. Originally SANs were based on Fibre Channel protocols; however,
today we are starting to see TCP/IP based SANs using the iSCSI and/or FCoE protocols. SANs
provide greater connectivity by allowing more hosts to share the same frontend port and
greater distance between host and storage. The management efficiencies of consolidated
storage are well proven.
In the Symmetrix, devices are presented through Fibre Channel, iSCSI or FCoE frontend
directors by assigning either a channel address or dynamic LUN address. In almost all cases,
multiple hosts will share the same ports; therefore, masking is configured to restrict which
host has access to which specific volumes.
Additionally, frontend directors were designed to support a large number of SCSI variants;
therefore, each port must be configured to support a subset of the SCSI, IP, and Fibre
Channel protocols required for specific hosts.
3

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

In SAN environments there are two types of devices: SCSI Initiators and SCSI Targets. Host
Bus Adapters (HBA) in a server are called initiators because they initiate IO operations.
Frontend ports on the storage array are SCSI targets, and are the target of IO operations.
In a Fibre Channel SAN environment, all devices are identified by a unique identifier called a
WWN. The screen capture here is from the Emulex HBAnyware utility and shows a host with
two HBAs. Each HBA is connected to a different frontend port. This connection is made by
physically connecting the HBA and frontend ports to the SAN switch(es) and configuring
logical zoning so they see each other. It is not unusual for a host to be configured with two,
four, or more HBAs with these zoned to multiple frontend directors on the Symmetrix.
Devices are mapped to frontend director ports and masked to specific initiators. On the
VMAX3, this is performed using the auto-provisioning group feature.
The Symmetrix has an active-active architecture where a device may be actively accessed on
more than one frontend port. EMC PowerPath, or other path management software,
coordinates access to the devices, providing availability through path failover and optimum
performance by spreading the workload across multiple HBAs and frontend ports.
Reference: Host Connectivity Guides on support.emc.com.
Note: HBAnyware is a host-based tool from Emulex.

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

There are three things that must be configured when connecting an open systems host to the
Symmetrix: HBA, SAN, and the Symmetrix. We will look briefly at the HBA and the SAN but
the focus of this discussion is the Symmetrix.
On the VMAX, storage provisioning is performed using the auto-provisioning group feature.
The approach is very similar for both VMAX and VMAX3; however, on VMAX3 device mapping
is implemented as part of the masking record for most environments.

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Before a host can see any devices on the Symmetrix, the HBA and frontend port must
initialize themselves and perform fabric and port logins. For this to happen there must be
connectivity from the Host Bus Adapter to the FA port. This requires physical cabling to the
switch and a logical path controlled by fabric zoning.
When the HBA and FA ports are powered on, or when the cables are connected, a Fabric Log
In, or FLOGI, occurs. This is a standard part of the Fibre Channel protocol where devices
identify themselves to the Fabric using their World Wide Name (WWN), a unique 128 bit hex
identifier. The switch will update the Name Server database and assign the device a Fibre
Channel address. After the FLOGI, the HBA will query the name service on the switch for the
addresses of target devices (FA ports), and then attempt to log into each port. This is called a
Port Log In, or PLOGI. After the HBA has logged into the FA port, the HBA will do a device
discovery.
Note: The FA port maintains a persistent login table on the FA ports so you can identify which
HBAs have logged in the past and the current state of the connection.

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Most HBA vendors provide tools to view the status of their HBAs. In addition, the Solutions
Enabler syminq command can be used to query the configuration and login status of an HBA.
In Fibre Channel, all devices are identified by their unique World Wide Name. Actually there
are two World Wide Names: the Worldwide Node Name (WWN) and the World Wide Port Name
(WWPN). The WWPN is what is typically used when configuring fabric zoning and device
masking.
In the example shown, the port state (online) indicates that the FLOGI process was successful
and the HBA has logged into the Fabric.

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

On the Symmetrix, you can also determine the WWPN and status of the FA port using the
symcfg command.
On the VMAX3, a port is online if the port is online and connected. If the status shows
PendOn, this indicates the port is online but not connected to a switch.

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

You can take different approaches to verifying connectivity status: Starting at the host and
working toward the Symmetrix or starting at the Symmetrix and working toward the host.
Another approach is to start in the middle. In Fibre Channel SANs, the switch provides a login
service. When an HBA or FA port is connected to the switch, it performs a Fabric Login and
identifies itself by its WWN, specifies operating parameters such as buffer credits, link speed,
and class of services, and is assigned a Fibre Channel address. This information is maintained
in the switches name service database.
In the example,, we see that both the HBA and FA ports have logged into the switch. Note:
The example is for a Brocade switch but other switch vendors have commands that provide
similar information.

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Zoning controls access in a Fibre Channel network by allowing or restricting access between
end devices in a FC SAN. There are several ways of defining zones, but the most common is
using the World Wide Port Name, which uniquely identifies a device in a Fibre Channel
network.
In the example, we see several different zones defined. Each zone includes a single HBA and
an FA port. While it is possible to have a zone with multiple members, EMC best practice is to
configure single initiator zoning, which is a single HBA per zone. This approach eliminates
unnecessary interactions between devices.

10

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

If both the HBA and FA ports have logged into the switch, and the switch is zoned correctly,
the HBA should log in to the FA. This can be seen using the command symaccess list
logins.
The identifier field lists the HBAs that have logged into the FA port. User-generated node and
port names are identified as the AWWN or AISCSI. Columns labeled On Fabric and Logged
In indicate whether the HBA is connected to a fabric and whether it is currently logged in to
the Symmetrix system. You can use the verbose (-v) option to view the time of the last active
login information.
Note: In a large environment, there could be many frontend ports and hundreds of initiators
logged into each port.

11

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

The Login History Table is very useful for troubleshooting connectivity issues. The Login record
is created the first time an initiator logs into an FA port and is persistent and shows the
current login state.
There can be many stale records as the system does not delete records when a host is
removed from the environment. This is by design as it allows an administrator to easily
identify problems when an initiator shows a logged in status of no. To keep this clean, Login
records for hosts that are no longer connected can be deleted using the command: symaccess
sid <SID> -wwn <WWN> -dirport <dir:port> -login remove.
Aliases are user defined names for WNNs and make it easier to identify end devices. These
are sometimes updated when a port login occurs but can also be set manually.

12

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Storage Area Networks (SAN) provide a fan-out capability where it is likely that more than
one host is connected to the same frontend port. The actual number of HBAs that can be
configured to a single port is operating system and configuration dependent but fan-out ratios
as high as 256:1 are currently supported. Reference the EMC Support Matrix for specific
configuration limitations.
Each port may have as many as 4096 addressable volumes presented. When several hosts
connect to the same frontend port, an access control conflict can occur because all hosts have
the potential to discover and use the same devices. However, by creating entries in the
Symmetrix device masking database (ACLXDB), you can control which host sees which
volume.
With Fibre Channel, device masking uses the UWWN (Unique Worldwide Name) of Host Bus
Adapters. In iSCSI, the iSCSI Qualified Name (IQN) is used. Regardless of the protocol, the
concepts are the same. The device-masking database (ACLXDB) on each Symmetrix system
specifies the devices that a particular WWN or IQN can access through a specific Fibre port.
Device masking is optional but typically used whenever a frontend port is shared by multiple
hosts. Masking is enabled on a port basis by setting the ACLX port flag.
Device masking is independent from zoning but the two are typically used together in an
environment. Zoning provides access control at the port level and restricts which HBA sees
which port on the storage system; device masking restricts which host sees which specific
volumes presented on a port.
13

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

If a frontend director is going to be shared, which is typical of most environments, masking is


enabled by setting the ACLX flag on the frontend director and configuring mapping and
masking using the auto-provisioning group feature.
Rarely, a frontend director port is dedicated to a single server or cluster of servers.
Configuring ACLX and performing mapping and masking is still an option, but it is also
possible to disable ACLX on the port and simply map devices to frontend ports; any host
zoned to see the port will have access to all devices that are mapped.

14

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

SCSI is a standards-based protocol that has been around for over 30 years and its command
set and nexus is flexible enough to support many different types of storage devices and host
operating systems. Although nearly every server vendor supports SCSI, unfortunately not
every vendor implements SCSI in exactly the same way. For example, while both HP-UX and
IBM AIX support the SCSI protocol, they support a different subset of the operational
parameters.
Fibre Channel and IP are transport protocols used with the SCSI protocol and they too have a
number of configurable protocol and link parameters.
The Symmetrix was designed to support a wide variety of host operating systems and
connection topologies. To accommodate this, the frontend directors can be configured for
different SCSI protocol options, link characteristics, and other capabilities. These settings are
called Port Flags or Director Bit Settings.
Historically these were set on a port basis, which meant that a port was set for a specific
operating system and would not support a heterogeneous environment. Today, if masking is
enabled on a port, these setting can be configured as part of the masking view and can
override how the ports are configured at the hardware level. As a result, heterogeneous host
are supported on the same ports. For example, the same port can support both AIX and HPUX hosts.

15

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Port flags may be set in the IMPL.bin file under DirEdit view. These define the operating
characteristics of the FC Link and the SCSI protocol. The settings also enable and disable
masking on a port. These are enabled or disabled using the space bar. If the flag is shown in
UPPERCASE it is enabled and if it is shown in lower case it is disabled.
Historically there was a wide variety of settings for different host environments, but today the
default settings are appropriate for most environments. One exception is HP-UX, which
requires Volume Set Addressing. If this flag is enabled, other operations such as windows,
Linux, and ESX would not work on that port, unless the flag were overridden in the masking
record.
In addition, there are Init parameters that effect how devices are presented. For example, in
SymmWin Initialization Parameters FBA, there are additional flags. An example of this is the
Dual Port Registration bit. This should be set to on if the same HBA is presented twice
through masking and zoning to the same FA.

16

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

The emulation code in the Symmetrix frontend director provides the flexibility to configure the
operating characteristics on a port basis in order to support different host configurations.
These settings are called port flags and can be displayed using the symcfg command.

17

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

The screen capture shows how the director flags could be set for an initiator as part of the
masking record. This would override the settings in the IMPL.bin for a specific set of initiators.

18

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

The Simplified Support Matrix available on support.emc.com and other EMC websites includes
the director bit settings. There are two tables in this document: One for fabric attached using
FC switches and the other for direct attached in an arbitrated loop topology. Note: The
document includes foot notes not shown in this slide.

19

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Because a Symmetrix can have many thousands of devices, it is not possible to configure
every device so that it is associated to every frontend port. Instead, specific devices are
mapped to specific ports by assigning a channel address. Historically this was accomplished
by associating a channel address with a device and frontend port. This is referred to as hard
addressing and is done in the IMPL.bin file and an online configuration change. The hard
address can be overridden using Dynamic LUN Addressing as part of the masking record. This
provided the flexibility to give each host their own LUN address space starting with zero,
rather than to use the channel address that was assigned when doing the mapping.
With the VMAX3, the association of devices with ports and initiators is done entirely in the
masking record; hard addressing is not required unless ACLX is disabled on a port.
Host systems discover and access Symmetrix devices using the LUN address. For open
systems hosts, the LUN number is the SCSI ID. Normally a host uses a combination of the
controller, target, and logical unit number to address a disk device. The controller number is
the Host Bus Adapter, the target is the port on the storage system and the logical unit
number is the assigned LUN Address. An example of this is /dev/dsk/c1/t1/d0. The c# refers
to the controller or HBA, the t# is the target device or frontend port, and the d# is the device
address, either the hard channel address or the dynamic LUN address.

20

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

On the VMAX3, hard addressing is used on ports that have ACLX disabled. Devices are made
available on a port by assigning a channel address to a device on that port. Typically the
devices are mapped to two or more ports using the same address for all ports. Valid
addresses are 0-FFF.
If ACLX is enabled on a port, the address field will be grayed out to prevent physical port
mapping. An X indicates a device is masked to that port and a Dynamic LUN Address is
assigned in the masking view.
Note: On VMAX3 a port can have a maximum of 4096 devices mapped and no more than
16384 devices on all ports for a single director instance.

21

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

The Auto-provisioning Group feature was first introduced with the VMAX and radically changed
how storage was provisioned. Prior to VMAX, provisioning was performed by defining each
point-to-point connection between the host and the array. While the concept was
straightforward, it was cumbersome when dealing with hosts with multiple HBAs and cluster
environments.
Today, most applications running on Symmetrix systems require high availability connectivity,
with multiple HBA connected through redundant fabrics to multiple HBAs on the Symmetrix.
In addition, clustered hosts, where multiple servers share access to the same storage, are
common. The auto-provisioning groups feature was developed to make storage allocation
easier and faster, especially with these types of configurations.
VMAX3 extends the concept by eliminating the need to do a configuration change in order to
map devices to frontend ports. On VMAX3, mapping is performed in the masking record.
VMAX3 also provides more flexibility in managing storage groups.

22

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

The basic concept of Auto-provisioning Groups is to group devices that are to be provisioned
to a host into storage groups, group a set of frontend ports used by a server into a port
group, and group all the HBAs for a host or cluster of hosts into an initiator group. Once the
groups are created, a masking view that defines a many-to-many association between
devices, ports and initiators can be created. When the masking view is created, the devices in
the storage group are mapped to the ports in the port group, masking the initiators in the
initiator group, and assigning dynamic LUN addresses to each device.
Use the command symaccess to perform provisioning operations for auto-provisioning
groups. Unisphere for VMAX may also be used; it provides an intuitive wizard based interface.

23

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

A storage group is a collection of devices that are used together. From prior discussions we
know that storage groups are used to set SLOs for a set of devices. Here we see that storage
groups are also used when doing provisioning and define the sets of devices that are visible to
a host.
A storage group contains zero or more devices, and can be configured in a cascaded
configuration where the storage group contains other storage groups.
Storage groups are assigned names and, using standard naming conventions, help document
the environment.
Storage groups can also be created and managed using the Solutions Enabler symsg
command.

24

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Common operations that are performed on storage groups using the Solutions Enabler symsg
and symaccess commands include:
Creating storage groups and optionally populating them using device lists, device ranges,
device groups, device file.
Removing devices from storage groups. On VMAX3, devices can optionally be unmapped
when they are removed from a storage group that is part of a view.
Adding devices to Storage groups. When a storage group is associated with a masking
view, dynamic LUN address are assigned automatically. The user can also specify an
alternative address to use.
The contents of a storage group can be listed in varying amounts of detail.
It is also possible to list out the storage group(s) that a particular device belongs to.
A storage group can be renamed or deleted.
Additional capabilities include:
Moving devices between storage groups.
Converting storage groups from cascaded and standalone and vice versa.
Most operations can also be performed using Unisphere for VMAX.
25

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Port groups are a logical grouping of frontend director ports that are used together. Ports are
typically shared, with many hosts accessing devices through the same set of ports. There are
two approaches to configuring port groups: Creating a separate port group for each host or
cluster and reusing director ports in multiple port groups, or creating single port group with a
set of ports and sharing it when defining masking views for multiple hosts or host group.
Port groups are assigned names, and, using standard naming conventions, help document the
environment.

26

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

A best practice is to include at least four frontend ports that are on different directors when
performing masking. When used with host based multi-pathing software such as PowerPath,
this distributes the workload and allows the array to better respond to bursts of IO activity.
Operations that are performed on Port Groups include:
Creating a port group and optionally populating it using a list of ports
Removing ports from port groups. Optionally, on VMAX3 if a device port is part of a
masking view, devices can be unmapped from the removed ports.
Add ports to port groups
Display the contents of the port group in varying amounts of detail
List the port group(s) that a particular port belongs to
Port groups can be renamed or deleted.
Can be created using SYMCLI or Unisphere for VMAX.

27

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

An initiator group is a collection of HBAs, identified by either the Fibre Channel WWPN or the
iSCSI iqn. Initiator groups are typically created for each host, or group of hosts, in a cluster
configuration.
There are different approaches to creating initiator groups. You can create the initiator group
and specify the first WWPN when you create the group, and, using separate commands, you
can add additional WWPNs one at a time. Oftentimes it is easier to create a file with a list of
all the WWPNs for a host or cluster and use that file as input when creating the initiator group
in a single operationas shown in the example here.
An initiator can only belong to a single initiator group. However, initiator groups can be
configured in a cascaded configuration where the initiator group contains other initiator
groups.
Again, initiator groups are assigned names and, by using standard naming conventions, help
document the environment.
Note: When creating initiator groups using Unisphere for VMAX, the term Host or Host
Groups is used to define a set of HBAs that are used together. To many, this is more intuitive
as you would normally think about provisioning storage to a host rather than an initiator.

28

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Operations that are performed on initiator groups include:


Creating the group and optionally populating it using one initiator, a file containing a list of
initiators, or creating a cascaded initiator group by populating the initiator group with
another initiator group.
Initiators can be added or removed from initiator groups.
The contents of the initiator group can be listed in varying amounts of detail.
Initiator Groups can be renamed or deleted.
HBA flags can also be set for an initiator group to override the settings in the IMPL.bin file
and allow heterogeneous hosts to share the same frontend ports.

29

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Creating storage groups, port groups, and initiator groups only creates the group objects. The
actual provisioning is accomplished when a masking view is created. This takes the devices in
the storage group and maps them to the frontend director ports, masks the devices in the
storage groups to the initiators in the initiator group, and assigns a dynamic LUN address.
On VMAX3, mapping is performed in the masking record; however, on prior VMAX mapping
would invoke a configuration manager session to first assign a hard channel address. This
increased the time it took to perform a provisioning operation.
Using Unisphere for VMAX, provisioning operations are performed using a wizard that
provisions storage to a host or host group, where a host is defined as a collection of initiators
and created previously. The wizard steps the user through the creation of a storage group and
the creation or selection of a port group; the masking view is created automatically.

30

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

The greatest advantage of the Auto-provisioning Groups approach to provisioning is when the
storage administrator needs to make changes to an existing configuration. In most
environments, adding more capacity to an existing host or cluster is a regular occurrence.
With auto-provisioning groups, this is simply a matter of adding devices to the storage group
that is associated with a masking view. The devices are automatically mapped and masked,
greatly simplifying the task of adding capacity.
Similarly, if it is determined that additional frontend ports would provide better performance,
the administrator would perform the necessary zoning, and then add the ports to the port
group; again, the devices would automatically be mapped and masked to take advantage of
the additional ports.
The same is true for initiators. While is it less likely to dynamically add HBAs to a host, it is a
common occurrence to add or remove hosts in a cluster. To add a new host to a cluster,
simply add the initiators to the initiator group that is part of the masking view, and the host
will have access to the same devices as the existing hosts.

31

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

An initiator group could contain all HBAs for a server, all HBAs for a cluster of servers, or a
subset of HBAs on a single server in order to isolate specific workloads through specific HBAs.
However, an HBA can only belong to a single initiator group. An
initiator group can be a member of another initiator group in a cascading configuration with
one level of cascading. This is useful when multiple masking views are required for a server
using the same HBAs. A good example of this is a cluster. Clusters are a collection of servers
that work together. Examples of clusters include high availability servers, data base clusters
such as Oracle RAC, compute clusters, and virtualization clusters such as VMware ESX. Unique
to these environments is they typically require shared access to a set of disks.
Using cascaded initiator groups, an initiator group would be created for each server that
contains its HBAs. That initiator group would be used to create masking views for each server
that storage groups with only the boot device . Then another initiator group would be created
as a cascaded initiator group; it would contain the initiator groups for each server. The
cascaded initiator group would be used to create a masking view using a storage group that
contains all the shared devices. In this configuration each server would have two masking
views: One that contains devices that are used exclusively by that host, and another masking
view that includes the other servers and a storage group that contains shared devices.

32

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Similar to initiator groups, storage groups can be configured to contain other storage groups.
As seen in earlier discussions, storage groups are also used when setting Service Level
Objectives (SLO). Cascaded storage groups provide the flexibility to create a different storage
group for each application and each storage group could have a different SLO. When doing
provisioning, a cascaded storage group would be used that would contain the storage groups
for each application.
The storage group containing other storage groups is referred to as the parent, and the
storage group containing devices is referred to as child. Only a single level of cascading is
allowed and a parent storage group cannot be a child of Storage Group. Storage Group can
only contain devices or other storage groups. Mixing devices and storage groups in a storage
group is not allowed.
A parent can have up to 32 child storage groups. Empty storage groups can be added to a
parent storage group, but the parent storage group must inherit at least one device when it is
used to create a masking view. Also, the same device cannot be a member of more than one
storage Group that is part of a parent storage group.
Service Level Objectives are defined for child storage groups and not for parents.

33

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

The VMAX3 supports the ability to limit the IO/Sec and throughput to a set of devices by
applying an IO Limit on a storage group. Limits are applied to all ports within the masking
view with the limits being evenly distributed across the available directors within the
associated port group.
Emulation code on the frontend director monitors and enforces performance against these set
limits to ensure that applications do not exceed limits. This reduces the potential of less
important applications with bursts of IO impacting other more important applications that are
using the same director ports. This provides greater levels of control in multi-tenant or cloud
environments and ensures the predictability needed to service more customers on the same
array. It also provides a tool to manage expectations of application administrators with regard
to performance and allows customers to provide incentives for their users to upgrade their
performance service levels.
Limits may be configured for standalone storage groups or cascaded storage groups. With
cascaded storage groups, multiple limits may be placed across the parent and child storage
groups. Limits set on the parent storage group must be less than or equal to the settings on
any child storage groups.
In the event of a director failure, the limits dynamic failover and are redistributed to
remaining online directors. Limits are dynamically redistributed to active directors based on
port activity.

34

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Rather than simply put a graphical user interface in front of the command line interface,
Unisphere for VMAX provides an intuitive interface and presents tasks from a user
prospective, shielding the underlying complexities.
When provisioning storage using Unisphere for VMAX, the same steps are required; however,
they are combined and presented in a manner that hides complexity and includes checks that
reduce risk of error.
Rather than create an initiator group, a user would go to the Host view and select either
Create Host or Create Host Group. They would be presented a dialog similar to the one
shown here where they would specify a host name, and, from the list of available HBAs, would
add them to the host. This would create a job that can be run either now or added to the Jobs
List to be executed later.
Note: This dialog also allows a user to specify the required host flags that would set or
override the director bit settings in the IMPL.bin file.

35

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Once a host has been created, the user can select Provision Storage to Host from the host
list to launch the storage provisioning wizard.
The first dialog in the wizard is Create Storage. Notice that you specify the number and size
of the devices and the SLO in terms of service level and workload type. Based on this, the
expected response time is displayed. The dialog also allows the user to specify additional
devices with different SLOs. Additionally, in this dialog the user can configure IO Limits to
limit the number of IO/sec and throughput allowed to these devices. This would be done if it is
appropriate to throttle the host and restrict the system resources it consumes.
In the second dialog of the wizard, the user can Select Port Group, either an existing port
group, or create a new one from the list of available ports.
While the Unisphere for VMAX wizard reduces the complexity to the user, under the covers it
generates tasks that create the same objects as are created using the Solutions Enabler CLI.
This includes creating TDEV devices of the appropriate size, creating a storage group, and
assigning a SLO. If the host will have devices with more than one SLO, it will create a
cascaded storage group, with each child having a different SLO. It will also configure IO Limits
for the storage group if specified, create a port group, and build the masking view.

36

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Unisphere for VMAX integrates the process for fulfilling the provisioning request with capacity
planning and service level management. When performing a provisioning operation, make
sure that you have both the capacity in terms of GB and TB and, even more important, the
capacity to handle the workload in terms of IO/Sec and throughput, with a response time that
will meet the specified SLO.
When using the storage provisioning wizard, during the Review step the user is given an
option to run the Suitability Check. This verifies that the system can handle the additional
workload as a result of this new provisioning request, and meet the requested Service Level
Objective.
This step is unique to Unisphere for VMAX and there is not an equivalent check with the CLI.
This is because Unisphere for VMAX maintains real time, diagnostic, and historical
performance data that can determine compliance with existing SLOs.

37

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

To allow for initial configuration, there is a special device referred to as the Access Control
Logix (ACLX) device. This is a special device that is visible on a director port regardless of
masking and is intended to be used by a management server as a gatekeeper device to
initially configure the system.
This device is visible like any other device except it is write disabled. Once other gatekeeper
devices have been provisioned to the management server, the ACLX device is not needed and
can be disabled. Continuing to present the ACLX device on a port would mean that every host
that is zoned to that port will see the device. While it will do no harm, server administrators
are sometimes annoyed when there is a small device presented that they cannot use.
The SHOW ACLX DV flag in the IMPL.bin file determines if the device is visible on a frontend
port; if it is, it is assigned the address specified in the IMPL Initialization parameters. The
default factory configuration enables this on the lowest port number available on the FA
director. The default setting can be modified to make the device accessible on a different port.

38

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

The Access Logix Database persistently maintains the access control information, including all
groups and masking view details in the system metadata. An ACLX device or any other
gatekeeper device is used by Solutions Enabler and Unisphere for VMAX to access the
database.
.

39

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

If the Show ACLX DV flag is enabled, the ACLX device will be visible to all host connected to
the port. By default the ACLX device is enabled on the first director port.
While this is configured in the IMPL.bin file, it can also be enabled using the Solutions Enabler
CLI.
To see if enabled, use the command:
symcfg list fa all port detail
To enable it, use the command:
symconfigure cmd set port 1D:4 SHOW_ACLX_DEVICE=Enable|Disable: commit

40

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

The default address used by a host to access the ACLX device is 000. This can be changed if
appropriate but it is important to verify that the LUN address chosen is valid for the host that
will be the management server. For example, some host OS types will not recognize
addresses greater than FF.

41

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

The masking database is a critical piece of configuration information and if for some reason
the database got corrupted, would result in a major Data Unavailability event. While it is very
unlikely that this would occur, the database is backed up regularly on MMCS. A local copy can
also be created on a local management server using the symaccess command. This copy
could be used to restore the database.
The copy could also be used to look at the Auto-provisioning Group information offline. This is
done using the file <File Name> with the symaccess command. Rather than querying the
live information, it will query the information in the backup file.

42

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

The WWPN of VMAX frontend ports are derived from the serial number of the Symmetrix
frame and the director and port numbers. Sometimes when planning the zoning of a VMAX
system you may need the WWPN and do not have direct access to the system. There is an
online tool available on gscloud.emc.com that, given a serial number, director and port, can
be used to decode the WWPN. It also works in reverse: Given a WWPN, it will decode the
serial number of the Symmetrix that it is associated with.
Reference https://support.emc.com/kb/191408 VMAX3: How to decode a Symmetrix World
Wide Name (WWN) on a VMAX3 and https://support.emc.com/kb/193262 VMAX3: How are
Symmetrix port names presented to the SAN Switch NameServer?
Note: The Atlas site that hosts the WWN tool supports only the most recent versions of
Chrome, Firefox, and Safari. Internet Explorer is not supported.

43

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

Connecting a host to the Symmetrix involves more than connecting a cable between the host
and the Symmetrix. On the host, it is necessary to physically install a supported HBA and load
the appropriate device driver. The full list of host environments that are supported can be
found in the EMC Support Matrix.
In most environments, a switch will be used. This is what provides the fan-out capability and
allows a single FA port to be shared among many HBAs. The switch is zoned to allow or
restrict access between specific HBAs and specific FA ports. Zoning is normally done using
Fibre Channel World Wide Port Names (WWPN).
On the Symmetrix, three things need to be configured; port flags, mapping, and masking.
Port flags define the SCSI and Fibre channel operating characteristics. Mapping assigns
channel addresses to devices on FA ports. Masking allows access for specific devices to
specific HBAs.
While the focus of this module has been on Fibre Channel connectivity in an open systems
environment, the same concepts apply to iSCSI environments as well.
One of the most important aspects of storage provisioning is planning and ensuring there are
enough system resources available to meet the required service level objectives.

44

Copyright 2015 EMC Corporation. All Rights Reserved.

Symmetrix VMAX3 Internals: Essentials


Storage Provisioning For Open Systems

45

Copyright 2015 EMC Corporation. All Rights Reserved.