NTSP Student Handout Rev2-5c

Nimble Technical Sales
Professional Accreditation
Nimble Storage Array Introduction,
Installation, and Maintenance
Revision 2‐5
Checking Your Enrollment
SSID
Classroom Network
Password
1. Login to http://university.nimblestorage.com/
2. Click “My Account”
3. Verify today’s course is listed and then click “Go”
4. Ensure your status states “Enrolled” with an “X” next to it (Don’t click the “X”)
5. If your screen looks different, ask your instructor for instructions 1
Nimble Technical Sales

Professional Accreditation
Nimble Storage Array Introduction,
Installation, and Maintenance
1
Revision 2‐5
Insert Instructor Name

Insert Instructor email
© 2012 Nimble Storage, Inc.

3
Introductions
 Name
 Company
 Position
 Data storage background
 What do you hope to get out of the course?
2
Revision 2‐5
Topics
In this course the following subjects will be discussed:
 Section 1: CS-Series Array  Section 8: Connecting to Hosts

Introduction  Section 9: Snapshots
 Section 2: Scale-to-Fit  Section 10: Replication
 Section 3: CASL Architecture  Section 11: Data Protection and DR
 Section 4: Networking and Cabling  Section 12: Maintenance and
 Section 5: Initial Installation Troubleshooting
 Section 6: Array Administration  Section 13: Support
 Section 7: Working with Volumes
Section 1: CS-Series Array Introduction
3
Revision 2‐5
Raw versus Usable versus Effective Capacity
Subtract
capacity
for Add storage
RAID-6 capacity due
Raw
parity, to inline Effective
Capacity Usable compression
spares & Capacity
(typical 30%
system Capacity
reserves to 75%)
Effective: 33 TB
Raw: 24 TB Usable: 17 TB
(assuming 50% compression)
Nimble Storage CS210 At a Glance
Model CPU DDR3 Ethernet Cache Data Effective Capacity

Memory Ports SSD HDD 0 2x
CS210 1 12GB 4x 1GbE 2x 80GB 8x 1TB 4TB 9TB
or or
160GB 8TB RAW
No 10 GbE
option
Capacity Expansion Add up to 1 additional shelf
Scaling Performance NOT SUPPORTED

8
4
Revision 2‐5
Nimble Storage CS220 at a Glance
Model CPU DDR3 Ethernet Cache SSD Cache Data Eff. Capacity
Memory Ports Total HDD 0 2x
CS220 6x1GbE 12x1TB
CS220G 2x1GbE or
1 12GB 4x80GB 320GB 8TB 16TB
2x10GbE 12 TB
RAW
Scaling Performance Scale Compute and Cache

© 2012 Nimble Storage, Inc. 9
CS240 6x1GbE 12x2TB
CS240G 2x1GbE or
1 12GB 4x160GB 640GB 17TB 33TB
2x10GbE 24 TB
RAW

5
Revision 2‐5
CS260 6x1GbE 12x3TB
2x1GbE or
1 12GB 4x300GB 1.2TB 25TB 50TB
CS260G 2x10GbE 36 TB
RAW

CS420(1) 12TB 8TB 16TB
6x1GbE 640GB
CS440 4x160GB or
2 24GB to 24TB 17TB 33TB
CS460 2x1GbE 4x300GB
2.4TB
2x10GbE 36TB 33TB 50TB
(1) Sold only with X2 or X4 options

6
Revision 2‐5
Hardware Tour - Front
3U
Hardware Tour Controller Unit– Front
HDD blank SSD blank HDD

PWR
LEDs LEDs
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
CS-Series 210
7
Revision 2‐5
Hardware Tour Controller Unit– Front
HDD SSD HDD
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
CS220 and higher
Disks
 Disks:
– 10 or 16 hot swappable drive bays populated with:
» 8 or 12 SATA (with SAS interposers) or SAS disks
» 2 or 4 solid-state drives (SSD)
When replacing a drive, ensure you replace drives with

the appropriate type!
8
Revision 2‐5
Nimble ES-Series External Storage Shelf
 Connect up to three additional shelves

– Scale storage capacity non-disruptively
 Uses 4 Lane 6Gb SAS connectivity from controller to shelf
– Support redundant data paths from controller to
shelves
 Each shelf is its own RAID Group
 Spares assigned for each shelf
1TB 2TB 3TB
ES1-H25 ES1-H45 ES1-H65
Raw Capacity 15 TB 30 TB 45 TB
Effective Capacity
11 – 22 TB 23 – 45 TB 34 – 68 TB
(w/ 0x-2x compression)
Flash 160 GB 300 GB 600 GB
Connectivity 2x 6Gb SAS / IO module
IO Modules Dual hot-swappable SAS controllers
17
Hardware Tour – Front
SSD
HDD HDD
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Expansion Shelf
9
Revision 2‐5
Hardware Tour - Back
1 2
3 4 8
5
6 7
Hardware Components
 Power supplies – 2X 100-240V, 50-60Hz, 4-10 Amp

– Power requirement – 500 watts
10
Revision 2‐5
Section 2: Scale to Fit
Nimble Scaling for Mainstream Applications
Coming Soon
Mainstream
Applications
PERFORMANCE
+ NODES
CAPACITY
11
Revision 2‐5
Nimble Scaling for Mainstream Applications
Real-time Analytics
VDI
Oracle
Mainstream
Applications
SQL Server
PERFORMANCE
` Exchange
SharePoint
Backup, DR
Archival, Cheap and Deep

CAPACITY
Scale Capacity by Adding Disk Shelves
 Add capacity non-disruptively by

Sufficient adding external disk shelves
performance,
but need – A disk shelf contains high
more capacity capacity HDDs and a SSD
P
 Add multiple disk shelves per
Add
Nimble Storage array
C
– Add up to three shelves
» Only one shelf for the CS-210
Once Expansion Shelves have – Mix and match different capacity
been added they cannot be shelves
removed.
12
Revision 2‐5
Scale Capacity Cabling
4 Lane 6Gb cable

1 to 3 meters in
length
Do not connect
SAS cables to an
expansion shelf
until after the array
has been
upgraded to 1.4
Adding a Shelf
1. Check to see if you have proper Nimble OS version

2. Cable new expansion shelf and power on
3. The new shelf is discovered by the control head
– Newly discovered shelves will be shown as “Available” in the GUI/CLI
4. Using the GUI or CLI, activate the new shelf
© 2012 Nimble Storage. Proprietary and 26

confidential. Do not distribute.
13
Revision 2‐5
Adding a Shelf
Discovering
Is there
data on No
Available Activate In Use
the
disks?
Yes Force
Foreign
Activate
Faulty
Scale Capacity Storage Pool
 Storage pool grows when an expansion shelf is activated.

– Segment Layer is updated with new ending block data
» The segment layer provides a map between the Nimble file system addresses
and disk locations
– The map is dynamically created for each incoming write request
» Essentially the segment layer works as a traffic cop by directing writes to the
proper disks

14
Revision 2‐5
Expanding Existing System
1 Fill Expansion
Shelf until 2 Then balance capacity between them
capacity
utilization
matches the
control head
50% Capacity 50% Capacity
Expansion Shelf Controller Shelf

Managing Internal Storage and Expansion Shelves
15
Revision 2‐5
Power On/Off Order
 On  Off
– Power expansion shelves – Power off the controller
first, then the controller shelf and then the
shelves expansion shelves

Scale Compute – The 400 Series Controllers
+
 Provides additional processing power and memory
– Provides two CPU’s each with:
» 6 cores
» 12 GB of DRAM
– Scales performance
– Replaces existing controllers
» A CPU is not installed into current controllers
16
Revision 2‐5
Nimble Array Scaling Compute
 Upgrading a CS220 to a CS420-X2 or -X4

– Cache is upgraded when scaling compute
on the CS220 array
» There is no CS420 array
 Upgrading compute on CS240 and
CS260
– Compute can be upgraded without
upgrading cache
33
Controller Upgrade
 Before you start:

– Ensure you have Nimble OS version 1.4 or later
– Ensure one controller is in active mode and the other is in standby
» Note which controller is active and which is in standby
– Note your current controller shelf model
– Note your current SSD size
17
Revision 2‐5
Controller Upgrade CS200 >> CS400
1. Halt (shut down) the standby controller

– In the GUI go to Manage >> Array
2. Disconnect cables
3. Remove the controller
4. Insert the replacement controller
5. Connect all cables
6. Verify the controller powers up and is in standby mode
7. Perform a failover to the new controller
– In the GUI go to Manage >> Array and click the Failover button
8. Repeat steps 1 – 7
9. Verify the model number has changed from 200 series to a 400 series
Scale Cache – X2 and X4
+
 Provides additional cache
– Scales performance
 There are two variations:
– -x2 doubles the standard cache size
– -x4 quadruples the standard cache size
18
Revision 2‐5
Nimble Storage Cache at a Glance
Model Cache SSD Cache Total

Any of these CS220 4x80GB 320GB
could be a CS220-X2 & CS420-X2 4x160GB 640GB
G-Series CS220-X4 & CS420-X4 4x300GB 1.2TB

CS240 & CS440 4x160GB 640GB
CS240-X2 & CS440-X2 4x300GB 1.2TB
CS240-X4 & CS440-X4 4x600GB 2.4TB

CS260 & CS460 4x300GB 1.2GB
CS260-X2 & CS460-X2 4x600GB 2.4GB
© 2012 Nimble Storage, Inc. Note there is no –X4 option for the CS460 arrays 37
Scale Cache Upgrade
1. Remove the bezel

2. Starting from the right remove the first SSD
3. Wait until the red LED under the slot lights up
4. Install the new larger SSD into the slot
5. Wait until the red LED turns off
6. Repeat steps 1-5 for remaining SSDs
7. Verify that the model number of the controller shelf and the capacity of
the SSDs have changed to –x2 or –x4
8. Replace the bezel
19
Revision 2‐5
Section 3: Cache Accelerated Sequential Layout (CASL)
Choices we need to make when choosing storage
Rank in order
• Performance
• Capacity
• Cost
• Reliability
20
Revision 2‐5
Data Layout – Write in place file system (EMC, EQL)
Pros Cons
• Simple to implement, long history • Poor random write

• Good sequential read performance performance
without cache • Slow, high overhead
compression
Data Layout – Hole filling (WAFL, ZFS)

WAFL – Write Anywhere File Layout
ZFS – Copy on write transactional model
Pros Cons
• Good random write performance • Performance degrades over

until disk fills up time
• More efficient redirect- • Slow, high overhead
on-write snapshots compression
21
Revision 2‐5
Data Layout – Always write full stripes (CASL)
Pros
• Good AND consistent write • Ground up design

performance relies on flash
• Very efficient snapshots • Enables variable block size
• Fast inline compression • Uses a sweeping process to
• Efficient flash utilization, ensure full stripe write space
long flash life
Sweeping
• Data blocks are indexed as they are written

• Over time the deletion of snapshots and data
leaves stale data blocks
• Sweeping removes stale blocks and forms new
stripe writes with the remaining active blocks
22
Revision 2‐5
Building a New Array
How would you design a storage

solution around SSDs?
• As a bolt on flash tier?
 No flash optimization - SSDs grouped using RAID
 Requires more expensive SSDs to obtain the high endurance required
 Performance increase only seen on the flash tier
• As a bolt on read cache?
 No flash optimization - SSDs grouped using RAID to form a read
cache LUN
 Required SLC SSDs to obtain the high endurance required
 No improvement to write performance
Solid State Drives - Tale of the Tape

SLC MLC
Density 16 Mbit 32 Mbit 64Mbit
Read Speed 100ns 120ns 150ns
Block Size 64Kbyte 128 Kbyte
Endurance 100,000 cycles 10,000 cycles
Operating Temp Industrial Commercial
SLC MLC
High density
Low cost per bit
Endurance
Op temp range
Low power consumption
Write/Erase speeds
Source:
Super Talent SLC vs. MLC: An Write/Erase endurance
Analysis of Flash Memory
23
Revision 2‐5
The Nimble Way – Purpose Built CASL
• Flash is highly optimized - writes matched to erase block size which

minimizes amplification
• Erase block size – When data is written to flash it is written a byte at a time.
• But when data is erased it is erased a block at a time. Thus if one bit changes
the entire block must be read, the cells erased and the remaining data written
back down along with the change
1 1 1 1
bit bit bit bit
1 Block
1
cell 1
cell 1
cell 1
cell
bit bit bit bit

Discussion: Disk Storage
What type of Do we use

RAID should multiple RAID
be supported? groups or a single
storage pool?
24
Revision 2‐5
The Nimble Way – Purpose Built CASL
• Fine movement of data (4KB real time movement)

• Utilizes Cost-effective MLC flash without RAID
• Provides a high level of write acceleration with the write-optimized layout
on flash AND disk

Data Path – Write
Write Operation
1. Write is received by active
Inline Compression
NVRAM
NVRAM
NIMBLE controller’s NVRAM (1GB)

ARRAY 2. Write is mirrored to partner
DRAM controller’s NVRAM
3. Write is acknowledged
4. Write is shadow copied to DRAM
Universal Compression:
5. System uses Lempel-Ziv 4 for
 Variable-size blocks enable
inline compression and a
fast inline compression, saving
30-75% modified LZ compression for pre
1.4 software releases.
 Elimination of read-modify-write
penalty allows compression of
• Variable block based;
all applications compresses all data into
stripes
25
Revision 2‐5
What you need to know about Lempel-Ziv 4
 LZ4 is a fast lossless  Mixing the new algorithm with

compression algorithm the old only affects replication
– Provides compression speeds
of 300 MB/s per CPU core
– Provides a fast decoder that
provides speeds up to and
beyond 1GB/s per CPU core.
It can reach RAM speed limits
on multi-core systems.
 Prior to the 1.4 Nimble software
release the array used a
modified LZ algorithm.
© 2012 Nimble Storage. Proprietary and confidential. Do not distribute. 51
Compression
 New LZ compression in 1.4

– Enabled by default (nothing to do.)
– ~21%(Read) and ~15%(Write) performance improvement w/ 7% - 24% additional compression benefit (depending.)
– Uses LZ4 compression for new volumes (including new cloned volumes).
– Uses old compression algorithm for existing volumes.
» Later release will enable LZ4 compression for all volumes.
26
Revision 2‐5
Application Compression with Nimble
Taken from InfoSite Feb 2013
53
Data Path – Write
4.5 MB stripes
Many IOPS sent as a stripe reduces IOPS
between controller and disks
Write Operation
2 6 8 1 21. 8 Write is received
1 by active
Inline Compression K
18K
K K
7K
K K K
21 K
KRandom
NVRAM
NVRAM
3 3 4 2controller’s
4 NVRAM
18K NIMBLE 21K
11K
K
ARRAY K 2.K
11K
K Random
KWrite is mirrored to partner
DRAM controller’s
32K NVRAM
32K 32K 32K 32K
3.
32K
Seq’l
Write is acknowledged
Universal Compression:
4. Write is shadow copied to DRAM
 Variable-size blocks enable
5. System uses a modified Lempel-
fast inline compression, saving
30-75% Ziv for inline compression.
• Variable block based;
 Elimination of read-modify-write
penalty allows compression of
compresses all data into
all applications stripes
27
Revision 2‐5
Data Path – Write
Write Optimized Layout

 Random writes always organized
into large sequential stripes
Inline Compression
NVRAM
NVRAM
NIMBLE  All data is written sequentially in full
DRAM
ARRAY RAID stripes to disks. Because of
compression and the stripe write
All Data there are fewer write operations
 Large stripe written to disk in
one operation: ~250x faster
High-Capacity Disk Storage than “write in place” layout
 Use of low-cost, high-density HDDs
coupled with compression lowers
costs substantially
Data Path – Write
Smart Caching
Inline Compression
NVRAM
NVRAM
NIMBLE  MLC flash: Converting random writes to

ARRAY sequential writes minimizes “write
DRAM
amplification”, allowing the use of MLC
All Data Cache- Large Adaptive Flash Cache SSDs
worthy
Data  No RAID overhead: Using flash as a
read cache avoids the overhead of RAID
High-Capacity Disk Storage protection
 Compression: Data on flash is
compressed, saving space
 Metadata in cache accelerates all reads
28
Revision 2‐5
Data Path – Write
Accelerated Reads
 All random writes and any “hot” data is
Inline Compression written to Flash Cache.
NVRAM
NVRAM
NIMBLE
ARRAY  Serves hot data from flash;
DRAM responds rapidly to changes
All Data Cache- Large Adaptive Flash Cache  Reads 50x faster than disk
worthy
Data
(200us vs. 10ms)
High-Capacity Disk Storage
Data Path – Reads
Read Operation
1 Inline Compression 1. Read from Shadow NVRAM (RAM)
NVRAM
NVRAM
NIMBLE
2
DRAM
ARRAY 2. If not found, check DRAM
3. If not found, read from cache
All Data Cache- 3 Large Adaptive Flash Cache
• If found, validate checksum,
worthy
uncompress, and return data
4High-Capacity Disk Storage 5
Data
4. If not found, read from disk
• If found, validate checksum,
uncompress, and return data
5. And, if cache-worthy, write to cache

29
Revision 2‐5
Performance
IOmeter Nimble CS2xx – 1.3 Nimble CS2xx – 1.4 Nimble CS4xx – 1.4

4K RND write IOPS 14,113 15,278 (8%) 26,031 (84%)
4K RND read IOPS 15,924 18,193 (14%) 41,963 (164%)
 4K RND read/write IOPS
Random io on 100g volume, block-size=4k, q=64 / 14,256
Sequential io on 100g volume,15,111 (6%)
block-size=32k, q=32 / 32,342 (127%)
2x compressible data
256k Seq write throughput
(MB/S) 261 317 (21%) 556 (113%)
256k Seq read throughput
(MB/S) 557 735 (32%) 966 (74%)
 2 volume, 2 test files, each 30G / 4 threads, with 8 outstanding io for each thread
© 2012 Nimble Storage. Proprietary and confidential. Do not distribute.
How does the read operation compare to others?
• How might inline compression Vs. full stripe compression effect the
read?
• How do you think a changed block is handled?
30
Revision 2‐5
Compression Performance Comparison During

Changed Block Operation
Fixed block Architecture CASL Variable Blocks
8 blocks grouped Individual blocks
& compressed 1 2 3 4 5 6 7 8
compressed and
Group placed into coalesced into stripe
N fixed size slots 1 12 2 33 4 5 56 67 87 8
Updated data block
Entire group read compressed and
& uncompressed coalesced into new stripe
Block updated
with new data
New group compressed
& re-written
Other Array Manufacturers Nimble Storage
Compression Performance Comparison For A Changed

Block
Fixed block Architecture CASL Variable Blocks
8 blocks grouped Individual blocks
& compressed compressed and
Group placed into Cost of fixed block architecture relative to CASL: coalesced into stripe
N fixed size slots 1. Additional M blocks read from disk
2. Additional CPU cycles for decompression & Updated data block
Entire group read compressed and
& uncompressed
recompression of all N blocks
3. Additional M-1 blocks written to disk coalesced into new stripe
Block updated
with new data
New group compressed
& re-written
Other Array Manufacturers Nimble Storage
31
Revision 2‐5
Ideal for Exchange
Gain Performance Save and Protect

MAILBOXES PER DISK COMPRESSION CUSTOMERS RETAINING
Published and verified
SNAPSHOTS FOR
Nimble 3,333
Microsoft Exchange ESRP 12 disks
1.8x >1 MONTH
benchmark results.
for 40,000
mailboxes 10-24x
Actual results 48% Actual results

across all across all
EMC Equalogic
NetApp Nimble Nimble
Compellent 34 disks for 32 disks for
72 disks for 64 disks for 10,000 customers customers
10,000
10,000 12,000 mailboxes mailboxes deploying deploying
mailboxes mailboxes Exchange Exchange
139 187 294 312 2010 2010
“With Nimble, we were able to run 3 snapshot backups a day and

“We started out deploying SQL workloads primarily on replicate offsite twice daily. Exchange users notice no performance
the Nimble array. Very quickly we realized we had degradation. Backups take minutes, not hours. Snapshot backups
enough performance headroom to consolidate our very demanding require very little space and are recoverable and mountable locally and
Exchange 2010 deployment on the same array.” remotely. A mailbox or Exchange system can be recovered in literally minutes.
–Ron Kanter, IT Director, Berkeley Research Group Best of all, we can regularly test our procedures for Disaster Recovery.”
–Lucas Clara, IT Director, Foster Pepper LLC
Data Security
 Data on disk with no dirty cache - ever

– RAID6
» Tolerates 2 simultaneous disk failures
– Checksum per block (data and index)
» Checked on every read and by background scrubber
» Mismatch triggers RAID-based reconstruction of stripe
– Self-description per block (LUN, offset, generation)
» Detects mis-directed reads and writes
 Data on flash
– Checksum per block (data and index)
» Checked on every read
» Mismatch causes removal from cache
 Data in NVRAM
– Mirrored to peer NVRAM
– Dual failure → data lost, but consistency preserved to last N minutes
32
Revision 2‐5
Summary
 Intelligent data optimization

 Sweeping
 Inline data compression for primary storage optimization
 The combination of SSDs and high-capacity disks in one device
 Instant, integrated backups
Summary
3 Unique Elements of Nimble CASL Technology

 Fully integrated flash (unlike bolt on offerings)
– Ground up data layout for flash AND disk to maximize flash benefit
 A fully sequentialized write layout on disk
– Dramatic price/performance advantage WITH inline compression
 Highly efficient snapshots (space AND performance)
33
Revision 2‐5
34
Revision 2‐5
Section 4: Nimble Array Networking and Cabling
Understanding IP’s
 The array management IP address

– Best Practice: This IP address can be used for data, but this is not
desirable: specific target IP addresses of the interface pairs should
be used instead.
 The target discovery IP address
– Best Practice: This IP address can be used for data, but this is not
desirable: specific target IP addresses of the interface pairs should
be used instead.
 The data IP addresses
 The two controller diagnostic IP addresses
1
Revision 2‐5
Networking Terminology
Host
 Interface Pairs eth1 eth2
– Controller A eth1 & Controller B eth1

– IP addresses float between cross
Active link
Standby link
Controller A Controller B
Set iSCSI Timeout in Windows
• Set the LinkDownTime to 45 seconds

• The NWT can set the timeout value for you
• Or set it manually:
• see the Microsoft iSCSI guide at http://technet.microsoft.com/en-
us/library/dd904411(WS.10).aspx
 HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4D36E97B-
E325-11CE-BFC1-08002BE10318}\instance-number\Parameters
 MaxRequestHoldTime set to 60 seconds (0X3C)
 LinkDownTime set to 45 seconds (0X2D)
 HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Disk
 TimeOutValue set to 60 seconds (0X3C)
2
Revision 2‐5
MPIO
 Setup MPIO on your server and make sure it’s active.

 Review the “Using MPIO” section of the User Guide
 This will require a reboot of the server to make both the registry
edit and the MPIO active so do this ahead of time so as to not
delay installation.
Other OS Timeout Value Changes
 Changing iSCSI Timeouts on Vmware

– None Needed
 Changing iSCSI Timeouts on Linux (iscsid.conf)

– For Linux guests attaching iSCSI volumes
» node.conn[0].timeo.noop_out_interval = 5
» node.conn[0].timeo.noop_out_timeout = 60
3
Revision 2‐5
Network Best Practices
 Best Practice Details

Do not use Spanning Tree Do not use STP on switch ports that connect to
Protocol (STP) iSCSI initiators or the Nimble storage array
network interfaces.
Configure flow control on Configure Flow Control on each switch port that
each switch port handles iSCSI connections. If your application
server is using a software iSCSI initiator and
NIC combination to handle iSCSI traffic, you
must also enable Flow Control on the NICs to
obtain the performance benefit.
Network Best Practices
 Best Practice Details

Disable unicast storm Disable unicast storm control on each switch that
control handles iSCSI traffic. However, the use of broadcast
and multicast storm control is encouraged.
Use jumbo frames Configure jumbo frames on each switch that handles
when applicable iSCSI traffic. If your server is using a software iSCSI
initiator and NIC combination to handle iSCSI traffic,
you must also enable Jumbo Frames on the NICs to
obtain the performance benefit (or reduce CPU
overhead) and ensure consistent behavior. Do not
enable Jumbo Frames on switches unless Jumbo
Frames is also configured on the NICs.
4
Revision 2‐5
Vmware Settings
 Review the Nimble Vmware Integration Guide
 Configure Round Robin ESX 4.1 only (4.0 will be different)
 To set the default to Round Robin for all new Nimble volumes type the following, all on one line:
 esxcli nmp satp addrule --psp VMW_PSP_RR --satp VMW_SATP_ALUA --vendor Nimble
 Configure Round Robin ESXi5 only

 To set the default to Round Robin for all new Nimble volumes type the following, all on one line:
 esxcli storage nmp satp rule add --psp=VMW_PSP_RR --satp=VMW_SATP_ALUA --
vendor=Nimble
 ESXi5.1 use the GUI to set Round Robin
 Push out Vcenter plugin

 With Nimble OS version 1.4 and higher use Administration >> Plugins
 vmwplugin --register --username arg --password arg --server arg
Cisco UCS and ISNS
 Cisco UCS Support

– Formal Cisco UCS certification program – Nimble will be listed.
– “Boot-from-SAN” now officially supported in 1.4
– Supported adapters: Palo-only (no other mezzanine adapters.)
– Supported UCS version: UCS Manager v2.0(3)
» Cisco UCS firmware version 2.02r – full version string is 5.0(3)N2(2.02r)
– Supported OS’s: VMware esx4.1u1, ESX5.0u1, Windows 2008 R2, RHEL6.2, SUSE 11 Update 1
 iSCSI iSNS Support

– Protocol used for interaction between iSNS servers and iSNS clients.
– Facilitates automated discovery, management, and configuration of iSCSI devices on a TCP/IP network.
– Primary driver  Microsoft HCL Certification requires it.
– Managed via the Nimble CLI only.
5
Revision 2‐5
Cabling Multi-switch Connectivity
Host
• Even ports to one switch
eth1 eth2
• Odd ports to the opposite switch
eth5 eth6 eth5 eth6
Active link
Standby link
What is wrong with this configuration?
Host
eth1 eth2
If a switch fails controllers
cannot perform a proper
failover since their sibling
interface does not have
connectivity. eth5 eth6 eth5 eth6
6
Revision 2‐5
Section 5: Installation
First Steps
 Call Nimble Support and request a login to the Support

Portal: (877) 364-6253 X2
Once you have this log on to the support site and download the
following: http://support.nimblestorage.com/download/download.html
– Latest Release Notes
– Latest User Guides
– Latest CLI Reference Guide
– Nimble Windows Toolkit
– VMware Integration Toolkit (if applicable)
– Related Best Practice Guides
7
Revision 2‐5
Pre-Install Checklist
 Complete Checklist and review in advance

of on-site visit
 Send to Nimble Support for review
 Create physical topology with customer and
validate against best practices
 Perform an on-site visit prior to the
installation
Pre-Install Checklist
 Collect all necessary data to perform an installation

 Organized in the same order that you will be entering in the data
 Can be left with the customer
Pre-Installation Checklist

8
Revision 2‐5
Before You Begin
Important: The computer used to initially configure the array must

be on the same physical subnet as the Nimble array, or have
direct (nonrouted) access to it.
• Ensure Adobe Flash Player is installed
Nimble Windows Toolkit Installation
The Nimble Windows Toolkit (NWT) includes

Nimble Protection Manager (NPM)
9
Revision 2‐5
Installer needs to modify a few iSCSI timeout values.

Installer will update them only if they are smaller than
recommended values. Do you want installer to update
the values? Click Yes to update and continue. Click No
to continue without updating the values.
10
Revision 2‐5
Perquisites
 Before launching the NWT:

– Set a static IP: Set your IP address to the same subnet as your array
management IP address will be on.
– Have your array controllers A & B correctly cabled to your switch fabric per
the previous drawings.
– Complete all your switch configurations for Flow Control, Jumbo Frames,
Spanning tree, Unicast, etc.
– Install the Windows Integration Tools (WIT) on the Laptop or Server your
installing with.

11
Revision 2‐5
NWT– Nimble Array Setup Manager
1) Start the “Nimble Array Setup Manager” 2) Select the Array to install and click “Next”
3) Enter the array name

• Make it useful such as
row and rack number
4) Set your management IP

address Subnet mask &
Default Gateway
5) Enter and confirm your

array password
6) Click “Finish”
12
Revision 2‐5
You should get this screen

after a few seconds.
7) Click “Close”. Your

default browser window
will be opened and
directed to the
management IP. If it
does not open a browser
and point it to your
management IP address
Nimble Install – Nimble Array Setup Manager
Enter Management IP
If you get this screen click

this selection to continue
NWT should take you straight to the Nimble

Array Setup Manager screens. If not, you may
see this screen.
13
Revision 2‐5
Log in with the password

you just set

One shared network (combined Management and Data network)
Two dedicated networks(separate Management and Data networks)
Advanced configuration
Use if you are setting up three or

more subnets
14
Revision 2‐5
Set the iSCSI

Discovery IP
Set the physical

IP addresses
Set the physical

IP addresses
Typical CS240 Configuration
Array Management IP Address and Target IP Address

(floating, shared by controllers)
Data ports
Management
& replication
CONTROLLER A CONTROLLER B
Diagnostic IP 1 (associated Diagnostic IP 2 (associated

with any physical port) with any physical port)
15
Revision 2‐5
Set the Domain

and DNS servers
Set Time Zone

and NTP server
16
Revision 2‐5
Set ‘From’ Address

Set ‘Send To’ Address’
Check – send copy to
Set SMTP Server Nimble storage
Ensure Auto Support in

enabled
If using an HTTP Proxy check here
Your Nimble Storage array is ready to use. Before you start using your
array, there are a couple of things you should do to ensure smooth
operations.
You must add the management IP address and the controller support
addresses you provided to
Your mail server’s relay list.
You will also need to open the following firewall ports:

• SSH 2222 hogan.nimblestorage.com (secure tunnel)
• HTTPS: 443 nsdiag.nimblestorage.com (software downloads,
autosupport, heartbeat.
17
Revision 2‐5
Nimble Install - WEB
Nimble Install – Post Initial Setup
Open the AutoSupport screen:

Administration >> Autosupport/HTTP Proxy
18
Revision 2‐5
Nimble Install – Post Initial Setup
Check – Send AutoSupport

1
data to Nimble Storage
Check – Enable Secure
2
Tunnel
Verify all items state

4
“Passed”
Click – Send
3 Click – Test AutoSupport Settings 5
AutoSupport
You should receive an email with a case for the test
Post-Install Checklist
 Verify with Nimble Support that an Autosupport was received and request
an Install Health-Check
– Don’t leave site without performing this step!
 Ensure you have updated firmware

 Ensure you perform a failover of the controllers
 Check VMware paths (to be discussed in later section)

19
Revision 2‐5
Incoming ports use to be aware of
To use: Open local For local IP addresses: Notes:

port:
SSH 22 Array management IP
HTTP 80 Array IP
GUI (HTTPS) 443 Array management IP HTTP (port 80) communication is
redirected to HTTPS
iSCSI 3260 Discovery and data IP Needed for data access
SNMP 4290 SNMP daemon
GUI charts, 4210 Array management IP
NPM
Control 4211 Array management IP
Replication 4213 Array management IP
(data)
Outgoing ports use to be aware of
To use: Open For local IP addresses: Notes:

local port:
External NTP 123 NTP server IP UDP port
External DNS 53 DNS server IP UDP and TCP port
SMTP Usually 25 Mail/SMTP server IP Needed for email alerts
SNMP 162 Needed for traps

SSH/SSHD 22 support.nimblestorage.com Needed for manual SCP of
diagnostic information

20
Revision 2‐5
Section 6: Array Administration
GUI Interface
https://{arrays management IP address}
21
Revision 2‐5
GUI Tour
GUI Tour
22
Revision 2‐5
Hardware Icons
GUI Tour – Hardware Icons
 Disk drive is healthy

 Disk drive is designated as a spare
 SSD is healthy
 Disk is failed or missing
 Rebuilding disk
 Foreign disk
 Empty slot
 A fan is faulty
 A hardware event has occurred
23
Revision 2‐5
GUI Tour – Volume Icons
 Volume is online
 Volume is offline
 Volume is offline due to a fault
 Volume replica
 Volume collection
 Volume is running out of space
GUI Tour – Replication Task Icons
 A replication task is scheduled

 A replication task is pending
 A replication task is in progress
 A replication task failed
24
Revision 2‐5
GUI Navigation
 Links
 Side menus
 Pull down menus
Performance Monitoring – Monitor >> Performance
25
Revision 2‐5
Performance Monitoring – Interfaces
Space Usage Graphs
26
Revision 2‐5
Command-Line Interface At-a-Glance

 Admin (same password as GUI) for customer/SE use
 CLI Access
– Everything is available in the CLI
– ssh (putty) to management to Support IP addresses
– Serial access using “dongle”
» 115200, 8bits, No parity, 1 stop, Null-modem cable
» Never leave home without it!
 Commands
– All commands follow similar form:
» <cmd> --list ; --info ; --edit ; --help
» man <cmd>
– vol, ip, subnet, route, nic, volcoll, stats …
– help (to see them all)
 Refer to the Nimble CS-Series Command Reference Guide
MIB II
 MIB II Support
– Customers use SNMP to view their Nimble array with existing Management Software
» E.g. Solarwinds, Nagios, Cacti, MG-SOFT MIB Browser
– MIB II is the second version of MIB
– Mandatory for every device that supports SNMP
– Support for SNMP v1 and v2, but not v3
27
Revision 2‐5
Section 7: Working with Volumes
Volumes Overview
Physical storage
resource
Volume Logical storage

resource
RAID 6
Storage Pool
28
Revision 2‐5
Thin Provisioning
Consumed Space
Volume
RAID 6
Storage Pool
Space from the pool

is consumed as data
is written
Volume Reserves
Volume
Volume Reserve
RAID 6
Storage Pool
A reservation reserves a
guaranteed minimum
amount of physical space
from the pool for a volume
29
Revision 2‐5
Volume Quotas
Volume Reserve
Volume Quota
Volume
Pool
A quota sets the amount of a volume that can be consumed before an alert is
sent and writes are disallowed.
Viewing Volume and Replica Usage
30
Revision 2‐5
Performance Policy
 Select a pre-defined policy or

create a custom policy
 Custom policies:
– Provide a name based on the
application
– Block size should be < = the
application block size
– Compression on/off
– Caching on/off
 Block size cannot be changed
on a volume without data
migration
Access Control (Initiator Groups)
 Access control – which hosts have access to a volume

 Best Practice: Always limit access to a host initiator group
 Allow multiple initiator access – For use with clusters, not MPIO
31
Revision 2‐5
Initiator Groups
 A set of host initiators (IQN’s) that are allowed to access a specified

volume.
 Can be created at volume creation or as a separate task
– Manage >> Initiator Groups
 An IQN can only be assigned to one initiator group.
Initiator Groups
• Initiator Name is the real IQN –

not an arbitrary name
• Case sensitive
• Seldom use IP
• Multiple initiators for ESX,

MSCS
• Initiator Groups are applied to:

• Volumes
• Volumes+Snapshots
• Snapshots only
32
Revision 2‐5
Volume Collection
 A grouping of volumes that share

snapshot/replication schedules
– All volumes in a group will be snapped and
replicated as a group
 Best practice: Create a volume collection
for each application
– Oracle Database and log files
 Ensure you do not create overlapping
Volume Collection
schedules
Volume Collection >> App Synchronization
• App flushes/quiesces I/O

while we take a snapshot
and then unfreezes
• VMFS consistent
snapshots
• SQL consistent snapshots
• Exchange consistent
snapshots
• SQL/Exchange uses MS
VSS framework and
requires NPM on the
Application Host – more
later
33
Revision 2‐5
Protection Template
 Protection templates are sets of defined schedules and retention limits

 Created apart from a volume
– Manage >> Protection >> Protection Templates
Creating a Volume
Demonstration
and Lab
34
Revision 2‐5
35
Revision 2‐5
Section 8: Connecting to Hosts
Section 8: Connecting to Hosts
Connecting the host
Initiator Target
iSCSI Portal: A targets IP and TCP port number pair (default – 3260)
Discovery: The process of an initiator asking a target portal for a list of it's
targets and then making those available for configuration
1
Revision 2‐5
iSCSI IQN
 iSCSI Qualified Name
iqn.2007-11.com.nimblestorage
iqn.2007-11.com.nimblestorage:training-vob104e23787e0f74.00000002.736e4164
1 2 3 4
1. Type – iqn or IEEE EUI-64 (eui)

2. Date – Year and month the naming authorities domain name was registered
3. Naming Authority – Domain name for this target
4. String – After the colon anything the naming authority wants to include
Connecting to Windows Hosts
2
Revision 2‐5
Windows iSCSI Target/Initiator
1. Open your iSCSI initiator management

tool and select the Discovery tab.
2. Click Discover Portal and add the
management IP address of the array
into the field. Click OK. The IP address
now appears in the list of targets.
3. Tab to Targets and click Refresh (in
the Discovered Targets area).
4. Select the volume to connect and click
Connect or Log In. If there are no
discovered entries in the list, type the
IP address or host name into the
Target field to discover them. Do not
select the control target. You may also
need to enter a CHAP secret or other
access record information.
Windows iSCSI Target/Initiator
5. On the dialog that is launched for connection, click the Advanced button to specify
physical port connections as described in Understanding IP addressing on page 29.
6. Select the adapter to connect with (usually Microsoft iSCSI adapter) and the target portal
IP address to use for the connection, then click OK.
7. Leave the volume selected for Add this connection to the list of Favorite targets if you
want the system to automatically try to reconnect if the connection fails, and select Enable
Multipath if the connection should use MPIO, then click OK.
8. Click OK to close the Initiator Properties dialog.
9. Move to the Disk Management area of your operating system to configure and map the
volume. Select Control Panel > Administrative Tools. Move to Computer Management
> Storage > Disk Management.
10. Right-click and initialize the new disk (volume). Important: Use the quick format option
when initializing a volume on Windows.
3
Revision 2‐5
Connecting to VMware Hosts
Integration Guides
 VMware Integration Toolkit Guide

 Hyper-V Integration Guide
 Nimble Storage Best Practices for Hyper-V
4
Revision 2‐5
VMware Networking Best Practices
 Do not use NIC Teaming on the iSCSI network

 VMware and Nimble recommendation
 Use 1:1 VMkernel to Physical ports

 Even Better - 1:1:1 VMkernel to vSwitch to Physical
 No additional steps to turn off NIC teaming in this case
 VMkernel ports must be bound to the iSCSI Initiator

 CLI command only in ESX 4.1 and in GUI for ESX 5
 esxcli swiscsi nic add -n <vmknic> -d <vmhba>

 VMware Multi-Path must be Round-Robin
 Ensure you hit the “Change” button when
setting
 Jumbo Frames (if used)

 Must be set everywhere (array,
VMware, switches)
 Every volume on array should be in an
Initiator Group
 With ESX must use multi-initiator with
the Nimble Volumes
 If you fail to do this and you have to
restart your ESX hosts one of your
hosts will become unusable due to
lack of access its VMDK
5
Revision 2‐5
VMware Networking Setup
 Only work with one Volume and one ESX host at a time
 Given a vSwitch with two Physical Adapters vmnic1 and 2
configure them for iSCSI use:
1. Select the ESX Host and click on the configuration tab
2. Click on Networking in the Navigation Pane
3. Use the “Add” button to create two VMkernel ports and enable for
iSCSI and vMotion and name them iSCSI0 and iSCSI1
4. Disable NIC teaming
5. Enable iSCSI SW initiator if not already done
6. Add VMkernel Ports to iSCSI using CLI command if you are working
with ESX 4.1 or with the vSphere GUI if using ESX 5
Verify the number of expected paths for each Volume
In the array navigate to:

Monitor >> Connections
1 row for each volume initiator combination
Verification Formula
( ESX Hosts * Physical Ports per Host * Array Data Ports )
( Count of subnets * Switches per Subnet )
Note:
2 switches with same VLAN/subnet trunked together is 1 switch
2 switches with same VLAN/subnet NOT trunked is 2 switches
6
Revision 2‐5
Verify the number of expected paths for each Volume
In vSphere navigate to:

Select host >> Configuration >> Storage Adapter >>
iSCSI Software Adapter >> click rescan
Verification Formula
( ESX Hosts (always 1) * Physical Ports per Host * Array Data Ports )
( Count of subnets * Switches per Subnet )
Note:
2 switches with same VLAN/subnet trunked together is 1 switch
2 switches with same VLAN/subnet NOT trunked is 2 switches
How many… Subnet 1
ESX Host 1 ESX Host 2

ESX hosts connected? 2
NIC1 NIC1 NIC1 NIC1
Physical ports per host? 2
Number of subnets? 1 Switch 1 Switch 2

Switches per subnet? 1
Eth Eth Eth Eth Eth Eth Eth Eth

Array data ports? 4 1 2 3 4 1 2 3 4
2X2X4 1 Volume
Expected paths? = 16
1X1
7
Revision 2‐5
How many… Subnet 1

Subnet 2

ESX hosts connected? 2
NIC1 NIC1 NIC1 NIC1
Physical ports per host? 2
Number of subnets? 2 Switch 1 Switch 2

Switches per subnet? 1

Array data ports? 4 1 2 3 4 1 2 3 4
2 X 2 X 4 16 1 Volume
Expected paths? = =8
2X1 2
What if… Subnet 1

Subnet 2
You lost a NIC, link or misconfigured the IP? ESX Host 1 ESX Host 2
• Where could you look to discover which
NIC1 NIC1 NIC1 NIC1
paths are missing?
• The two easiest points to check would
be the switches view of the links and
the arrays view of the links. Switch 1 Switch 2
What would your path count be in the

iSCSI software adapter screen? Eth Eth Eth Eth Eth Eth Eth Eth
1 2 3 4 1 2 3 4
How many paths should there be? 8 Controller A Controller B

How many paths are lost due to the failure? 2
1 Volume
= 6 Paths
8
Revision 2‐5
What if… Subnet 1

Subnet 2

You lost a Eth 2 or misconfigured the IP
what would your path count be in the NIC1 NIC1 NIC1 NIC1
iSCSI software adapter screen?
How many paths should there be? 8

How many paths are lost due to the failure? 2 Switch 1 Switch 2
= 6 Paths

1 2 3 4 1 2 3 4
1 Volume
 Additional Troubleshooting
 Verify physical connectivity (draw a picture)
 May want to use switch commands to print connected MAC addresses
and compare with MAC address of array ports.
 nic --list
 Verify VLAN/Subnets are correct on all ports
 Verify links are UP and IPs are correct on array
 Under GUI navigate to Manage >> Array
 ip --list
 Clear all appropriate iSCSI static connections in VMware
before all rescans
9
Revision 2‐5
Work on only one system at a time and check the following before moving to
another:
 Check src/dest IP addresses of all connections on array:
 GUI: Monitor::Connections
 CLI: vol --info <volume name>
 Check paths on VMware
 Storage Adapters
 iSCSI SW Initiator
 Right click on device and select Manage Paths
 Force a failover and check you still have the correct number of connections
 As Root on active controller
 ctrlr –-list – will display the active controller
 reboot –-controller A or B whichever is the active controller from above
Testing Performance
10
Revision 2‐5
Performance Metrics
 Measuring Random I/O
Random I/O
Random I/O
max IOPS max IOPS
– I/O’s per second (IOPS)
– Typically, 1K—100K IOPS
– Small I/O size; e.g., 4KB
– High QD; e.g., 16 I/O size QD
Seq throughput
Seq throughput
 Sequential throughput
– MBytes per second (MBps) max MBPS max MBPS
– Typically, 100—1000 MBps
– Large IO size; e.g., 256KB
– High QD; e.g., 16
I/O size QD
 Latency
– Milliseconds (ms)
latency
latency
– Typically, 0.1—10ms
– Random
min latency min latency
– Small IO size; e.g., 4KB
– QD = 1 I/O size QD
Tips on Measuring Throughput
 Use high QD.

– Enterprise workload has many outstanding I/Os.
– A file copy is NOT representative.
 For random I/O
– Use small I/O size (e.g., 4KB) to increase randomness.
» Be wary of high IOPS claims with 512B. Apps don’t go that low.
– Use IOPS, not MBps. MBps is unstable at small I/O sizes.
 For sequential throughput
– Use large I/O size (e.g., 256KB) to improve sequentially.
– Use MBPS, not IOPS, since IOPS is unstable at large I/O sizes.
 Value random throughput over sequential throughput.
– Most apps (Exchange, SQL, VMware) need random throughput.
– Legacy storage systems provide sufficient sequential throughput.
11
Revision 2‐5
Tips on Measuring Latency
 Use low QD.

– By definition: latency = QD / IOPS.
– Upper bound on IOPS implies latency rises unboundedly with QD.
 Focus on avg latency instead of max latency.
– Max latency meaningful only for few apps; e.g., trading.
Performance
IOmeter Nimble CS2xx – 1.3 Nimble CS2xx – 1.4 Nimble CS4xx – 1.4

4K RND write IOPS 14,113 15,278 (8%) 26,031 (84%)
4K RND read IOPS 15,924 18,193 (14%) 41,963 (164%)
 4K RND read/write IOPS
Random io on 100g volume, block-size=4k, q=64 / 14,256
Sequential io on 100g volume,15,111 (6%)
block-size=32k, q=32 / 32,342 (127%)
2x compressible data
256k Seq write throughput
(MB/S) 261 317 (21%) 556 (113%)
256k Seq read throughput
(MB/S) 557 735 (32%) 966 (74%)
 2 volume, 2 test files, each 30G / 4 threads, with 8 outstanding io for each thread
12
Revision 2‐5
DO NOT leave until you have…
 Updated the array to the current release – this should be done before
anything else
 AutoSupport enabled and confirmed that it is working with Nimble
support - test via button in GUI
 Confirmed the system heartbeat is working with Nimble Support
 Email alerts enabled and confirmed they are working with Nimble
Support- test email alert button
 Confirm VMware connection count
– Failover controllers and re-check
Section 9: Snapshots
13
Revision 2‐5
Snapshots
Snapped Data
New Data
(non-snapped)
27
Discussion: What are snapshots & how do they work?
What is a
COW?
14
Revision 2‐5
COW Snapshots
Changed Block
Snapped Data
New Data
(non-snapped)
Snapshot Reserve
29
Discussion: What are snapshots & how do they work?
What is a
ROW?
15
Revision 2‐5
ROW Snapshots
Changed Block
Snapped Data
New Data +
Changed Blocks
31
File and Snapshot Lifecycle 09:00
filename A B C D State of data at 09:00 filename 4-block file created

A B C D
16
Revision 2‐5
filename A B C D State of data at 09:00 filename 4-block file created

Snap10 A B C D 10:00 snapshot A B C D
10 snap!
filename A B’ C D State of data at 10:20 filename 4-block file created

Snap10 A B C D 10:00 snapshot A B C D B’
10 snap!
If block B is changed, the original
state can be recovered by rolling
back to the snap taken at 10:00
17
Revision 2‐5
filename A B’ C D State of data at 11:00 filename 4-block file created

Snap10 A B C D 10:00 snapshot A B C D B’
Snap11 A B’ C D 11:00 snapshot
10 10
11 snap!
The next snap taken captures the
change made to block B
filename A B’ C E filename 4-block file created

Snap10 A B C D 10:00 snapshot A B C D B’ E
Snap12 A B’ C D E 12:00 snapshot
Snap13 A B’ C D E 13:00 snapshot 10 10
11 12
10 10 14 snap!
13 10
Snap14 A B’ C D E 14:00 snapshot If block D is deleted, it can still be
recovered using the SNAP taken at
14:00.
18
Revision 2‐5
filename A B’ C E’’ filename 4-block file created

Snap10 A B C D 10:00 snapshot
A B C D B’ E E’ E’’

10 10
11 12
10 10
13 10
14 15 10
10 16 10
17
Snap14 A B’ C D E 14:00 snapshot Note that the snapshot pointers
never change, they are immutable.
Snap15 A B’ C E’ 15:00 snapshot (update E)
They disappear when the snapshot
Snap16 A B’ C E’’ 16:00 snapshot (update E 2X) reaches the end of its retention
cycle, at which point the snapshot is
Snap17 A B’ C E’’ 17:00 snapshot deleted along with its pointers.
8 8 39
Full Backups Actual Apparent
Blocks Stored Blocks Stored


10 10
11 12
10 10
13 10
14 15 10
10 16 10
17
Snap15 A B’ C E’ 15:00 snapshot (update E) Restoring is easy. If the deletion of

Block D signalled the start of a
Snap16 A B’ C E’’ 16:00 snapshot (update E 2X) sequence of unwanted events, then a
roll back to the snapshot taken at 14:00
Snap17 A B’ C E’’ 17:00 snapshot is required.
8 8 39
19
Revision 2‐5


10 10
11 12
10 10
13 10
14 15 10
10 16 10
17
Snap15 A B’ C E’ 15:00 snapshot (update E) By restoring just the pointers from the
14:00 snapshot to the active file (or
Snap16 A B’ C E’’ 16:00 snapshot (update E 2X) filesystem or LUN), the state file (or
filesystem or LUN) at 14:00 can be
Snap17 A B’ C E’’ 17:00 snapshot
restored almost instantly, without
8 8 39 having to move any data.
Snapshots
RARELY USED
Snapped
Volume Snapshot Reserve – An accounting
for a set amount of space that will be
guaranteed available for the
Snapshot Reserve snapshot.
Snapshot Quota – An accounting for
Snapshot Quota the total amount of space a snapshot
can consume.
20
Revision 2‐5
Zero Copy Clone
 Snapshots are ROW

– Snapped data is held as a single dataset
– New writes are directed to available space in the storage pool
 Zero copy clone
– Allows a volume to be created for online use based on a snapshot
– Any changed data is handled like a ROW snapshot
– Occupies no additional space until new data is written or changed
Best in Class Space Efficiency

Raw Capacity Usable Capacity (as % of raw) Effective Capacity (as % of raw)
8% 7.5%
Equallogic
52%
56%
68% 72%
75% 68%
60% 74%
80%
100% 4%
3% 144%
150%
Equallogic
44% NetApp
32% 25% NetApp
w/
w/ dedupe
dedupe
All vendors NetApp Equallogic Nimble
With
No snapshots
snapshots
Nimble Stores More Data Nimble

Nimble
Without any snapshots With 30 days of snapshots w/compression
w/compression
Effective Capacity (w/compression)
Usable (physical) Capacity
1.85x more than NetApp 1.92x more than NetApp
Snapshot space
Parity, Spares and Overheads
2.2x more than Equallogic 19x more than Equallogic
21
Revision 2‐5
Industry Leading Snapshot Efficiency
 Production Exchange 2010 Exchange 2010 Snapshot Usage

120
deployment 102.1
100
– 20GB Exchange Infostor
 Used DAG to mirror database 80
Usage (GB)
to Equallogic and Nimble 60
CS-Series 40
 Captured daily snapshots on 20

2.9
both systems for ten days 0
Nimble Leading Vendor
 Equallogic used 30x more
space than Nimble for the
same ten snapshots
Section 10: Replication
22
Revision 2‐5
Replication Overview
What is replication
and how does it
work?
Replication Overview
Network
Snapshot Replica
Partners
Replica
23
Revision 2‐5
How Replication Works
1. Create a replication partnership

Network
2. Define replication schedule

Snapshot 3. At first replication the entire
Replica volume is copied to the replica
partner
4. Subsequent replicas contain only
changes that have occurred
Replication Partner Notes
• Replication happens
on Management IP
• You can have many

replication partners
• You can pause

replication by partner
but NOT by Volume
Collection or schedule
24
Revision 2‐5
Replication QOS – Bandwidth Limit
• Support Multiple
QOS Policies
• Applies to Partner
• Can define a Global

QOS for all partners
– Under Manage
Replication Partner
Replication Schedules
• Replication configured
using Volume Collection
schedule attributes
• Different Schedules in
the same Collection must
replicate to the same
partner
• Calculate your change

rate and bandwidth – can
you get it all done??!!!
25
Revision 2‐5
One-to-One Replication
N
E
Single volume assigned to Multiple volumes assigned
T “Hourly” to “Daily”
W Replica of volume Replicas of volumes
assigned to “Hourly” assigned to “Daily”
O
R
K
Reciprocal Replication
N
E
T
W
O
R
K
26
Revision 2‐5
Many-to-One Replication
Volumes assigned Volumes assigned

to “SQL” to “Hourly”
N
E
Replica of volumes Replica of volumes
T
assigned to “SQL” assigned to “Hourly”
W
O Replica of volumes Replica of volumes
R assigned to “Outlook” assigned to “datastore1”
K
Volumes assigned to Volumes assigned to
“Outlook” “datastore1”
Replication (DR) Primitives
 Handover (Used for a planned outage)

– When you perform a handover of a volume or volume collection, the system:
» takes all associated volumes offline
» takes a snapshot of all associated volumes
» replicates these snapshots to a downstream replication partner
» transfers ownership of the volume collection to the partner
» brings the newly replicated volumes online
» reverses replication roles/direction
27
Revision 2‐5
Replication (DR) Primitives
 Promote (Used during unplanned outage)

– When you promote a downstream replication partner, the
system:
» Suspends the replication relationship associated with the volume collection.
» Creates a second (local) instance of the volume collection and assumes
ownership.
» Brings the most recently replicated snapshots online as volumes. The contents of
the newly available volumes are then consistent with the last replicated
snapshots.
 Demote
Replication Status
• Replication tab of
Volume Collection
• Can use “stats”

command on CLI
to view throughput
history
28
Revision 2‐5
29
Revision 2‐5
Section 11: Data Protection & DR
Recovery Scenarios
 Recovery from local snapshots

– Single volume, volume collection
– Replacing entire volume
 Testing my DR site without interrupting replication

– Use of clones
 Full Disaster Recovery
1
Revision 2‐5
Recovery Scenarios – Recovery from local snapshots
1. Clone the snapshot (creates a first-class volume)
Recovery Scenarios – Recovery from local snapshots
2. Add/adjust ACLs on the volume (host initiators)

3. Mount the volume
4. Copy the data or Storage vMotion the VM back to original
volume
5. Unmount the cloned volume
6. Delete the cloned volume
2
Revision 2‐5
Recovery Scenarios
 Restore to previous snapshot

1. Quiesce applications
2. Unmount the active volume(s) from the host(s)
3. Select the snapshot/snap-collection to restore
4. Click Restore
5. Mount the volume(s)
6. Start applications
Recovery Scenarios
 Testing my DR site without interrupting replication

1. Go to downstream replica
2. Clone the snapshot (create a first class volume)
3. Add/adjust ACLs on the volume
4. Mount the volume
5. Interrogate/Test the data and applications (via Windows,
ESX, etc.)
6. Unmount the volume
7. Delete the cloned volume
3
Revision 2‐5
Recovery Scenarios
 Full Disaster Recovery (Primary site is inaccessible)

– Failover to DR site
1. Promote downstream volume collections at DR site
2. Add/adjust ACLs on the volumes
3. Mount volumes to application servers (Windows/ESX)
4. Start production environment at DR site
Recovery Scenarios
 Full Disaster Recovery (Primary site is inaccessible)

– Failback to Primary site
1. Install new array and configure as downstream partner
2. Allow replication of volumes while still running at DR site
3. Gracefully shutdown apps at DR site
4. Perform Handover to primary site
5. Start production environment at primary site
4
Revision 2‐5
Nimble + CommVault = Comprehensive Backup

Management
 1st partner/charter member certified through the CommVault IntelliSnap

Connect Program
 Extensive integrations of hardware snapshots with SnapProtect
 Benefits:
– Centralized management of backups
– Support for long-term retention and compliance
– Integrated catalog across primary, secondary, and other forms of media
– Granular recoveries of snapshots, application objects, files, VMs, and volumes
– Frequent recovery points irrespective of the volume of data to be protected
– Rapid restores that require no data movement
– Support for app and VM consistent recoveries
speeding up application and VM restore times

Nimble + CommVault

5
Revision 2‐5
Application-Integrated Data Protection
NPM Protection for Windows Applications

Backup NPM leverages VSS (Volume Shadow-Copy Service)
Server
How it works:
1. NPM schedule triggers snapshot process
2. NPM agent talks to MS VSS service.
3. VSS tells Exchange to “quiesce” mail stores.
3 Mail stores
VSS 4. VSS tells NTFS to flush buffer cache.
4 NTFS
 5. VSS tells Nimble array to take a snapshot.
6. Nimble array captures near instant snapshots of all volumes
2 9 in collection.
1 7. Optional: NPM runs database verification on predefined
5 schedule to ensure consistency and truncates logs
8. NPM triggers WAN efficient replication on pre-defined
schedule
9 9. Optional: Existing backup software mounts snapshot for
Tape
6 weekly archive copy to tape
10. When needed, snapshots provide fast restore capability
snapshots
Improved protection with fast snapshots,
© 2012 Nimble Storage. Proprietary and confidential. Do not distribute. Efficient capacity and bandwidth utilization
6
Revision 2‐5
VMware Synchronized Snapshots
 Nimble OS can take a VM

application consistent
snapshot
– Define the vCenter host
– At first use you will also
need to provide:
» Username with
administrator access
» Password for the
administrator
VMware Synchronized Snapshots
 Return to the details page of the volume collection and click Validate to
ensure:
– username and password are correct
– user has the correct permissions
7
Revision 2‐5
SRM with Nimble Storage Replication Adapter
 VMware vCenter Site Recovery Manager (SRM)

– Host-based application that lets you set up disaster recovery plans for the
VMware environment before you need them.
– SRM is a vCenter plug-in that allows disaster recovery tasks to be managed inside
the same GUI tool as your other VM management tasks.
 SRM, when used with the Nimble Storage Replication Adapter, lets you create
and test a Nimble array-based DR recovery plan without having an impact on
your production environment.
 In DR scenarios, your Nimble CS-Series arrays keep your data protected and
replicated for immediate availability from the DR replication partner.
 Requires VMware vCenter Site Recovery Manager 4.1 and later
VMware SRM + Nimble Replication

Efficient DR Automation
Site A (Primary) Site B (Recovery)

 Nimble arrays support SRM v4.1 and v5.0
 Many new features in 5.0
VMware Site Recovery VMware Site Recovery
vCenter Server Manager vCenter Server Manager – Planned migration (vs. unplanned)
» Use-case: Disaster avoidance, datacenter
relocation
– Re-protection
VMware vSphere VMware vSphere » Use-case: After a successful failover, reverse
roles of active/replica sites
– Failback
» Use-case: For disaster recovery testing with live
Servers Servers
environments with genuine migrations – return to
their initial site
– Disaster Recovery Event
» An initial attempt will be made to shut down the
Storage Replication protection group’s VMs and establish a final
synchronization between sites
– Scalability (# of VMs, protection groups etc.)
8
Revision 2‐5
SRM - Planned Migration
 Ensures an orderly and

pretested transition from a
protected site to a recovery site
– Ensures systems are quiesced
– Ensures all data changes
have been replicated
– Will halt the workflow if an
error occurs allowing you to
evaluate and fix the error
– Start the virtual machines at
the recovery site
– Systems will be application
consistent
SRM - Reprotection
 Reverses replication Site A (Primary) Site B (Recovery)
– Ensures continued protection VMware Site Recovery VMware Site Recovery

vCenter Server Manager vCenter Server Manager
– For use after a recovery plan

or planned migration
» Selecting reprotect will
VMware vSphere VMware vSphere
establish synchronization and
attempt to replicate data back
to the primary site
Servers Servers
Storage Replication
9
Revision 2‐5
SRM - Failback
 Failback will run the same workflow used to

migrate the environment to the recovery site
– Will execute only if reprotection has successfully Failback cannot be
completed used with vSphere
– Failback ensures the following: Replication–protected
» All virtual machines that were initially migrated to the virtual machines.
recovery site will be moved back to the primary site.
» Environments that require that disaster recovery
testing be done with live environments with genuine
migrations can be returned to their initial site.
» Simplified recovery processes will enable a return to
standard operations after a failure.
» Failover can be done in case of disaster or in case of
planned migration.
SRM - Disaster Recovery Event
 When a disaster even occurs SRM will:

– attempt to shut down the protection groups VMs
– Attempt to establish a final synchronization between sites
 Designed to ensure that VMs are static and quiescent before running the
recovery plan
 If the protected site is not available the recovery plan will run to
completion even if errors are encountered

10
Revision 2‐5
vStorage APIs for Array Integration (VAAI)
 vStorage APIs for Array Integration (VAAI)

– VAAI is a feature in the Nimble OS and supports:
» Zero Blocks/Write Same primitive (fundamental operation)
hardware acceleration feature
» Hardware Assisted Locking
» SCSI Unmap
VAAI – Write Same
 When performing a deletion VMware I/O operations like VM creation,

cloning, backup, snapshots, and VMotion require that data be zeroed via
a process called “block zeroing”
 The write same API speeds up this process by:
– moving a large block of zeros and executing repeated writes on the array
 The array handles this operation rather than having the host send
repetitive commands to the array
11
Revision 2‐5
VAAI Support - Hardware Assisted Locking
 Hardware Assisted Locking (ATS)

– Enabled if array supports it
– Allows an ESX server to offload the management of locking to the storage hardware
» avoids locking the entire VMFS file system
– Without ATS
» a number of VMFS operations cause the file system to temporarily become locked for
exclusive write use by an ESX node. These can include:
– Creating a new VM or template
– Powering ON/OFF a VM
– Creating deleting a file or snapshot
– Moving VMs via vMotion
VAAI Support - SCSI Unmap
 SCSI Unmap (space reclamation) –

– vSphere 5.0 introduced VAAI Thin Provisioning Block Space Reclamation Primitive (UNMAP) designed to efficiently
reclaim deleted space to meet continuing storage needs. In ESX5.1 it is enabled by default.
Before SCSI Unmap With SCSI Unmap

Host deletes data… Host deletes data…
• Array doesn’t • Array understands
understand that data and releases the
is no longer relevant. space.
• Data remains on • Space is now
array consuming reclaimed and now
space. useable.
Object-1 erased,
Object-1 erased, server storage understands
does not inform storage and releases space
that space can be used
released KB from VMware regarding SCSI Unmap:
http://kb.vmware.com/selfservice/microsites/
search.do?language=en_US&cmd=displayK
C&externalId=1021976
12
Revision 2‐5
Nimble vCenter Plugin
 The Nimble vCenter Plugin works with vCenter to:

– clone datastores and snapshots
– resize datastores
– edit protection schedules
– take snapshots and set them on/offline
– restore from a snapshot
– delete snapshots
The vCenter plugin requires ESX 4.1 or later

Registering the plugin
The plugin is not

supported for multiple
datastores located on
one LUN, one datastore
spanning multiple LUNs,
or if the LUN is located
on a non-Nimble array.
vmwplugin --register --username <username> --password

CLI
<password> --server <server_hostname-address-or-URL>
Review the Vmware Integration Guide 1.2 - Using Nimble's vCenter plugin, for details on
using this plugin
13
Revision 2‐5
Section 12: Maintenance
Hardware FRUs
 Disk Drives (SSD and HDD)
 Controllers
 Power Supplies
All are non-disruptive replacements
14
Revision 2‐5
Replacing a Drive
 Indications that a drive has failed:

– The LED is solid red or off (while others are on)
– An event appears in the Event list
– You received as alert email from the array
Only replace a disk drive with another disk drive, or

an SSD with another SSD. Do not replace an SSD
with a disk drive.
Replacing a Drive
1 2
3 4
15
Revision 2‐5
Replacing a Controller
 Indications that a controller has failed:

– If the controller was in the active role, the LEDs indicate that no activity is
occurring
– The power LED indicator is off
– An even appears in the Event list
– You received an alert email from the array
Replacing a Controller
1 2
3 4
16
Revision 2‐5
Replacing a Power Supply
 The following indicate that a power supply has failed:

– The LED on the power supply is red or unlit
– An event appears in the Event list in the hardware category

– An email has been received from the array indicating a power supply failure
Replacing a Power Supply
17
Revision 2‐5
Upgrade firmware
 Before you begin:

– Check your current version
– Obtain the most recent version
– Check system health
Industry Leading Supportability Features
One-click, zero-downtime software upgrades

 Before you begin:
– Check your current version
– Obtain the most recent version
– Check system health
18
Revision 2‐5
Firmware Upgrade Process
1. Load new firmware to standby

Firmware 2. Reboot standby to run new rev.
3. Load new firmware to other controller
4. Reboot active to activate new rev. –
causes failover and the standby
becomes active
Active
Standby Standby
Active
Failover
Section 13: Support
19
Revision 2‐5
Real-time monitoring & alerts
Instant Email Alerting

 Easy and flexible setup
 System alerts to Nimble Support and the
customer
– All warning level and alerts
– Proactive disk monitoring
– System utilization
– Protection (Snapshot) monitoring
From: <customer>
Sent: Monday, June 20, 2011 2:03 PM
 Automated case created To: alerts
Subject: Nimble Alert on <array name> - WARNING: System temperature is warm
 Nimble support proactively Time: Mon Jun 20 14:03:01 2011
contacts customer
Type: 12323
Id: 76501
 Ability to ‘remote’ in for real- Message: current temperature for sensor bp-temp1 at backplane left side is 44 C
time troubleshooting
Array name: <name>
Serial: <AA..>
Version: 1.0.6.1-9650-opt
Proactive Wellness
Accelerate Protect Empower
Customer Site All Customer Sites
Comprehensive
Telemetry Data
5-minute heartbeats
Nimble
Support
Real-time analysis of over

150,000 heartbeats per day

20
Revision 2‐5
Proactive Wellness
Customer Site Opportunities to All Customer Sites

free up space
Connectivity and
health checks before
software upgrades
Replication 5-minute heartbeats

Nimble
conformance
to RPO
Support Proactive wellness,
Automated case creation
Alerts for MPIO

unprotected Misconfiguration
volumes Warnings

Proactive Wellness
Customer Site All Customer Sites
5-minute heartbeats
Nimble
Support Proactive wellness,
Automated case creation
Secure remote support

21
Revision 2‐5
Real-time monitoring & alerts

Heartbeat Monitoring
 Monitored systems send back a heartbeat every 5 minutes!
 Nimble support monitors and reacts to heartbeat info collected:
– System state (critical, error and warning counts)
– Capacity information (space usage and compression)
– Performance (R/W sequential and IOPs)
Etc.......
Daily Health Check

Daily Auto-Support
 Detailed system telemetry sent back every 24 hours
– Analysis and resolution of all log events and alerts
– Proactive disk monitoring, System performance and utilization analysis
 Comprehensive automated daily health check
– Alerts Support Engineers to issues that can be proactively resolved before
they affect business critical operations
 Daily review of all

incidents (auto and
human initiated)
22
Revision 2‐5
Secure Remote Support
VPN Tunnel
 Ability to establish secure VPN tunnel from customer array to Nimble
Support to enable real-time diagnostics and problem resolution
 Ability to download and run custom corrective scripts to prevent
conditions that may result in downtime
 Customer has full control over enabling/disabling the tunnel
Can you
please open Nimble Support uses a
the tunnel? DSA public key to
connect to the array
Customer authorizes and

WAN enables VPN tunnel which
gives Nimble Support remote
access for troubleshooting
Comprehensive Analysis and Reporting
System Data Collected
Array status
Array alerts
Performance policies
Protection templates
Snapshots
23
Revision 2‐5
Internal Analysis and Reporting
Case Tracking History

 Look back on all ‘Critical’ & ‘Non-critical’
support cases
 Track issues and trends over <n> days for
ALL customer cases
Allows Nimble to track availability

statistics and dynamically assess
quality metrics
Support Process
 Support Process
– Call 1-877-3NIMBLE (US, Canada only) or +1 408-636-6347 worldwide
– Email support@nimblestorage.com
 Coverage
– Telephone Support: 24x7
– Email support: 8:00AM-6:00PM PST M-F
– Support website: 24x7
– Engineering Escalation: 24x7 on-call availability
 SLA
– P1 telephone response less than 30 minutes (current average is <5min)
» Immediate escalation to Engineering.
– P2 response less than 4 business hours
– P3 response less than 8 business hours
 Severity Definitions
– P1 – Not serving data; severe performance degradation; single controller not operational
– P2 – Performance degradation, intermittent SW faults, network degradation
– P3 – Problem or defect causing minimal business impact; request for information
24
Revision 2‐5
Escalation Process
 Escalation Process
– Call Nimble Support for 24x7 response
– Indicate nature of issue and business impact. Request immediate escalation as desired.
– Escalation team, account team and executive management notified immediately
following problem description and Support case creation.
– Immediate diagnosis via remote diagnostics tunnel and or WebEx with customer by
support team and engineering resource as required.
– Case worked until resolution.
Parts Delivery
 NBD Parts
– North America
» Part must be identified by 3PM PST
– Europe
» Part must be identified by 3PM CET
 4 Hour Parts
– We partner with a global logistics provider with over 700+ worldwide depots, and as
we expand, we are constantly adding new ones
» Contact your sales team to locate your nearest depot
25
Revision 2‐5
Taking the Exam
© 2012 Nimble Storage, Inc.

51
Taking the Exam
 Your are encouraged to use your notes and course book/PDF

 Do not discuss answers with other students
 You have 1 hour 30 minutes to complete
 You will be given two chances to answer each question correctly (with
the exception of true/false)
 You must score a 75% or higher to pass

26
Revision 2‐5
Log in to Nimble University
University.nimblestorage.com
Upon Completion
1. Have your instructor to document your outcome

– If taking the exam from home – take a screen capture for your records
2. Close the exam window
– IMPORTANT: do not close the course portal window
3. The course portal will display:
4. DO NOT close the course portal window until the following is displayed:

27
Revision 2‐5
28

NTSP Student Handout Rev2-5c

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

NTSP Student Handout Rev2-5c

Transféré par

Droits d'auteur :

Formats disponibles

Nimble Technical Sales

Checking Your Enrollment

5. If your screen looks different, ask your instructor for instructions 1

Nimble Technical Sales

Insert Instructor Name

© 2012 Nimble Storage, Inc.

In this course the following subjects will be discussed:

 Section 1: CS-Series Array  Section 8: Connecting to Hosts

Section 1: CS-Series Array Introduction

Raw versus Usable versus Effective Capacity

Nimble Storage CS210 At a Glance

Model CPU DDR3 Ethernet Cache Data Effective Capacity

Capacity Expansion Add up to 1 additional shelf

Scaling Performance NOT SUPPORTED

Nimble Storage CS220 at a Glance

Capacity Expansion Add up to 3 additional shelf

Scaling Performance Scale Compute and Cache

Nimble Storage CS240 at a Glance

Capacity Expansion Add up to 3 additional shelf

Scaling Performance Scale Compute and Cache

Nimble Storage CS260 at a Glance

Capacity Expansion Add up to 3 additional shelf

Scaling Performance Scale Compute and Cache

Nimble Storage CS420 at a Glance

Capacity Expansion Add up to 3 additional shelf

Scaling Performance Scale Compute and Cache

Hardware Tour - Front

Hardware Tour Controller Unit– Front

HDD blank SSD blank HDD

Hardware Tour Controller Unit– Front

HDD SSD HDD

CS220 and higher

When replacing a drive, ensure you replace drives with

Nimble ES-Series External Storage Shelf

 Connect up to three additional shelves

Hardware Tour – Front

Hardware Tour - Back

 Power supplies – 2X 100-240V, 50-60Hz, 4-10 Amp

Section 2: Scale to Fit

Nimble Scaling for Mainstream Applications

Nimble Scaling for Mainstream Applications

Archival, Cheap and Deep

Scale Capacity by Adding Disk Shelves

 Add capacity non-disruptively by

Scale Capacity Cabling

4 Lane 6Gb cable

1. Check to see if you have proper Nimble OS version

© 2012 Nimble Storage. Proprietary and 26

Scale Capacity Storage Pool

 Storage pool grows when an expansion shelf is activated.

© 2012 Nimble Storage. Proprietary and 28

Expanding Existing System

50% Capacity 50% Capacity

Expansion Shelf Controller Shelf

© 2012 Nimble Storage. Proprietary and 29

Managing Internal Storage and Expansion Shelves

Power On/Off Order

© 2012 Nimble Storage. Proprietary and 31

Scale Compute – The 400 Series Controllers

Nimble Array Scaling Compute

 Upgrading a CS220 to a CS420-X2 or -X4

 Before you start:

– Note your current SSD size