Solaris 10 and

Brocade SAN Setup

This is some information/walkthrough of getting Solaris 10 hooked up to a SAN.
Also info on SUN/STK NAS devices. I think the SUN/STK NAS’s are way nicer
then NetApps. Also covers SUN 6000 series array setup. A few Brocade notes
also. Covers UFS and Veritas vxvm and vxfs setup for new LUNs. This is a
walkthrough/notes I created while setting up some equipment, enjoy. I am
seeing a good deal of visits to this page, and I am glad it is of service.

Some info about author of this stuff. Drop me Questions at main site:

Creative Intent!
Click to read: Creative Intent!
Get Creative and Have Fun!!!

Brocade SilkWorm 4100 SAN Switch................................................................................................2

Documentation for Brocade Silkworm 4100 Switch.......................................................................2
Setting the IP Address.....................................................................................................................3
To set/change the Domain ID..........................................................................................................3
To set/change switchname...............................................................................................................4
Setting the Date and Time...............................................................................................................4
To synchronize local time with an external source......................................................................4
To set the date and time manually...............................................................................................4
SUN 6130 Array..................................................................................................................................5
SUN 5320 NAS...................................................................................................................................5
Documentation for SUN 5320 NAS................................................................................................5
Ensure the following OS packages/patches are installed.................................................................5
Patch install order........................................................................................................................5
Mounting NAS.................................................................................................................................6
From Solaris.................................................................................................................................6
From Windows............................................................................................................................6
L500 Tape Library...............................................................................................................................7
Documentation for L500 Tape Library............................................................................................7
SUN/Solaris.........................................................................................................................................7
Ensure the following OS packages/patches are installed.................................................................7
View HBA’s....................................................................................................................................7
Before assigning LUN’s..................................................................................................................8
Assigning LUN’s.............................................................................................................................8
Configure New LUN’s on Solaris...................................................................................................8
Create Label for LUN......................................................................................................................9
Run format command..................................................................................................................9
 Get LUN Info for Mounting..........................................................................................................10
luxadm probe.............................................................................................................................10
Create New Filesystem on LUN/Volume......................................................................................10
run newfs...................................................................................................................................10
edit /etc/vfstab to mount the new LUN..........................................................................................11
Run mount command.................................................................................................................11
Veritas Volume Setup....................................................................................................................12
If you use Veritas follow these steps.........................................................................................12
# instruct veritas to scan for new luns........................................................................................12
# check for new luns on veritas level.........................................................................................12
# initialize new disks.................................................................................................................12
# initialize new disk group with disk c3t3d30...........................................................................12
## if group already exists...........................................................................................................12
# check the status of the new disks............................................................................................12
# Check freespace of the diskgroup...........................................................................................13
## Size can be gotten from vxdg free........................................................................................13
## If not all space is used, or task is to grow volume................................................................13
# check the new volume.............................................................................................................13
# create a filesystem on the new volume...................................................................................13
# make a mount point................................................................................................................13
# mount the new filesystem at the new mount point.................................................................13
# verify the new mounted filesystem.........................................................................................13
# verify the size on solaris level.................................................................................................13
# make permenent......................................................................................................................14
# test vfstab entry.......................................................................................................................14
Creative Intent Click to read: Introduction.....................................................................................14

Brocade SilkWorm 4100 SAN Switch

####################################################################
############

Documentation for Brocade Silkworm 4100 Switch

All Brocade documentation downloaded to: \\srvapps02\Gtri
employee's\Emikulka\Kuwait\Brocade

Tasks:
Assign IP
Change Domain ID
Change Hostname
Setting the Date and Time

A new switch (before IP) will need to be configured through serial port.

Default username and password on new Brocade switch is:

username: admin
password: password

Domain ID must be different on each switch.

Default Domain ID is 1.
One of the switches will need a different Domain ID.

The current setup in non-classified:

J2_SANSW01
IP: 192.16.50.5
Domain: 3

J2_SANSW02
IP: 192.16.50.6
Domain: 1

# Note: Brocade will not allow a hostname longer then above.

A "switchShow" command will show config of switch

############################

Setting the IP Address

After connecting with serial cable, use the "ipaddrset" command to set the IP address.

(example)
J2_SANSW01:admin> ipaddrset 172.16.50.5
Ethernet IP Address [172.16.50.5]:
Ethernet Subnetmask [255.255.255.0]:
Fibre Channel IP Address [0.0.0.0]:
Fibre Channel Subnetmask [0.0.0.0]:
Gateway IP Address [172.16.50.1]:
Issuing gratuitous ARP...Done.
IP address is being changed...Done.
Committing configuration...Done.

Just hit enter if correct.

Not using fibre Channel IP setup in this configuration. Leave blank.

Now the switch can be telneted to.

############################

To set/change the Domain ID

1. Connect to the switch and log in as admin.

2. Enter the switchdisable command to disable the switch.

3. Enter the "configure" command.

4. Enter y after the Fabric Parameters prompt:

Fabric parameters (yes, y, no, n): [no] y

5. Enter a unique domain ID at the Domain prompt. Use a domain ID value from 1
through 239 for
normal operating mode (FCSW compatible). For example:

Domain: (1..239) [1] 3

6. Respond to the remaining prompts (or press Ctrl-d to accept the other settings and
exit).

7. Enter the "switchenable" command to reenable the switch.

############################

To set/change switchname
Enter the "switchname" command at the command line, using the following syntax:

switchname “newname”

Where "newname" is the new name for the switch.

(example)
J2_SANSW01:admin> switchname J2_SANSW01

############################

Setting the Date and Time

To synchronize local time with an external source

1. Connect to the switch and log in as admin.
2. Enter the following command:
tsclockserver ipaddr

(example)
J2_SANSW01:admin> tsclockserver 172.16.10.1
Updating Clock Server configuration...done.

To set the date and time manually

1. Connect to the switch and log in as admin.
2. Enter the date command at the command line using the following syntax:

date “MMDDhhmmYY”

The values represent the following:

• MM is the month; valid values are 01 through 12.
• DD is the date; valid values are 01 through 31.
• hh is the hour; valid values are 00 through 23.
• mm is minutes; valid values are 00 through 59.
• YY is the year; valid values are 00 through 99 (values greater than 69 are
interpreted as
1970 through 1999, and values less than 70 are interpreted as 2000-2069).
Example
You can synchronize the local time
SUN 6130 Array
All SUN documentation downloaded to: \\srvapps02\Gtri
employee's\Emikulka\Kuwait\StorEdge 6130

SUN 5320 NAS

Documentation for SUN 5320 NAS
All SUN documentation downloaded to: \\srvapps02\Gtri
employee's\Emikulka\Kuwait\Sun StorageTek 5320 NAS

Ensure the following OS packages/patches are installed

TABLE 5-2       Required Sun StorageTek 6130 Array Software and

Firmware

Version
Software (minimum) Patch ID

Sun StorageTek 6130 array 1.3 118164-

management software 06

Controller CRM-F firmware 06.12.09.10 117856-

18

Array firmware installer   118185-

14

Patch install order

The patches for the NAS need to be installed in a specific order. Also note that newer
versions of some of these patches are available and were installed. Those patch #’s
are noted below.

These have been downloaded to: \\srvapps02\Gtri

employee's\Emikulka\Kuwait\Patches needed for NAS

Install order is:

118164-06
118185-15
117856-19

118164-06
118164-06 can be confusing.
In the Readme file the Problem Description is
“Change the PTX Host Type to a different value:Sun Storedge NAS Gateway”
The Host Type for the NAS should be selected as PTX. The patch does not change
that value, in the pulldown to Sun Storedge Gateway. The pull down menu stays PTX.
The patch changes some parameters to that value but pulldown stays the same.

118185-15
This install is straight forward. This installs a firmware installer for the 6130 Array. This
will be used to upgrade to latest microcode installed in 117856-19.

117856-19
This is the latest firmware. After this patch is installed the firmware must be installed.
This is covered in the README file. The firmware is install by:
cd /var/sadm/swimages/118185-15/bin
./install
The README file covers this in detail.
It can take a few minutes to get a response during different steps of upgrade.

Mounting NAS
In this configuration the NAS allows shared filesystems on the 6130 to be shared over
IP network.
The Solaris servers ues NFS.
The Windows servers mount shares on NAS as shared folders.

From Solaris
The Solaris servers are NFS clients to the NAS server.

First make sure the NAS is mountable manually.

Steps needed to mount NAS filesystems manually

make local directory (example) /mnt/admin
mount j2-nas01:/admin /mnt/admin

Steps to make NFS filesystems automount at bootup

To make this mount permanent, you must edit the /etc/vfstab for each filesystem.

Edit /etc/vfstab: (j2-nas01:/admin - /mnt/admin nfs - yes rw)

The parameters in /etc/vfstab are listed

#device         device          mount           FS      fsck    mount   mount
#to mount       to fsck         point           type    pass    at boot options

A successful entry consists of the following

j2-nas01:/admin - /mnt/admin nfs - yes rw

From Windows

Steps needed to mount NAS filesystems

L500 Tape Library

Documentation for L500 Tape Library

http://sunsolve.sun.com/handbook_pub/Systems/L500/L500.html

SUN/Solaris
####################################################################
############

Ensure the following OS packages/patches are installed

This one should be on the
first Solaris install DVD

 SUNWqlc

These two dowload from SUN if needed

 120965-01
 119131-13

# Note: If HBA's are in server when Solaris is installed

        these should all have been installed

############################

View HBA’s
Once the OS is updated and a reboot -- -r has been done check if SUN/Solaris is
seeing the HBA's.  

run the luxadm command:

luxadm -e port

(example)
luxadm -e port
/devices/pci@1d,700000/SUNW,qlc@2,1/fp@0,0:devctl                  CONNECTED
/devices/pci@1d,700000/SUNW,qlc@2/fp@0,0:devctl                    CONNECTED

Above is showing two HBA's 2 and 2,1

OK, good to go.

############################

Before assigning LUN’s

Collect Current HBA information
It is good to collect this data for diff later

fcinfo hba-port -l |grep HBA

HBA Port WWN: 210000e08b1c829a
HBA Port WWN: 210000e08b1c2395

Collect LUNs Solaris already knows about

fcinfo remote-port -sl -p 210000e08b0c5518 > 210000e08b0c5518.out
fcinfo remote-port -sl -p 210100e08b2c5518 > 210100e08b2c5518.out

cfgadm -al -o show_SCSI_LUN > currentLUNs.out

(Note added LUNS when available will be seen here as "unconfigured"

Scan deeply LUNs attached to each HBA

This shows HBA’s
luxadm -e port
/devices/pci@1c,600000/pci@1/SUNW,qlc@4/fp@0,0:devctl              CONNECTED
/devices/pci@1c,600000/pci@1/SUNW,qlc@5/fp@0,0:devctl              CONNECTED

Run following command on each Controller

luxadm -e dump_map /devices/pci@1c,600000/pci@1/SUNW,qlc@4/fp@0,0:devctl
luxadm -e dump_map /devices/pci@1c,600000/pci@1/SUNW,qlc@5/fp@0,0:devctl

Assigning LUN’s
Now is the time to assign LUNs
Assign LUNs.

Configure New LUN’s on Solaris

LUNS should “Just Show Up” on Solaris 10.

Cgfadm –al
Example output
c1::2200000c50401277 disk connected unconfigured unknown

New LUNs show as unconfigured until cfgadm is used.

When LUNs appear configure them

cfgadm -c c1::2200000c50401277
Also command can be done globally for each controller:
cfgadm -c configure c1
cfgadm -c configure c2
It does not effect previously configured LUNs.

If they do not, a few things to try if LUNs don’t show up

Check for legacy txt in sd.conf. Solaris 10 does not need this, and it just slows up
booting.
Txt in this file “my theory is” can mess finding new LUNs in Solaris 10.
Update sd.conf
vi /kernel/drv/sd.conf
Add new LUN IDs created on Hitachi
After Solaris 9, this should not be needed. 10 does not need this.

Instruct Solaris to re-read sd.conf

update_drv -f sd
9 and 10 can do this. 8 will failed.

scan scsi bus so Solaris can see the new luns

devfsadm

Find new LUNs

cfgadm -al -o show_SCSI_LUN
(Note added LUNS when available will be seen here as "unconfigured"

If LUNs do not appear, server reboot will be needed.

reboot -r

############################

Create Label for LUN

This is all only valid if you Are Not running Veritas

Now run a format command

# Note: the disk only needs to be labeled on once. The following servers only need
mount it

Run format command

format

A disk looking like the following should show up.

c6t600A0B800021E8B90000536B456B26B3d0 <SUN-CSM100_R_FC-0612 cyl
32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g600a0b800021e8b90000536b456b26b3

Select the disk number. It will need a volume name if it does not already have one.
Since this is the Export directory for Solaris "export" is a good volume name. Solaris
does
not like disks without a volume name (label).

Some commands in format to look at are:

format> current
Current Disk = c6t600A0B800021E8B90000536B456B26B3d0
<SUN-CSM100_R_FC-0612 cyl 32766 alt 2 hd 512 sec 64>
/scsi_vhci/ssd@g600a0b800021e8b90000536b456b26b3
            (SUN and Solaris see this as a known disktype)

format> type
select type 19

format> volname
Enter 8-character volume name (remember quotes)[""]:"export"
Ready to label disk, continue? y

format> save
Saving new disk and partition definitions
Enter file name["./format.dat"]:

format> quit

This only needs to be done from one machine

############################

Get LUN Info for Mounting

run the luxadm command to get LUN info

luxadm probe

(example)
root@j2-apps01 # luxadm probe
No Network Array enclosures found in /dev/es

Found Fibre Channel device(s):

  Node WWN:200400a0b821eab1  Device Type:Disk device
    Logical Path:/dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2

Each server will have a different Path, to the same LUN

############################

Create New Filesystem on LUN/Volume

Now create a new filesystem (format)

run newfs command on LUN found in luxadm probe command:

newfs /dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2

(example)
root@j2-apps01 # newfs /dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2
newfs: construct a new filesystem
/dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2: (y/n)? y
/dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2:      1073676288 sectors
in 32766 cylinders of 512 tracks, 64 sectors
        524256.0MB in 10922 cyl groups (3 c/g, 48.00MB/g, 5824 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
 32, 98400, 196768, 295136, 393504, 491872, 590240, 688608, 786976, 885344,
Initializing cylinder groups:
...............................................................................
...............................................................................
............................................................
super-block backups for last 10 cylinder groups at:
 1072703520, 1072801888, 1072900256, 1072998624, 1073096992, 1073195360,
 1073293728, 1073392096, 1073490464, 1073588832,

Now the disk is mountable and solaris can understand it

This only needs to be done from one machine

A LUN is a physical disk to Solaris at this point

############################

edit /etc/vfstab to mount the new LUN

Now edit /etc/vfstab to mount the new LUN

The new LUN/disk in this example (from luxadm) is

c6t600A0B800021E8B90000536B456B26B3d0s2

The device path is:

/dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2

Add the folling line to /etc/vfstab:

/dev/dsk/c6t600A0B800021E8B90000536B456B26B3d0s2       
/dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2       /export/home     ufs    
1       yes      logging

Edit this to match LUN ID of your system

This will mount at boot

############################

Run mount command to test if LUN mounts:

mount /export/home

There is no output if it works

Run the mount command again to see new LUN and mount point

mount

(example)

root@j2-apps01 # mount |grep export

/export/home on /dev/dsk/c6t600A0B800021E8B90000536B456B26B3d0s2
read/write/setuid/devices/intr/largefiles/logging/xattr/onerror=panic/dev=1d80022 on
Tue Nov 28 15:33:06 2006

############################
Veritas Volume Setup

If you use Veritas follow these steps

# instruct veritas to scan for new luns

vxdctl enable

# check for new luns on veritas level

vxdisk -o alldgs list

c7t2d11s2 auto:none - - online

invalid
c7t1d12s2 auto:none - - online
invalid

# initialize new disks

/etc/vx/bin/vxdisksetup -i c7t2d11

# initialize new disk group with disk c3t3d30

vxdg init oraclelogs c7t2d11=c7t2d11

## if group already exists

vxdg -g oraclelogs adddisk c7t1d12=c7t1d12

# check the status of the new disks

vxdisk -o alldgs list

# Check freespace of the diskgroup

# vxdg free

## Size can be gotten from vxdg free

# make a volume of max size (41853696)
vxassist -g oraclelogs make oralogvol01 41853696

## If not all space is used, or task is to grow volume

## After making 19g check free space of new Volume
# vxassist -g oraclelogs maxgrow oralogvol01 Volume oralogvol01
can be extended by 2007040 to: 41852928 (20436Mb) # growto
space available vxassist -g oraclelogs growto oralogvol01
41852928
# check the new volume
vxprint -htr

# create a filesystem on the new volume

mkfs -F vxfs /dev/vx/rdsk/oraclelogs/oralogvol01
mkfs -F vxfs /dev/vx/rdsk/oraclelogs/oralogvol02

# make a mount point

mkdir /oralog01

# mount the new filesystem at the new mount point

mount -F vxfs /dev/vx/dsk/oraclelogs/oralogvol01 /oralog01

# verify the new mounted filesystem

cd /oralog01
ls

# verify the size on solaris level

df -h

# make permenent
vi /etc/vfstab
/dev/vx/dsk/oraclelogs/oralogvol01
/dev/vx/rdsk/oraclelogs/oralogvol01 /oralog01 vxfs 2 yes suid

# test vfstab entry

# umount /oralog01
# mount /oralog01

Support the info checkout my book:

Creative Intent
Click to read: Introduction