Procedure to add a hard disk and to build the ZFS mirroring to a SUN station T5120 with only one

hard disk installed.

Rev. No. Release Date Responsible Approved by Description of Changes

V01 V02 V03 V04

29/01/2013 04/02/2013 12.01.2013 12.01.2013

Julian Manole Julian Manole Julian Manole Julian Manole

Initial release Adding command in 3.c Adding the disk labeling, adding clarifications. Correction rootmirror in paragraph 6.

1. General a. Scope: The procedure below contains the steps for installing a second hard disk to a SUN station T5120 that runs the EOPS application with only one hard disk. b. The procedure has four steps: i. Physical addition of the new hard disk ii. Partitioning the new hard disk for allowing the ZFS mirroring iii. Building the ZFS mirroring iv. Setting the environment parameters in order to allow the boot from the second disk in case when the first disk is removed. Important notes: The first three steps may be implemented while the application is running. The forth step requests the server stopping. Therefore a maintenance window is required. The procedure must be implemented through the ILOM. In the examples inserted in this document the commands are in red fonts.

2. Initial conditions a. The procedure implementation is possible only if the SOLARIS and OSC installation have been performed in accordance to the specific ECI Installation procedure. b. The procedure is possible only if there is no raid volume previously installed in the server. c. The procedure includes the commands to be done for assuring the two conditions above. If one of the conditions below is not accomplished the procedure cannot be implemented (Call the ECI Support for checking for a solution).

d. In order to prevent any situation of losing the information please perform a database backup and export the CIT information. Also, please record all the internal routes (with the command SetRoute) and the IP addresses (with the command ConfNet). 3. Step # 1: Physical addition of the new HD a. Before inserting the new HD do the following steps: Login to the ILOM and open the serial console to the server Login as root (recommends to type bash after the logging in) Retrieve the existent hard disk it should be as presented below:

bash-3.00# format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c1t0d0 <LSILOGIC-LogicalVolume-3000 cyl 65533 alt 2 hd 16 sec 273> disk0001 /pci@0/pci@0/pci@2/scsi@0/sd@0,0 Specify disk (enter its number):

Note: In this example the disk volume had been previously named (see highlighted in Green). It is not mandatory, but recommended. If the disk has no name you may do the volume naming later (it will be detailed in a further step). Chose the disk 0 and type two times p to get the partitioning see below (from this step well not show the whole printed information, only the partitioning itself).
Specify disk (enter its number):0 selecting c1t0d0: disk0001 [disk formatted] /dev/dsk/c1t0d0s0 is part of active ZFS pool rpool. Please see zpool(1M). /dev/dsk/c1t0d0s2 is part of active ZFS pool rpool. Please see zpool(1M). FORMAT MENU: disk - select a disk type - select (define) a disk type partition - select (define) a partition table current - describe the current disk format - format and analyze the disk repair - repair a defective sector label - write label to the disk analyze - surface analysis defect - defect list management backup - search for backup labels verify - read and display labels save - save new disk/partition definitions

inquiry - show vendor, product and revision volname - set 8-character volume name !<cmd> - execute <cmd>, then return Quit format> p PARTITION MENU: 0 - change `0' partition 1 - change `1' partition 2 - change `2' partition 3 - change `3' partition 4 - change `4' partition 5 - change `5' partition 6 - change `6' partition 7 - change `7' partition select - select a predefined table modify - modify a predefined partition table name - name the current table print - display the current table label - write partition map and label to the disk !<cmd> - execute <cmd>, then return quit partition> p Volume: disk0001 Current partition table (original): Total disk cylinders available: 65533 + 2 (reserved cylinders) Part Tag Flag Cylinders Size Blocks 0 root wm 0 - 65532 136.49GB (65533/0/0) 286248144 1 unassigned wm 0 0 (0/0/0) 0 2 backup wm 0 - 65532 136.49GB (65533/0/0) 286248144 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0

Retrieve the zpool status it should be as presented below. If it is different stop the procedure and call the ECI Support center.

bash-3.00# zpool status pool: rpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 c1t0d0s0 ONLINE 0 0 0 errors: No known data errors

Check if a raid installed volume exist. The response should be as presented below.

bash-3.00# raidctl Controller: 1 Disk: 0.0.0 Disk: 0.1.0

If a volume exists (INDICATED UNDER THE CONTROLLER) stop the procedure and call ECI Support.

b. Insert the new hard disk in the free slot. The green led in the hard disk should start blinking. c. Do the command
bash-3.00#

devfsadm

d. Check the new disk identification as presented below:

bash-3.00# format -e Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c1t0d0 <LSILOGIC-LogicalVolume-3000 cyl 65533 alt 2 hd 16 sec 273> disk0001 /pci@0/pci@0/pci@2/scsi@0/sd@0,0 1. c1t1d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848> /pci@0/pci@0/pci@2/scsi@0/sd@1,0 Specify disk (enter its number):

4. Step #2 Second disk labeling and partitioning a. Labeling: i. Chose disk 1 (see above) and type the command label. ii. Enter the SMI label as described in the example below:
Specify disk (enter its number): 1 selecting c1t1d0 [disk formatted] FORMAT MENU: disk - select a disk type - select (define) a disk type partition - select (define) a partition table current - describe the current disk format - format and analyze the disk repair - repair a defective sector label - write label to the disk analyze - surface analysis defect - defect list management backup - search for backup labels

verify - read and display labels save - save new disk/partition definitions inquiry - show vendor, product and revision scsi - independent SCSI mode selects cache - enable, disable or query SCSI disk cache volname - set 8-character volume name !<cmd> - execute <cmd>, then return quit format> label [0] SMI Label [1] EFI Label Specify Label type[0]:

Press Enter, same for the further default options. b. Partitioning. i. Do again the command format, chose disk 1 and type p two times for getting the partitioning (for saving space well present only the final output):
Part Tag Flag Cylinders Size Blocks 0 root wm 0 0 (0/0/0) 0 1 swap wu 0 0 (0/0/0) 0 2 backup wu 0 - 14086 136.71GB (14087/0/0) 286698624 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 usr wm 0 - 14086 136.71GB (14087/0/0) 286698624 7 unassigned wm 0 0 (0/0/0) 0

Note: The partitioning is not something strictly defined, the output may be different, depending on the origin of the new disk. ii. The partitioning must be changed in order to similar with the existing disk. Do the disk partitioning in two sub steps: Sub-step 1: At the end of this step the final partitioning should be:
Current partition table (unnamed): Total disk cylinders available: 14087 + 2 (reserved cylinders) Part Tag Flag Cylinders 0 unassigned wm 0 1 unassigned wm 0 2 unassigned wm 0 3 unassigned wm 0 4 unassigned wm 0 5 unassigned wm 0 6 unassigned wm 0 7 unassigned wm 0 Size 0 0 0 0 0 0 0 0 Blocks (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0

1. Substep 2: The partitioning at the end of this steps should be similar with the first disk:
0 1 2 3 4 5 6 7 root wm 0 - 14086 136.71GB (14087/0/0) 286698624 unassigned wm 0 0 (0/0/0) 0 backup wm 0 - 14086 136.71GB (14087/0/0) 286698624 unassigned wm 0 0 (0/0/0) 0 unassigned wm 0 0 (0/0/0) 0 unassigned wm 0 0 (0/0/0) 0 unassigned wm 0 0 (0/0/0) 0 unassigned wm 0 0 (0/0/0) 0

Presented below few examples how to set the partitioning in order to get the requested partitioning:
AVAILABLE DISK SELECTIONS: 0. c1t0d0 <LSILOGIC-LogicalVolume-3000 cyl 65533 alt 2 hd 16 sec 273> disk0001 /pci@0/pci@0/pci@2/scsi@0/sd@0,0 1. c1t1d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848> /pci@0/pci@0/pci@2/scsi@0/sd@1,0 Specify disk (enter its number): 1 selecting c1t1d0: disk0002 [disk formatted] FORMAT MENU: disk - select a disk type - select (define) a disk type partition - select (define) a partition table current - describe the current disk format - format and analyze the disk repair - repair a defective sector label - write label to the disk analyze - surface analysis defect - defect list management backup - search for backup labels verify - read and display labels save - save new disk/partition definitions inquiry - show vendor, product and revision volname - set 8-character volume name !<cmd> - execute <cmd>, then return quit format> p PARTITION MENU: 0 - change `0' partition 1 - change `1' partition 2 - change `2' partition 3 - change `3' partition 4 - change `4' partition 5 - change `5' partition 6 - change `6' partition 7 - change `7' partition select - select a predefined table modify - modify a predefined partition table

name - name the current table print - display the current table label - write partition map and label to the disk !<cmd> - execute <cmd>, then return quit partition> p Volume: Current partition table (unnamed): Total disk cylinders available:

14087 + 2 (reserved cylinders)

Size 0 0 0 0 0 0 0 0 Size 0 Blocks (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 Blocks (0/0/0) 0

Part Tag Flag Cylinders 0 unassigned wm 0 1 unassigned wm 0 2 unassigned wm 0 3 unassigned wm 0 4 unassigned wm 0 5 unassigned wm 0 6 unassigned wm 0 7 unassigned wm 0 partition> 2 Part Tag Flag Cylinders 2 unassigned wm 0

Enter partition id tag[unassigned]: backup Enter partition permission flags[wm]: Enter new starting cyl[0]: 0 Enter partition size[0b, 0c, 0e, 0.00mb, 0.00gb]: partition> q

14087c

Pay attention: The number of the cylinders that you have to insert should be identical to the one presented when you do the command format see the numbers highlighted in yellow. After the setting it will show one cylinder less it is OK. In the places when the settings where the default value is the requested one, just press enter (in our example the permission flag was already wm and doesnt need change)
Part Tag Flag Cylinders 0 unassigned wm 0 1 unassigned wm 0 2 backup wm 0 - 14086 3 unassigned wm 0 4 unassigned wm 0 5 unassigned wm 0 6 unassigned wm 0 7 unassigned wm 0 Size Blocks 0 (0/0/0) 0 0 (0/0/0) 0 136.71GB (14087/0/0) 286698624 0 (0/0/0) 0 0 (0/0/0) 0 0 (0/0/0) 0 0 (0/0/0) 0 0 (0/0/0) 0

partition> 0 Part Tag Flag Cylinders 0 unassigned wm 0

Size 0

Blocks (0/0/0)

Enter partition id tag[unassigned]: root Enter partition permission flags[wm]:

Enter new starting cyl[0]:

Enter partition size[0b, 0c, 0e, 0.00mb, 0.00gb]: 14087c partition> p Volume: disk0002 Current partition table (unnamed): Total disk cylinders available: 14087 + 2 (reserved cylinders) Part Tag Flag Cylinders Size Blocks 0 root wm 0 - 14086 136.71GB (14087/0/0) 286698624 1 unassigned wm 0 0 (0/0/0) 0 2 backup wm 0 - 14086 136.71GB (14087/0/0) 286698624 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0 partition> q FORMAT MENU: disk - select a disk type - select (define) a disk type partition - select (define) a partition table current - describe the current disk format - format and analyze the disk repair - repair a defective sector label - write label to the disk analyze - surface analysis defect - defect list management backup - search for backup labels verify - read and display labels save - save new disk/partition definitions inquiry - show vendor, product and revision volname - set 8-character volume name !<cmd> - execute <cmd>, then return quit

All the partitions should be set on the same way. At the end save the configuration youll be requested to name the partition. Name it as below:
format> save Please name this partition type before saving it format> volname "disk0002" Enter 8-character volume name (remember quotes)["disk0002"]:Ready to label disk, continue? y

5. Step #3: Building the ZFS mirroring Check again the partitioning of both disks and assure that is correct. Return to the root prompt (by typing q twice).

Type the command below for building the ZFS mirroring Note: The name of the disks are the same as in the response of the command format with the addition s0 (see the parts highlighted in green):

bash-3.00# zpool attach -f rpool c1t0d0s0 c1t1d0s0 Please be sure to invoke installboot(1M) to make 'c1t1d0s0' bootable. Make sure to wait until resilver is done before rebooting.

Check the zpool status few times and wait to get the following response (the resilvered size may be different, assure that the resilver completed and No known data errors see highlight:

bash-3.00# zpool status pool: rpool state: ONLINE scrub: resilver completed after 0h8m with 0 errors on Sun Feb 3 10:57:00 2013 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c1t0d0s0 ONLINE 0 0 0 c1t1d0s0 ONLINE 0 0 0 20.4G resilvered errors: No known data errors bash-3.00#

Type the command below for doing the second disk bootable:

bash-3.00# installboot -F zfs /usr/platform/`uname -i`/lib/fs/zfs/bootblk /dev/rdsk/c1t1d0s0 bash-3.00#

6. Step #4: Setting the environment parameters NOTE: THIS STEP REQUESTS THE SERVERS STOP, IT SHOULD BE DONE ONLY DURING MAINTENANCE WINDOW. Stop the server by doing the command init 0; wait to get the ok prompt.

Identify the paths of the two disks (disk0 and disk1) see below:

{8} ok devalias ttya /ebus@c0/serial@0,ca0000 nvram /virtual-devices/nvram@3 net3 /pci@0/pci@0/pci@1/pci@0/pci@3/network@0,1 net2 /pci@0/pci@0/pci@1/pci@0/pci@3/network@0 net1 /pci@0/pci@0/pci@1/pci@0/pci@2/network@0,1 net0 /pci@0/pci@0/pci@1/pci@0/pci@2/network@0 net /pci@0/pci@0/pci@1/pci@0/pci@2/network@0 cdrom /pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/device@4/storage@0/disk@0:f disk3 /pci@0/pci@0/pci@2/scsi@0/disk@3 disk2 /pci@0/pci@0/pci@2/scsi@0/disk@2 disk1 /pci@0/pci@0/pci@2/scsi@0/disk@1 disk0 /pci@0/pci@0/pci@2/scsi@0/disk@0 disk /pci@0/pci@0/pci@2/scsi@0/disk@0 scsi /pci@0/pci@0/pci@2/scsi@0 virtual-console /virtual-devices/console@1 name aliases

Define nvalias (use the two paths retrieved above: :

ok nvalias rootdisk /pci@0/pci@0/pci@2/scsi@0/disk@0,0:a ok nvalias rootmirror /pci@0/pci@0/pci@2/scsi@0/disk@1,0:a

{8} ok printenv Variable Name

Do the command printenv in order to see the environment parameters:

Value Default Value false true No default No default No default min true false true false 7 No default false No default false true 80 34 9600,8,n,1,virtual-console virtual-console

ttya-rts-dtr-off false ttya-ignore-cd true keyboard-layout US-English reboot-command security-mode none security-password security-#badlogins 0 verbosity min pci-mem64? true diag-switch? false local-mac-address? true fcode-debug? false scsi-initiator-id 7 oem-logo oem-logo? false oem-banner oem-banner? false ansi-terminal? true screen-#columns 80 screen-#rows 34 ttya-mode 9600,8,n,1,output-device virtual-console input-device virtual-console

auto-boot-on-error? false load-base 16384 auto-boot? true network-boot-arguments boot-command boot boot-file boot-device disk:a disk net multipath-boot? false boot-device-index 0 use-nvramrc? false nvramrc error-reset-recovery boot

false 16384 true boot disk net false 0 false boot

Do the parameters setting as described below:

{8} ok setenv use-nvramrc? true use-nvramrc? = true {8} ok setenv nvramrc devalias rootdisk /pci@0/pci@0/pci@2/scsi@0/disk@0,0:a devalias rootmirror /pci@0/pci@0/pci@2/scsi@0/disk@1,0:a nvramrc = devalias rootdisk /pci@0/pci@0/pci@2/scsi@0/disk@0,0:a devalias rootmirror /pci@0/pci@0/pci@2/scsi@0/disk@1,0:a {8} ok setenv boot-device rootdisk rootmirror net boot-device = rootdisk rootmirror net

8} ok printenv Variable Name

Ceck again the parameters it should be as below (see the highlighted):

Value Default Value

ttya-rts-dtr-off false false ttya-ignore-cd true true keyboard-layout US-English reboot-command security-mode none No default security-password No default security-#badlogins 0 No default verbosity min min pci-mem64? true true diag-switch? false false local-mac-address? true true fcode-debug? false false scsi-initiator-id 7 7 oem-logo No default oem-logo? false false oem-banner No default oem-banner? false false ansi-terminal? true true screen-#columns 80 80 screen-#rows 34 34 ttya-mode 9600,8,n,1,9600,8,n,1,output-device virtual-console virtual-console input-device virtual-console virtual-console auto-boot-on-error? false false load-base 16384 16384 auto-boot? true true network-boot-arguments boot-command boot boot boot-file boot-device rootdisk rootmirror net disk net

multipath-boot? false false boot-device-index 0 0 use-nvramrc? true false nvramrc devalias rootdisk /pci@0 ... error-reset-recovery boot boot

Do the command:
ok boot

Check correct booting from the rootdisk. Connect the global zone in the normal mode (as ops), do the command OpsShell.sh and wait until both zones are in the status running.