Volumemanager example commands PATH=$PATH:/usr/lib/vxvm/bin:/opt/VRTS/bin:/usr/lib/fs/vxfs using XP san: look at disk LUNs with command: xpinfo -d -t | awk '{FS=","} {print

$3,$1,$2,$4,$5 }' | sort -n | more prints out SAN disks sorted by LUN, can check against vxdisk list. Disks usually have two paths but vxdisk list only shows 1 path. pkginfo -l VRTSvxvm #find out veritas version running /sbin/vxlicrep # print out licences /opt/VRTS/bin/vxlicinst #install licence vxsvc - graphic UI daemon (must be running to use vea) /opt/VRTSob/bin/vea #GUI vea server, kill with /opt/VRTSob/bin/vxsvc -k can remove lock file under /var/vx/isis/vxisis.lock if it says running when it isn't. vxconfigd - vol config daemon, enable with below command vxiod - IO manager, reads and write to volumes vxrelocd - Hot swap manager which uses hotswap if a disk fails. vxdctl enable #rescan for all attached disks vxdisk -o alldgs list # show all dgs even if not imported DEVICE is disk accessname, could be c2t4d0s2 alt. enclosure based ENC1_05 *NOTE: different hosts can have the different DEVICE names for the same disk, even enclosure names can be different. DISK is the disk media name (dm), like data01 for dg data, can exist the same na me in different dgs. Can use either name in disk operations. vxdisk -s list ENC1_05 - lists info on 1 disk, only disk_id guaranteed unique to each disk, use this to get detail info on a single disk vxprint -htg my_dg # view vols, plexs, sd for my_dg dg diskgroup e.g. datadg dm diskmedia name, ie. temp01 device c0t1d0s2 or enc. based SENA1 v volume vol01 pl plex vol01-01 sd subdisk i.e SENA1_13 disk access name is: c0t3d5s2 Length is disk capacity in sectors (divide by 2 to get kbytes) length does not indicate how full the disk is. vxdisksetup -g DATADG -i c1t2d0 #create private and public areas on new disk and put in new diskgroup vxdg -g CASHIP01dg free # check if any imported disks are free

vxdg init newdg c3t3d13 # initialise newdg with free disk , disk cannot be in a DG beforehand # use vxdisk -o alldgs list to check that disks are free before initialising # as vxdiskadm doesn't show this info. Also run all commands on the same box # as different boxes can give different disks the same names vxdg -g datadg adddisk c3t3d13 #put disk in dg vxinfo -g datadg # print volume state info, ie. started, offline etc. vxtask list # print current tasks , like remirroring etc. Adding new disks ================ devfsadm -v (was drvconfig and disks) use format to label disks vxdctl enable vxdiskadm # add new disks,adddisk, vxassist -g ENTMTP01dg -U fsgen make bin92 14g #create 14Gb partition #-U fsgen|gen fsgen= normal volume, gen=raw volume # make a newfs mkfs -F vxfs -o largefiles /dev/vx/rdsk/ENTMTP01dg/bin92 # mount it mount -F vxfs /dev/vx/dsk/ENTMTP01dg/bin92 /oracle1/bin to add manydisks do this: /etc/vx/bin/vxdisksetup -i c5t5d160 # use for loop to add many vxdg -g CCV00P02dg adddisk c5t5d163s2 #alternative to add new disks interactively vxdiskadd c1 # lists available disks on controller c1, or give c1t5d0 etc # now answer the questions about what diskgroup etc. Replacing disks =============== use vxevac to remove all subdisks from disk ---------- use either vxdiskadm or cli 1: vxdiskadm method vxdiskadm -> option 4 (Remove a disk for replacement) vxdiskadm -> option 11 (Disable (offline) a disk device) disk should now appear as offline in vxdisk list, take the DEVICE name: now swap the disks vxdiskadm -> option 5 Replace a failed or removed disk choose the failed disk to replace (use list to see the ones marked failed e.g. b illingdg01) replace with the disk you put in, use list to see name i.e. c1t2d0 it will then ask to bring online, initialize etc.

-----------2: using CLI vxdg rmdisk rep03 (if you wish to clean the disk of veritas then do vxdiskunsetup <DEVICE_NAME> before offlining vxdisk offline <DEVICE_NAME> i.e. LOIS2_8 or c2t43d0s2 -----------renaming enclosure from veritas vxdmpadm setattr enclosure LOIS2 name=MELONS from luxadm luxadm enclosure_name MELONS LOIS2 don't know if this is correct order, there were no live disks when I did this ------------sccli to look at T3 enclosure ------------vxdisk list LOIS2_6 gives last line: c2t41d0s2 state=enabled #this is your device access name now find the wwn: ls -l /dev/dsk |grep c2t41d0s2 lrwxrwxrwx 1 root other 76 May 25 08:19 c2t41d0s2 -> ../../devices /pci@8,700000/pci@4/SUNW,qlc@4/fp@0,0/ssd@w22000004cf4cc7e6,0:c the wwn here is 4cf4cc7e6 now find the slot position from: luxadm display LOIS2 luxadm remove_device enclosurename,[f|r]slot# luxadm remove_device LOIS2,f3 # it then expects you to remove the disk Moving individual subdisks ========================== 1. view volume subdisks vxprint -htv u04-snapshots01 pl u04-snapshots01-04 u04-snapshots01 ENABLED ACTIVE LOGONLY CONCAT sd rep22-28 u04-snapshots01-04 rep22 1485 66 LOG PLUM_0 note: subdisk rep22-28 has length 66, this is the one we want to move 2. find space on disk to use vxprint -st | grep report08 RW ENA

sd report08-21 u04-user01-04 report08 71131989 33 offset for next subdisk is 71131989 + 33 = 71132022 3. create new subdisk

LOG

SENA1_10 ENA

vxmake -g rep sd report08-22 disk=report08 offset=71132022 len=66 subdiskname diskname from step 2. from step 1. it should look like this: sd report08-22 report08 71132022 66 SENA1_10 ENA

4. move subdisk rep22-28 to report08-22 /usr/sbin/vxsd -d -g rep -o rm mv rep22-28 report08-22 vxprint step 1. will show it moved. to remove a subdisk use vxedit, example shown earlier. Volumes ======= Layered volumes: a layered volume is a concat-mirror, stripe-mirror as opposed to a mirror-concat or mirror-stripe. the difference being that for layered the subdisks are mirrored and built into a plex whereas normally the subdisks are built into plexes and mirrored. This gives a 50% increase in redundancy for layered. vxassist -g datadg make concat_vol 50m #create a concat volume size is 50MB vxassist -g datadg make strip1_vol 20m layout=stripe,mirror ncols=2 stwidth=128k c3t3d15 c3t3d14 c3t3d13 c3t3d12 #create stripe mirror over 2 columns, width 128k, using 4 disks vxassist -g datadg make raid1_vol 50m layout=raid5 ncol=4 disk1 disk2 disk3 dis k4 disk5 # create raid5 volume with 4 columns over 5 disks, you get a log plex with # raid 5, so specify ncol 4 for 5 disks and the log will land # on the last disk (disk5 in this case) vxassist -g testdg mirror volume1 layout=stripe ncols=4 stwidth=64k MELONS_16 M ELONS_17 MELONS_18 MELONS_19 # add a striped mirror to existing volume using specified disks and cols. vxassist -g datadg remove volume strip1_vol # delete/remove a volume vxdisk -e list #print device name and path, like this: SENA0_18 sliced bostmp07 bos

online

c4t87d0s2

vxdisk rm ENC1_1 # remove a disk from veritas all together, note disk should not be used, imported or have subdisks when done.

vxassist -g datadg mirror Vol_01 bostemp11 # add a mirror to a running volume vol_01, use disk bostemp11 vxassist -g datadg remove mirror strip1_vol # remove a mirror from a volume vxassist remove mirror rootvol !rootmirr # split a root mirror from a rootvol vxassist -g testdg remove mirror volume1 !disk01 # remove the mirror on disk01 - untested #vxdg destroy datadg # this is very dangerous to cut and paste by accident! #destroy diskgroup vxvol -g testdg stop u01_mirror # stop volume before trying to remove it vxplex -g testdg -o rm -v volume01 dis plex01 #remove and destroy plex01 from volume01 - frees up subdisks vxplex -g datadg -v u04-betssnap06 dis u04-betssnap06-03 #dis plex from volume, use det for detach vxedit -g testdg -r rm plex01 # remove and destroy a stand alone plex vxedit -g datadg rm volume1 # remove a volume Moving Plexes ============= first create mirror plex then remove unwanted plex 1. vxassist -g datadg mirror <volume> <diskname> 2. vxplex -g rep -o rm -v <volume> dis <plexname> Volume creation, different ways =============================== #using vxassist: vxassist -g rep -U gen make u05-games 32g layout=mirror rep24 report06 #using vxmake # make plex pl-games-01 from vmdisk rep23 vxmake -g rep plex pl-games-01 sd=rep23 len=67108864 # make plex pl-games-02 from vmdisk report07 vxmake -g rep plex pl-games-02 sd=report07 len=67108864 # make volume vol-games using plex pl-games-01 vxmake -g rep -U gen vol vol-games plex=pl-games-01 # attach plex pl-games-02 to make mirrored volume vxplex -g rep att vol-games pl-games-02 # change volume user and group id: so oracle can mount it vxedit -g rep set user=oracle group=oinstall u03-games #create link for raw device

ln -s /dev/vx/rdsk/rep/u03-games /GAMES/u03-games.dbf

Volume Operations ================= vxdg split datadg splitdg vol01 source dest volume # extract vol from a dg and put in another dg vxrecover -g splitdg -sE vol01 # start a stopped volume, e.g after splitting it out vxsd -g diskgroup -v vol2 mv c3t3d14-02 c3t3d15-02 /usr/sbin/vxsd -t taskid_119 -g rep -o rm mv trep0_8-20 rep26-01 # move subdisks /usr/sbin/vxsd -g datadg -o rm dis trep0_8-2 #dissassociate the subdisk trep0_8-22, i.e. remove an unwanted drl /usr/sbin/vxedit -g datadg -rf rm report08-22 # destroy the subdisk report08-22 entirely #Warning!!! # on using vxevac, if you use vxdisk list to find a free disk from # one machine and then use vxevac on another machine, it might not # be the same disk, but it can have the same DEVICE name. this is # because each machine can rename enclosure named disks or have them # on different controllers, always check with vxdisk list diskname # and look at the disk id. vxevac -g bos bosdrl02 PLUMSD_4 # evacuate subdisks from dm bosdrl02 to PLUMSD_4 vxvol -g datadg rdpol round strip1_vol #change read policy to round robin vxedit -g datadg set spare=off c3t3d11 vxedit -g datadg set nohotuse=on c3t3d11 #set spare flag on/off for a disk # a disk set with the spare flag on is available for hot relocation # setting the disk as spare=off makes it part of the free disk pool # setting the nohotuse=on prevents it from being used for hot relocation vxedit -g datadg rename c3t3d11 data-01 #use vxedit to rename disks, volumes, plexes etc #reserve a disk (on/off)so vxassist won't grab it, # specify by name to use it later vxedit set reserve=on c3t3d11 #remove a subdisk vxedit -g datadg -rf rm disk01-01 # start all vols in imported dg vxvol -g testdg startall Using vxmake ============

# use vxmake to create a subdisk (sd) of VMdisk disk01 with 0 offset vxmake sd disk01-01 disk=disk01 offset=0 len=17678493 subdisk sd-name vmdiskname,offset,size # make a plex called "vol01-01" vxmake plex vol01-01 #put subdisks into a plex vol01-01 vxsd assoc vol01-01 sd1 sd2 sd3 .. #alt. make plex and assign sd at once: vol01-01 using subdisk sd1 vxmake -g datadg plex vol01-01 sd=sd1 #create a volume vol01 using plex vol01-01 vxmake -U fsgen vol01 plex=vol01-01 # to make a raw vol use -U gen #start volume, then do newfs and mount it. vxvol start vol01 #to create striped plex use vxmake plex vol01-01 layout=stripe ncolumn=4 stwidth=32k sd=sd1,sd2,sd3,sd4 Snapshot ======== vxassist -g datadg snapstart vol01 #start the snapshot process vxassist snapwait # the snapstart command exits before it's fully sync'd, so snapwait means do not allow a snapshot until the sync is done. vxassist -g datadg snapshot vol01 snap_vol01 #create snapshot volume snap_vol01 vxassist -g datadg -o resyncfromreplica snapback snap_vol01 # replace data in original vol with snapshot volume data vxassist -g datadg snapclear snap_vol02 # disassociate snap_vol02 from original volume, after this the resyncfromreplica won't work anymore, this has the effect of creating a freestanding volume with the snapshot data. vxassist -g datadg snapabort vol01 #abort snapshot process Fast resync =========== vxvol -g datadg set fastresync=on fmron #enable nonpersistent fast resync vxprint -g datadg -F%fastresync fmron # is fast resync enabled? rtns on else off vxprint -g datadg -F%hasdcolog fmron #whether volume has persistent fast resync (rtrns on) otherwise rtrns off

Recovery ======== vxassist -g datadg addlog strip1_vol logtype=drl c3t3d11 # add dirty log to volume strip1_vol # Dirty Region Logging only speeds up volume start by ensuring mirrors are #consistent with each other, it replays the log on the unsynced mirror. #there is no data protection here. vxdctl enable #Rescan for all attached harddisks vxdg -g datadg -k adddisk c3t3d13 #add new hardisk and use old hd name etc (-k option) vxrecover -g datadg recvol # recover vol, spin up disk, reattach plexes etc. # fro plexes marked Detached stale do this # set plex clean vxmend -g bos fix clean plex # reattach it to volume vxplex -g bos att vol plex

use vxdiskadm to replace a failed disk or to mirror a root disk or use /etc/vx/bin/vxrootmir rootmirr # where rootmirr is the mirror disk name to replace: vxdiskadm 4 Remove a disk for replacement 11 Disable (offline) a disk device use luxadm to quiesce bus and blink e.g luxadm led /dev/rdsk/c1t3d0s2 # led_blink # led_off # change disk the led led status blink led stop blinking

vxdiskadm 5 Replace a failed or removed disk to use the new disk instead Volume resize ============= vxassist maxgrow vol_01 #returns the max a volume can grow by. vxassist maxgrow vol_01 disk1 disk2 disk3 disk4 # this will grow the volume to fill the specified disks # inc. vol_01 with 5Meg grows both FS and volume (use this) # allegedly "-o verify" is used to test only, but did the operation anyway! vxresize -F vxfs -g datadg vol_01 +5m or #resize a volume, but only use contiguous diskspace, here vol_01 by 1GB vxassist growby vol_01 1g layout=nodiskalign

#specify size to grow to (here to 5g) vxassist growto vol_01 5g #resize filesystem, also known as fsadm_vxfs, not necessary with vxresize fsadm -F vxfs -b 97656 /rezmnt # shrinking can be done similarly with shrinkto and shrinkby # resize keeping plexes on specific controllers (untested) vxresize -x -F vxfs -g bos u08 <newlength> <disk1> <disk2> <disk3> <disk4> mirro r=ctlr ctlr:c2 ctlr:c3 Trouble shooting ================ check support.veritas.com for help docs. if a volume is listed as DISABLED ACTIVE or a disk group is online disabled fro m vxprint -ht or vxdg list then you can deport/import it again. Use vxrecover to recover and start the volumes. +++ To fix a volume with stale plexes and 1 plexes marked RECOVER: Ignore the LOGONLY plexes Set the STALE plex to CLEAN vxmend -g datadg fix clean <volumename> start the volume vxvol start <volumename> fix a volume with 2 recover states: vxmend off <plexname> vxmend on <plexname> vxmend fix active <plexname> vxvol start <volumename> Misc ===== from http://seer.support.veritas.com/docs/237510.htm get size of all vm diskspace in a dg vxprint -g datadg -dF "%publen" | awk 'BEGIN {s = 0 } {s += $1} END {print s}' this adds up the columns in vxprint labelled length (capacity) convert sectors to meaningful sizes (on sun systems ):

1 sector = 512 bytes, 2 sectors = 2x512bytes = 1kbyte divide: sectors / 2 = kb sectors / 2048 = Mb sectors / 2097152 = Gb RAID types: raid0 striping raid1 mirroring raid 0+1 striping + mirroring raid 5 striping + parity checked DMP === Dynamic Multipathing - more than 1 path to diskbox. to see what paths are available: vxdisk list c3t3d0 advanced vxprint ================ use vxprint -F to format output, see what fields are available: vxprint -g datadg -m "subdisk01" then vxprint -g datadg -h -F "%vname %daname" this gives a volume name (%vname column 2) and what subdisks (%daname column 8) it uses. using vol_pattern to get what you want .i.e I want to see volumes with 4 plexes only: vxprint -g datadg -hv -e v_nplex=4 put it together: vxprint -g datadg -hv -e v_nplex=4 -F "%vname %daname" shows the subdisks on volumes with only 4 plexes or a more complete example: vxprint -g rep -htv -e v_nplex=3 -F "%vname %daname %dm %is_log" gives list vol vmdisk log_02 log_02 log_02 log_02 log_02 log_02 log_02 subdisk logging :

- - - - SENA0_19 trep0_9 off - - SENA0_8 rep00 off - - LOIS2_0 rep01 on

vxprint -g datadg -e '( pl_sd ~ /bos16/ )' #print plexes using subdisk bos16 vxprint -g bos -e '( pl_sd = "bos16-01" )' # exact match #see what volumes are on a device, are they drl vols? vxprint -g bos -e '( sd_device = "LOIS2_12" )' -F "%vname %daname %is_log" volume device name bos-stat LOIS2_12 off u07-temp LOIS2_12 on bos-stat LOIS2_12 on to see if any volumes are using the same three disks: vxprint -g bos -e '( sd_device = "LOIS2_12" ) || ( sd_device = "LOIS2_14" ) || ( sd_device = "LOIS2_17" ) ' -F "%vname %daname" using && (and) would give the answer as those that are, or nothing. find volumes which have only 2 plexes but ignore log plexes: vxprint -g bos -htv -e '( v_nplex=4 ) && ( ! v_plex.pl_subdisk.sd_is_log ) ' F "%vname %daname %dm %is_log" |grep off | awk '{ print $1 }' | uniq -c | sort Disaster recovery ================= backup configs with: vxprint -mpvshr -g diskgroup_name > file1 vxprint -ht > file2 or /etc/vx/bin/vxconfigbackup Execute as required manually (if vxconfigbackupd is not running), when you req uire a backup of the configuration data /etc/vx/bin/vxconfigrestore Execute as required manually when you require to restore corrupted configuratio n data The "-l" option allows you to specify an alternative directory for the location of the backup configuration files other than the default location, /etc/vx/cbr/b k. ## End