Veritas Volume Manager

Handy commands

 * - displays attached shelf, also shows the temperature in each drive slot (probe -p shows physical path name)
 * - displays contents to NFS0 (from 'probe' above)
 * - blink LED or ok-to-remove on v880.
 * - Offline and online disk for replacement, equiv to remove_device and insert_device.
 * - displays master/slave for shared disks, do operations on master. "mode: enabled: cluster active - MASTER"


 * - list current disks and their status. HANDY!
 * - Will show you imported and deported disk groups.
 * - list the current plex/subplex stuff a bit like metastat
 * - after replacing disk, reread veritas internal device tree so it can see the disk.
 * - after replacing disk, if disk is showing as 'DISABLED' do this.
 * - Lists controllers used with DMP
 * - Lists disk devices and their DMP nodename.
 * - Performs a 'full' fsck of a vxfs filesystem, do this if advised by a normal fsck.

Reattaching disks
If you loose access to storage temporarily you will need to reattach it. You can do this with the very handy 'vxreattach' command.


 * -c Checks to see if it can be reattached and lets you know the diskname and the diskgroup.
 * -b Backgrounds the attach (good idea)
 * -r automatically recovers the plex.

Here's an example:

root@# vxreattach -c fabric_28 oradata-stag-dg fabric_13 root@# vxreattach -br fabric_28

Resizing a volume
This example resizes a volume that is current constructed from a stripe across 4 mirrors (dgn dgnm). There is existing space on the disks making up the mirror to use for the extra space:-

vxassist -g oracledg growto bckpvol 120g alloc="oracledg07,oracledg08,oracledg09,oracledg10, oracledg07m,oracledg08m,oracledg09m,oracledg10m" layout=stripe-mirror mirror=ctlr

Good examples of relaying out volumes:

If the user space tools stop running...
It's possible for the userspace tools to stop running (vxrelocd, vxconfigbackupd, vxconfigd). The mounted volumes will not be affected but things like hot-relocation won't work. Do the following to get it going again:-

/sbin/vxconfigd -syslog vxrelocd root & vxconfigbackupd &

Resyncing logs
Sometimes after a disk has been replaced the logs will not sync up again. You need to delete and recreate them, do the below on the highest level volume that has a problem, eg if every sub volume in a stripe has a problem then do this on the stripe itself.

vxassist -g [disk group] remove log [volume]

...then:-

vxassist -g [disk group] addlog [volume] alloc="[disk to store logs on]"

Disk weirdness
Quite often when Sun replace a disk in an A5200, Veritas refuses to see it, and complains:-

A reboot fixes this, but obviously that might not be desirable. Instead, try this:-


 * Remove the disk. Do it twice, since there's multiple entries.

vxdisk rm c1t5d0s2 vxdisk rm c1t5d0s2


 * Find out which disk/array it belongs to, and remove it.

luxadm remove_device ARRAY,f5


 * Refresh device paths.

devfsadm -C vxdctl enable1


 * Do a vxdisk list, and it should show up only once. Delete it again.

vxdisk rm c1t5d0s2


 * Offline the disk completely. If there's multiple paths, do it for all of the paths.

luxadm -e offline /dev/dsk/c1t5d0s2


 * Refresh device paths... again.

devfsadm -C vxdctl enable


 * You should now be able to use option 5 (Replace disk) from vxdiskadm.

Copying Volumes with vxfs
vxdump -0f - /dev/vx/dsk/rootdg/oracle_inst | (cd /a; vxrestore -rf -)

Using PowerPath with VxVM
If you have new PowerPath devices they will not show in 'vxdisk list' until you initialise them. To do this do powervxvm install

Then you can either initialise all, or do them one at a time. In a clustered environment do one at a time and skip the quorum device.

Once initialised you need to add them to the disk group. This doesn't work with vxdiskadm as it is brain dead. Do

vxdg -g mydg adddisk medianame=emcpower0c

To remove a device again do

vxddladm rmforeign blockpath=/dev/dsk/emcpower19c charpath=/dev/rdsk/emcpower19c

Other Random Stuff
You can get some high level info on the whole Veritas setup with vxassist help

Topics for the vxassist help command: Usage: vxassist help

usage         Usage of the vxassist command. options       Options to the vxassist command. attrs         Attributes for creation and allocation. alloc         Allocation attributes. layout        Mirror layout attributes. mirror        Mirroring attributes. showattrs     Display current attributes. space         Display current disk allocation information.

vxassist help

example:

Disk: rootdisk len=0 used=0 free=0 (0.47%) Attributes: dm:rootdisk device:c1t0d0 da:c1t0d0s2 enclosure:SENA0 ctlr:c1 target:c1t0 ctlr:c2 target:c2t0 ctype:photon tray:c0tray0 diskparam:14087.24.424.10025 diskgeom:14087.24.424 diskrpm:10025 vendor:NON_ARRAY protection:NON_ARRAY Free regions: 142657344,671616
 * 1) vxassist help space

Disk: disk01 len=0 used=0 free=0 (0.47%) Attributes: dm:disk01 device:c1t3d0 da:c1t3d0s2 enclosure:SENA0 ctlr:c1 target:c1t3 ctlr:c2 target:c2t3 ctype:photon tray:c0tray0 diskparam:14087.24.424.10025 diskgeom:14087.24.424 diskrpm:10025 vendor:NON_ARRAY protection:NON_ARRAY Free regions: 142657344,671616

Disk sets: ctlr:c1 space=0 used=0 free=0 (0.47%) ctlr:c2 space=0 used=0 free=0 (0.47%) ctype:photon space=0 used=0 free=0 (0.47%) da:c1t0d0s2 space=0 used=0 free=0 (0.47%) da:c1t3d0s2 space=0 used=0 free=0 (0.47%) device:c1t0d0 space=0 used=0 free=0 (0.47%) device:c1t3d0 space=0 used=0 free=0 (0.47%) diskgeom:14087.24.424 space=0 used=0 free=0 (0.47%) diskparam:14087.24.424.10025 space=0 used=0 free=0 (0.47%) diskrpm:10025 space=0 used=0 free=0 (0.47%) dm:disk01 space=0 used=0 free=0 (0.47%) dm:rootdisk space=0 used=0 free=0 (0.47%) enclosure:SENA0 space=0 used=0 free=0 (0.47%) protection:NON_ARRAY space=0 used=0 free=0 (0.47%) target:c1t0 space=0 used=0 free=0 (0.47%) target:c1t3 space=0 used=0 free=0 (0.47%) target:c2t0 space=0 used=0 free=0 (0.47%) target:c2t3 space=0 used=0 free=0 (0.47%) tray:c0tray0 space=0 used=0 free=0 (0.47%) vendor:NON_ARRAY space=0 used=0 free=0 (0.47%)