problem with LUNS from cx500 to v440. (LONG)
Sorry for this long post.
I've got some strange behavior when adding or removing LUNS from my v440 (solaris8) that
are presented from our Clariion CX500. I'm using PowerPath 4.5.0_b169, SUN 2GB PCI HBA's (Qlogic)
SG-XPCI1FC-QF2 (two cards in the host and each card is zoned seperately). There are two McData 4500's in
the fabric, each card cabled to a different switch.
When I add a LUN to the storage group, PowerPath still doesn't pick it up even after running
powermt config, powercf -q, devfsadm -v. It is not until I issue a format command that it picks it up
and even then it only sees one SP, one path per zone to that SP.
#format (output snipped)
Searching for disks...done
c3t500601603022643Fd6: configured with capacity of 10.00GB
c4t500601613022643Fd6: configured with capacity of 10.00GB
AVAILABLE DISK SELECTIONS:
15. c3t500601603022643Fd6 <DGC-RAID5-0219 cyl 32766 alt 2 hd 64 sec 10>
/pci at 1c,600000/SUNW,qlc at 1/fp at 0,0/ssd at w500601603022643f,6
28. c4t500601613022643Fd6 <DGC-RAID5-0219 cyl 32766 alt 2 hd 64 sec 10>
/pci at 1d,700000/SUNW,qlc at 2/fp at 0,0/ssd at w500601613022643f,6
34. emcpower5a <DGC-RAID5-0219 cyl 53246 alt 2 hd 256 sec 20>
/pseudo/emcp at 5
Specify disk (enter its number): ^D
root at balt-db2:~#powermt display dev=all
-- output snipped --
CLARiiON ID=APM00060500873 [BALT-DB2-Qual]
Logical device ID=600601601D96170034CA91025DBBDA11 [LUN 151]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP A
==============================================================================
---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3072pci at 1c/SUNW,qlc at 1 c3t500601603022643Fd6s0 SP A0 active alive 0 0
3073pci at 1d/SUNW,qlc at 2 c4t500601613022643Fd6s0 SP A1 active alive 0 0
powercf -q will now add the pseudo device.
root at balt-db2:~#powercf -q
Could not find config file entry for:
---------------------------------------
volume ID = 600601601d961700
---------------------------------------
adding emcpower6
but the paths to SPB are still not there.
root at balt-db2:~#powermt display dev=emcpower6a
Pseudo name=emcpower6a
CLARiiON ID=APM00060500873 [BALT-DB2-Qual]
Logical device ID=600601601D96170034CA91025DBBDA11 [LUN 151]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP A
==============================================================================
---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3072pci at 1c/SUNW,qlc at 1 c3t500601603022643Fd6s0 SP A0 active alive 0 0
3073pci at 1d/SUNW,qlc at 2 c4t500601613022643Fd6s0 SP A1 active alive 0 0
t at balt-db2:~#powermt config
root at balt-db2:~#powermt display dev=emcpower6a
Pseudo name=emcpower6a
CLARiiON ID=APM00060500873 [BALT-DB2-Qual]
Logical device ID=600601601D96170034CA91025DBBDA11 [LUN 151]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP A
==============================================================================
---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3072pci at 1c/SUNW,qlc at 1 c3t500601603022643Fd6s0 SP A0 active alive 0 0
3073pci at 1d/SUNW,qlc at 2 c4t500601613022643Fd6s0 SP A1 active alive 0 0
if I use cfgadm to display all the devices, followed by powermt config/check. the
paths to spb show up. Is this normal expected behavior with PP?
root at balt-db2:~#cfgadm -al -o show_FCP_dev
Ap_Id Type Receptacle Occupant Condition
c3 fc-fabric connected configured unknown
c3::500601603022643f,6 disk connected configured unknown
c3::500601693022643f,6 disk connected configured unknown
c4 fc-fabric connected configured unknown
c4::500601613022643f,6 disk connected configured unknown
c4::500601683022643f,6 disk connected configured unknown
root at balt-db2:~#powermt config
root at balt-db2:~#powermt check
root at balt-db2:~#powermt display dev=emcpower6a
Pseudo name=emcpower6a
CLARiiON ID=APM00060500873 [BALT-DB2-Qual]
Logical device ID=600601601D96170034CA91025DBBDA11 [LUN 151]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP A
==============================================================================
---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3072pci at 1c/SUNW,qlc at 1 c3t500601603022643Fd6s0 SP A0 active alive 0 0
3072pci at 1c/SUNW,qlc at 1 c3t500601693022643Fd6s0 SP B1 active alive 0 0
3073pci at 1d/SUNW,qlc at 2 c4t500601613022643Fd6s0 SP A1 active alive 0 0
3073pci at 1d/SUNW,qlc at 2 c4t500601683022643Fd6s0 SP B0 active alive 0 0
root at balt-db2:~#format
Searching for disks...done
c3t500601603022643Fd6: configured with capacity of 10.00GB
c3t500601693022643Fd6: configured with capacity of 10.00GB
c4t500601613022643Fd6: configured with capacity of 10.00GB
c4t500601683022643Fd6: configured with capacity of 10.00GB
emcpower6a: configured with capacity of 10.00GB
-- output truncated for brevity. --
15. c3t500601603022643Fd6 <DGC-RAID5-0219 cyl 32766 alt 2 hd 64 sec 10>
/pci at 1c,600000/SUNW,qlc at 1/fp at 0,0/ssd at w500601603022643f,6
16. c3t500601693022643Fd6 <DGC-RAID5-0219 cyl 32766 alt 2 hd 64 sec 10>
29. c4t500601613022643Fd6 <DGC-RAID5-0219 cyl 32766 alt 2 hd 64 sec 10>
/pci at 1d,700000/SUNW,qlc at 2/fp at 0,0/ssd at w500601613022643f,6
30. c4t500601683022643Fd6 <DGC-RAID5-0219 cyl 32766 alt 2 hd 64 sec 10>
/pci at 1d,700000/SUNW,qlc at 2/fp at 0,0/ssd at w500601683022643f,6
37. emcpower6a <DGC-RAID5-0219 cyl 32766 alt 2 hd 64 sec 10>
/pseudo/emcp at 6
Now I can configure a slice, newfs and mount it fine.
But if I subsequently remove the LUN from PP,the storage group, and Unbind the LUN on the cx500,
the old device paths are not cleared up by devfsadm -C -v until I reboot.
root at balt-db2:~#powermt remove dev=emcpower6a
root at balt-db2:~#powercf -q
Could not validate the entry:
---------------------------------------
emcpower6: user ID = 11300000030
---------------------------------------
removing emcpower6
root at balt-db2:~#powermt check
root at balt-db2:~#powermt config
root at balt-db2:~#powermt check
root at balt-db2:~#powermt display dev=emcpower6a
Bad dev value emcpower6a, or not under Powerpath control.
root at balt-db2:~#powermt save
root at balt-db2:~#devfsadm -C -v
root at balt-db2:~#format
Searching for disks...done
-- output snipped --
AVAILABLE DISK SELECTIONS:
15. c3t500601603022643Fd6 <drive not available: formatting>
/pci at 1c,600000/SUNW,qlc at 1/fp at 0,0/ssd at w500601603022643f,6
16. c3t500601693022643Fd6 <drive type unknown>
/pci at 1c,600000/SUNW,qlc at 1/fp at 0,0/ssd at w500601693022643f,6
29. c4t500601613022643Fd6 <drive not available: formatting>
/pci at 1d,700000/SUNW,qlc at 2/fp at 0,0/ssd at w500601613022643f,6
30. c4t500601683022643Fd6 <drive type unknown>
/pci at 1d,700000/SUNW,qlc at 2/fp at 0,0/ssd at w500601683022643f,6
root at balt-db2:~#cfgadm -al -o show_FCP_dev
Ap_Id Type Receptacle Occupant Condition
c3 fc-fabric connected configured unknown
c3::500601603022643f,6 disk connected configured failing
c3::500601693022643f,6 disk connected configured failing
c4 fc-fabric connected configured unknown
c4::500601613022643f,6 disk connected configured failing
c4::500601683022643f,6 disk connected configured failing
After a reboot, devfsadm -C -v removes all the device paths and the disks dissappear
from format the the cfgadm and luxadm output.
I'd hate to think that I'll have to reboot my production DB server just to remove a LUN completely.
Anyone have any insight on what I'm doing wrong?
Thanks
Allen
