Unsolved
This post is more than 5 years old
51 Posts
0
3513
metas not visible to host in cluster environment
Hi, may be someone came across this problem before. I've opened a case on this as well, but any other suggestions will not hurt. We have a cluster system with two nodes (both see the same metas). We can see the metas from the host side, but not able to write to them. When trying to format a disk get this message "Warning: error writing VTOC", when trying to create a file system:
# newfs -m 1 /dev/rdsk/c3t20d51s0
newfs: construct a new file system /dev/rdsk/c3t20d51s0: (y/n)? y
write error on sector 16: I/O error
# newfs -m 1 /dev/rdsk/c3t20d51s0
newfs: construct a new file system /dev/rdsk/c3t20d51s0: (y/n)? y
write error on sector 16: I/O error
ble1
2 Intern
2 Intern
•
14.3K Posts
0
November 8th, 2006 13:00
GlenH
141 Posts
1
November 8th, 2006 15:00
Have you labelled the device using format ? You need to do that first if its a new device.
If you are re-using an existing device that's been previously assigned to another host, then this can happen if the other host has left a SCSI reservation on the device.
A good telltale sign of this is if you run "inq" and the capacity is not shown.
You can also check to see if that is the case by running a "symdev list -resv" and see if your device appears in the list.
If there is a reservation, and you are absolutely certain no-one else is using it, log a call and get your EMC CE to clear it for you.
Glen.
diskextender
51 Posts
0
November 9th, 2006 06:00
The symdev list -resv returned - "No devices were found"
The syminq returns all the drives and shows their capacity.
The problem isn't that the volumes are not visible. The volumes are visible, but cannot be accessed. I'm unable to label a disk - getting the error "Warning: error writing VTOC - Label failed". I'm unable to create a filesystem or dd the disk.
To answer the other question:OS level: Solaris 9. There is no other software installed on the system, just powerpath and the OS.
Type of disk: We have 3 quorum devices assigned to both systems, several 2gb mirrored metas and several 34gb mirrored metas (all seen by both systems). The thing is, only 34gb metas are not writable, the rest of them are fine. PSE lab checked the metas and there are NO celerra flags on them ( I know it might cause the problem). Other thing: we have one 34gb meta assigned exclusively to each cluster node, and we don't have any issues writing to those metas.
owen4
2 Posts
0
November 9th, 2006 15:00
I've opened a case on this as well, but any other
suggestions will not hurt. We have a cluster system
with two nodes (both see the same metas). We can see
the metas from the host side, but not able to write
to them. When trying to format a disk get this
message "Warning: error writing VTOC", when trying to
create a file system:
# newfs -m 1 /dev/rdsk/c3t20d51s0
newfs: construct a new file system
/dev/rdsk/c3t20d51s0: (y/n)? y
write error on sector 16: I/O error
Did you check whether the metas are actually Read/Write?
symdev list should show RW, not WD in the next to last column (STS).
diskextender
51 Posts
0
November 10th, 2006 05:00
Read/Write?
symdev list should show RW, not WD in the next to
last column (STS).
Yes, we did check the metas and all of them are showing is read/write. The wierd thing is that we were able to see those metas a while ago. I rember back then we install Veritas Oracle RAC cluster, then we rebuilt the systems and installed Oracle OCF cluster software. Not sure if these disks were initiallized under Veritas if something was done to them. Seems like something was changed on the host side, not the SAN side.
diskextender
51 Posts
0
November 10th, 2006 13:00
JasonBailey
147 Posts
0
November 11th, 2006 21:00
diskextender
51 Posts
0
November 13th, 2006 05:00
C:\Documents and Settings\erduncann>symdev show 2d3
Symmetrix ID: XXXXXXXXXX
Device Physical Name : \\.\PHYSICALDRIVE7
Device Symmetrix Name : 02D3
Device Serial ID : 53002D3000
Symmetrix ID : XXXXXXXXXX
Attached BCV Device : N/A
Attached VDEV TGT Device : N/A
Vendor ID : EMC
Product ID : SYMMETRIX
Product Revision : 5671
Device WWN : 60060480000187870253464241324433
Device Emulation Type : FBA
Device Defined Label Type: N/A
Device Defined Label : N/A
Device Sub System Id : 0x0003
Device Block Size : 512
Device Capacity
{
Cylinders : 73656
Tracks : 1104840
512-byte Blocks : 70709760
MegaBytes : 34526
KiloBytes : 35354880
}
Device Configuration : 2-Way Mir (Meta Head,
Non-Exclusive Access)
Device is WORM Enabled : No
Device is WORM Protected : No
SCSI-3 Persistent Reserve: Enabled
Dynamic Spare Invoked : No
Dynamic RDF Capability : None
Device Service State : Normal
Device Status : Ready (RW)
Device SA Status : Ready (RW)
Front Director Paths (6):
{
----------------------------------------------------------------------
POWERPATH DIRECTOR PORT LUN
--------- ---------- ---- -------- ---------
PdevName Type Type Num Sts VBUS TID SYMM Host
----------------------------------------------------------------------
\\.\PHYSICALDRIVE7 PARENT FA 07B:1 RW 000 00 054 054
Not Visible N/A FA 08B:1 RW 000 00 02D N/A
Not Visible N/A FA 09B:1 RW 000 00 02D N/A
Not Visible N/A FA 10B:1 RW 000 00 054 N/A
Not Visible N/A FA 07C:1 RW 000 00 009 N/A
Not Visible N/A FA 10C:1 RW 000 00 009 N/A
}
Meta Configuration : Striped
Meta Stripe Size : 960k (2 Cylinders)
Meta Device Members (4) :
{
----------------------------------------------------------------------
BCV DATA RDF DATA
---------------------------- --------------------------
Sym Cap Std Inv BCV Inv Pair R1 Inv R2 Inv Pair
Dev (MB) Tracks Tracks State Tracks Tracks State
----------------------------------------------------------------------
--> 02D3 8632 - - N/A - - N/A
02D4 8632 - - N/A - - N/A
02D5 8632 - - N/A - - N/A
02D6 8632 - - N/A - - N/A
----------------------------------------------------------------------
34526 - - - -
}
Mirror Set Type : [Data,Data,N/A,N/A]
Mirror Set DA Status : [RW,RW,N/A,N/A]
Mirror Set Inv. Tracks : [0,0,0,0]
Back End Disk Director Information
{
Hyper Type : Data
Hyper Status : Ready (RW)
Disk [Director, Interface, TID] : [11B, D, 4]
Disk Director Volume Number : 276 (0x113)
Hyper Number : 4
Disk Capacity : 69879m
Disk Group Number : 2
Hyper Type : Data
Hyper Status : Ready (RW)
Disk [Director, Interface, TID] : [02D, C, 4]
Disk Director Volume Number : 36 (0x23)
Hyper Number : 4
Disk Capacity : 69879m
Disk Group Number : 2
}
Message was edited by:
Julie Gibson
I XXXX'd out the symmetrix ID, that's what we do in Support Solutions so it seems best in this environment too.
diskextender
51 Posts
0
November 16th, 2006 11:00
One solution of course if VTOCing all the devices from the Symmetrix side, but it would require our CE involved and a lot of work
What worked for us is to change SCSI persistent reserve flag back to "disabled" and it solved the issue: we were able to write to those devices again.
JasonBailey
147 Posts
0
November 16th, 2006 12:00
VTOC is used to clear celerra signatures. The signature is actually written to the disk by the celerra, its not a flag, so thats why a VTOC is needed. I think you can also DD to the disk as an alternative.
The celerra signature problem would not cause you to be unable to write to the disk, it would just mean Powerpath would hide the disk from the O/S. Makes for very confused and frustrated SAN admins calling in
So the solution to remove the PR was to remove the PER bit from the device, which is not a great solution but it works. I would have hoped it could be cleared with the symld command like other reservations (AIX / MSCS).
The alternative is to make sure that you run the appropriate Veritas / Sun Cluster command to release the PR on the device before you give it to another server
diskextender
51 Posts
0
November 16th, 2006 13:00
appropriate Veritas / Sun Cluster command to release
the PR on the device before you give it to another
server
Our problem was that our Unix admins were not aware of any commands that would clear the PR on the devices. Could you recommend what should be done (what commands to run) from the Solaris host side to clear PR?
JasonBailey
147 Posts
0
November 16th, 2006 22:00
VERITAS Storage Foundation 4.1 for OracleRAC Installation and Configuration Guide Solaris
http://seer.support.veritas.com/docs/275731.htm
Search on "Removing Existing Keys From Disks" and/or "Using vxfenclearpre Command"