metas not visible to host in cluster environment

What is the OS? What kind of disk?

DX,

Have you labelled the device using format ? You need to do that first if its a new device.

If you are re-using an existing device that's been previously assigned to another host, then this can happen if the other host has left a SCSI reservation on the device.

A good telltale sign of this is if you run "inq" and the capacity is not shown.

You can also check to see if that is the case by running a "symdev list -resv" and see if your device appears in the list.

If there is a reservation, and you are absolutely certain no-one else is using it, log a call and get your EMC CE to clear it for you.

Glen.

Thank you for the suggestions. It was the most promising resolution yet, but didn't work.

The symdev list -resv returned - "No devices were found"

The syminq returns all the drives and shows their capacity.

The problem isn't that the volumes are not visible. The volumes are visible, but cannot be accessed. I'm unable to label a disk - getting the error "Warning: error writing VTOC - Label failed". I'm unable to create a filesystem or dd the disk.

To answer the other question:OS level: Solaris 9. There is no other software installed on the system, just powerpath and the OS.
Type of disk: We have 3 quorum devices assigned to both systems, several 2gb mirrored metas and several 34gb mirrored metas (all seen by both systems). The thing is, only 34gb metas are not writable, the rest of them are fine. PSE lab checked the metas and there are NO celerra flags on them ( I know it might cause the problem). Other thing: we have one 34gb meta assigned exclusively to each cluster node, and we don't have any issues writing to those metas.

Did you check whether the metas are actually Read/Write?

symdev list should show RW, not WD in the next to last column (STS).

Yes, we did check the metas and all of them are showing is read/write. The wierd thing is that we were able to see those metas a while ago. I rember back then we install Veritas Oracle RAC cluster, then we rebuilt the systems and installed Oracle OCF cluster software. Not sure if these disks were initiallized under Veritas if something was done to them. Seems like something was changed on the host side, not the SAN side.

We tried one more thing: I unmasked one of the "problem" metas from the solaris cluster and assigned it to a windows host. I'm able to discover and see the meta, but not able to write a signature to it (or in other words initialize it). Getting the following error message:"An unexpected error has occured. checkc they system even log for more info" Event log shows error event of source LDM, event ID 2. The description of the event is the following: "Unspecified error (80004005)"

would you mind posting a symdev show {symdev number} ?

Here is the output of symdev show on the device that I assigned to a windows host (it is assigned to the host where i executed the command):
C:\Documents and Settings\erduncann>symdev show 2d3

Symmetrix ID: XXXXXXXXXX

Device Physical Name : \\.\PHYSICALDRIVE7

Device Symmetrix Name : 02D3
Device Serial ID : 53002D3000
Symmetrix ID : XXXXXXXXXX

Attached BCV Device : N/A

Attached VDEV TGT Device : N/A

Vendor ID : EMC
Product ID : SYMMETRIX
Product Revision : 5671
Device WWN : 60060480000187870253464241324433
Device Emulation Type : FBA
Device Defined Label Type: N/A
Device Defined Label : N/A
Device Sub System Id : 0x0003

Device Block Size : 512

Device Capacity
{
Cylinders : 73656
Tracks : 1104840
512-byte Blocks : 70709760
MegaBytes : 34526
KiloBytes : 35354880
}

Device Configuration : 2-Way Mir (Meta Head,
Non-Exclusive Access)

Device is WORM Enabled : No
Device is WORM Protected : No

SCSI-3 Persistent Reserve: Enabled

Dynamic Spare Invoked : No

Dynamic RDF Capability : None

Device Service State : Normal

Device Status : Ready (RW)
Device SA Status : Ready (RW)

Front Director Paths (6):
{
----------------------------------------------------------------------
POWERPATH DIRECTOR PORT LUN
--------- ---------- ---- -------- ---------
PdevName Type Type Num Sts VBUS TID SYMM Host
----------------------------------------------------------------------
\\.\PHYSICALDRIVE7 PARENT FA 07B:1 RW 000 00 054 054
Not Visible N/A FA 08B:1 RW 000 00 02D N/A
Not Visible N/A FA 09B:1 RW 000 00 02D N/A
Not Visible N/A FA 10B:1 RW 000 00 054 N/A
Not Visible N/A FA 07C:1 RW 000 00 009 N/A
Not Visible N/A FA 10C:1 RW 000 00 009 N/A
}

Meta Configuration : Striped
Meta Stripe Size : 960k (2 Cylinders)
Meta Device Members (4) :
{
----------------------------------------------------------------------
BCV DATA RDF DATA
---------------------------- --------------------------
Sym Cap Std Inv BCV Inv Pair R1 Inv R2 Inv Pair
Dev (MB) Tracks Tracks State Tracks Tracks State
----------------------------------------------------------------------
--> 02D3 8632 - - N/A - - N/A
02D4 8632 - - N/A - - N/A
02D5 8632 - - N/A - - N/A
02D6 8632 - - N/A - - N/A
----------------------------------------------------------------------
34526 - - - -
}

Mirror Set Type : [Data,Data,N/A,N/A]

Mirror Set DA Status : [RW,RW,N/A,N/A]

Mirror Set Inv. Tracks : [0,0,0,0]

Back End Disk Director Information
{
Hyper Type : Data
Hyper Status : Ready (RW)
Disk [Director, Interface, TID] : [11B, D, 4]
Disk Director Volume Number : 276 (0x113)
Hyper Number : 4
Disk Capacity : 69879m
Disk Group Number : 2

Hyper Type : Data
Hyper Status : Ready (RW)
Disk [Director, Interface, TID] : [02D, C, 4]
Disk Director Volume Number : 36 (0x23)
Hyper Number : 4
Disk Capacity : 69879m
Disk Group Number : 2
}

Message was edited by:
Julie Gibson
I XXXX'd out the symmetrix ID, that's what we do in Support Solutions so it seems best in this environment too.

To anyone who would be interested on what the resolution to this problem is.
One solution of course if VTOCing all the devices from the Symmetrix side, but it would require our CE involved and a lot of work
What worked for us is to change SCSI persistent reserve flag back to "disabled" and it solved the issue: we were able to write to those devices again.

Actually, a VTOC would not have cleared the persistent reservation (PR).

VTOC is used to clear celerra signatures. The signature is actually written to the disk by the celerra, its not a flag, so thats why a VTOC is needed. I think you can also DD to the disk as an alternative.

The celerra signature problem would not cause you to be unable to write to the disk, it would just mean Powerpath would hide the disk from the O/S. Makes for very confused and frustrated SAN admins calling in

So the solution to remove the PR was to remove the PER bit from the device, which is not a great solution but it works. I would have hoped it could be cleared with the symld command like other reservations (AIX / MSCS).

The alternative is to make sure that you run the appropriate Veritas / Sun Cluster command to release the PR on the device before you give it to another server

Our problem was that our Unix admins were not aware of any commands that would clear the PR on the devices. Could you recommend what should be done (what commands to run) from the Solaris host side to clear PR?

I looked it up for VCS. Try the following manual:

VERITAS Storage Foundation 4.1 for OracleRAC Installation and Configuration Guide Solaris

http://seer.support.veritas.com/docs/275731.htm

Search on "Removing Existing Keys From Disks" and/or "Using vxfenclearpre Command"