How to identify a slot which NVMe SSD for vSAN is installed

Environment
This article uses the case referring to the storage device of ESXi from vSphere Web Client as an example.
Steps in this article show how to identify in which slot the NVMe SSD device shown in the device details list is installed when multiple NVMe SSDs are installed.
This article identifies the device "vmhba1:C0:T0:L0" as an example.

Table of Contents
1.　Identify the Disk Bus ID
2.　Identify the Physical Disk from iDRAC CLI (racadm)

Identify the Disk Bus ID

Since the device C0:T0:L0 has been confirmed exist on vmhba1, use the following lspci or esxcli command to confirm the target VMHBA bus ID.
The followings are each output example.

lspci output example:

[root:~] lspci |grep -i nvme
0000:86:00.0　　Mass storage controller: Dell Express Flash NVMe [vmhba1]
0000:87:00.0　　Mass storage controller: Dell Express Flash NVMe [vmhba2]

esxcli output example:

[root:~] esxcli storage core adapter list |grep -i nvme
vmhba1　　nvme　　link-n/a　　pscsi.vmhba1　　Data Integrity　　(0000:86:00.0) Dell Express Flash
vmhba2　　nvme　　link-n/a　　pscsi.vmhba2　　Data Integrity　　(0000:87:00.0) Dell Express Flash

This result identifies the vmhba1 is the device installed in Bus 86.
Since this number is hexadecimal convert it to decimal and BUS 134 is the bus ID.

Identify the Physical Disk from iDRAC CLI (racadm)

Access to the iDRAC IP address with telnet or SSH and run the racadm hwinventory command.
Since the output results will be large it is recommended to use output saving feature if the terminal software to be used has it.

Search by the string "NVMe" on the output result and confirm the target bus ID from before and after the search result.
Although the output results are different depends on the server model and/or configuration, output result may be like the following results will be output.

Extracts from output example:

-------------------------------------------------------------------
[InstanceID: Disk.Bay.8:Enclosure.Internal.0-1:PCIeExtender.Slot.1]
Device Type = PCIDevice
LastUpdateTime = 2018-10-21T21:47:15
LastSystemInventoryTime = 2019-01-17T23:16:47
SlotType = PCI Express Gen 3 SFF-8639
SlotLength = 2.5 Inch Drive Form Factor
DataBusWidth = 4x or x4
Description = Express Flash NVMe XS1715 SSD 400GB
Manufacturer = Samsung Electronics Co Ltd
PCISubDeviceID = 1F95
PCISubVendorID = 1028
PCIDeviceID = A820
PCIVendorID = 144D
FunctionNumber = 0
DeviceNumber = 0
BusNumber = 134
DeviceDescription = PCIe SSD in Slot 8 in Bay 1
FQDD = Disk.Bay.8:Enclosure.Internal.0-1:PCIeExtender.Slot.1
InstanceID = Disk.Bay.8:Enclosure.Internal.0-1:PCIeExtender.Slot.1
-------------------------------------------------------------------

According to the output example above, it is identified that the vmhba1:C0:T0:L0 on Bus 134 is NVMe SSD #8.

Case where hardware logs show an error state but vSAN show the NVMe device as healthy.

We have seen cases in the field where the hardware (TSR logs) have flagged an NVMe device in an error state while the vSAN shows no issues.  In such a instance, the process to identify the NVMe device is reversed.  However, racadm hwinventory splits the required information into several places.  to identify the device, first search for the device serial number in the text.  You will find a section similar to the following:

[InstanceID: Disk.Bay.7:Enclosure.Internal.0-2:PCIeExtender.Slot.3]
Device Type = PCIeSSD
DeviceDescription = PCIe SSD in Slot 7 in Bay 2
Slot = 7
RAIDType = Unknown
SystemEraseCapability = 2
FreeSizeInBytes = 0
RaidStatus = Unknown
HotSpareStatus = No
FailurePredicted = NO
NegotiatedSpeed = 8 GT/s
MaximumCapableSpeed = 8 GT/s
PCIeCapableLinkWidth = x4
PCIeNegotiatedLinkWidth = x4
ProductID = a822
DeviceProtocol = NVMe-MI1.0
State = Unknown
Manufacturer = SAMSUNG
PrimaryStatus = Error
Revision = 1.1.1   
Model = Dell Express Flash PM1725a 3.2TB SFF    
RemainingRatedWriteEndurance = 100%
MediaType = Solid State Drive
DriveFormFactor = 2.5 inch
SerialNumber = S3B0NX0JB00320
BusProtocol = PCIE
InstanceID = Disk.Bay.7:Enclosure.Internal.0-2:PCIeExtender.Slot.3
FQDD = Disk.Bay.7:Enclosure.Internal.0-2:PCIeExtender.Slot.3
SizeInBytes = 3200631791616
-------------------------------------------------------------------

Note the PrimaryStatus in this case is in an Error state.  Note the InstanceID.  Further search the text file for other occurrances of the InstanceID until you find a segment similar to the following:

[InstanceID: Disk.Bay.7:Enclosure.Internal.0-2:PCIeExtender.Slot.3]
Device Type = PCIDevice
SlotType = PCI Express Gen 3 SFF-8639
SlotLength = 2.5 Inch Drive Form Factor
DataBusWidth = 4x or x4
Description = Express Flash PM1725a 3.2TB SFF
Manufacturer = Samsung Electronics Co Ltd
PCISubDeviceID = 1FDB
PCISubVendorID = 1028
PCIDeviceID = A822
PCIVendorID = 144D
FunctionNumber = 0
DeviceNumber = 0
BusNumber = 102
DeviceDescription = PCIe SSD in Slot 7 in Bay 2
FQDD = Disk.Bay.7:Enclosure.Internal.0-2:PCIeExtender.Slot.3
InstanceID = Disk.Bay.7:Enclosure.Internal.0-2:PCIeExtender.Slot.3
LastUpdateTime = 2019-03-22T08:27:12
LastSystemInventoryTime = 2019-03-22T09:27:13
-------------------------------------------------------------------

From here you can see the BusNumber = 102.  This BusNumber should then be converted to it's hexidecimal value, (in this case 66), which can then be used to locate the vmhba as before.

If you have any questions or doubt to identify, please contact Technical Support with Support Assist log of iDRAC and command result confirmed in "Identify the Disk Bus ID".
If you are in trouble with the physical issue, Technical Support will confirm the issue and arrange an engineer to repair according to identify physical disk has issue or contracts.

Please refer to the following articles to gather the Support Assist log of iDRAC (checking the "OS and Application Data" option is not required for both steps).

• How to export a SupportAssist Collection and the RAID Controller Log via iDRAC 7 and 8?
• How to manually create the SupportAssist Collection with iDRAC 9?