Dell PowerFlex 4.5.x Install and Upgrade Guide

Prepare an NVDIMM as a DAX (acceleration) device

Prepare a new or replacement NVDIMM as a DAX or acceleration device before adding it to the SDS in a Linux system.

1. Using SSH, log in to the Linux server.
2. View information for the new or replacement NVDIMM:

   dmidecode --type memory | grep "Non-" -B 3 -A 3 | grep -E 'Locator|Serial' | grep -v Bank

   Output similar to the following appears:

   Locator: A7
   Serial Number: 17496594
   Locator: B7
   Serial Number: 174965AC
   

3. Find the serial number in the output and record it in the NVDIMM information table.
4. Display the correlation between the ID and NMEM device name of each NVDIMM mounted on the server:

   ndctl list -Dvvv | jq '.[].dimms[]'

   Output similar to the following appears:

   {
     "dev": "nmem1",
     "id": "802c-0f-1722-174965ac",
     "handle": 4097,
     "phys_id": 4370,
     "health": {
       "health_state": "ok",
       "temperature_celsius": 255,
       "life_used_percentage": 30
     }
   }
   {
     "dev": "nmem0",
     "id": "802c-0f-1711-16492521",
     "handle": 1,
     "phys_id": 4358,
     "health": {
       "health_state": "ok",
       "temperature_celsius": 255,
       "life_used_percentage": 30
     }
   }
   

5. In the output from the previous step, find the device (dev) with the id that partially correlates with the serial number you discovered previously for the failed device.
   For example:

   • The NVDIMM output displays serial number 16492521 for the NVDIMM device.
   • In the previous step, the output displays the ID of device nmem0 as 802c-0f-1711-16492521.

   Serial number	Device ID
   Locator: A7 Serial Number: 16492521 Locator: B7 Serial Number: 1649251B	"dev": "nmem0", "id": "802c-0f-1711-16492521",

   In the above example, the NMEM name is nmem0.

6. Record the NMEM name in the Device name row of the NVDIMM information table.
7. Correlate between the NMEM DIMM and the namespace/DAX device (acceleration device):

   ndctl list -Dvvv | jq '.[].dimms[]'

   Output similar to the following appears:

   {
     "dev": "nmem1",
     "id": "802c-0f-1722-174965ac",
     "handle": 4097,
     "phys_id": 4370,
     "health": {
       "health_state": "ok",
       "temperature_celsius": 255,
       "life_used_percentage": 30
     }
   }
   {
     "dev": "nmem0",
     "id": "802c-0f-1722-17496594",
     "handle": 1,
     "phys_id": 4358,
     "health": {
       "health_state": "ok",
       "temperature_celsius": 255,
       "life_used_percentage": 30
     }
   }
   
   ndctl list -Dvvv | jq '.[].regions[].namespaces[]'
   
   {
     "dev": "namespace1.0",
     "mode": "raw",
     "size": 17179869184,
     "sector_size": 512,
     "blockdev": "pmem1",
     "numa_node": 1
   }
   {
     "dev": "namespace0.0",
     "mode": "raw",
     "size": 17179869184,
     "sector_size": 512,
     "blockdev": "pmem0",
     "numa_node": 0
   }
   

8. To identify the new NVDIMM. You can run the command below to find the NVDIMM region with no associated namespace.
   For example, region1 use to have namespace1.0 until the NVDIMM was replaced.

   ndctl list --regions --verbose
   {
     "regions":[
       {
         "dev":"region1",
         "size":17179869184,
         "available_size":17179869184,
         "max_available_extent":17179869184,
         "type":"pmem",
         "numa_node":1,
         "iset_id":-4649799281728009192,
         "persistence_domain":"unknown"
       },
       {
         "dev":"region0",
         "size":17179869184,
         "available_size":0,
         "max_available_extent":0,
         "type":"pmem",
         "numa_node":0,
         "iset_id":4335710697273129127,
         "persistence_domain":"unknown",
         "namespaces":[
           {
             "dev":"namespace0.0",
             "mode":"devdax",
             "map":"dev",
             "size":16909336576,
             "uuid":"265ef925-571d-4a0a-bb96-dca4a3873000",
             "raw_uuid":"ff7b05b9-510e-4604-8329-7f3dccf10300",
             "daxregion":{
               "id":0,
               "size":16909336576,
               "align":4096,
               "devices":[
                 {
                   "chardev":"dax0.0",
                   "size":16909336576
                 }
               ]
             },
             "align":4096,
             "name":"",
             "numa_node":0
           }
         ]
       }
     ]
   }
   

   The NVDIMM that was replaced or sanitized will reuse the same region[x] but will not have a configured namespace.

   On storage-only nodes, the region carries over from the old NVDIMM to the newly installed or sanitized NVDIMM. This is the device that you want to create a namespace for.

   For example, the new NVDIMM shows up in the command above as region1. The command below will create a dax device and namespace based on the region.

9. Run this command to create the namespace, and DAX on the new NVDIMM labeled region1.

   ndctl create-namespace --mode=devdax --region=region<X> --align=4k

   For ndctl build 61.2 or later, run:

   ndctl create-namespace --mode=devdax --region=region<X> --align=4k --no-autolabel

   NOTE:--no-autolabel is only required for Red Hat Enterprise Linux 7.6.

   For example,

   ndctl create-namespace --mode=devdax --region=region1 --align=4k
   {
     "dev":"namespace1.0",
     "mode":"devdax",
     "map":"dev",
     "size":"15.75 GiB (16.91 GB)",
     "uuid":"e47c3ce0-8a98-4256-8fed-ff2bacead155",
     "daxregion":{
       "id":1,
       "size":"15.75 GiB (16.91 GB)",
       "align":4096,
       "devices":[
         {
           "chardev":"dax1.0",
           "size":"15.75 GiB (16.91 GB)"
         }
       ]
     },
     "align":4096
   }
   

10. Run this command again to validate the namespace was created. In this example, you can see namespace1.0 and dax1.0 was created for region1 NVDIMM.

    ndctl list --regions --verbose
    {
      "regions":[
        {
          "dev":"region1",
          "size":17179869184,
          "available_size":17179869184,
          "max_available_extent":17179869184,
          "type":"pmem",
          "numa_node":1,
          "iset_id":-4649799281728009192,
          "persistence_domain":"unknown"
        },
      {
        "dev":"namespace1.0",
        "mode":"devdax",
        "map":"dev",
        "size":16909336576,
        "uuid":"e47c3ce0-8a98-4256-8fed-ff2bacead155",
        "chardev":"dax1.0",
        "align":4096
      },
        {
          "dev":"region0",
          "size":17179869184,
          "available_size":0,
          "max_available_extent":0,
          "type":"pmem",
          "numa_node":0,
          "iset_id":4335710697273129127,
          "persistence_domain":"unknown",
          "namespaces":[
            {
              "dev":"namespace0.0",
              "mode":"devdax",
              "map":"dev",
              "size":16909336576,
              "uuid":"265ef925-571d-4a0a-bb96-dca4a3873000",
              "raw_uuid":"ff7b05b9-510e-4604-8329-7f3dccf10300",
              "daxregion":{
                "id":0,
                "size":16909336576,
                "align":4096,
                "devices":[
                  {
                    "chardev":"dax0.0",
                    "size":16909336576
                  }
                ]
              },
              "align":4096,
            }
          ]
        }
    

11. Record the acceleration device name in the NVDIMM information table.
12. Find the full acceleration device path:

    	ls -ltr /dev/ | grep dax

    The following output should appear:

    /dev/daxX.X

    For example:

    /dev/dax0.0

13. Record the acceleration device path in the NVDIMM information table.