PowerFlex Manager - Connection Failure To VCenter

When trying to update details, add/change options, or resource discovery related to the vCenter resource, the operation fails.

asmManager.log

Example of an unsuccessful discovery and connection:

 2024-01-24 13:37:48,530 DEBUG [JobMgrQuartzScheduler_Worker-76,tid=100] (PuppetDevice.java:176): Discovering puppet device: 10.122.1.1, type=VM Manager
2024-01-24 13:37:48,542 DEBUG [JobMgrQuartzScheduler_Worker-76,tid=100] (PuppetModuleUtil.java:244): In PuppetModuleUtil.discoverAndShowFacts, request.getCredentialId(): b1fc43b0-8bdb-47dc-a35a-c0b7d674f312
2024-01-24 13:37:50,651 DEBUG [JobMgrQuartzScheduler_Worker-76,tid=100] (PuppetModuleUtil.java:373): elapsed time in sec: 2 to inventory vcenter-my.vcenter.hostname
2024-01-24 13:37:54,664 DEBUG [JobMgrQuartzScheduler_Worker-76,tid=100] (PuppetModuleUtil.java:373): elapsed time in sec: 6 to inventory vcenter-my.vcenter.hostname
2024-01-24 13:38:02,674 DEBUG [JobMgrQuartzScheduler_Worker-76,tid=100] (PuppetModuleUtil.java:373): elapsed time in sec: 14 to inventory vcenter-my.vcenter.hostname
2024-01-24 13:38:18,684 DEBUG [JobMgrQuartzScheduler_Worker-76,tid=100] (PuppetModuleUtil.java:373): elapsed time in sec: 30 to inventory vcenter-my.vcenter.hostname
2024-01-24 13:38:18,711 ERROR [JobMgrQuartzScheduler_Worker-76,tid=100] (PuppetDevice.java:320): error discovering Puppet device with IP:10.122.1.1, jobId=Job-018a6ca7-69e3-4750-9bdb-25baa4e2ff29=10.122.1.1
com.dell.asm.asmcore.asmmanager.exception.AsmManagerRuntimeException: Puppet discovery job failed for AsmDeployerDevice{certName=vcenter-my.vcenter.hostname, host=10.122.1.1, discoveryStatus=null}: AsmDeployerDevice{certName=vcenter-my.vcenter.hostname, host=10.122.1.1, discoveryStatus=FAILED}, elapsed time in min = 0
    at com.dell.asm.asmcore.asmmanager.util.PuppetModuleUtil.discoverAndShowFacts(PuppetModuleUtil.java:386)
    at com.dell.asm.asmcore.asmmanager.util.discovery.providers.PuppetDevice.discoverDevices(PuppetDevice.java:278)
    at com.dell.asm.asmcore.asmmanager.tasks.callable.DiscoverDeviceCallable.call(DiscoverDeviceCallable.java:72)
    at com.dell.asm.asmcore.asmmanager.tasks.DeviceInventoryJob.executeSafely(DeviceInventoryJob.java:107)
    at com.dell.asm.asmcore.asmmanager.tasks.AsmDefaultJob.execute(AsmDefaultJob.java:40)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
2024-01-24 13:38:18,715 DEBUG [JobMgrQuartzScheduler_Worker-76,tid=100] (PuppetDevice.java:323): Finished discovering Puppet device with IP:10.122.1.1, status: Error discovering, jobId=Job-018a6ca7-69e3-4750-9bdb-25baa4e2ff29=10.122.1.1
2024-01-24 13:38:18,717 INFO  [JobMgrQuartzScheduler_Worker-76,tid=100] (DeviceInventoryDAO.java:497): DeviceInventoryDAO.updateDeviceInventory for device with refId vcenter-10.122.1.1
2024-01-24 13:38:18,717 INFO  [JobMgrQuartzScheduler_Worker-76,tid=100] (DeviceInventoryDAO.java:498): DeviceInventoryDAO.updateDeviceInventory for device with ipAddress 10.122.1.1
2024-01-24 13:38:18,717 INFO  [JobMgrQuartzScheduler_Worker-76,tid=100] (DeviceInventoryDAO.java:499): DeviceInventoryDAO.updateDeviceInventory for device with serviceTag udmvsca01.prod.treasurywineestates.com
2024-01-24 13:38:18,726 ERROR [JobMgrQuartzScheduler_Worker-76,tid=100] (DeviceInventoryJob.java:117): Discovery by IP range failed for device: serviceTag: null, ipAddress: 10.122.1.1, message: The device with the IP address: 10.122.1.1 cannot be discovered because Puppet module vcenter was not available or failed to return the device information., jobId=Job-018a6ca7-69e3-4750-9bdb-25baa4e2ff29=10.122.1.1

Example of a successful discovery and connection:

 2024-01-24 00:00:00,117 DEBUG [JobMgrQuartzScheduler_Worker-21,tid=45] (PuppetDevice.java:176): Discovering puppet device: 10.122.1.1, type=VM Manager
2024-01-24 00:00:00,118 DEBUG [JobMgrQuartzScheduler_Worker-32,tid=56] (QuartzListener.java:286): --> Entering jobAdded() listener: DeviceInventoryJob.Job-5c629bb1-ef36-4645-9817-92fc7a6f5b37
2024-01-24 00:00:00,120 DEBUG [JobMgrQuartzScheduler_Worker-32,tid=56] (QuartzListener.java:276): --> Entering jobScheduled() listener: trigger: DeviceInventoryJob.Trg-967f3461-0804-4ba6-a0e8-cb04f92631f3, nextFireTime: Wed Jan 24 00:00:00 AEDT 2024
2024-01-24 00:00:00,131 DEBUG [JobMgrQuartzScheduler_Worker-21,tid=45] (PuppetModuleUtil.java:244): In PuppetModuleUtil.discoverAndShowFacts, request.getCredentialId(): b1fc43b0-8bdb-47dc-a35a-c0b7d674f312
2024-01-24 00:00:00,138 DEBUG [JobMgrQuartzScheduler_Worker-1,tid=25] (QuartzListener.java:230): --> Entering triggerFired() listener: 
      JobExecutionContext:
        fireId: NON_CLUSTERED1692107784684
        trigger: DeviceInventoryJob.Trg-967f3461-0804-4ba6-a0e8-cb04f92631f3
        job: DeviceInventoryJob.Job-5c629bb1-ef36-4645-9817-92fc7a6f5b37
        scheduledFireTime: Wed Jan 24 00:00:00 AEDT 2024
        previousFireTime: null
        nextFireTime: null
        isRecovering: false
        refireCount: 0
2024-01-24 00:00:00,138 DEBUG [JobMgrQuartzScheduler_Worker-1,tid=25] (QuartzListener.java:88): --> Entering jobToBeExecuted() listener: DeviceInventoryJob.Job-5c629bb1-ef36-4645-9817-92fc7a6f5b37

Note: The vCenter object name is the certName.

Device_Inventory_logs/<certName>

 The inventory operation failed due to "an incorrect user name or password":

 An error occurred while collecting vcenter_facts.
InvalidLogin: Cannot complete login due to an incorrect user name or password.    <<<<<
/opt/puppetlabs/puppet/lib/ruby/gems/2.4.0/gems/rbvmomi-2.2.0/lib/rbvmomi/connection.rb:63:in `parse_response'
/opt/puppetlabs/puppet/lib/ruby/gems/2.4.0/gems/rbvmomi-2.2.0/lib/rbvmomi/connection.rb:92:in `call'
/opt/puppetlabs/puppet/lib/ruby/gems/2.4.0/gems/rbvmomi-2.2.0/lib/rbvmomi/basic_types.rb:213:in `_call'
/opt/puppetlabs/puppet/lib/ruby/gems/2.4.0/gems/rbvmomi-2.2.0/lib/rbvmomi/basic_types.rb:76:in `block (2 levels) in init'
/opt/puppetlabs/puppet/lib/ruby/gems/2.4.0/gems/rbvmomi-2.2.0/lib/rbvmomi/vim.rb:62:in `block in connect'
/opt/puppetlabs/puppet/lib/ruby/gems/2.4.0/gems/rbvmomi-2.2.0/lib/rbvmomi/vim.rb:45:in `tap'
/opt/puppetlabs/puppet/lib/ruby/gems/2.4.0/gems/rbvmomi-2.2.0/lib/rbvmomi/vim.rb:45:in `connect'
/etc/puppetlabs/puppet/modules/vcenter/bin/discovery.rb:768:in `block in '
/opt/puppetlabs/puppet/lib/ruby/2.4.0/timeout.rb:93:in `block in timeout'
/opt/puppetlabs/puppet/lib/ruby/2.4.0/timeout.rb:33:in `block in catch'
/opt/puppetlabs/puppet/lib/ruby/2.4.0/timeout.rb:33:in `catch'
/opt/puppetlabs/puppet/lib/ruby/2.4.0/timeout.rb:33:in `catch'
/opt/puppetlabs/puppet/lib/ruby/2.4.0/timeout.rb:108:in `timeout'
/etc/puppetlabs/puppet/modules/vcenter/bin/discovery.rb:767:in `' 

Impact

Inability to perform operations involving the vCenter or resource discovery.

Due to external activities, either the vCenter username or password was modified, resulting in the PFxM not being updated accordingly.

The user is responsible for updating the PFxM with any changes made outside of PFxM.

Update the PFxM saved username and password, or change the vCenter username or password manually, to match the one stored in PFxM. 

Impacted Version

N/A - working as expected