Avamar: Hyper-V 2012 proxy backups fail with "Failed to get the HA VMs from the Avamar Windows Cluster Service"

One of the VMs specified in the dataset might be removed/retired from the CSV cluster for whatever reason.

Hyper-V CSV backups may fail almost immediately and may put the Hyper-V VSS writer on the proxy node in a failed state.

Backups of Hyper-V CSV using Hyper-V 2012 proxy may fail with error messages similar to the following:
 

There can be several contributing factors to this problem.

1) Powershell and/or WMI service(s) maybe in a failed or hung state.

2) It may also be security/permissions issue and the user account that is configured for the Avamar Windows Cluster Service doesn't have sufficient access to the other remote Hyper-V CSV node virtualization namespaces.

3) Multiple Proxy hosts are configured as "Primary" proxy node i.e. "avhypervvss.cmd" file on secondary nodes also contain "--proxy"

4) Finally, it may also be due to the dataset being configured to backup specific VMs; however, if one those VMs no longer exists in the CSV cluster, it will fail the backup.

1)  Run the following command from a Powershell prompt:

Get-WmiObject -Query "Select * From mscluster_resource where Type='Virtual Machine'" -Namespace root\mscluster

If the command fails to run, then rebooting the Hyper-V node maybe needed to fix both the WMI and backup error.

2)  If it still fails, then add the domain user account (the one that is configured for the Avamar Windows Cluster service) to the local Administrators and WinRMRemoteWMIUsers groups on each node of the Hyper-V CSV cluster.  Make sure that the domain user account is part of the Domain Admins group in Active Directory.

3)  Only one node i.e. Primary node should have the following settings in it.

--multi-proxy=node1,node2,node3,node4
--proxy

Rest of the participating/secondary nodes should not have anything at all in their avhypervvss.cmd file.

4)  Edit the dataset and remove the VM that is no longer clustered.  You can verify that the VM is no longer clustered or in a highly-available (HA) state by checking in the Failover Cluster Management tool.