Symptoms
Setup PPME migration handle and didnt exclude the 2 nodes of the 6 node cluster that did not have access to the source and target device
In a MSCS with SQL Always On cluster ran PPME migration handle setup
powermig setup -techType hostcopy -src harddiskX -tgt harddiskXX -cluster mscd
handle was able to be setup fine
PS C:\Windows\system32> powermig sync -handle c7
Start sync for handle c7 ? [yes]/no: y
PPME error(79): Not all nodes have reached the correct state to complete this operation, please try this or the recover command again later
Cause
Applicable to SQL AlwaysOn environment only.
Cluster had 6 nodes seen by MS Failover cluster and implementing SQL AlwaysOn. 2 of the 6 nodes were part of a different DataCenter but were all part of the same cluster.
It was assumed that since 2 of the 6 nodes are in a different Datacenter and that since those 2 nodes would never see the same luns as nodes 1-4 that those nodes weren't needed to be excluded during the PPME handle setup phase. It was assumed that the nodes couldn't even talk to each other across datacenters in PPME migration. It was assumed that the additional 2 nodes weren't even part of the same cluster.
Resolution
Applicable to SQL AlwaysOn environment only.
If PPME luns are being migrated i.e. are being used in the ppme migration they must be seen by all nodes in the cluster
If you have nodes in that cluster that cant see the luns in question, the handle will setup fine but once attempting a sync on that handle it will report "PPME error(79): Not all nodes have reached the correct state to complete this operation, please try this or the recover command again later."
You must either make access to the source and target device to all nodes when in a Geo Cluster setup or
exclude the nodes that did not have access to the source and target device in geo-cluster or SQL AlwaysOn, or
exclude the nodes that wont participate in the PPME migration
When setting up PPME handles, if part of a cluster, all nodes must have access to all the devices.
Using the exclude_nodes option, you can exclude nodes that do not have access to the source and target device when in a Geo Cluster setup
Ran powermig setup -techType hostcopy -src harddiskX -tgt harddiskXX -cluster mscd -exclude_nodes NODE5 NODE6
Excluded nodes 5 and 6 and were able to sync the handle successfully
So in order do the following:
powermigcl display
powermigcl config or powermigcl config -all -force
powermigcl display
powermig setup -techType hostcopy -src harddiskX -tgt harddiskX -cluster mscd -exclude_nodes NODE5 NODE6
handle created successfully
powermig info -all
validate handle created and handle number
powermig sync handle X
powermig query -handle X
when the migration is completed and commited
very important "always cleanup handle 1st, then run the powermigcl unconfig*
powermig cleanup -handle
powermig info -all
handles should be removed now
then you can do your powermigcl unconfig -all or powermigcl unconfig -all -force
powermigcl display
make sure resources show as unconfigured
Now you are free to move your cluster resources without handles getting stuck within Powerpath PPME
Beginning with PowerPath for Windows 6.0 SP1, you can perform migrations in Geo Cluster (multi-site cluster) SQL Always On environment only setup where in not all nodes have access to all the devices. In earlier PowerPath releases, all nodes needed to have access to all devices for the migration to succeed. Using the exclude_nodes option, you can exclude nodes that do not have access to the source and target device when in a Geo Cluster setup. Migration Enabler will not look for any status or updates from excluded nodes and the nodes that are excluded cannot have migration devices connected during a migration. You cannot exclude the migration node. You will receive an error if the node cannot be excluded or if the excluded node is not present in the cluster or is not reachable. Migration Enabler supports only SQL AlwaysOn replication technology in a Geo Cluster setup.
https://support.emc.com/docu56488_PowerPath_Migration_Enabler_6.x_User_Guide.pdf?language=en_US
EMC PowerPath Migration Enabler Version 6.x User Guide
https://support.emc.com/docu56486_PowerPath_Family_6.x_CLI_and_System_Messages_Reference.pdf?language=en_US
PowerPath Family Version 6.x CLI and System Messages Reference
**in EMC hostgrab you can validate the cluster nodes info in cluster_mps_information.txt file
Products
VMAX, VNX/VNXe