Feedback on VS2 to VS6 conversion

Hi We recently convert a 4 engines Metro cluster from VS2 to VS6, unfortunately without any recover point inside.

Last Monday we proceed to cluster 1 conversion , it take around 12 Hours with almost 3 hours lost on cables issues.

Today , we convert cluster 2

Procedure start with about 3 hours to prepare the conversion  ( health check, backup config , install FC-IOM in slot 4 of the VS6, cabling local comm from VS6 to VS2 .. )

Then the conversion itself is quite fast ( 10 to 15 mn per directors )

We spent anther slot of 2 hours , to switch IP add on management station and restart properly Vpn  and Ipsec

Then the removal parts of FC4-IOM in slot 4 of VS6 take 15 mn per director.

No specific issue during the process

new VS6 cables testing should be a parts of the preparation if you dont want to loose many time during the process itself

one issue to restart VPN dialog from VS2 cluster 2 and VS6 cluster 1 after switching management server IP add.

The old VS2 Cluster 1 was always involved in the dialog , so we have to:

stop vpn on old VS2 Cluster 1

restart vpn on new VS6 Cluster 1 and VS2 Cluster 2

After this action , all was up and running , this parts is not well documented in the procedure

Regards

Christian

Hello @dupasc

Glad to know that it was a great success ,appreciate the details too .

Indeed few more question to the point

     >>I assume it has with quite moderate number of volumes and CGs ?OR the VS2 was pretty less on load ?

     >>On cable check which you highlighted are you talking about FE ,WANCOM and Admin ports ?

     >>Did you rack stack install VS6 in advance and just deal the software level migration or the hours you described included the physical stack and other operations ?

     >>Also wonder and i assume you are experiencing all the advantages of VS6 comparing with Vs2 by now ?

hi ,

for this first migration , yes this is a small and pretty less loaded config.

This is a customer pré production, we plan to transform production  next month ( 2 other Metro )

Cable problems were on FE and Wan

My previous message was for the conversion of Cluster 1 only , the cluster 2 was done  last Wednesday.

Unfortunately this one was not so good.

Field has decide to win some time and install the Fiber channel in slot 4 of VS6 before the request , they have also prepare the local com cables between the VS2 and VS6. Cables was seated on the B300 side and loosely seated on the VS6 side .

Unfortunately one of the cables was in fact tightly seated by mistake and the VS2 has discovered the unexpected link  before the script prepare the VS2 context for this discovering.

The Vplex VS2  quorum was failing ( discover 9 local directors instead of 8 ), the vplex code has protected the data by stopping all the FE on the VS2.

We have to rollback the situation and waiting now a new slot to convert this side of  the Metro.

Ok and unfortunate that you could not get this thorugh this time .I kindly request you to keep update here once you done with the conversion .

Thanks alot

I posted about our upgrade here http://thefluffyadmin.net/?p=929

The upgrade was not completely smooth. The current version of the upgradescript that EMC uses is not very robust. Meaning that if it runs into 1 issue, it will fail and you cannot easily go back or redo anything.  EMC engineering then has to go into the JSON file and reset things, or due various steps like the i/o transfer step manually.

Cool ,So that assumed to be a successful conversion or did you face some sort of concerns over upgrade ?

Yes conversion was completely successful. We are operating the new VS6 cluster just like it was the old one. But with twice the performance.  it does not impact you operationally at all.

My main concern was loosing both directors at the same time. Considering the manual cabling work required, and the fickleness of the upgrade script, this is a risk. But as we have a metro cluster, I was able to move all production VMs to the other datacenter first. So all that was left was local, non-replicated VMs that where not production anyway.

The reason the script failed on the second day with the second cluster is not entirely clear. As the VS6 director was coming online for the i/o transfer, it timed out in the upgrade script with an error about not being able to establish quorum. This may have been as a result of either a wan-com port being incorrectly closed, or local-com (the Infiniband ports) being incorrectly down.  When the script fails halfway, it can leave the directors and settings in an inconsistent state.

We had engineering on a webex with us while the local EMC guys did the upgrade. This was crucial in saving time. I would much recommend you insist that engineering is on standby or with you on webex when you do the upgrade.