VPLEX: Fabric Merge

Sean,

Thanks for such a quick response... This confirms what I have been reading about... so my issue is, is the cross connect required? I am struggling to understand the concept (I am NOT a SAN guy, more VMware)

Is the cross connect required to allow me to vmotion from one site to another?

Can I do everything Uniform using a Non-Uniform design?

Can the cross connect be implemented without merging fabrics?

Am I missing any features if go with Non Uniform (no cross connect?) rather than Uniform?

As you can tell, i'm confused my brain is failing to allow me to understand this, hence the 'noddy' questions!

I'm going to read those links you have sent me in the meantime. Thanks for posting up.

Roo

Roo,

With VPLEX Metro, you do NOT need to merge fabrics across your two sites. This is generally referred to as "Non-uniform Host Access," as the hosts at a given site only "see" storage volumes through their local VPLEX cluster.  The alternative is "Uniform Host Access", wherein the hosts at a given site can see the storage clusters at both sites. Typically this latter option requires merging fabrics. In VPLEX parlance, a Uniform Host Access environment uses what we call "cross-connect."

VPLEX supports both Uniform (cross-connect) and Non-uniform Host Access, but in my experience most customers tend to lean towards Non-uniform Host Access (independent fabrics at each site), as it is simpler and easier to manage.

Here are a couple resources with more info:

VMware KB: Implementing vSphere Metro Storage Cluster (vMSC) using EMC VPLEX

http://www.emc.com/collateral/software/white-papers/h11065-vplex-with-vmware-ft-ha.pdf

Thanks,

- Sean

Hi

You can certainly vmotion without cross connect. You would build your data stores on Distributed virtual volumes.

You cannot implement cross connect without merging fabrics.

Steve

Sent from my iPad

Hi Roo,

Cross-connect is definitely not required, and you get the same functionality both with and without cross-connect -- you can still use Vmotion, HA, DRS, etc. in both cases. Generally you cannot implement cross-connect without merging fabrics. The exception to this would be employing something like inter-VSAN routing, in which case you still have Fibre Channel connectivity between sites, but you're not merging logical fabrics -- you're routing between logical fabrics with IVR (kind of like routing between two distinct subnets in the IP world -- the subnets are not merged, but they can still communicate with each other at a higher layer). With Non-uniform Host access (no cross-connect), you do not need any FC connectivity between sites at all.

Basically the advantage to cross-connect is a failure scenario where an entire VPLEX Cluster at one site fails, but the servers at that site survive. I've never seen it happen, but in this rare situation, the workloads running on the surviving servers would be able to use the cross-connect zones to communicate with the remote VPLEX Cluster, so workloads running on those servers/VMs would remain running.

Without the cross-connect zone (Non-uniform host access), the workloads running on those surviving servers would terminate, and server clustering would kick in to restart them at the other site (e.g. via VMware HA or Windows Failover Clustering) -- essentially it would mimic the behavior of an ESXi or Windows cluster node server failing; it would just happen across sites.

The downside to cross-connect is essentially complexity. You have double the number of zones/paths (which all count against VMware's path limits), and because half of the paths are across sites, you would not normally want to use the cross-site zones (due to the higher latency associated with distance). So you have to set the cross-sites paths into standby mode, so that they're only used in the event that the primary paths fail. If your MPIO stack happens to be EMC PowerPath or PowerPath/VE, this is pretty simple -- because PowerPath understands VPLEX, and will automatically set the paths with higher latencies into standby mode (aka "Proximity-based auto standby"). For other MPIO stacks (e.g. VMware's NMP), you have to do this manually, which is labor-intensive and prone to error. Additionally, the VMware Round Robin PSP can't even do this manually, so you have to use the Fixed PSP -- which means any given LUN only has one active path, and you must manually balance/rotate the active path for each LUN to achieve any semblance of balance.

Most (if not all) alternatives to VPLEX Metro do not support Non-uniform Host Access, so we generally consider this capability to be a differentiating advantage of VPLEX's architecture.

Hope that helps,

- Sean

OMG Sean! What do you do for a living to be able to explain with such ease!?!

Right then... I think I'm at a place where it's starting to make sense and I haven't read those links you sent me yet...

so...

If I don't have cross connect (the additional pathing does concern me and the crossing of a WAN link to get to the disks also concerns me from an Application performance perspective, SQL servers maybe if anyone has dared virtualising SQL and rely on VMware technology to protect it) and my vplex cluster fails in one site but servers stay alive, rather than rely on VMware HA can VMware FT bring the servers up on the other side without downtime to the VM? I guess what we are saying in this scenario is the VM will think its disks have been yanked right, what that does at the hypervisor level I have no idea, but certainly all virtual machine files will be on that local vplex that's just been yanked from it. Is this a reasonable question to ask or am I just thinking about this too much?

On the other hand if I do have cross connect via IVR, how do I get myself out of this criss-cross situation where the vplex in one site fails (but servers don't) and I'm now traversing the IVR to get to my disks? simply vmotion? I guess the priority in this case would be to get the re-alignment back...

Hey intech

thanks for the response... I've been told by our SAN/Storage team we have Inter VSAN routing in place, when I asked for what reason, I got blank faces and no one could tell me why! ... this might help me a great deal I'm guessing...

would I need IVR for any other reason than trying to implement VPLEX... I really don't like the idea of presenting disks from one site to servers in another... unless its a last last resort

Roo

Hi Roo,

Glad to help... Explaining this sort of thing is my day job

You can run VMware FT over VPLEX Metro, but typically this is deployed in a cross-connect configuration (and it requires less than 1ms round-trip time between sites). Incidentally, cross-connect also has a <1ms RTT requirement -- so if your latency between sites is greater than 1ms RTT, then both FT and cross-connect are out. Running VMware FT over VPLEX Metro without cross-connect may be possible too, but we'd need to go through an RPQ process to qualify support for it -- your EMC SE can help with this if it's something you wanted to pursue.

If you do move forward with it, I'd also highly recommend PowerPath over native MPIO stacks -- as PowerPath automates the process of setting the cross-connect paths into standby mode, which really helps to reduce complexity & manual labor.

And yes, if you ended up in a state where your VMs/apps are accessing their storage over the cross-connect, you can just Vmotion those VMs to the other site (or fail over your cluster resources for WFC).

Thanks,

- Sean

Sean,

Thank you so much... so can I just ask a couple more inline questions please, to hopefully wrap things up?

You can run VMware FT over VPLEX Metro, but typically this is deployed in a cross-connect configuration (and it requires less than 1ms round-trip time between sites). Incidentally, cross-connect also has a <1ms RTT requirement -- so if your latency between sites is greater than 1ms RTT, then both FT and cross-connect are out.

So my options to host a single VM with RTO=0 using VMware FT could well be limited I operate at >1ms RTT? I guess I would need to assess the application service as a whole and build it out cross datacenter using Global site type load balancers... although this will never work well for SQL driven applications.

Running VMware FT over VPLEX Metro without cross-connect may be possible too, but we'd need to go through an RPQ process to qualify support for it -- your EMC SE can help with this if it's something you wanted to pursue.

Do you know if this has been successful with other companies and normally what factors are considered as to whether we get quaified or not? just high level stuff will do for me.

If you do move forward with it, I'd also highly recommend PowerPath over native MPIO stacks -- as PowerPath automates the process of setting the cross-connect paths into standby mode, which really helps to reduce complexity & manual labor.

Is this something that needs to be deployed to the ESXi build? Do you have anything basic I could take away to read? I've always relied on ESXi to just sort my pathing out for me...

And yes, if you ended up in a state where your VMs/apps are accessing their storage over the cross-connect, you can just Vmotion those VMs to the other site (or fail over your cluster resources for WFC).

Perfect! Thank you very much

Sean,

I believe all my 'high level' questions have been answered... better than what I expected. May I take this opportunity to thank you for your time and detailed responses in helping me understand these concepts. Very much appreciated and hopefully enough for me get VPLEX implemented into our environment. I'll be sure to let you know how I get on with the business.

Many many thanks for the invaluable knowledge transfer...

Closing this thread out... absolutely awesome speaking to you!

Roo

Thanks for the kind words Roo! Glad I was able to help. Great chatting with you too, and best of luck with your VPLEX implementation!

Thanks,

- Sean