This post is more than 5 years old
112 Posts
1
7752
VPLEX + ESXi Round-Robin Multipath
I’m interested in the new feature allowing ESXi hosts using a “Round-Robin” policy for multi-pathing to VPLEX instead of fixed-path.
This has been presented during "VPLEX advanced configuration" during last EMCworld 2014.
Since "fixed-path" config was the "best-practise" for EMC, where can I find some more info about the new feature? (VPLEX version, ESXs version, benefits, pros and cons, etc.)
Thank you.
garyo
89 Posts
1
May 22nd, 2014 23:00
Hi,
Internally we're working to update our documentation around recommending round-robin over fixed path. It'll be probably a couple of weeks before it's published publicly. The EMC VMware host connectivity guide will be updated.
The main benefit of recommending round-robin over fixed path, is that for fixed path it was very manual and difficult to get the lun ownership balanced properly across all VPLEX directors. For hundreds of hosts and thousands of volumes you can imagine how difficult of a task this was. Not to mention that even if you did happen to perfectly manually load balance things, then on a path failover from the host it is completely arbitrary which path was chosen as the next path.
Round-robin makes all this much easier, and behaves in a similar manner to any round-robin multipathing policy which is what PowerPath Adaptive (intelligent round-robin policy based upon path latency and outstanding I/O's) has been doing from the very beginning. There is the potential for some additional inter-director cache-coherency traffic however that additional traffic is very minimal and far outweighs the potential for imbalance that existed with Fixed path policy.
VMware's Compatibility Guide will outline which ESXi versions go with which GeoSynchrony versions.
VMware Compatibility Guide: Storage/SAN Search
In short, GeoSynchrony 5.2 and 5.3 will be round-robin preferred along with newer ESXi versions (5.1, 5.5.)
Hope that helps,
Gary
matteo_marchett
112 Posts
0
May 23rd, 2014 01:00
Perfect! Thanks a lot for your clear explanation. I'll keep on eye on the compatibility guide and host connectivity guide.
matteo_marchett
112 Posts
0
July 24th, 2014 07:00
Any official news? From support.emc.com I can see that
"Host Connectivity Guide for VMWare ESX Server" has not been updated yet.
Thank you
SprinkleJames
23 Posts
0
August 7th, 2014 08:00
I'd like to see the EMC docs updated too. But if you want something official, the VMware Compatibility Guide does list Round Robin. So I'm going to go ahead and recommend RR, except in the case of cross-connected Metro Cluster.
Here's the link.
It would be nice if we could define a preferred director or preferred ports for each volume in a storage view, and have VPLEX advertise those ports as ALUA optimized. That would be a way to eliminate the manual load balancing on the host side, and the issue with inter-director cache traffic.
garyo
89 Posts
0
August 19th, 2014 08:00
Hi - the targeted release update to the ESX Host connectivity guide stating that RR is preferred for VPLEX is Q3/14.
Gary
1gar2nMRBp12266
14 Posts
1
November 25th, 2014 07:00
Yes. This applies to all supported (on VPLEX support matrix) versions of VPLEX code.
-Don
matteo_marchett
112 Posts
0
November 25th, 2014 07:00
So, in my case I have ESXi 5.1 and 5.5 hosts and Geosynchrony 5.2.
Can you double-confirm the current recommended setting is Round-Robin?
Thanks again.
1gar2nMRBp12266
14 Posts
1
November 25th, 2014 07:00
The VPLEX best practices guides have been recently updated and are the most accurate info available. Here is a link to the latest guide, see page 11:
http://www.emc.com/collateral/technical-documentation/h13547-vplex-host-multipathing-best-practices.pdf
matteo_marchett
112 Posts
0
November 25th, 2014 07:00
Hi garyo, can you please confirm that as I can see from here
https://support.emc.com/docu5265_Host-Connectivity-Guide-for-VMWare-ESX-Server.pdf?language=en_US
Round Robin multipathing is now the recommended setting? Does it apply for VPLEX 5.2 as well?
Thanks.
bhalilov1
114 Posts
0
November 25th, 2014 09:00
Please correct me if I'm wrong, but wouldn't round robin cause really inefficient caching ?
garyo
89 Posts
1
November 25th, 2014 09:00
Hi there,
Yes, as Don says, Round-robin is preferred over Fixed.
But I stress that there's technically nothing functionally incorrect with Fixed, it's just that round-robin is a much easier means to load balance. Fixed policy has so much manual effort to get right (every host, every volume) and non-deterministic failover properties (no idea which path will get picked should the primary fail) that round-robin is simply "set it and forget it".
Gary
matteo_marchett
112 Posts
0
November 26th, 2014 03:00
Garyo, the one you described is just my case. I've recently presented new luns with vplex and I've added new physical hosts to the environment. Keeping everything manually balanced with fixed path policies it's really really hard. I have to do thousands of clicks or copy-and-paste luns IDs in a script code to set it up.
That is really time consuming and prone to errors.
I'll go with the Round-Robin configuration on all my physical hosts, it's really easier to manage.
Thanks for your feedback.
garyo
89 Posts
0
November 27th, 2014 08:00
Hi Burhan,
I think inefficient is too strong of a word. Yes, you have the potential for more duplicated read cached pages among directors with a round-robin policy, but I believe that the probability of that happening is low. (A host frequently reading the same page over and over again surely has it cached locally?)
Writes that have invalidate cache pages on other directors do increase the possibility of additional inter-director messages in round-robin that you otherwise don't get in a Fixed policy environment.
Do you get more inter-director cache-coherency chatter among directors in round-robin situation vs. Fixed? Sure, but I believe that the additional overhead of that is small. In our performance lab, we test the differences between the two policies, and the latency differences are very small - near negligible in my opinion. Inter-director messaging within a cluster is usually quite small - tens of microseconds usually, typically less than say 100usec.
If anything round-robin-ing across more directors has the potential to increase the total available cache for a given volume. Each VS2 director offers roughly 32GB of usable cache space. Now, how the cache gets used is entirely dynamic and naturally is highly an "it depends" kind of situation - number of volumes, size of volumes, total addressable space of those volumes, read vs. write ratio, etc... So a four director storage-view just might increase the total available space for caching.
VPLEX cache stats reporting does leave a lot to be desired, since right now it doesn't easily show you things like percent cache hits, or duplicate cache pages. (It'd be nice if I could quote some metrics.) This is something we're working on improving.
Finally, an absolutely perfect caching model would be for each volume to only direct I/O to an individual director, but doing that manually is next to impossible and would be a forever losing battle. Folks are certainly are welcome to do that, but as shown for any sizable VPLEX deployment - hundreds of hosts and thousands of volumes the manual effort required is substantial. It's also not guaranteed to be right because different volumes and different hosts are active at different times - some frequently, others not at all...
Round-robin is like the "easy button" for balancing work across all the directors :-)
Gary
Wolfi72
12 Posts
0
January 27th, 2015 05:00
Interesting discussion. I have an aspect to add to the question "fixed or - new recommendation - round robin": it's an NDU. IHAC with a bunch of ESXi 5.x Hosts attached to a Metro Cluster running Geosynchrony 5.1.0.4 and we have an update to 5.3 SP2 being scheduled.
Now SolvE is apparently not (yet?) aware of the new recommendation or at least doesn't distinguish between whether or not the hosts are being cross-connected.
I recently generated the procedure for the upgrade to be able to fulfill all the prereqs and in the document, under "Step 1 - Verify the Host environment" for VMware there is clearly stated:
"VMware: Set Native Multipathing to Fixed"
So since I have to take care of preventing the attached servers from crashing and in this environment, the hosts are actually on round robin, I got anxious and had to doublecheck with EMC and was told "you're fine with RR unless being cross-connected. If you are, reconfigure to fixed". I was keen on getting this piece of information in a written form, providing a statement that RR in my case is officially being supported to get support in the case of issues during the NDU.
As of now, I don't have this statement yet, but i'm assuming that "for NDUs with NMP in cross-connected topologies fixed only" is due to preventing the hosts from chosing paths across the link b/c NMP doesn't provide an auto-standby feature as PP/VE does. So it's to circumvent a mess in terms of local and remote path connectivity. Is it that simple? There's a couple of statements in the mentioned best practice WP with regards to this under "RR - New Policy Recommendation".
It's not explicitly stated, but I draw the obvious corollary that in a non-cross-connected topology, RR is still ok, even for an NDU if i'm on NMP. So the new recommendation holds true.
garyo
89 Posts
0
February 5th, 2015 15:00
Thanks, the inconsistency in the Solve Desktop document has been raised and an internal defect has been filed to have it corrected.
> As of now, I don't have this statement yet, but i'm assuming that "for NDUs with NMP in cross-connected topologies fixed only" is due to preventing the hosts from chosing paths across the link b/c NMP doesn't provide an auto-standby feature as PP/VE does. So it's to circumvent a mess in terms of local and remote path connectivity. Is it that simple? There's a couple of statements in the mentioned best practice WP with regards to this under "RR - New Policy Recommendation".
In a cross-connected environment with NMP RR you wouldn't want the host to choose a remote path, but in the case of path failure you really have no idea which path it's going to pick (local vs. remote.) But if your inter-cluster latency isn't that bad, then having a host choose a remote path during an NDU may not necessarily be the end of the world or result in any kind of failures. And so if failback is working properly you might get back the local cluster as the primary path. Again, all random though and requires manual balancing. Use PP/VE with auto-standby for cross-connect to avoid all this.
Gary