Fabric Across the Datacenter

what kind of switches ?

we currently have this:

site A:

Fabric A

Fabric B

site B:

Fabric A

Fabric B

currently Fabric A spans both sites

currently Fabric B spans both sites

while this works just fine, we have very low latency, DWDM. I really want to get away from spanning fabrics between two sites. If you have any kind of link flopping between the sites ..it will affect systems at each site. So i am planning to have unique fabric A and B at each site and use transit VSAN (i am on Cisco MDS) to facilitate communication between the two sites. IVR will be used to setup zones between different VSANs.

What is your connectivity between sites? What is your average latency between sites?  What type of latency can your systems tolerate?

If the connectivity is fiber: Do you have strands of single-mode fiber between sites that you can use?  Single-mode fiber will require you to purchase some SFPs for your switch.  If you have connectivity that will let you use multi-mode - then your current SFPs will work fine.

If you don't have fiber connectivity - Do you have the ability to use Ethernet-based protocols (FCIP) on your FC switches?

It would not be a good idea to have Fabric Even on one site and Fabric Odd at another site.

You'd rather have either:

Fabric1 - Switch1

Fabric2 - Switch3

Fabric3 - Switch2

Fabric4 - Switch4

or - If you have connectivity between sites.

Fabric1 - Switch1, Switch2

Fabric2 - Switch3, Switch4

The benefits you would get from having the fabric span multiple sites are mainly:

You could have hosts at site A access targets in Site B (Low latency connections only)

You could easily/quickly replicate data from one target to another.  Some replication products insist on FC (SanCopy) connectivity.  Most arrays come with built-in IP replication nowadays though.

Having one fabric at each site would not be good at all.  That would mean your hosts are physically connecting to only one fabric, which isn't in line with best practices for high-availability.

Hi,

Thanks for your inputs Dynamox and driskollt and i fairly understood your points...

I agree to what Dynamox says " if we have link floppings between the sites then it might impact both the sites" and thats was one of my concern.

Also for driskollt , I wanted to know even if we want to have Host1 in SiteA to see the targets in SiteB , would't that be possible with the Fabric ISL's between the two sites where each switch in a unique fabric is visible to other switches in other fabric and hence the targets as well ?? Why do we need to have fabrics across sites to achieve this ?? We might loose few ports for ISL in this case i guess.

For Reduandancy , i got two fabrics at one SiteA with Switch 1 and Switch 3 and the same on other SiteB. For replication , say using SRDF we have the connectivity b/w the frames and it goes through well.

I don't think there is much of a problem with the links b/w the sites , i think we have bought enough which might be good even if we have fabric across the sites,

I am just looking at a reason which would benefit me if i use fabric across the sites and i cannot achieve that using fabric within same site.

I don't know and i m sorry for my ignorance but hope you understand my reason !!

you did not mention what kind of switch infrastructure do you have and what kind of connectivity between the sites ?

I'd go for the transit VSAN (Cisco) to isolate any dark fiber problems from the fabrics at both sites.

man ..you have an enterprise switch with all the bells and whistles ..use it !!! . Even if you start with one VSAN it's ok because it's much harder to reconfigure later on.

this is what i am doing, each site has fabric A and fabric B. You have transit VSANs (200 and 300) that are used to setup communication via IVR. These transit VSANs will carry your replication traffic (SRDF, MirrorView ..etc). For example in VSAN 20 i have my backup server and in VSAN 120 in have my tape library ..you can zone them together via IVR and all traffic will flow through VSAN 200. If for some reason that link will start flopping, you are only affecting your backup server and your tape library ..not every single host in vsan 20 and 120. If you look at Cisco best practices ..this is what they recommend anytime you stretch for any significant distance. (mine is 11 miles between datacenters)

We have got Cisco MDS 9513 and we got DWDM links between the two sites.

same here ...do you have enterprise license ?

Yes, are you hinting at having VSAN feature we got ...i thought the same in that case, but somehow my clients don't wana use VSAN and want to keep it as standalone switches.

Let me know if otherwise..

Thanks dynamox , thats exactly what i would like to do myself and i understand i got everything to use all that and come up with this design.Unfortunately, my clients are like rigid since they have never used this concept and you know & i know how difficult it is to change people's minds when they r used to age old methods ....

Thanks again for clearing my doubts...let me see if i can achieve this, I will keep u guys posted.

Cheers !!

Rahul