Oracle RAC on vSphere 5 Performance Study

Hi guys, sorry but this whitepaper so poor for detailed info, after reading i got more questions than answers. Can you answer some my questions:

First of all, Why you are using "Mixed" cluster

"This testing, which was done with an Oracle RAC cluster made up of six 32-CPU-based nodes, four of them virtual and two of them physical"

i can imagine this test like:

1. shutdown physical nodes, run test.

2. shutdown virtual nodes, run test

same hardware, same software (quote from white paper), It's true ?

2nd question, why virtual nodes x2 greater ?

3rd, which OS for guests you are using RHEL\OEL 5\6 update ?

4th, memory configuration like a size of huge pages are using.

The whitepaper is certainly descriptive and could have benefited from providing more details on what was done, that said, the best practices section does provide some insight into important areas to ensure improved performance of the RAC interconnect, exploiting NUMA architectures and enabling hyperthreading in vSphere.

A recent study by Principled Technologies, commissioned by VMware, perhaps provides the detail you are looking for on the steps to configure and test Oracle RAC workloads with additional focus on demonstrating the impressive capabilities of VMware vSphere 5.1 with vMotion in moving large, critical workloads with no application downtime - http://www.principledtechnologies.com/VMware/vMotion_Oracle_RAC_1013.pdf

We have just published a more recent performance study of VMware virtualization of Oracle RAC.

Good document, but the themes are completely different.

"The physical vs. virtual server (RAC)" and "RAC vmotion Performance Testing"

I love coffee too and taken in moderation it's good for you. I'll try to answer a few questions:

1. In having two physical non-virtualized RAC nodes with two virtualized RAC nodes the authors were able to accomplish:

• Showing a mixed RAC architecture works ~ This is important as some customer will be interested in migrating to virtualization in a phased approach moving from physical to virtual one server at a time.
• Most importantly comparisions can be made between the performance of the physical nodes and the virtual nodes. The authors for example were able to observe differences in performance between physical and virtual nodes as shown in the graph on page 5 of the study.

The authors don't detail which operating system was used but based on the fact VMware virtualization supports x86 OSs my guess would be Linux Red Hat or maybe Oracle Eterprise Linux. The most interesting part of this paper, for me, is the best practices: NUMA architecture, VMXNET3 and hyperthreading. Using these best practices assist with architecting a virtualized Oracle RAC infrastructure.

This is a rather short paper that has some some good points but not some of the detail you are looking for.

Cheers,

-SL

Hi Sam,

Let start again:

"1. In having two physical non-virtualized RAC nodes with two FOUR virtualized RAC nodes the authors were able to accomplish:"

Why 4 ? Why not 2 vs 2 ?

"Showing a mixed RAC architecture works ~ This is important as some customer will be interested in migrating to virtualization in a phased approach moving from physical to virtual one server at a time."

Ok, no problem, it works. But while testing all six nodes worked out or only 4 virtual OR 2 physical ? If all 6 at a time, then it's makes me sad =(.

IloveCoffee,

Just had my two cups this morning. In general the guideline is between 3% to 7% of CPU overhead is needed for the hypervisor. Typically, this isn't a problem as most databases and applications run well below 90% CPU utilization so the instructions used by the hypervisor do not significantly impact database performance.

The other consideration is the management of interrupts. An OS can be interrupted by some devices (HBA, NIC, and even the keyboard) connected to the server. Interrupts are offloaded to the x86 (AMD and Intel) to the CPUs with virtualization turned on so generally not much impact to the IO to and from the storage array.

One tip: the faster the CPU the faster virtualization. If possible place virtualized Oracle databases on the faster CPUs for faster virtualization and databases.

The amount of performance overhead required by a particular layer of your stack can be referred to as the "tax" for that layer. For example, Oracle CRS has a tax, commonly referred to as the "RAC tax". According to my RAC training class (admittedly a while ago), the RAC tax is around 25% of one core on most boxes. (Very environment and workload specific, but a good rule-of-thumb.) You measure the RAC tax by running a workload (say, TPC-C) on a box with RAC enabled, and on the same box with RAC disabled.

The hypervisor layer is similar. Each hypervisor has a specific overhead characteristic. In our experience, the vSphere tax is pretty low. I think Sam's estimate of less than 10% of the CPU is pretty conservative. It depends on the specific workload, of course, but a  good comparison was the physical vs. virtualized Oracle RAC performance study on the VNX7500 that my team did back in 2010. That showed a 4% difference overall in TPC-C performance.

It is possible for a specific workload (especially if it can be partitioned among a larger number of individual, small VMs) to be faster when virtualized vs. physical. But that is fairly rare. Most of the time, virtualization will exact a bit of overhead. That is generally small, though, in our experience.