In the new digitally driven world, IT and business leaders are working overtime to design, deploy and capitalize on applications driven by artificial intelligence. This pursuit requires leading-edge, high-performance computing systems that are built and tuned for the rigors of data-intensive machine learning workloads.
But how do you find the best system for the task at hand? This is where the MLPerf Inference benchmark suite comes into play. MLPerf Inference gives organizations an objective way to measure how fast systems can run trained machine learning models under different deployment scenarios. It gauges the performance of models running with real-world datasets for a wide variety of applications and form factors, and optionally includes system power measurement.
The news: MLCommons, the open engineering consortium behind the benchmark, just released new results for the MLPerf Inference v1.1 machine learning inference performance benchmark suite. And these results confirm that Dell Technologies is a leader in systems for powering machine learning workloads. (See details at “MLCommons™ Releases MLPerf™ Inference v1.1 Results.”)
For MLPerf Inference v.1.1, Dell Technologies was the only company with 234 results from tests that include GPU and CPU-only, new GPUs, Triton, and multi-instance GPU (MIG). We tested a variety of Dell PowerEdge servers designed for edge, bare-metal, virtualized and cloud environments, all to provide customers the data they need to make the best choice for their needs.
The results show that Dell Technologies is No. 1 in many categories, including image recognition, speech-to-text, object detection, recommendation engines, natural language processing and more. In general, these results document some rather amazing inference performance and performance/watt results for the tested Dell systems, including the PowerEdge R750xa, R750, R7525, XE8545, and DSS 8440 servers, along with XE2420 and XR12 edge servers. For a look at some noteworthy results for these servers, see “Introduction to MLPerf™ Inference v1.1 with Dell Servers.”
Dell Technologies differentiators
What’s behind the Dell Technologies advantage in machine learning performance? Here are some of the differentiators for Dell systems.
-
- PCIe accelerator support — Standard-form-factor PCIe-based accelerators provide optimal performance for inference workloads. Our engineers continually tune our systems and MLPerf Inference models for maximum performance on these accelerators. As a result, the DSS 8440 server is at the top of the performance rankings for PCIe-based accelerator platforms for image classification, speech to text and object detection workloads.
-
- Virtualization support — One issue that always comes up with virtualization is the performance overhead introduced by the virtualization layer. To address these concerns, Dell collaborated closely with VMware to run SSD and BERT benchmarks on Dell PowerEdge servers configured with VMware ESX to characterize performance of virtualized AI workloads. The results demonstrated that the performance of virtualized systems for the tested workloads was within 6% of the bare-metal equivalent.
-
- Virtual GPU support — As the GPUs get more performant and as users expand the use of virtualization to their AI environments, we see an increasing demand to partition the GPUs — so more users can gain the benefits of accelerators. In order to demonstrate the performance with virtual GPUs, we used NVIDIA MIG technology and demonstrated that it is feasible to effectively run inference workloads on partial GPUs, thereby increasing the overall utilization of these highly capable acceleration engines.
-
- Power efficiency — Power consumption is an important design consideration, so we optimize the power and cooling algorithms for system efficiency. We captured and reported power consumption data as part of our MLPerf Inference benchmark submission, and we applaud the two others who submitted power consumption metrics.
Key takeaways
At Dell Technologies, we continue to push the AI technology envelope, and we are committed to working with our customers to tune our infrastructure to fit their workloads and environments. That’s the case when it comes to designing and optimizing systems for machine learning and other AI applications. As the diverse MLPerf Inference benchmarks show, the Dell Technologies portfolio of systems optimized for AI has what it takes to build robust solutions for today’s AI needs.
For a more detailed look at the benchmarks and results for the tested Dell PowerEdge servers, see the blog “Introduction to MLPerf™ Inference v1.1 with Dell Servers.” Then take a test drive in one of our worldwide Customer Solution Centers, collaborate with our HPC & AI Innovation Lab experts or tap into one of our HPC & AI Centers of Excellence.
