Dell Re-takes Performance Lead on TPCx-Big Bench Benchmark at Scale Factor 10000 Using 14G Hardware Platform
Computer benchmarking is the practice of discovering, measuring and assessing the relative performance of a system for a specific workload. Industry standard benchmarks allow researchers to compare the best performance being achieved by their systems to those of another entity (competitor) or to an entirely different industry. This information can then be used to identify gaps in an organization’s performance processes to achieve a competitive advantage. It is also used to compare the performance of the various vendor offerings within the Industry.
Dell uses industry-standard benchmark tests to provide objective and verifiable performance data to its customers as it relates to its hardware platforms and solutions. Customers can use this data when designing and sizing a solution to deliver optimal performance for their business use cases and to scale as needed.
As big data analysis systems (BDAS) mature, the pressure to evaluate and compare both the performance and the price performance of these systems rises. To address this, the Transaction Processing Performance Council (TPC) developed the TPCx-Big Bench (TPCx-BB), a benchmark standard for comparing BDAS solutions. Detailed information about the TPCx-BB specification can be found at http://www.tpc.org/tpcx-bb/default.asp.
On March 22, 2018, Dell published a SF10000 TPCx-BB benchmark result that occupies the number 1 position for both the Performance and Price/Performance tables on the TPC website. With this result, and as at that date, Dell topped the Performance and Price/Performance tables of the TPCx-BB benchmark at SF10000. This is the 2nd result from Dell following the pioneering result it published in the summer of 2017. There are two other results published at SF10000, both from HPE as shown on this page http://www.tpc.org/tpcx-bb/results/tpcxbb_perf_results.asp.
System Under Test
For this submission, the System Under Test (SUT) comprised 1 x DELL EMC PowerEdge R640 Server (Master node) and 18 x Dell PowerEdge R740xd Servers (Slave nodes) as depicted in the System Configuration diagram below:
What was tested and measured?
The benchmark is designed to stress the CPU and IO systems of a Big Data Cluster using a single stream (power test) and concurrent streams (throughput test) of 30 queries (workloads). Each run is performed under 3 phases: Load, Power and Throughput. For the test to pass an audit, 2 sequential performance runs must be executed.
Results
The overall TPCx-BB performance data for the Dell R640/R740xd configuration is summarized in the table below:
| Load Test | 2,190.66s |
| Power Test | 18,949.43s |
| Throughput Test | 70,134.07s |
| Performance Metric | 1,660.75 BBQpm@SF10000 |
| Total System Cost | $908,125 |
| Price/Performance | 546.82 $/BBQpm@SF10000 |
| Availability Date | March 20, 2018 |
Detailed results and documentation are available at the TPC website.
Competitive Landscape
The TPCx-BB benchmark standard has published results on SF1000, 3000, 10000 and 30000. Dell has one SF10000 published result submitted in May, 2017 based on 13G hardware (Intel Xeon Broadwell based processors). As of April 04, 2018, only Dell and HPE have published results at SF10000 based on servers with Intel Xeon Skylake processors. The published results for SF3000 and 1000 from HPE and Huawei are based on Intel Broadwell and Haswell family of processors. The only result the Dell 14G performance data can realistically be compared to is the latest HPE SF10000 result:
Conclusion
Ever since the release of the TPCx-Big Bench benchmark standard in December 2015, Dell has been an active member of the TPCx-Big Bench Technical Committee and will continue to collaborate with other stakeholders within the industry to maintain the TPCx-BB specification.
Dell uses the TPCx-BB workloads to compare and differentiate its data analytics solutions and will additionally use them for performance characterizations and optimizations for the benefit of its customers.
Industry standard computer benchmarking should not be viewed as a purely academic exercise or a competition among vendors. Studying the design of top performing systems has significant benefits for customers that now, with the industry collaboration on the TPCx-BB benchmark, can advance the state of the art of performance engineering for Data Analytics.
