PowerEdge

Generative AI Release Updates With Dell and AMD

Dell Technologies Generative AI Solutions with AMD now available featuring PowerEdge XE9680 and Instinct MI300X Accelerator.
By   |  

As AI continues to rapidly evolve, the fusion of open-source technologies and cutting-edge hardware acceleration is driving industry innovation. Dell Technologies and AMD are at the forefront, offering on-premises infrastructure solutions that are up to 75%1 more cost-effective than public cloud IaaS and tailored to empower generative AI applications in the enterprise.

We’re excited to share a series of updates powering the deployment of open AI ecosystems and frameworks. First, the Dell PowerEdge XE9680 is now shipping with the AMD Instinct™ MI300X accelerators, enhancing AI performance options for our customers. Second, we’ve published the Dell Validated Design for Generative AI with AMD, enabling custom applications development with open-framework foundations. And third, our new deployment services, available in September, will ensure smooth integration of these innovations into your operations.

Now Shipping – PowerEdge XE9680 With AMD Instinct MI300X Accelerators

The PowerEdge XE9680 with AMD Instinct MI300X accelerators offers high-performance capabilities designed for enterprises leveraging generative AI and features eight MI300X accelerators, a combined 1.5 TB of HBM3 memory and 42 petaFLOPS of peak theoretical FP8 with sparsity precision performance.2

This powerful configuration enables faster training and inferencing of large language models helping organizations deliver AI-driven insights and innovative applications more efficiently. Recent testing on a single node configuration displayed a leading total cost of ownership value:

  • Deployed the Llama 2 70B parameter model on a single AMD Instinct MI300X accelerator on the Dell PowerEdge XE9680 Server.3
  • Deployed eight concurrent instances of the model by utilizing all eight available AMD Instinct MI300X accelerators on the Dell PowerEdge XE9680 Server.3
  • Fine-tuned the Llama 2 70B parameter model with FP16 precision on one Dell PowerEdge XE9680 Server with eight AMD Instinct MI300X accelerators.3

With simplified deployment through Dell OpenManage Enterprise, intelligent automation via APEX AIOps software and enhanced security featuring integrated cyber recovery and a Zero Trust approach, the XE9680 empowers businesses to rapidly implement and scale their GenAI solutions while maintaining a robust security posture.

Available Today – the Dell Validated Design for Generative AI with AMD

Announced in May, and available today, the Dell Validated Design for Generative AI with AMD is the next step of Dell Generative AI Solutions making it easier for organizations to deploy trustworthy GenAI. This design guidance gives organizations and developers comprehensive directions to implement LLM inferencing and model customization, as well as advanced techniques like fine-tuning and retrieval augmented generation (RAG). Built on open standards and reducing the need for proprietary AI software suites, developers can simplify development and freely customize workflows with open-source LLM models from partners including Hugging Face and Meta.

Accelerate modern workloads. Innovation and scale that enable efficient, agile businesses and outcomes.

  • Powered by AMD Instinct MI300X accelerators, the Dell Validated Design enables near-linear scaling and low latency distributed GenAI training and inferencing.
  • The PowerScale F710 delivers faster time to AI insights with massive gains in streaming performance that accelerates all phases of the AI pipeline.
  • The Dell PowerSwitch Z9664F-ON, offering 64 ports of 400GbE, delivers low latency and high throughput Ethernet fabrics for modern AI clusters.
  • The Broadcom Thor2 AI optimized NIC delivers 400G, interconnecting MI300X accelerators with the industry’s lowest power requirements.4

Boost application development. Open-source software and ecosystems allow developers and data scientists to innovate freely.

  • AMD ROCm-powered frameworks extend the Dell Generative AI Solutions ecosystem and include support for open-source large language models like PyTorch, TensorFlow, ONNX-RT and JAX, as well as the full stack of drivers, dev toolkits and APIs for AMD Instinct accelerators.
  • Dell Omnia streamlines the creation and management of AI clusters automating configuration for efficient workload processing.
  • Enterprise SONiC distribution by Dell Technologies delivers a scalable networking solution that combines the open-source SONiC platform with Dell PowerSwitch, offering advanced features and enterprise-grade support.

Dell Validated Designs for Generative AI make it simple for Dell customers to build GenAI platforms tailored to their needs by taking the guesswork out of integration, performance and sizing considerations.

Services to Get Started

Manage the AI lifecycle with confidence, with a future-proofed, open development framework developed for AI. Aligned to this new Dell Validated Design, we’ll be introducing new platform implementation services in September. Trusted experts will help you quickly establish a fully operational platform that is primed for innovation, implementing the necessary tools and framework into your environment and sharing best practices to maintain secure, streamlined operations.

Not sure where to begin? Try an Accelerator Workshop, a half-day facilitated event that is a great first step in determining how your organization can maximize value from AI. These are collaborative and engaging sessions involving key stakeholders, designed to focus on key challenges to help you achieve clarity for your vision. With over 1,000 workshops delivered per year and decades of experience, we’ll help you accelerate AI success.

More Resources

At Dell Tech World we demoed the Industry’s First Multimodal RAG on Dell PowerEdge XE9680 Server with AMD Instinct MI300X Accelerators

The advent of Llama3 has attracted much interest in the generative AI domain, we’ve also created a blog to help with the deployment of Llama on PowerEdge XE9680 with MI300X. Check it out at Run Llama 3 on Dell PowerEdge XE9680 and AMD MI300x with vLLM.

AMD, the AMD Arrow logo, AMD Instinct and combinations thereof are trademarks of Advanced Micro Devices.

1 https://www.delltechnologies.com/asset/en-us/solutions/business-solutions/briefs-summaries/esg-inferencing-on-premises-with-dell-technologies.pdf
2 https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/data-sheets/amd-instinct-mi300x-platform-data-sheet.pdf
3 https://infohub.delltechnologies.com/en-us/p/silicon-diversity-deploy-genai-on-the-poweredge-xe9680-with-amd-instinct-mi300x-accelerators/
4 https://techfieldday.com/video/broadcom-thor-2-high-performance-ethernet-nic-for-ai-ml/

About the Author: Luke Mahon

Luke Mahon is the Director of the Dell AI Solutions Technical Marketing Engineering team, leading a team that works hand in hand with product management, engineering, marketing, sales, pre-sales and our customers to help foster a deep technical understanding of our AI offerings. Luke is an 18-year Dell veteran who has worked in a number of key areas of the business. Prior to this role he led the APEX TME team and before that he was responsible for our Dell Technologies messaging strategy and the workforce transformation messaging pillar. In the past he developed and led our converged infrastructure marketing campaign and prior to moving to marketing he spearheaded our technical private cloud initiative in EMEA as an Enterprise Technologist. He has also held several key technical pre-sales positions over the years, specializing in cloud, server technology, systems management and automation. Having studied Manufacturing Engineering at Brunel University in London, Luke spent several years working as an IT journalist, working for PC Magazine in both his native Britain and Dubai before to moving into his first technical pre-sales role. Luke is an avid music fan, a collector of vintage computers and classic cars as well as a keen foodie and a passionate Global traveler.