Intel aims to compete with Nvidia H100 with Gaudi 3 accelerators

Intel wants to compete with the Nvidia H100 with its new Gaudi 3 accelerators, currently the most powerful card for training AI models. According to Intel, it can sell a kit consisting of eight cards and a baseboard for two-thirds of what Nvidia charges.

The processor designer has its Gaudi 3 cards produced on TSMC's N5 node. They contain 64 of Intel's tensor cores and 8 matrix engines. The package consisting of chiplets also includes 128GB of HBM2e memory with a bandwidth of 3.7TB/s and 96MB of SRAM cache at 12.8TB/s. For connectivity, Intel uses 16 PCIe 5.0 lanes and 24 200GbE ports, which the company emphasizes are part of an open standard, unlike Nvidia's InfiniBand.

According to the maker, a cluster of 8,192 Gaudi 3 accelerators is 40 percent faster when training a GPT3-175B model than a comparable cluster of Nvidia H100s. With 64 accelerators and the Llama2-70B model, Intel's advantage would be 15 percent. Inferencing should be on average twice as fast on Gaudi 3, depending on the parameters chosen and the AI ​​model used.

A Gaudi 3 kit consisting of 8 accelerators and a baseboard has a suggested retail price of 125,000 dollars. According to Intel, this would offer a 130 percent better price-performance ratio than a comparable Nvidia H100 kit. In addition, Intel emphasized its good software support during the announcement, in which it collaborates with well-known model makers to be able to offer good support for new AI models on 'day zero'.

Dell, HPE, Lenovo and Supermicro already sold systems with Intel's older Gaudi accelerators, but ASUS, Foxconn, Gigabyte, Inventec, Quanta and Wistron are now joining them. This increases the number of Gaudi 'integrators' to ten suppliers. Intel's Gaudi branch emerged from the acquisition of the originally Israeli Habana Labs in 2019.