AMD Instinct MI300X

The MI300X is AMD's answer to NVIDIA's H100/H200 in the generative AI accelerator market. It uses a chiplet design combining CDNA 3 compute dies with I/O dies on a single package, and unlike the MI300A (which pairs GPU chiplets with Zen 4 CPU cores for HPC workloads), the MI300X is a GPU-only part aimed squarely at LLM training and inference.

We'd point teams running memory-bound inference workloads, think 70B+ parameter LLMs, at the MI300X first among AMD's lineup. The 192GB of HBM3 and 5.3 TB/s of bandwidth let you serve larger models per GPU than an H100 can, which cuts down on the number of GPUs needed to fit a model in memory. It's a weaker fit for pure FP64 HPC simulation work, where the older MI250X still has a role, since MI300X trades some double-precision throughput for AI-focused FP8/FP16 performance.

Against the NVIDIA H100 (80GB), the MI300X wins on raw VRAM (192GB vs 80GB) and memory bandwidth, which matters a lot for inference on large models, but NVIDIA's CUDA/cuDNN software stack and broader framework support still give H100 the edge in day-to-day training ergonomics. Against the newer MI325X, the MI300X is the same CDNA 3 compute die but with less memory (192GB vs 256GB HBM3E) and lower bandwidth (5.3 TB/s vs 6.0 TB/s), so if budget allows, MI325X is the better buy for new deployments. Against NVIDIA's H200 (141GB HBM3E), the MI300X still leads on capacity but the H200 closes the bandwidth gap significantly. Pricing on all of these is enterprise-quote-only, there's no public retail price.