AMD’s MI350 launch made memory, cooling and software part of the comparison
AMD launched its MI350 Series in June 2025 with CDNA 4, 288GB HBM3E and air- or liquid-cooled configurations, making rack design central to any performance claim.
Historical reporting: AMD launched the Instinct MI350 Series on June 12, 2025. Performance, tokens-per-dollar and availability statements are AMD claims with disclosed test conditions or forward-looking language, not independent measurements.
Key takeaways
- MI350X and MI355X shared CDNA 4 and memory capacity but supported different cooling and density choices.
- Memory bandwidth, precision and rack design must be evaluated with the software path and target workload.
- Vendor comparisons should preserve test conditions, pricing assumptions and the difference between rollout plans and observed service results.
AMD launched a series and an infrastructure story[source 1][source 2]
AMD launched the Instinct MI350 Series at its Advancing AI event on June 12, 2025. The company identified MI350X and MI355X accelerators and placed them alongside ROCm software, rack-scale designs and an MI400/Helios preview. The event therefore presented a stack of accelerator, software and system choices rather than a chip whose value could be inferred from a single performance number.
AMD reported a four-times generation-on-generation AI-compute increase and up to a 35-times inference increase. Its release notes state that results depend on model, precision, driver and optimization choices, and that server configurations may vary. Those qualifications should travel with the headline figures whenever they are used in a planning discussion.
Memory capacity and thermal design separated deployment paths[source 1][source 2]
AMD’s technical post described both MI350X and MI355X as CDNA 4 designs with 288GB of HBM3E and up to 8TB/s of memory bandwidth. It also described air-cooled and direct-liquid-cooled configurations, with up to 64 GPUs in an air-cooled rack and up to 128 in a direct-liquid-cooled rack. These are product-design and vendor-performance statements, not proof of a reader’s facility readiness.
Cooling is an architectural input, not only a facilities afterthought. Higher density may change rack power delivery, heat rejection, maintenance windows and failure domains. A decision to fit a model into fewer accelerators can be favorable for communication overhead while creating a different availability or serviceability profile.
Analysis: model fit is a memory-and-runtime question[source 1][source 2]
Start with a transparent memory budget: weights by precision, optimizer state for training, activations, key-value cache, communication buffers and operational reserve. Then test the exact runtime and distributed strategy. The same nominal model can have different capacity and latency behavior under another quantization, sequence length, tensor-parallel layout or serving engine.
Treat software maturity as part of the acceptance scope. Pin ROCm, framework, kernel and model versions; run functional checks before performance tests; and retain a small regression suite after upgrades. This is independent engineering guidance. It does not claim that any MI350 configuration supports every framework or produces the vendor’s reported result.
Analysis: preserve comparison conditions and rollout status[source 1][source 2]
A fair economics comparison includes the selected model, input and output lengths, concurrency, target latency, precision, hardware count, cloud price date and utilization assumption. AMD’s release discloses conditions for some comparisons, including particular Llama serving tests and pricing assumptions. Repeating the headline without those terms can turn a bounded result into an unsupported general claim.
The June event also included rollout and broad-availability expectations. A plan, a partner integration and an operating production cluster are different states. Verify current provider inventory, deployment architecture, support ownership and failure behavior before treating the announcement as procurement proof. Check facility power, cooling service, replacement paths, monitoring coverage, maintenance windows and firmware-update responsibility as well. Confirm trained operators can respond to an alert without depending on undocumented vendor procedures. The historical launch is useful context; the decision still needs live, workload-specific evidence.
Sources
Continue reading
Related reading
News analysisAMD showed Helios as an Open Rack Wide reference platform, keeping its 2026 volume expectation separate3 min read
News analysisNVIDIA’s memory codesign post highlights a practical boundary between capacity and usable agent throughput3 min read
News analysisNVIDIA supply-chain capacity plans make packaging, test and optics explicit infrastructure constraints3 min read