SemiAnalysis Reports AMD MI325X Servers Running vLLM Can Generate Up to 3 Dollars Per Hour Per Chip in Profit Serving Kimi K3 2.8T
PROFIT ALERT🚨: AMD @vllm_project can generate up to $3/hr/chip in pure profit serving the open-source Kimi K3 model, even after accounting for the rapid increase in chip rental prices post-Claude Code. Kimi K3 2.8T is a great model for AMD, as it has over 2.8 trillion parameters that can take advantage of AMD's 288GB per chip. B200 CANNOT fit Kimi K3 in a single server, so it has to use tensor parallelism or pipeline parallelism for communication across 400Gbit/s InfiniBand/Ethernet, which is 18x slower than the BW within an NVLink domain.
Context
Ultra large open weight mixture of experts models like Kimi K3 2.8T favor accelerators with massive high bandwidth memory capacity per server node, giving AMD's 288GB chips a sweet spot where single node Nvidia B200 systems run out of memory.
Moonshot AI's Kimi K3 is an open weight mixture of experts model with more than 2.8 trillion total parameters, requiring well over 1.4 terabytes of high bandwidth memory just to hold quantized weights in a single server node.
Deep dive
Semiconductor research firm SemiAnalysis reported on Sunday night that inference providers running open source vLLM on AMD accelerators with 288GB of memory per chip can generate up to 3 dollars per hour per GPU in net profit serving Moonshot AI's open weight Kimi K3 2.8T model, even after factoring in recent increases in hourly GPU rental rates driven by coding agent demand.
SemiAnalysis explained that the advantage comes down to single node memory capacity. Because Kimi K3 has more than 2.8 trillion parameters, an eight GPU AMD server with 288GB of HBM per chip provides 2.3 terabytes of unified node memory, allowing the entire model to fit inside one server box.
By contrast, a standard eight GPU Nvidia B200 HGX server lacks enough total memory to hold Kimi K3 2.8T in a single node. As a result, B200 deployments without multi node NVLink racks must split the model across two servers using tensor or pipeline parallelism over 400 gigabit InfiniBand or Ethernet networking, which runs roughly 18 times slower than intra node interconnect bandwidth.
What to expect next
Inference API hosts on OpenRouter and enterprise clusters serving multi trillion parameter open models will look to high memory AMD nodes to avoid multi node network bottlenecks, while software teams continue optimizing vLLM kernels for ROCm. As open weight mixture of experts models continue scaling past two trillion total parameters, memory capacity per server node will remain a primary differentiator in total cost of ownership between competing AI accelerator architectures.