HBM grew from 52% to 63% of AI chip component costs

Epoch AI's Venkat Somala put real numbers on a story we already half-knew: high-bandwidth memory is now the single largest component cost in AI accelerators. HBM rose from 52 percent of AI chip component spending in Q1 2024 to 63 percent in Q4 2025, while the absolute spend by Nvidia, AMD, Google, and Amazon climbed from about $12 billion in 2024 to $32 billion in 2025. Advanced packaging slipped from 19 to 15 percent of the mix, auxiliary parts fell from 15 to 9, and logic dies held steady near 13. The breakdown is on Epoch's data-insights page at epoch.ai.

The shape of that shift matters. A logic die is the thing usually called "the chip", but the bill of materials for an accelerator now leans more on stacked DRAM than on the compute silicon itself. That helps explain why memory makers like SK hynix, Micron, and Samsung keep raising HBM prices into 2026, why Microsoft and Meta have walked up their capex forecasts, and why DDR5 for ordinary servers and laptops is getting squeezed. Memory fabs are being pulled toward whoever pays the most per wafer, and right now that is the AI accelerator buyers.

For anyone trying to forecast where inference costs land, this is the cleanest data point in months. It suggests the floor on per-token cost is being set in the HBM stack, not in GPU compute or in advanced packaging, and that the slope of price decline depends on memory yields more than on Nvidia's roadmap.

Why it matters

If you build on hosted LLMs or budget GPU capacity, the cost of inference over the next two years tracks HBM supply more than anything else. Treat memory pricing, not raw compute, as the leading indicator for when API prices start moving again.