Memory, Not GPUs, Now Sets the Price of AI

I.The bill arrives at the most profitable company in the buildout

Nvidia's second-quarter results on 26 August were, on the surface, the best in its history: revenue of $96.2 billion, up 106% year on year, with data-centre revenue of $89.0 billion. The number that moved the story was smaller. GAAP gross margin came in at 75.0%, the company guided the next quarter to 74.0%, and on the call CFO Colette Kress went further — margins bottom "in Q4 in the 71 to 72% range before settling at 72 to 73% in fiscal year '28". The cause she named was memory.

Three days earlier, Bloomberg reported that Nvidia had notified its largest buyers of price increases of more than 15% on Grace Blackwell and Vera Rubin systems shipping from early 2027, sized according to product generation and memory configuration. Taken together these are the same event seen from both sides of the income statement: Nvidia is absorbing part of a memory cost shock and passing the rest downstream. Roughly four points of gross margin at a $400-billion-a-year revenue run-rate is real money, and the company chose to disclose the trough rather than discover it quarter by quarter.

II.The chokepoint moved, and nobody re-priced it

For three years the binding constraint on AI was advanced packaging and GPU die supply — a bottleneck Nvidia controlled and monetised. That is no longer where the scarcity sits. Goldman Sachs' teardown of the Vera Rubin platform, published 10 August, puts memory at about 62% of total material cost, against 53% for the GB300 generation, with memory content alone approaching $2 million of a roughly $7.8 million rack and memory spend per system up more than 435% year on year.

When one input crosses 60% of a system's bill of materials, pricing power migrates to whoever supplies it. That is now three firms: Samsung, SK hynix and Micron. The shift is visible in Nvidia's own engineering. Analysts at GF Securities report that Nvidia has halved SOCAMM module capacity from 192GB to 96GB, cutting CPU-attached memory per rack from 54–55TB to about 28TB, on Bernstein estimates that HBM4 could reach $53 per gigabyte in 2027. A chip designer trimming memory out of its flagship to protect a cost target is not an optimisation. It is a supplier setting the specification.

III.The honest read: a real constraint inside a famous cycle

The physical shortage is well documented. TrendForce expects server DRAM contract prices to rise 13–18% quarter on quarter in 3Q26, with RDIMM bit supply growing only 15–20% year on year — materially slower than server CPU shipments, which is why the firm expects tightness to persist into 2027. Buyers are already rationing: TrendForce observes customers migrating from 96GB and 128GB modules down to 32GB and 64GB. Consumer markets show the pass-through unfiltered, with a 32GB DDR5-6000 kit at around $392 in August against $110–140 a year earlier.

Three caveats deserve equal weight. First, the rate of increase is decelerating sharply — Fig. 1 is a plateau forming, not an accelerating spiral, and TrendForce notes consumer buyers are "reaching their affordability limit", which caps how far prices can run. Second, the pain is unevenly distributed: several large US cloud providers hold long-term agreements that contractually bar suppliers from raising their prices, so the increases land on everyone else. Third, memory is the most reliably cyclical business in semiconductors. Deloitte estimates combined capex at Micron, Samsung and SK hynix rising nearly 340% between 2024 and 2027. Every prior cycle ended the same way — new fabs land, supply floods, prices collapse. The counter-argument is that cleanroom, power and water constraints push meaningful new capacity to 2028 at the earliest.

IV.Two curves are now moving in opposite directions

The same fortnight that hardware got more expensive, intelligence got cheaper. Google released Gemini 3.7 Flash on 13 August at $0.75 per million input tokens and $3.75 per million output tokens — roughly half its predecessor. This is not a contradiction; it is the central fact. Epoch AI's measurement of inference pricing found declines of between 9x and 900x per year at fixed capability, varying by task, driven by distillation, sparsity and better serving stacks — with the explicit caveat that the fastest of those drops may not persist.

Algorithmic deflation and hardware inflation are now running simultaneously, and the gap between them is being financed rather than earned. Alphabet raised 2026 capital spending to $195–205 billion from $180–190 billion. So long as capital is abundant, a token price can fall while the asset producing it costs more, because the shortfall lands in depreciation schedules rather than in the price list. That works until the cost of capital changes or the efficiency curve flattens — and one of those two is now measurably harder to sustain.

V.Who actually pays

The incidence is the story. Hyperscalers with long-term memory agreements and in-house accelerators are largely shielded; Nvidia's >15% increase reaches them through server makers but their memory input is pre-priced. The exposed parties are the ones without that cover: GPU neoclouds reselling capacity on thin spreads, enterprises buying racks outright, sovereign compute programmes with fixed budgets in national currency, and any AI company whose gross margin assumed hardware costs falling 20–30% per generation. For those buyers, a 15% price increase on a system whose memory content has been cut is a double hit — more expensive, and less capable per rack for memory-bound workloads such as long-context serving and large KV caches.

The second-order effect is a divergence between capex measured in dollars and capacity measured in useful compute. When hyperscalers report record spending this autumn, the meaningful question is no longer how large the number is but how many accelerators and megawatts it buys. Rising input prices mean the same headline capex delivers less deployed capability — a distinction that flatters the investment narrative while quietly slowing the capability roadmap.

VI.The precedent cuts both ways

Memory has run this loop before. Demand surges, prices spike, everyone builds fabs, supply overshoots, prices collapse — Samsung, SK hynix and Micron all posted losses in 2022–23 after pandemic demand faded. That history is the single strongest reason to treat the current shortage as temporary. What makes this cycle arguably different is the shape of the demand: AI memory demand is contracted years forward, sits inside multi-year infrastructure commitments, and comes from buyers whose alternative to paying is not buying less but building less. SK hynix has earmarked ₩54.3 trillion (about $38.3 billion) through 2031 against that demand. The honest position is that both readings are live: the constraint is real for the next four to six quarters, and the cycle has never once failed to turn.

VII.How it plays out: three scenarios

Probabilities below are analytical judgment, not measured frequencies, over a horizon to end-2028.