For a decade the story of AI infrastructure was compute. More FLOPs, bigger clusters, faster interconnects. In 2026 the binding constraint quietly changed, and most capacity plans have not caught up. The scarce resource now is memory — the DRAM and high-bandwidth memory that feed the accelerators — and its price has moved in a way that reprices the entire stack above it. A new preprint from Satoshi Matsuoka, who runs RIKEN’s R-CCS and has spent his career at the top of high-performance computing, puts numbers to the shift and argues it will restructure the industry between now and 2030. It is not peer-reviewed, and I will flag where its conclusions are calculations rather than settled facts. But the underlying prices are real, and if you architect systems on top of these models, they are already in your bill.
The price shock nobody budgeted for
The headline number is brutal. Conventional DRAM contract prices rose on the order of 90% in the first quarter of 2026 against the previous quarter — not year on year, quarter on quarter. Matsuoka’s paper pins memory at roughly 40–50% of an AI accelerator’s bill-of-materials, and a rising share of total data-center capital cost. Treat that 40–50% as a mid-range estimate rather than gospel — the exact fraction varies by accelerator and it is the kind of figure that moves — but even the low end reframes how you should think about a GPU. You are not mostly buying compute any more. You are buying memory with some compute attached.
The supply side offers no quick relief. SK hynix chairman Chey Tae-won, speaking at Computex on 2 June 2026, warned that the shortage could persist to 2030. High-bandwidth memory has eaten a growing slice of total DRAM wafer output — the paper cites TrendForce data putting HBM at around 23% of it — which means the memory AI needs most is competing for the same fab capacity as the memory everything else needs. Every HBM stack a foundry commits to an accelerator is conventional DRAM it does not make, so the shortage propagates outward into ordinary server memory too. When the people who make the chips are talking in multi-year horizons, this is not a spot-price blip you wait out. It is a planning assumption.
The demand side is the other blade
Scarcity only bites because demand is vertical. The paper cites Google’s disclosed trajectory: from 9.7 trillion tokens per month in May 2024 to 3.2 quadrillion by May 2026. That is roughly a 330-fold increase in two years from a single provider. Every one of those tokens has to pass through memory — the model weights, the KV cache for the context window, the activations. Token demand and memory demand are the same curve wearing different clothes.
This is the mechanism I described in the piece on inference cost and the energy wall, seen from the memory side. Falling per-token prices drive exploding token volume, and exploding volume slams into a physical resource that cannot scale on the same timeline. The energy wall and the memory wall are the same wall photographed from two angles.
The solvency corridor — a calculation, not a prophecy
Here is where I want to be careful with the framing. Matsuoka does not merely observe high prices; he builds a scenario model of whether the announced AI buildouts can stay solvent, and this is his calculation, not an established outcome. The result is a narrow corridor: the industry’s planned capacity pencils out only if token demand keeps growing on the order of 2× per year for four consecutive years, with premium pricing staying sticky. The paper puts the sensitivity band at roughly 1.6× to 2.4× depending on how inference efficiency trends.
Sit with what that means. If demand compounds at 2× a year, the memory gets absorbed and the economics hold. If it sags toward 1.6×, or if per-token prices fall faster than volume rises, some of the capital committed to these buildouts does not earn its return. The paper also sketches a bifurcation in training: a “luxury tier” of frontier runs costing 18–38 billion dollars apiece by 2030, alongside a mass tier where previous-frontier parity falls toward a few million. I do not know whether the corridor holds — nobody does — but I know that betting your architecture on the optimistic end of someone else’s solvency model is a choice, and you should make it deliberately.
What memory scarcity means for architects now
You cannot fix the DRAM market. You can stop it from quietly eating your unit economics. Every one of these levers is a way of buying less memory per unit of work.
- Treat context length as a budget, not a setting. The KV cache grows with every token you keep in context, and that cache lives in the most expensive memory you are renting. A 1M-token window is a capability, not an instruction to fill it. Retrieve narrowly, summarise aggressively, and pay for the context you actually need.
- Make caching a first-class architectural concern. Prompt caching, prefix reuse and result caching are no longer micro-optimisations — they are the difference between paying for memory once and paying for it on every call. Design your prompts so the expensive shared prefix is stable and cacheable.
- Right-size the model to the task. Memory cost scales with parameters resident on the accelerator. The reflex to route everything to the largest model is now a line item. Most production traffic does not need a frontier model, and I laid out how to make that call in the 2026 LLM stack piece.
- Re-run the on-prem versus API math. When memory is 40–50% of accelerator cost and prices are volatile, the build-versus-rent calculation shifts. API providers are absorbing the memory shock and amortising it across tenants; owning hardware means owning the DRAM curve directly. Neither answer is universal — but if you last ran this model in 2024, the inputs have changed by half.
- Plan capacity in memory, not just GPUs. “How many H-class cards” is the wrong unit. “How many gigabytes of HBM at what price, secured for how long” is the question your finance team is about to start asking. Get ahead of it.
The deeper point is a change of mental model. We spent years optimising for compute because compute was scarce and memory was cheap enough to ignore. That era is over. Whether or not Matsuoka’s solvency corridor holds to the decimal, the direction is unambiguous: memory is now the resource that decides what your system costs and what you can afford to build on it. Architect as if every token has to squeeze through a wall — because in 2026, it does.