Riyad / Interests / Semiconductors & Regret
Interests
A sober, entirely serious look at the semiconductor memory industry, written by someone who is definitely not using an HBM-powered GPU to browse Twitter while pretending this is research.
Memory Cycle Tracker
Analysts have published three conflicting reports this week alone. Inventory is both too high and too low. No one knows anything.
[ Strategy: Stare at a chart until you see a pattern. Then ignore it. ]
HBM: The Thirst is Real
At some point in 2023, the collective semiconductor industry woke up and realized that AI accelerators need memory bandwidth in amounts normally associated with a small country's internet backbone. The solution? Stack DRAM dies on top of each other until the thermal density approaches the surface of the sun, then call it HBM3E and ship it.
The current state of play: every HBM supplier is sold out through 2027. NVIDIA is buying every micron (pun intended) of capacity. SK hynix achieved HBM market leadership by simply not messing up their TSV process, which turns out to be harder than it sounds. Samsung is doing great if you ignore their HBM3E qualification delays. Micron is trying really hard and that's what counts.
~1.6 TB/s
HBM3E Bandwidth
100%
HBM Capacity Sold Out
~40%
HBM Share of DRAM Bits
Fab Dreams and CapEx Realities
Building a leading-edge fab costs approximately $20 - 30 billion and takes 3 - 5 years, assuming no geological surprises, trade embargoes, or unexpected shortages of ultra-pure water. By the time the fab comes online, the node it was designed for is already two generations behind, but don't worry - the industry just renames everything so your 3nm fab is actually 2nm now. Problem solved.
EUV lithography tools, each costing around $200 million and requiring a vacuum system that could double as a space simulator, have a lead time measured in quarters. ASML ships roughly 40 - 50 High-NA EUV tools per year. TSMC buys most of them. Samsung and Intel fight over the remaining ones like siblings arguing over the last piece of cake, except the cake costs a quarter billion dollars and requires its own cleanroom.
Fab construction begins. Groundbreaking ceremony. Speeches about the future of semiconductors.
Building shell completed. Tool installation begins. Delays from missing parts (obviously).
Fab starts qualification runs. Yield is terrible. Everyone expected this. Everyone is still disappointed.
Fab reaches volume production. The node it was designed for is now called 'legacy.' A new fab is already being planned.
A Brief Chronicle of the Current Cycle
Everyone says memory is a cyclical nightmare. Build more fabs anyway.
Memory prices collapse. Everyone acts surprised. The cycle works exactly as it always has.
AI appears. Suddenly 8-stack HBM is the most important thing in the known universe.
HBM is sold out until 2027. Samsung, SK hynix, and Micron begin a holy war over channel bandwidth.
HBM4 announced. Everyone pretends HBM3E is obsolete. The cycle resets but faster.
Current year. The cycle enters its most advanced phase yet: 'It's Complicated, But With AI.'
Field Guide to Industry Terms
- Node
- A marketing term loosely correlated with physical transistor dimensions. 3nm means whatever we want it to mean.
- EUV
- Extremely Expensive Ultraviolet. A machine that costs more than a naval destroyer and requires its own gravitational field to operate.
- Yield
- The fraction of dies on a wafer that are not decorative. Single most prayed-over metric in the industry.
- Bandwidth
- A number that is never, ever large enough. You could have a petabyte per second and you'd still complain about memory-bound kernels.
- Capex
- The art of spending $25 billion on a building that will be obsolete before the concrete cures.
- HBM
- High Bandwidth Memory, or How to Stack DRAM Dies Until They Thermally Self-Destruct. Currently the only game in town.
- Lead Time
- The interval between placing an order and forgetting why you placed it. Measured in quarters, sometimes geologic eras.
- Inventory Correction
- Industry-speak for 'we made a truly unreasonable amount of DRAM and now we're eating the cost.' Happens every 18 months like clockwork.