Infrastructure

775 Microns Now Caps Every AI Memory Chip Nvidia Ships

SK Hynix and Samsung showed rival memory-packaging fixes at Hot Chips 2026, and SK Hynix is running the safe one through Nvidia's whole Rubin generation.

InfrastructureBy Servola Tech Desk2026-08-254 min read

AI-assisted, edited by humans. | Editor-in-Chief: Leon Soliman | Editorial standards

775 Microns Now Caps Every AI Memory Chip Nvidia Ships

Two Suppliers, One Hard Ceiling

Every high-bandwidth memory stack sold today runs into the same physical wall, and at Hot Chips 2026 the two companies that make nearly all of it admitted they have not broken through it, only worked around it differently. The limit is 775 microns, the standard thickness of a 300mm logic wafer, and it caps how tall an HBM stack can physically be: every additional layer of DRAM has to come from thinner dies and narrower gaps between them, not from simply adding more silicon.

SK Hynix, the incumbent supplier behind Nvidia's current and next-generation accelerators, and Samsung, the closest rival chasing its lead, both used the same August conference to reveal how they plan to keep HBM bandwidth climbing once that ceiling stops incremental fixes from working. Their answers point in opposite directions, and the choice each one made says as much about risk tolerance as it does about physics.

SK Hynix Runs the Safe Play Through Rubin

SK Hynix, the supplier Nvidia's Rubin-generation accelerators are already built around, told Hot Chips 2026 that hybrid bonding, the packaging technique the industry has anticipated for years, will not be ready for the next memory generation, HBM4E, and will arrive at the earliest with HBM5. Until then, the company is extending its existing MR-MUF bonding method, already used on its 16-high HBM3E stacks, through the entire Rubin cycle rather than risk switching to an unproven process mid-ramp.

The numbers show why the upgrade matters once it does arrive: hybrid bonding removes the microscopic bumps that currently separate stacked dies, letting core dies grow up to 24 percent thicker at 20 layers high and cutting thermal resistance by roughly 35 percent compared with MR-MUF at that height. Today's HBM4, still bonded the old way, packs more than 20,000 through-silicon vias and 16,148 base micro-bumps into a 12.8 by 11 millimeter footprint, targeting over 2 terabytes per second of bandwidth and a 48 gigabyte capacity. Each generation has roughly doubled peak bandwidth over the last one:

Generation	Peak bandwidth
HBM2E	460 GB/s
HBM3	717 GB/s
HBM3E	1,024 GB/s
HBM4	2,048 GB/s

Samsung Skips the Interposer Instead

Samsung took the opposite bet at the same conference: instead of refining how stacked memory dies get bonded together, it revealed zHBM, an architecture that stacks memory directly on top of the compute die and removes the 2.5D interposer that has physically separated the two components for the past decade. Samsung says the design cuts input/output power by removing the SerDes circuitry and data-alignment overhead a chip normally needs to talk across an interposer.

The company's own figures claim 70 percent higher power efficiency and 230 percent greater DRAM bandwidth versus a standard HBM4E stack, freeing roughly 8.3 percent more power for the compute chip itself; in one configuration, four zHBM stacks paired with a 1,200 watt GPU save about 100 watts. Samsung is pairing the idea with a custom HBM base die, called cHBM, that adds SoC-like logic functions using an advanced logic process rather than the passive data-and-test circuitry a standard HBM base die carries, alongside a Heat Path Block covering half the chip's PHY area to cut peak temperature by more than 35 percent.

What Multi-Year AI Capacity Plans Should Price In

Buyers signing multi-year AI capacity commitments are effectively betting on SK Hynix's roadmap whether they realize it or not, because Nvidia's Rubin generation, the hardware most large AI deployments through 2027 and 2028 will run on, is built around the packaging method SK Hynix just confirmed will not change. That is not a temporary shortage that eases once a new fab comes online; it is a known, physics-bound constraint the market leader has chosen to work around rather than solve for at least one more full product cycle.

Samsung's zHBM is the genuine wildcard in that plan. If it proves out at the scale Samsung is claiming, it would not just add supply, it would reshuffle who holds pricing and allocation power heading into the generation after Rubin, since it sidesteps the exact bottleneck SK Hynix is still managing around. A buyer weighing supplier diversification for 2027-28 capacity has a concrete reason to treat Samsung's roadmap as more than a hedge against SK Hynix's market share; it is a bet on a genuinely different fix for the same ceiling.

Frequently asked questions

Why can't HBM stacks simply keep growing taller?

Every HBM stack is capped at 775 microns, the thickness of a standard 300mm logic wafer, the physical ceiling SK Hynix and Samsung both cited at Hot Chips 2026. Adding another layer of DRAM within that limit means thinner dies with narrower gaps, not more silicon, which is why both companies are now competing on smarter packaging rather than simply stacking higher.

What is hybrid bonding and why is SK Hynix delaying it?

Hybrid bonding removes the microscopic bumps that currently separate stacked memory dies, letting core dies grow up to 24 percent thicker and cutting thermal resistance by roughly 35 percent at 20 layers high compared with today's MR-MUF method. SK Hynix told Hot Chips 2026 it is not ready for the next generation, HBM4E, and will instead extend MR-MUF through Nvidia's entire Rubin cycle, targeting hybrid bonding for HBM5 at the earliest.

What is Samsung's zHBM and how does it differ from SK Hynix's approach?

zHBM stacks memory directly on top of the compute die and removes the 2.5D interposer that normally separates them, which Samsung says cuts input/output power and delivers 70 percent higher power efficiency and 230 percent greater bandwidth than a standard HBM4E stack. It is a structural bet against the same 775-micron ceiling that SK Hynix is instead managing with incremental packaging changes.

Does any of this affect Nvidia's current chip roadmap?

SK Hynix has confirmed no near-term production impact, since it is deliberately staying on its proven MR-MUF method through Nvidia's Rubin generation rather than risk switching mid-ramp. The practical effect for buyers is that Nvidia's Rubin-era accelerators are provisioned against a known, unsolved memory constraint for the life of that platform, not a shortage a new fab announcement would quickly resolve.

AI infrastructure risk management has mostly watched fabs, export controls, and wafer yield. Hot Chips 2026 was a reminder to watch packaging too: the two companies that supply nearly all of the world's high-bandwidth memory just showed, in the same week, that neither has solved the physical ceiling everyone is building against, they have only chosen different ways to work around it, and Nvidia's next full hardware generation is already committed to one of those choices.

SK HynixSamsungHBM4Hot Chips 2026memory packagingAI accelerator supply chain

Three Firms Control 90% of Memory. Now a Court Will Ask Why It Costs 700% More.

A US class action accuses Samsung, SK Hynix and Micron of coordinating the shift to AI memory while starving DDR4 and DDR5, driving prices up 700%. What it means for buyers.

4 min read

Infrastructure

The 518 Billion Dollar Memory Fix Lands in 2028

South Korea unveiled a chip plan worth over 576 billion dollars, anchored by Samsung and SK Hynix fabs in Gwangju. For everyone buying memory, the key number is a date: no price relief before 2028.

3 min read

Decisions

The Third Pivot Was the Warning Magic Leap Missed

Magic Leap filed a WARN notice for 193 layoffs effective 1 October 2026, its fourth strategic pivot after $4 billion raised. The lesson is the pattern, not the pivot.