Samsung and SK Hynix split on what comes after HBM

The two Korean memory giants used the Hot Chips conference in late August to stake out diverging bets on AI memory's future. Samsung detailed its first commercial processing-in-memory product built on low-power DRAM, and is pushing more logic into the HBM base die; SK Hynix presented iHBM, a packaging approach it says cuts the thermal resistance of high-bandwidth memory stacks by more than 30 percent, according to reporting by the Korea JoongAng Daily and Digitimes.

The positions are rational given the standings. SK Hynix holds a majority of the HBM market by most estimates, and a leader defends the standard it dominates: its problems are the leader's problems — heat, and the difficulty of stacking DRAM dies ever taller without the middle layers cooking. A 30 percent cut in thermal resistance is not a glamorous claim, but thermals are the binding constraint on how tall stacks can go, and stack height is how HBM capacity grows.

Samsung, chasing, bets on changing the game. Processing-in-memory attacks the underlying problem from the other side: instead of moving data to the compute ever faster, it moves some of the compute into the memory, so the data does not travel at all. The physics case is old and sound — data movement, not arithmetic, dominates the energy budget of AI workloads. The history is the caution: PIM has been the next big thing in memory for two decades, and it has repeatedly foundered on the same rock, which is that it requires software to change. Accelerator vendors and framework authors have to program for it, and they have historically declined.

What is different now is the size of the prize and the pain of the status quo. When memory bandwidth is the ceiling on a hundred-billion-dollar accelerator market, the reward for breaking the wall rather than climbing it has never been higher. The next accelerator generation will show which bet the buyers made.