d-Matrix unveils Raptor 3D-DRAM inference chip at Hot Chips 2026, claiming 10x HBM4 bandwidth to break AI memory wall; Nvidia hikes server prices 15%

d-Matrix unveiled its Raptor 3D-DRAM inference chip at Hot Chips 2026, claiming to deliver 10x the bandwidth of HBM4. This innovation aims to tackle the AI memory wall, a critical issue as demand for memory bandwidth surges with the growth of large language models (LLMs).¹

The company’s 3DIMC technology is designed to enhance inference performance, particularly during the decode phase, which is memory-bandwidth-bound. d-Matrix asserts that improving decode bandwidth is essential for overall inference efficiency, as most inference time is spent in this phase.³

Raptor utilizes a TSMC N4 logic die stacked on a 3D DRAM die, achieving a bandwidth of 32.6 GB/s per mm², significantly outperforming HBM parts at 1.5 GB/s. This results in a 13.5x improvement in energy efficiency, consuming 2.96 mW per GB/s compared to HBM's 40 mW.

In parallel, Nvidia announced a 15% price hike for its AI servers, attributed to rising memory costs and supply chain constraints. This increase affects major configurations like Vera Rubin and Grace Blackwell, with some customers facing significant price adjustments for systems shipping early next year.²

As the demand for AI capabilities escalates, the competition between d-Matrix and established memory providers like Samsung, SK Hynix, and Micron intensifies, highlighting the urgent need for innovative solutions to overcome the memory bandwidth challenges in AI applications.