解决扩展AI内存的能源成本问题

tions |Semiconductor and Infrastruc-ture Systems January 22, 2026Industry Report Sebastien Naji+1 212 245 6508snaji@williamblair.com Total RecallHow AI Is Supercharging Ana Bilbao+1 312 364 8598abilbao@williamblair.com Please refer to important disclosures on pages 44-46. Analyst certification is on page 44.William Blair or an affiliate does and seeks to do business with companies covered in its research reports. As aresult, investors should be aware that the firm may have a conflict of interest that could affect the objectivity of thisreport. This report is not intended to provide personal investment advice. The opinions and recommendations here- Introduction.......................................................................................................................3Key Takeaways...................................................................................................................3Brief History of Memory....................................................................................................5Bits and Bytes: The Memory Hierarchy............................................................................9 While so much of the AI platform shift has been focused on computing demand, the traditionallyslower growth and less exciting memory market is starting to have its day in the sun. Advance-ments in AI compute are increasingly hindered by memory bottlenecks, revitalizing memory as anarea of innovation as vendors race to develop high-bandwidth solutions capable of addressing thedata-intensive needs of AI models. With AI inference demand still in a phase of early adoption, we The William Blair technology team has previously explored AI-driven advancements in compute,detailing how the AI platform shift require a deeply integrated network of next-generation tech-nologies (From Chips to Systems: How AI Is Revolutionizing Compute and Infrastructure) and of-fering a GenAI primer (Generative AI: The Next Frontier of Innovation). In this report, we offer In conjunction with this report, we are initiating coverage of three companies:Micron,Rambus, andSilicon Motion. Bandwidth the Key AI Memory BottleneckAI performance is increasingly limited by memory bandwidth, since large language models (LLMs) require moving enormous volumes of data for relatively little computation.. Training and inferenceinvolve repeatedly streaming massive amounts of data—e.g., model parameters, context, and KVcache—through accelerators with relatively low data reuse. As a result, performance is often lim-ited by how fast data can be delivered rather than how many compute floating-point operations persecond (FLOPS) are available. Since the 1990s, processor performance has scaled at a much fasterrate than memory bandwidth, leading to the development of the “memory wall.” While DRAM band- HBM Improves Memory Access for AI Accelerators To help address the memory wall problem, memory vendors have developed a new type of dy-namic random-access memory (DRAM), high-bandwidth memory (HBM). HBM helps address thememory wall by increasing effective memory bandwidth—DRAM memory stacks are integratedin-package with the processor and connected via thousands of short, low-power wires on a siliconinterposer (TSVs, or through-silicon vias). Because of its wide interfaces (16 times wider than tra-ditional double data rate [DDR] channels), HBM enables GPUs to quickly access large volumes of Memory Vendors Shifting Focus Toward More Profitable HBMThe success of HBM, first introduced by SK Hynix and followed by offerings from Micron and Sam- sung, has pushed the major memory vendors toward HBM because it delivers higher revenue and 3Sebastien Naji +1 212 245 6508 William BlairAI / INSIGHTS 1.2–1.3 TB/s, rising to more than 2.0 TB/s with HBM4, versus only about 50–100 GB/s per DDR5DIMM. Economically, HBM sells for ASPs that are roughly 3-4 times that of traditional DDR DRAM,and despite higher TSV and packaging costs and lower yields, gross margins are materially higherbecause AI customers value bandwidth far more than raw capacity. We estimate HBM gross mar-gins are in the 55% to 65% range today, compared to historical DRAM gross margins between 25%and 45%. This is driving a dramatic mix shift in capacity allocation at the major memory vendors.SK Hynix, which controls approximately 60% of the HBM3 market, now allocates more than half of HBM Base Die Becoming a Critical Value Capture Battleground The base logic die in HBM has evolved from a relatively simple interface into a strategically im-portant compute-adjacent component with room for significant customization. Thus far, HBMbase dies have relied on older memory nodes and handled basic functionality (e.g., managingelectrical connections, translating high-level GPU instructions, distributing signals across TSVs); most intelligence resides in the GPU’s memory controller. With HBM4, there is a shift to a morecapable and performant base die built on leading process