Table of Contents
Unveiling SanDisk’s Controversial Investor Day Presentation: A Closer Look at HBM Capacity, Bandwidth, and Write Endurance
SanDisk recently stirred up a storm in the tech world with its Investor Day presentation, which sparked a flurry of discussions and debates on social media. The presentation, initially perceived as routine, took an unexpected turn when questionable slides surfaced, triggering a backlash from the tech-savvy community.
SanDisk’s HBM Capacity, Bandwidth, and the Missing Puzzle Piece
An AI model is a complex system built on transformer blocks that consist of two crucial components:
- The Attention Layer: This layer focuses on the relationships between words in a sentence. It identifies connections such as linking “Paris” to “France” when given the prompt “Paris is the capital of France.” The layer stores contextual information in a KV cache, which grows as the context expands.
- The Feed-Forward Network (FFN): This network retains the model’s knowledge in the form of weights and utilizes intricate mathematical formulas to decipher the meaning behind individual words. For example, it would access its country encyclopedia when encountering the word “France.”
At present, both model weights and KV cache are stored in High Bandwidth Memory (HBM). However, expanding HBM capacity is challenging and costly, as it involves increasing GPU counts.
HBF: A Promising Solution
High-Bandwidth Flash (HBF) emerges as a potential solution, with SanDisk collaborating with SK hynix to develop a standard offering 512GB storage and bandwidth ranging from 0.4TB/s to 3TB/s. The commercial launch is slated for 2028/2029.
In essence, HBF stacks NAND die atop each other, connected by Through Silicon Vias (TSVs), with a logic die controlling this NAND array.
Despite its advantages, HBF faces challenges due to NAND cells’ slow write speeds and limited write endurance.
Comparing HBF to HBM
SanDisk’s presentation raised eyebrows due to inaccuracies and omissions, such as comparing HBF, set for launch in 2028/2029, to the 16Hi HBM4E, expected to dominate the market then. The latter boasts a bandwidth of 32TB/s, significantly higher than the 12.8TB/s projected by SanDisk.
Furthermore, SanDisk’s use of bf16 quantization, instead of the more common fp8 or fp4, added fuel to the fire of criticism.
Notably, the omission of HBF’s expected write endurance raises concerns about its suitability for KV cache.
In Conclusion
SanDisk’s recent missteps in accuracy and transparency have ignited discussions within the tech community. While the company may defend its presentation, the discrepancies uncovered paint a concerning picture. As technology continues to evolve rapidly, ensuring precision and foresight in product development and communication remains paramount.
About the author: With over six years of experience in financial and tech writing, I bring a nuanced perspective to each topic. As an alum of the University of Toronto, I strive to deliver in-depth knowledge and unique insights in every piece I craft. When not writing, I explore the world’s culinary delights as an aspiring food connoisseur.
Follow Wccftech on Google to stay updated with our latest news coverage.