Oxmiq Labs is presenting HBF in AI Compute at Hot Chips 2026, making the case for high-bandwidth flash as a capacity tier for AI inference. This ended up being a major discussion topic, so we wanted to get this one out and its own day to give it a bit more exposure.
We did this one live, but it got stuck in the queue.
Oxmiq Labs HBF in AI Compute at Hot Chips 2026
Oxmiq opens by laying out the HBF hardware specification across three grades. Maximum user bandwidth rises from 0.384 TB/s to 3.072 TB/s and UCIe rates climb from 8 GT/s to 32 GT/s, while capacity tops out at 512 GiB on a 16-high stack. This pitch is that HBF delivers 8 to 16 times the capacity of HBM at the same cost.
Oxmiq frames HBF within the memory technology landscape using alpha and beta, with beta tracking cost and alpha tracking bandwidth. A key point is that HBF is a capacity point on this landscape, not a cheaper version of HBM.
Next comes model personality, with a larger MoE model set against dense models in terms of batch size and tokens/s. MoE opens up a small batch capacity sub-phase, where the model is held in full and lower bandwidth is fine.
Oxmiq is now breaking down the cost of serving in terms of a token. They are arguing that the real metric is the cost of serving the token in terms of hold and feed (bandwidth).
Oxmiq lays out a few ways to deploy HBF, covering an HBM-only baseline, an all-HBF configuration, and a 2x HBF-6x HBM mix. All three sit in the same cost envelope, with the HBF options trading bandwidth for capacity. Peak bandwidth drops from 22 TB/s on the HBM baseline to 12.8 TB/s on all-HBF, while capacity jumps from 288 GB to 4 TB.
First, Oxmiq examines an all-HBF system with a short 256-by-256 context. HBF wins on cost at low batch, but with 85 percent “dead capacity.”
Moving to long context at 1M tokens in and 1K out, the picture shifts. Capacity wins only while the bandwidth demand remains low. Beyond that point, HBM is the better option.
The next case considers a mixed case in which HBM acts as a hot-expert cache. Expert popularity flattens across mixed queries, so caching pays off only at low batch sizes or when similar queries are batched together.
Oxmiq details its system and simulation setup at the rack level across a 72-GPU rack. Using a decode-centric Kimi-K2 1T model at FP4 with 1M tokens in and 1K out, the simulation finds that for the same rack and cost, HBF buys roughly 14x the capacity for about 0.6x the bandwidth.
A first sim check plots rack dollars per token against single-server capacity. Cheaper dollars per GB does not equal cheaper dollars per token. This is neat as the math exercise behind why not HBF, or maybe more where HBF is playing.
Going into that first chart they are mapping the same trade across batch sizes. HBM keeps its edge at the rack scale, while HBF fits best where capacity per box is the goal, a useful split for buyers choosing deployment scale.
Here is the other chart that zeroes in on the bandwidth-versus-capacity split within a single server. Matching memory to deployment scale matters as much as the sticker price per gigabyte.
Oxmiq then turns to the software constraints of HBF. Access happens in 64 KB chunks for maximum bandwidth, with roughly 24 hours of power-on data retention at 85 degrees Celsius and host-managed lifecycle handling. Since the memory is read-optimized and write-constrained, placement becomes a software problem.
The company walks through the inference engine ecosystem, ranging from production serving engines like vLLM and SGLang to vendor-optimized stacks such as TensorRT-LLM and AWS Neuron. Oxmiq treats vLLM as the production default and the focus for HBF.
Oxmiq then looks at where the memory actually goes in a model like Kimi-K3. Roughly 93 percent of bytes are MoE expert weights at 1.45 TB, so Oxmiq argues HBF suits the MoE expert pool and KV-cache offload, while HBM handles attention weights and hot data.
Inside vLLM, Oxmiq proposes a plugin that uses HBF in place of host CPU pinned memory for the KV cache and MoE expert pools. A GPU with 4x HBM and 4x HBF stacks would reach 2.2 TB with about 17.4 TB/s peak bandwidth, far more capacity than today’s offload paths.
Oxmiq sees attention sparsity as a genuine HBF fit. Sparse-attention models such as DeepSeek Sparse Attention and Compressed Sparse Attention play directly into HBF’s low-bandwidth capacity zone, though the fit stays conditional on that model style.
Oxmiq closes its case with expert parallelism. Cheap HBF capacity means fewer EP shards and less all-to-all communication, allowing 2 nodes to hold experts locally in HBF instead of sharding them across 8 GPUs and incurring all-to-all traffic at every layer.
Oxmiq wraps with a set of takeaways that frame HBF through a much narrower lens than some of the major NAND vendors. Really, it concludes that HBF wins only where bandwidth demand is low, such as MoE with small batch and long-context sparse KV.
Those observations are an interesting framing, to say the least, especially compared to many HBF promoters, who are less balanced in their messaging.
Final Words
HBF looks like a capacity instrument aimed at a narrow but real slice of inference work, particularly large MoE models where expert weights dominate. Matching memory type to workload and deployment scale matters more than raw dollars per gigabyte. On the software side, it seems like an HBF allocator and placement policy are the missing pieces before this becomes practical. That said, it is probably less of a lift than it would have been without AI-native coding tools to help. It was really neat to see a candid take like this.
