Remember when choosing a chip meant comparing two numbers? Clock speed and memory size, printed on a box, and you were done. That habit never really died. It just migrated to AI accelerators, where the marketing still lands on a headline compute figure and a memory capacity, and everyone nods along as if those two numbers explain the part.
They don’t. And the reason they don’t is the thing I keep circling back to when I evaluate accelerator tooling: the piece of an AI accelerator that decides whether it works is often not the logic die at all. It’s the packaging underneath it.
What the spec sheet leaves out
A modern accelerator is not one chip. Reporting on ABF substrates in data center silicon describes the current approach plainly, designers now place multiple large logic dies alongside memory on a single package, because that’s the only way to feed frontier models the compute and memory bandwidth they demand. The substrate is what holds all of that together and routes signals between the pieces.
Which makes it a load-bearing component in the most literal sense. The industry is pushing the limits of silicon packaging to meet growing compute demand, and that limit isn’t a transistor problem. Bloomberg’s 2026 outlook on AI accelerators frames the broader situation the same way, computing demand has pushed past what Moore’s Law alone can deliver, so the gains have to come from somewhere else. “Somewhere else” turns out to mean assembly, interconnect, and how much memory you can physically park next to a die.
None of that shows up in a benchmark chart. All of it shows up in whether you can actually buy the part.
Follow the money and it points at memory
The funding pattern in early 2026 is the clearest signal here. Positron AI raised $230 million in a Series B on February 4, 2026, for a memory-centric inference accelerator. Read that description again, the pitch isn’t “more FLOPS,” it’s a design organized around memory. Cerebras Systems closed $1 billion in a Series H on February 3, 2026, for a wafer-scale training and inference processor, which is essentially a bet that the packaging problem gets solved by refusing to cut the wafer into pieces in the first place.
Two very different answers to the same question. Both funded heavily within two days of each other.
Then there’s Microsoft Maia 200, unveiled in January 2026 as an inference accelerator built on TSMC’s 3nm process with 216GB of HBM3e. It’s in mass production and serving Microsoft 365 Copilot, with Anthropic reportedly in talks. The number I’d point at there isn’t the process node, it’s the 216GB. That much high-bandwidth memory on a package is a packaging achievement before it’s a compute achievement.
Why this matters if you’re picking tools, not chips
Most people reading a toolkit review are not buying accelerators by the rack. So why care about substrate engineering? Because it shapes three things you will absolutely feel:
- Availability. Packaging capacity is a physical constraint. When it tightens, the parts your inference provider promised you get scarce, and prices move.
- Fragmentation. Custom silicon is proliferating. Google’s TPU infrastructure, AWS Trainium clusters, and Meta’s MTIA accelerators are all in production, and TrendForce projects custom ASIC shipments from cloud providers to grow 44.6% in 2026. Your framework’s support matrix is going to get more crowded, not less.
- Memory ceilings. If the accelerator you’re deploying on has a hard memory limit set by what fits on the package, that limit becomes your model size limit, your batch size limit, and your context window economics.
The market leaders in 2026 remain Nvidia, AMD, and Broadcom, with real money going into new technologies and infrastructure, and a crowd of startups working on inference and custom silicon around them. That’s a healthy amount of competition. It’s also a lot of incompatible hardware arriving faster than tooling can absorb it.
My honest read
I’ve stopped treating accelerator comparisons as compute comparisons. When I look at a new part now, I want to know how much memory sits on the package, how many dies are in there, and whether the vendor can actually ship volume. Maia 200 being in mass production is a more useful fact about it than its process node. Positron’s memory-centric framing tells me more about where inference bottlenecks live than any throughput claim would.
The uncomfortable part is that the interesting engineering has moved somewhere buyers can’t easily inspect. You can benchmark a model. You can’t benchmark a substrate. You just find out later, when supply gets tight or your memory budget doesn’t stretch as far as the datasheet implied.
So when the next accelerator gets announced with a big compute number on the slide, my first question won’t be about the number. It’ll be about what’s holding it up.
🕒 Published: