On October 20, 2026, DeepSeek put the inference framework that runs its own production API on GitHub under Apache 2.0, and the most consequential line in the release was buried in the prerequisites. DeepSeek's DeepInfer names exactly two supported hardware platforms: Nvidia SM100 GPUs — the Blackwell data center architecture — and Huawei Ascend 950 NPUs. Hopper is not on the list. Neither is anything older. Four commits of history, 21.6k stars and 2.4k forks within days, and a compatibility table that functions as a geopolitical statement. The Hacker News thread, submitted by a user called ggcr, cleared 2,345 points and 891 comments to become one of the largest AI discussions of the month — and spent almost none of them on kernels. It spent them arguing about whether anyone in this industry is making money.
What DeepSeek Actually Shipped
This is not a reference implementation or a cleaned-up research artifact. DeepSeek describes DeepInfer as the high-performance inference framework powering its large-scale production services, released as an offline-inference build with the production optimizations left intact. The default checkpoint is deepseek-ai/DeepSeek-V4.1-Flash-1017 — the build that landed over the weekend of October 18, days before the framework that serves it. The prerequisites are narrow and specific: Linux, GCC 13.0 or newer, Python 3.12, and either CUDA 13 on Nvidia or CANN 9.2 on Ascend. There is no convenience wheel in the getting-started path. You build from source, then run the offline inference example script.
The Kernel Stack Is the Part You Cannot Rebuild
DeepInfer bundles kernels DeepSeek has been releasing piecemeal for more than a year — FlashMLA, DeepGEMM, DeepEP — alongside pieces that had not shipped before: TileKernels, DeepSelect, and custom kernels written in TileLang, the kernel language DeepSeek published on September 30, 2026 specifically to make Ascend support possible. Parallelism is handled by attention data parallelism with balanced context parallelism, plus mixture-of-experts expert parallelism with EPLB load balancing. The README is candid about lineage: DeepInfer was originally built primarily on vLLM, with inspiration drawn from SGLang and TensorRT-LLM. That matters for anyone weighing a migration. This is not a parallel universe of abstractions — it is the serving stack most teams already recognize, with DeepSeek's production tuning fused into it.
Two Optimizations With Numbers Attached
Most framework launches ship adjectives. Two of DeepInfer's features ship figures:
- DSpark speculative decoding. A draft model proposes five tokens per step, and a confidence head predicts which of them will be accepted, so verification is scheduled rather than run blindly. Trade reporting puts the effect at roughly 30% fewer draft tokens verified per step, translating into a 4–7% throughput gain at production load.
- Decoder SWA bounded replay. Instead of pushing the full prompt back through the decoder layers, only the final 128 tokens are replayed — reported as close to a 50% reduction in prefill computation.
Notice the honest shape of those numbers. A 30% cut in verified draft tokens buys 4–7% end to end. That is what mature optimization looks like at scale: the easy multiples are long gone, and a single-digit percentage across a fleet is worth open-sourcing a framework over. Asynchronous scheduling is enabled by default. Structured outputs cover regex, JSON schema, and EBNF grammar.
Why Huawei's Name in the README Is the Headline
Listing Ascend 950 beside SM100 is not a courtesy credit. It is DeepSeek telling Chinese operators that the stack serving one of the world's highest-traffic model APIs has a tested path off Nvidia silicon. The timing sharpens the point considerably: DeepInfer appeared one day after the U.S. Department of Homeland Security ordered federal civilian agencies to remove DeepSeek products. And the hardware commitment behind the line item has already been placed — Bloomberg reported in September that DeepSeek ordered 160,000 Ascend 950DT accelerators for a new data center in Inner Mongolia, and the lab has said it intends to begin pre-training on Ascend hardware before the end of 2026.
Equal Billing Is Not Equal Support
Read the support matrix closely and the parity turns out to be marketing-grade rather than engineering-grade. Two of DeepInfer's reproducibility features are Nvidia-only, and every published benchmark figure was measured on Nvidia hardware.
| Capability | Nvidia SM100 (Blackwell) | Huawei Ascend 950 NPU | Nvidia Hopper and older |
|---|---|---|---|
| Listed as supported | Yes | Yes | No — absent from the list |
| Required toolchain | CUDA 13 | CANN 9.2 | — |
| Optimized kernels (FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect) | Yes | Yes — TileLang path, published September 30, 2026 | — |
| DSpark speculative decoding | Yes | Yes | — |
| Asynchronous scheduling (default on) | Yes | Yes | — |
| Batch-invariant inference | Yes | No — Nvidia only | — |
| Deterministic sampling with fixed seeds | Yes | No — Nvidia only | — |
| Published performance benchmarks | Yes — 4 GPUs prefill, 64 decode | None released | — |
The empty column is the story. There is no public performance data for DeepInfer on Ascend 950 at all — no throughput figure, no latency figure, no Pareto curve. So the framework declares first-class support for a Chinese accelerator while shipping zero evidence of how it actually performs there. Anyone reading the README as proof of Nvidia-equivalence on Huawei silicon is mistaking a compatibility claim for a benchmark.
What Nvidia-Only Determinism Costs You
The two features restricted to Nvidia are the two that matter most to anyone running evaluations. Batch-invariant inference means a given input produces the same output regardless of what else is in the batch; deterministic sampling with fixed seeds means a run can be repeated exactly. Lose both and you lose the ability to reproduce an eval run bit for bit, which is the difference between a regression test and a vibe check. On Ascend, DeepInfer will serve tokens. It will not currently let you prove that yesterday's numbers were real.
What Is Not in the Box
DeepSeek is unusually explicit about the holes, and they happen to be the expensive parts. The README lists three components as required for a complete inference service and not included in this release:
- A KV cache system
- VE/Prefill/Decode disaggregation
- Load balancing across data-parallel replicas
According to trade reporting, the KV cache management and load-balancing code was withheld as too tightly coupled to DeepSeek's internal infrastructure. Guidance on prefill/decode separation is provided; the implementation is left to the operator. This echoes the reasoning DeepSeek gave in April 2025, when a partial engine release cited the resource constraints of a small research team focused on building better models rather than maintaining infrastructure for outsiders.
Scale is the other filter, and it is the harsher one. The README's performance benchmarks were measured on 4 SM100 GPUs for prefill and 64 for decode; at least one trade writeup instead described the minimum efficient deployment as 64 for prefill and 64 for decoding, so treat the prefill figure as unsettled pending a cleaner reading. Either way, the decode side is 64 Blackwell-class accelerators. "Anyone can now self-host DeepSeek's stack with its production optimizations intact" is entirely true in the license sense and almost entirely false in the procurement sense.
The Thread Stopped Talking About Software
DeepInfer is a serving framework. The 891-comment discussion it produced is an argument about whether the AI industry's unit economics work at all, and the pivot happened in the top comment. That is less of a digression than it looks: a company that publishes the exact engine behind its API is also, inadvertently, publishing evidence about what it costs to run one.
The $38 Number
Commenter schneehertz laid out DeepSeek's disclosed production figures — a cluster throughput of 155 trillion tokens per day, against roughly $38 of revenue per card per hour at off-peak pricing. abound ran the obvious arithmetic in reply: the amortized cost of owning a data center full of GPUs sits far below $38 per card-hour, so the margin is real rather than a rounding artifact. For a debate that usually runs on speculation about undisclosed cloud contracts, having a first-party number from the operator of a top-traffic API is genuinely new information.
The Objection: Inference Is Not the Business
Closi drew the distinction the entire thread turns on — inference looks profitable for major providers in isolation, and those same providers are unprofitable once training, research, and overhead are added back in. HarHarVeryFunny pushed it further: computing the margin on serving a model while ignoring what it cost to produce that model is a meaningless figure. v9v supplied the counterweight, citing a leaked Liang Wenfeng speech that put training-cost recovery at 8 to 12 months of inference revenue. If that holds, frontier AI is a financing problem rather than a broken business model — a distinction worth roughly a trillion dollars of market capitalization depending on which way it resolves.
Commodity Tokens, Trillion-Dollar Expectations
horsawlarway framed the bind most sharply: inference margins can be comfortably positive and still nowhere near what is needed to justify valuations built on selling a fungible commodity. tsunamifury described the trap as a cycle — you bleed revenue in the gap between releases, then bleed margin when you close that gap by training the next model. jeremyjh noted the commoditization is already measurable, with 20-plus providers on OpenRouter competing to serve the same open weights and prices moving in only one direction. jacquesm said open models now cover 95% to 99% of what he needs, with GLM 5.3 close behind DeepSeek. And WarmWash raised the asymmetry nobody in the thread had a clean answer to: Chinese labs operating with state resources and no obligation to return a profit are not playing the same game as providers who answer to investors every quarter.
What It Actually Changes for You
Strip away the geopolitics and three practical things are different this week.
- If you serve V4.1-Flash at scale, you can adopt DeepSeek's own tuning instead of approximating it — and you will still be writing your own KV cache system and your own load balancer.
- If you need reproducible evaluations, batch-invariant inference and fixed-seed deterministic sampling are both available, on Nvidia only.
- If you are planning hardware, the signal to read is Hopper's absence, not Huawei's presence. An entire Nvidia generation did not make the cut, which tells you what DeepSeek's production fleet now assumes.
One smaller piece deserves attention because it solves a problem that bites quietly: a companion library called deepseek-recipe converts requests into the token sequences matching DeepSeek's official API logic. That is the usual source of behavioral drift between a hosted endpoint and a self-hosted copy of the same weights, and shipping it alongside the engine is the difference between reproducing DeepSeek's API and merely resembling it.
The next thing worth watching is not a feature release. It is the first credible third-party benchmark of DeepInfer running on Ascend 950, published by someone with no stake in either chipmaker. Until that exists, Huawei's equal billing is a line in a README and every number anyone can check still came off Nvidia hardware. DeepSeek has said it intends to begin pre-training on Ascend before the end of 2026 — roughly ten weeks from now — and pre-training is a far less forgiving test than serving tokens. If that deadline holds, the README stops being a statement and becomes a measurement.
-EditorZ
Photo by Taylor Vick on Unsplash
Post a Comment