DeepSeek open-sources DeepInfer, the stack behind V4.1-Flash

DeepSeek published DeepInfer on 20 October 2026 under the Apache License 2.0. The repository describes it as the inference framework that powers DeepSeek’s large-scale production services. The supported model is DeepSeek-V4.1-Flash, and DeepSeek’s post says that is the model it is serving at scale. The release is an offline inference stack: optimised kernels, several ways of splitting work across chips, and options for deterministic sampling and structured output.

The supported hardware is NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. DeepSeek says a full service still needs pieces this release does not include: a KV cache system, which is the memory of an ongoing conversation, disaggregation of prefill and decode, and load balancing across replicas. It says most of the engine work for that split is already in the code. The default checkpoint is deepseek-ai/DeepSeek-V4.1-Flash-1017.

Alongside it, DeepSeek published deepseek-recipe under the MIT licence. That library turns chat and responses requests into the prompt and token IDs a DeepSeek V4 or V4.1 model expects, and turns the model’s output back into an API response. DeepSeek’s post calls it the production token encoding and decoding used to match the official API. The caller still has to run the model, execute tools and serve HTTP.

DeepSeek’s charts plot how prefill throughput trades off against latency, and how decode throughput trades off against per-user speed, for DeepSeek-V4.1-Flash.