DeepSeek open-sources DeepInfer, the inference engine behind its own service

DeepSeek has released DeepInfer on GitHub under the Apache 2.0 licence. It says this is the inference framework that powers its large-scale production services. This release covers offline inference only, supports one model (DeepSeek-V4.1-Flash) and runs on NVIDIA SM100 GPUs or Huawei Ascend 950 chips. Features include structured outputs (JSON schema, regex or grammar), deterministic sampling and batch-invariant results on NVIDIA. Parts you need for a full service are not included: a KV cache system, prefill and decode disaggregation, and load balancing across replicas. The README says it was originally built mainly on vLLM, and DeepSeek's speed charts were measured on 4 SM100 GPUs for prefill and 64 for decode. On Hacker News (2,108 points, 63 top-level comments when we read it) there is no consensus. One camp calls it a blow to US labs' pricing and valuations. Others push back: one commenter says more than 30 providers already serve DeepSeek with existing engines, and three point out that the KV cache and other serving pieces are missing. Three asked how it compares with vLLM or SGLang, and no top-level comment gave numbers. Huawei Ascend support drew attention, though one commenter said US rules keep those chips out of reach for American buyers. Another noted that a US host running DeepInfer would be running a DeepSeek product, which matters given the new federal removal order. Sorted take: for most teams this is not a drop-in tool. It needs top-end GPUs and you would still build the cache and routing yourself. The useful signal is price: cheaper serving code for DeepSeek models could mean lower prices at the hosts you already use.