DeepSeek DeepInfer-কে ওপেন-সোর্স করেছে, DeepSeek-V4.1-Flash-এর পেছনের ইনফারেন্স ফ্রেমওয়ার্ক

এটি ফিডে পাওয়া অংশবিশেষ। এটি সংক্ষিপ্ত বা প্রকাশকের মূল ভাষায় হতে পারে।

DeepSeek has open-sourced DeepInfer, the inference framework that runs its production services. The DeepSeek DeepInfer inference framework is the engine behind DeepSeek-V4.1-Flash. It ships under the Apache 2.0 license. DeepSeek also released deepseek-recipe, its MIT-licensed token encoding and decoding layer. For developers, this is a rare look at a frontier lab’s real serving stack. TL;DR Size / scope: 1 supported model (DeepSeek-V4.1-Flash), 1M-token context window per the model card. Runs on: NVIDIA SM100 (Blackwell) GPUs and Huawei Ascend 950 NPUs. Benchmarks used 4 GPUs for prefill and 64 for decode. Performance: Production-grade throughput; DeepSeek says 1 cluster serves 155 trillion tokens per day. Best: Above 25,000 output tokens/s/GPU at 256 requests per rank (EP64, 131,072 context). Worst: Offline inference only. KV cache system, P/D disaggregation and load balancing are not included. What is DeepInfer? DeepInfer is a high-performance inference engine from DeepSeek for its own models. According to it’s GitHub README, it powers DeepSeek’s large-scale production services. This release covers offline inference with the core features needed at scale. It was originally built primarily on vLLM, with ideas from SGLang and TensorRT-LLM. The deployment Q&A gives production context. A single cluster serves 155 trillion tokens per day. DeepSeek reports overall revenue of US$38 per GPU-hour at valley pricing. How does DeepInfer work? The engine stacks DeepSeek’s kernels: FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect and custom TileLang kernels. On top sit 5 system-level ideas. Parallelism: Attention DP plus Wide-EP Attention uses data parallelism. The MoE layers use expert parallelism with an expert-parallel load balancer (EPLB). Decode runs EP64 to EP128 inside a supernode. Prefill uses the smallest EP that memory allows, typically EP2 to EP8. DSpark: confidence-scheduled speculative decoding A draft model proposes 5 tokens per step. A Confidenc