DeepSeek DSpark: Semi-Autoregressive Speculative Decoding Explained

DeepSeek’s new DSpark module brings speculative decoding to DeepSeek-V4. It might look like a niche inference tweak, but in production it boosted per-user generation speed by 60 to 85 percent with no drop in model quality.

What sets DSpark apart is that it tackles two longstanding problems at once — weak draft quality and the waste of verifying drafts — where prior methods addressed only one. This article breaks down how it solves both and why that matters at production scale.

What Is Speculative Decoding?

LLM generation is slow because each token needs a full forward pass through the model. Speculative decoding speeds this up with a smaller draft model that predicts several future tokens at once, which the target model then verifies in a single pass.

If the draft model makes good predictions, several tokens can be produced from a single forward pass through the target model. If it makes poor predictions, it reverts to its normal pace. Output quality is still maintained because the target model verifies the predictions against its own probability distribution.

The key issue is developing an appropriate draft model:

• When it is sequential and accurate over long predictions, it cannot keep up with the target model and fails to produce multiple tokens before the target finishes.
• In this case, latency keeps increasing based on the number of blocks being processed.

By making the draft model faster and parallel rather than sequential, the predictions become less accurate in the latter part of the block. DSpark demonstrates a solution that addresses both factors at once.

The Core Idea: Semi-Autoregressive Drafting

In an autoregressive context (such as Eagle3), each generated token is conditioned on all previously generated tokens. While this is representative of traditional machine learning training, it is inefficient, since the model experiences a linear increase in latency the longer the number of tokens generated.

In a parallel context (such as DFlash), the model generates an entire block of tokens in a single forward pass. This produces very fast output. However, each token is estimated in isolation from the others in the block. The result can be an incoherent mix of words — “of” and “problem” each form reasonable phrases (“of course” and “no problem”), but combined (“of problem”) they no longer make sense.

Source: X

DSpark combines a largely parallel structure for speed with a thin sequencing layer that adds local dependencies between tokens. Together, it is a mostly-parallel approach with a thin layer of autoregression on top to fix incoherence across the sequence.

The paper presents two sequencing structures:

• A Markov head uses only the preceding token plus a low-rank matrix, achieving nearly no overhead.
• An RNN head maintains a minimal recurrent state across the block, giving it more context than the Markov head.

DeepSeek found the Markov head delivers essentially all the benefits at much lower complexity, so that is the one they put into production.

Getting Started with DeepSpec

DeepSeek has open-sourced the training and evaluation code for their draft models as DeepSpec. This is a complete repository for training any type of draft model — not just DSpark, but also DFlash and Eagle3. You can reproduce their comparisons of those models using this repo.

To install the dependencies and clone the repo:

git clone https://github.com/deepseek-ai/DeepSpec.git
cd DeepSpec
python -m pip install -r requirements.txt

This covers the installation for training and evaluating models with DeepSpec. You will still need to prepare your data separately by using a mechanism to infer outputs from the target model. For more information, consult the scripts/data/README.md within the repo.

Hands-On: Training and Evaluating a Draft Model

There are three stages in a DeepSpec workflow: preparing data, training your model from the draft, and evaluating it. The output of one stage becomes the input to the next.

Step 1: Picking a Config

Config files are in the config/ folder, with one file for every pair of algorithms and target models.

ls config/dspark/
# dspark_qwen3_4b.py  dspark_qwen3_8b.py  dspark_gemma4_12b.py

Each config file specifies the target model, block size, and which sequential head to use. To match the smallest benchmark described in the paper, use the dspark_qwen3_4b.py configuration file.

Step 2: Training the Model

To start training, run:

bash scripts/train/train.sh --opts config_path=config/dspark/dspark_qwen3_4b.py

The script creates one worker per GPU on your system. Checkpoint files are saved to ~/checkpoints///step_*. If you are using a single node, set the CUDA_VISIBLE_DEVICES variable to match the number of GPUs available.

The training process optimises three loss types simultaneously:

• A cross-entropy term for predicting the next token correctly
• A distribution-matching term directly related to the acceptance rate of generated content
• A confidence loss, which enables the scheduling trick described below

Step 3: Evaluation

bash scripts/eval/eval.sh \
  --target_name_or_path Qwen/Qwen3-4B \
  --draft_name_or_path ~/checkpoints/deepspec/dspark_block8_qwen3_4b/step_latest

Verification happens in one pass. Measure how many tokens are accepted across three task types: math, code, and chat. More accepted tokens means fewer wasted forward passes toward the target model.

Experimental Results

DSpark exceeded Eagle3’s accepted length by approximately 27–31% and exceeded DFlash’s output by 16–18%. Both improvements remained consistent across the Qwen3-4B, 8B, and 14B targets. DSpark performed similarly on Gemma4-12B as well, indicating the gains are not a quirk specific to Qwen’s architecture.

The cross-family outcome is meaningful: architecture-specific tricks typically break down when tested on a different model family, so consistent results across both Gemma and Qwen represent a more reliable signal than single-model comparisons.

Gotchas and Things That Trip People Up

A few important points to keep in mind regardless of your setup:

• Chat verifies differently from code: Chat has more valid next tokens and therefore a lower confidence rate than code. This means confidence decreases faster in chat tasks, and scheduling will prune draft tokens more aggressively. Expect different acceptance rate profiles across task types and tune your block size accordingly rather than using a single value for all workloads.

DSpark’s combination of semi-autoregressive drafting, confidence-based scheduling, and the lightweight Markov head makes it a practical and deployable advance in speculative decoding — one that holds up across model families and task types at production scale.