shanjiaz/qwen3_5_4b_perfectblend_regen_dspark

Qwen3.5-4B PerfectBlend DSpark

This is a five-layer DSpark speculative decoding draft model for Qwen/Qwen3.5-4B. It is the best checkpoint from the original five-epoch training run (epoch 5, zero-based checkpoint directory 4).

The model proposes eight speculative tokens per verification step and consumes five verifier hidden states from layers 19, 23, 27, 29, and 31.

Architecture

Setting	Value
DSpark transformer layers	5
Verifier hidden-state layers	19, 23, 27, 29, 31
Speculative tokens	8
Hidden size	2,560
Intermediate size	9,216
Attention heads / KV heads	16 / 4
Markov head	Vanilla, rank 256
Confidence head	Enabled, with Markov input
Training dtype	bfloat16
Loss	0.1 CE + 0.9 TV, confidence alpha 1.0

Autoregressive acceptance results

The primary evaluation used real autoregressive vLLM decoding. Accepted tokens were fed back into the subsequent decoding context; these are not teacher-forced measurements.

Evaluation configuration:

• Dataset: RedHatAI/speculator_benchmarks
• 9 standard subsets, 80 requests per subset (720 total)
• Temperature: 0.0
• Maximum concurrency: 8
• Maximum completion length: 4,096
• Maximum model context: 32,768
• Speculative tokens: 8
• Serving vLLM: 0.28.1rc1.dev267+g46c8a161f

Acceptance length is 1 + accepted_draft_tokens / draft_steps.

Subset	Acceptance length
HumanEval	4.9126
Math reasoning	5.4006
QA	5.7102
Question	4.0906
RAG	4.4461
Summarization	3.7130
Tool call	4.3839
Translation	2.8460
Writing	3.9882
Macro mean	4.3879
Draft-step weighted	4.1407

Aggregate positional acceptance

The following rates are weighted by the number of draft steps in each subset.

Draft position	Acceptance rate
0	77.87%
1	59.10%
2	45.73%
3	36.57%
4	30.23%
5	25.29%
6	21.30%
7	17.99%

Across all proposed draft tokens, 39.26% were accepted.

Probabilistic-sampling stress test

The same checkpoint was also evaluated with the request temperature omitted, which makes vLLM use temperature 1.0, top-p 1.0, and top-k 0.

Setting	Macro acceptance length	Draft-step weighted	Accepted draft tokens
Temperature 0.0	4.3879	4.1407	39.26%
Temperature 1.0	2.8853	2.7640	22.05%

This model was trained entirely on temperature-0 trajectories, so the greedy result is the matched evaluation. Users whose deployment relies on probabilistic sampling should benchmark the exact production temperature, top-p, and top-k.

Training data

The processed training dataset contains 10,000 examples from shanjiaz/qwen3_5_4b_perfectblend_regen. All trajectories were generated at temperature 0.0. Reasoning effort was approximately balanced:

Reasoning effort	Examples
Low	3,310
High	3,355
Max	3,335

Of the 10,000 source responses, 7,142 ended with finish_reason=stop and 2,858 ended at the generation-length limit.

Training validation

The saved epoch-5 validation metrics were:

Metric	Value
Validation loss	0.3213
Analytical acceptance rate	0.7488
Analytical acceptance length	5.4793
Greedy token accuracy	0.7653

The validation acceptance length is an in-domain, teacher-forced analytical estimate based on target/draft distribution overlap. It should not be compared directly with live autoregressive acceptance length.

Three additional low-learning-rate epochs were tested separately. That continuation lost to this checkpoint on seven of the nine matched autoregressive workloads, so this epoch-5 checkpoint is the recommended version.

Serving with vLLM

This model requires a vLLM build with DSpark support. The evaluation used:

vLLM 0.28.1rc1.dev267+g46c8a161f

Example command:

CUDA_VISIBLE_DEVICES=0 \
python -m vllm.entrypoints.cli.main serve Qwen/Qwen3.5-4B \
  --host 0.0.0.0 \
  --port 8000 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.90 \
  --speculative-config '{"method":"dspark","model":"shanjiaz/qwen3_5_4b_perfectblend_regen_dspark","num_speculative_tokens":8}'

For the matched greedy configuration, send "temperature": 0 in generation requests. Speculative decoding preserves the target model's output distribution; the draft model changes performance, not target-model semantics.

Reproducibility

• Training speculators commit: ddf9d6012b169849ea1dc53faffa41968b4e8f85
• Evaluation speculators commit: c59c5fbdf0a50a778188be43dcda54b40fff7ca2
• Training seed: 42
• Training epochs: 5
• Learning rate: 3e-4
• Packed sequence length: 8,192