After two days running the latest models on one DGX Spark, I am keeping the MiaAI-Lab DeepSeek V4 Flash recipe as my daily driver.
In my agent workload, a single request generates roughly 35–40 tokens/s for prose and code. The standardized server-side benchmark below lands at 24–27 tokens/s because it counts differently, but the result points in the same direction. I can also run four sequences at once without giving up useful speed.
Now I have the eval receipts to back that choice.
Sixcat: DeepSeek takes the lead
DeepSeek V4 Flash scored 87.5/100 in one sixcat run with 20 items per category:
- Math: 100
- Code: 95
- Tools: 90
- Truth: 85
- Instruction following: 80
- Knowledge: 75
The biggest trap was not the model. It was my LiteLLM configuration.
When reasoning_effort was missing, LiteLLM mapped it to OFF. Sixcat requested thinking, but the model returned no reasoning and scored 67.5 overall, including 40 on Code. Setting reasoning_effort=high raised the total by 20 points and Code by 55 points; reasoning appeared on 119 of 120 items. If you run this model behind a proxy, send the parameter explicitly.
Decode performance through 128k context
Next came the promised decode benchmark: DeepSeek on vLLM, 2k to 128k contexts, with 128 generated tokens per run.
- Generation: 23.9 tok/s at 2k, rising to 26.4 tok/s at 128k
- Prefill: roughly 1,000 tok/s across the full ladder
- TTFT: 2.1 seconds at 2k, rising to 132 seconds at 128k
- TPOT: roughly 38 ms throughout
- Concurrency: 27 tok/s with one stream, 57 aggregate at four streams (+2.1×), and 52 at eight
Decode never slows down. Long-context prefill is the cost.
I repeated the same ladder with the GLM EXL3 2-bit K2 pack and MTP k=2. This run used the refreshed vllm-exl3 0.2.1 runtime on 2026-09-01 and reached the full 131k window:
- Generation: 17.1 tok/s at 2k, 14.4 at 64k, and 14.9 at 128k
- Prefill: roughly 550 tok/s
- TTFT: 234 seconds at 128k
- Aggregate throughput: roughly 11 tok/s at every concurrency level because MTP limits
max-num-seqsto 1
GLM EXL3 ties on tool calling
I tested vcruz305's EXL3 K2 pack on vLLM. It uses 2-bit EXL3 routed experts, native MTP k=2, fused MoE, and KV fp8.
I ran both stacks through tool-eval-bench: 88 deterministic scenarios across eight complete trials, with hard mode enabled and seed 42. The battery covers tool selection, multi-step chains, safety boundaries, structured output, and adversarial statefulness.
DeepSeek V4 Flash scored 87.8 ± 1.9/100 (★★★★):
- Pass@8: 85.2% capability ceiling
- Pass^8: 69.3% reliability floor
- Outcomes: 71 passed, 13 partial, 4 failed
- Unstable scenarios: 18
- Median turn: 5.5 seconds
GLM-5.3 Flash EXL3 scored 88.8 ± 1.2/100 (★★★★):
- Pass@8: 88.6%
- Pass^8: 70.5%
- Unstable scenarios: 17
This is a statistical tie. The 1.9-point DeepSeek spread and 1.2-point GLM spread overlap, and both models rate ★★★★. The open question was whether a 2-bit EXL3 GLM build could compete as an agent model. It can.
Why DeepSeek stays loaded
Serving breaks the tie.
DeepSeek holds its decode speed from 2k through 128k context and batches well: aggregate throughput rises from 27 tok/s with one stream to 57 tok/s with four.
GLM EXL3 moves from 17.1 tok/s at 2k to 14.4 at 64k, then holds 14.9 at 128k. The refreshed runtime serves the full 131,072-token window with MTP k=2, and decode remains flat across it. Prefill runs at roughly 550 tok/s with a 234-second TTFT at 128k; DeepSeek reaches roughly 1,000 tok/s and 132 seconds.
Batching is the blocker. Native MTP k=2 pins the server to max-num-seqs=1, so aggregate throughput stays near 11 tok/s regardless of how many clients ask. Speculative decoding and concurrency still do not mix on this build.
DeepSeek therefore stays loaded: a tool-calling tie against the EXL3 K2 pack, flat decode through 128k on both stacks, and useful four-way concurrency on DeepSeek.
The EXL3 pack already matches DeepSeek on agentic quality and serves a verified 128k window. It still needs batching.
Next, I will test DeepSeek-V4-Flash-Vision-Exp on image understanding, OCR, chart analysis, and visual-agent tasks, then update this article with the results. More is coming.