Erfan Baghaei Potraghloo and colleagues submitted SafeStart to arXiv on 4 November 2026, reporting that reducing active experts in mixture-of-experts (MoE) language models can weaken refusal of harmful requests. Their proposed mitigation preserves computation at the start of each response while retaining most of the reported decoding speedup.[1]

MoE models route each token through a subset of expert subnetworks. According to the abstract, keeping about two thirds of the selected experts in vLLM on one RTX A6000 accelerated offline decoding by 15–28% at batch size 8. Under a jailbreak instructing models not to refuse, harmful compliance increased by 18.4 percentage points on Qwen1.5-MoE, 9.5 on OLMoE and 6.6 on DeepSeek-V2-Lite. These were the three of nine models with significant increases; MMLU accuracy fell by less than two points.[1]

The authors attribute most of the increase to prompt processing and the first few generated tokens. SafeStart runs all experts for the prompt and first eight generated tokens, then uses the reduced set through a fused vLLM kernel. They report retaining at least 85% of the offline decoding speedup at batch sizes 8 and 32. On 889 held-out requests, expert truncation increased harmful compliance by 5.8–21.4 points; SafeStart left increases of 0.6–2.6 points, meeting the authors’ non-inferiority criterion on two of three models.[1]

[1]: SafeStart: Mitigating the Overlooked Safety Cost of Active Expert Reduction in MoE LLM Serving. arXiv, 4 November 2026. https://arxiv.org/abs/2611.06147

Sources

  1. SafeStart: Mitigating the Overlooked Safety Cost of Active Expert Reduction in MoE LLM ServingarXiv · 4 Nov 2026

Check or correct this page

Verification record

The digest below shows whether the text you are reading matches the published source. It does not show that the reporting is correct. The build computes it from this page's Markdown source and lists it in the site manifest.

6337334f30fe2f0de25b024bf21154fcdbd0261306a590a69fc101771ef4bc01