2026.emnlp-main.24@ACL

Total: 1

#1 Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space [PDF] [Copy] [Kimi] [REL]

Authors: Yangyi Shen, Tianjian Feng, Jiaqi Han, Wen Wang, Tianlang Chen, Chunhua Shen, Jure Leskovec, Stefano Ermon

Diffusion Language Models (DLMs) offer order-agnostic generation that can explore many possible decoding trajectories. However, current decoding methods commit to a single trajectory, limiting exploration in trajectory space. We introduce Order-Token Search to explore this space through jointly searching over generation order and token values. Its core is a likelihood estimator that scores denoising actions, enabling stable pruning and efficient exploration of diverse trajectories. Across mathematical reasoning and coding benchmarks, Order-Token Search consistently outperforms baselines on GSM8K, MATH500, Countdown, and HumanEval (3.1%, 3.8%, 7.9%, and 6.8% absolute over backbone), reaching accuracy comparable to diffu-GRPO post-training. Our work establishes joint search as a key component for advancing decoding in DLMs.

Subject: EMNLP.2026 - Main