2512.10365

#1 GPG: Generalized Policy Gradient Theorem for Transformer-based Policies [PDF²] [Copy] [Kimi¹] [REL]

Authors: Hangyu Mao, Guangting Dong, Zhicheng Dou

We present the Generalized Policy Gradient (GPG) Theorem, specifically designed for Transformer-based policies. Notably, we demonstrate that both standard Policy Gradient Theorem and GRPO emerge as special cases within our GPG framework. Furthermore, we explore its practical applications in training Large Language Models (LLMs), offering new insights into efficient policy optimization.

Subjects: Machine Learning , Artificial Intelligence , Computation and Language

Publish: 2025-12-11 07:30:33 UTC

---