2503.20807

Total: 1

#1 Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models [PDF4] [Copy] [Kimi4] [REL]

Authors: Pin-Yu Chen, Han Shen, Payel Das, Tianyi Chen

Fine-tuning Large Language Models (LLMs) on some task-specific datasets has been a primary use of LLMs. However, it has been empirically observed that this approach to enhancing capability inevitably compromises safety, a phenomenon also known as the safety-capability trade-off in LLM fine-tuning. This paper presents a theoretical framework for understanding the interplay between safety and capability in two primary safety-aware LLM fine-tuning strategies, providing new insights into the effects of data similarity, context overlap, and alignment loss landscape. Our theoretical results characterize the fundamental limits of the safety-capability trade-off in LLM fine-tuning, which are also validated by numerical experiments.

Subjects: Machine Learning , Artificial Intelligence , Computation and Language , Machine Learning

Publish: 2025-03-24 20:41:57 UTC