2602.04294

Total: 1

#1 How Few-shot Demonstrations Affect Prompt-based Defenses Against LLM Jailbreak Attacks [PDF2] [Copy] [Kimi1] [REL]

Authors: Yanshu Wang, Shuaishuai Yang, Jingjing He, Tong Yang

Large Language Models (LLMs) face increasing threats from jailbreak attacks that bypass safety alignment. While prompt-based defenses such as Role-Oriented Prompts (RoP) and Task-Oriented Prompts (ToP) have shown effectiveness, the role of few-shot demonstrations in these defense strategies remains unclear. Prior work suggests that few-shot examples may compromise safety, but lacks investigation into how few-shot interacts with different system prompt strategies. In this paper, we conduct a comprehensive evaluation on multiple mainstream LLMs across four safety benchmarks (AdvBench, HarmBench, SG-Bench, XSTest) using six jailbreak attack methods. Our key finding reveals that few-shot demonstrations produce opposite effects on RoP and ToP: few-shot enhances RoP's safety rate by up to 4.5% through reinforcing role identity, while it degrades ToP's effectiveness by up to 21.2% through distracting attention from task instructions. Based on these findings, we provide practical recommendations for deploying prompt-based defenses in real-world LLM applications.

Subjects: Computation and Language , Artificial Intelligence , Cryptography and Security

Publish: 2026-02-04 07:54:51 UTC