2025.findings-naacl.94@ACL

Total: 1

#1 PairScale: Analyzing Attitude Change with Pairwise Comparisons [PDF] [Copy] [Kimi] [REL]

Authors: Rupak Sarkar, Patrick Y. Wu, Kristina Miler, Alexander Miserlis Hoyle, Philip Resnik

We introduce a text-based framework for measuring attitudes in communities toward issues of interest, going beyond the pro/con/neutral of conventional stance detection to characterize attitudes on a continuous scale using both implicit and explicit evidence in language. The framework exploits LLMs both to extract attitude-related evidence and to perform pairwise comparisons that yield unidimensional attitude scores via the classic Bradley-Terry model. We validate the LLM-based steps using human judgments, and illustrate the utility of the approach for social science by examining the evolution of attitudes on two high-profile issues in U.S. politics in two political communities on Reddit over the period spanning from the 2016 presidential campaign to the 2022 mid-term elections. WARNING: Potentially sensitive political content.

Subject: NAACL.2025 - Findings