G4JyhFXeu1@OpenReview

Total: 1

#1 Reach Into The Choir: Free-List Elicitation Uncovers Distinct Model Voices in LLM Ensembles [PDF1] [Copy] [Kimi] [REL]

Authors: Ben Wigler, Maria Tsfasman

Open-ended LLM homogeneity can create false plurality: several systems may appear to offer independent perspectives while returning the same familiar default. Single-pass answers obscure the distinction among genuinely narrow consensus, prompt-vocabulary echo, and broader answer spaces with stable alternatives beneath the surface. We introduce CHOIR (Collective Hierarchically-Ordered Inquiry Responses), a framework that adapts free-list elicitation from cognitive anthropology to LLM ensembles. CHOIR repeatedly elicits ranked lists, clusters items into prompt-level concepts, and measures concept salience across models, prompt variants, and persona conditions. We evaluate CHOIR on Infinity-Chat 100, an external prompt bank from recent work on open-ended model homogeneity, and on a 27-question targeted diagnostic bank designed to isolate mechanism-level contrasts. On Infinity-Chat 100, CHOIR reproduces high surface agreement (93/100 prompts above chance) while separating narrow prompts from broad prompts with recoverable depth. Across targeted probes and the external prompt bank, base-model identity remains the strongest recoverable signature, and persona prompts shift surfaced concepts within base-model signatures. A source-blind ranking module prioritises rare-but-stable candidates for later inspection. CHOIR reframes open-ended homogeneity as a diagnostic measurement problem: where models converge, why they converge, and what remains reachable under structured depth probing.

Subject: COLM.2026