2504.07087

Total: 1

#1 KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs [PDF11] [Copy] [Kimi12] [REL]

Authors: Elan Markowitz, Krupa Galiya, Greg Ver Steeg, Aram Galstyan

Knowledge graphs have emerged as a popular method for injecting up-to-date, factual knowledge into large language models (LLMs). This is typically achieved by converting the knowledge graph into text that the LLM can process in context. While multiple methods of encoding knowledge graphs have been proposed, the impact of this textualization process on LLM performance remains under-explored. We introduce KG-LLM-Bench, a comprehensive and extensible benchmark spanning five knowledge graph understanding tasks, and evaluate how different encoding strategies affect performance across various base models. Our extensive experiments with seven language models and five textualization strategies provide insights for optimizing LLM performance on KG reasoning tasks.

Subjects: Computation and Language , Artificial Intelligence , Information Retrieval

Publish: 2025-04-09 17:58:47 UTC