2026-08-11 | | Total: 3
Large Language Models (LLMs) are being considered for research evaluation, raising concerns about the introduction of AI bias. This study investigates whether ChatGPT research quality scores differ by first-author gender using 89,744 journal articles from the UK Research Excellence Framework (REF) 2021. Author information was withheld from ChatGPT to avoid direct gender bias. Nevertheless, male first-authored papers had slightly higher ChatGPT scores in most Units of Assessment (UoAs), especially in health, science and engineering-related subjects, and this pattern was often stronger for ChatGPT than for REF scores, based on a departmental-level proxy. Rank-based ChatGPT gains relative to REF scores were also more favourable for male first-authored papers in most UoAs, although the differences were generally small. Gender differences were not evident for solo research in the social sciences, arts and humanities, however. The male-favouring pattern for first-authored research was not explained by gender differences in writing styles, at least as reflected in abstract complexity. Some ChatGPT-REF differences may also reflect the departmental averaging process used to generate the REF proxy scores. Average ChatGPT scores may differ by first-author gender indirectly through other factors, such as field, topic, method, journal context or authorship structure. Thus, this is an additional reason to be cautious with AI-based research evaluation.
In this paper, we introduce StatCite, a large-scale citation network dataset covering publications in statistics and data science from 1981 to 2025. The dataset contains 189,101 research articles collected from 62 representative journals and provides bibliographic metadata, including title, author list, publisher, published year, abstract, keywords, and reference list. Based on the collected publications, we construct four complementary citation-based networks, namely the paper citation network, the co-citation network, the bibliographic coupling network, and the journal citation network. To illustrate the utility of the dataset, we present descriptive analyses of the constructed networks and investigate the community structure of the paper citation network. The results show that StatCite preserves key structural characteristics commonly observed in large-scale citation networks and captures several major research areas in statistics and data science. By integrating multiple network representations with rich textual metadata, StatCite provides a valuable resource for statistical analysis, knowledge discovery, and data-driven studies of scientific literature.
Scientific named entity recognition (SciNER) plays a crucial role in information extraction and knowledge discovery from scientific texts. Recently, large language models (LLMs) have demonstrated the capacity to achieve competitive SciNER performance with minimal human effort. Existing research highlights the importance of incorporating candidate entity type information for accurate entity recognition and classification by LLMs. However, when too many candidate entity types are provided in the prompt, LLMs struggle to accurately recognize and label entities in scientific texts, where entity types are more complex than in general domains. To address this challenge, we propose TdSciNER, a type-driven approach that effectively leverages entity type information to enhance SciNER performance. In TdSciNER, we first design an entity type filter model to identify the most likely entity types present in a given sentence. Subsequently, we introduce an auxiliary multi-class entity typing task within a multi-task learning framework alongside SciNER to obtain richer contextual representations. Then, we develop a novel demonstration selection strategy based on sentence similarity and entity type diversity to activate the in-context learning capabilities of LLMs, thereby improving entity recognition accuracy across diverse scientific domains. Experiments on three datasets demonstrate that our method achieves performance comparable to fully supervised models. Further analysis validates that each entity type-driven component in TdSciNER contributes to the improvement of SciNER performance. This work provides valuable insights for future advancements in SciNER and broader information extraction tasks in scientific text mining.