2506.12683

Total: 1

#1 Evaluating Cell Type Inference in Vision Language Models Under Varying Visual Context [PDF] [Copy] [Kimi1] [REL]

Authors: Samarth Singhal, Sandeep Singhal

Vision-Language Models (VLMs) have rapidly advanced alongside Large Language Models (LLMs). This study evaluates the capabilities of prominent generative VLMs, such as GPT-4.1 and Gemini 2.5 Pro, accessed via APIs, for histopathology image classification tasks, including cell typing. Using diverse datasets from public and private sources, we apply zero-shot and one-shot prompting methods to assess VLM performance, comparing them against custom-trained Convolutional Neural Networks (CNNs). Our findings demonstrate that while one-shot prompting significantly improves VLM performance over zero-shot (p1.005×105 based on Kappa scores), these general-purpose VLMs currently underperform supervised CNNs on most tasks. This work underscores both the promise and limitations of applying current VLMs to specialized domains like pathology via in-context learning. All code and instructions for reproducing the study can be accessed from the repository https://www.github.com/a12dongithub/VLMCCE.

Subjects: Computer Vision and Pattern Recognition , Quantitative Methods

Publish: 2025-06-15 01:50:16 UTC