SbJAGSdLhY@OpenReview

Total: 1

#1 Geometry-Aware Collaborative Multi-Solutions Optimizer for Model Fine-Tuning with Parameter Efficiency [PDF] [Copy] [Kimi] [REL]

Authors: Van-Anh Nguyen, Trung Le, Mehrtash Harandi, Ehsan Abbasnejad, Thanh-Toan Do, Dinh Phung

We propose a framework grounded in gradient flow theory and informed by geometric structure that provides multiple diverse solutions for a given task, ensuring collaborative results that enhance performance and adaptability across different tasks. This framework enables flexibility, allowing for efficient task-specific fine-tuning while preserving the knowledge of the pre-trained foundation models. Extensive experiments across transfer learning, few-shot learning, and domain generalization show that our proposed approach consistently outperforms existing Bayesian methods, delivering strong performance with affordable computational overhead and offering a practical solution by updating only a small subset of parameters.

Subject: NeurIPS.2025 - Poster