| Total: 21
Federated learning (FL) has emerged as a distributed machine learning paradigm due to its privacy-preserving advantages. Most FL studies assume offline labeled datasets are available at clients. In practice, however, client data often arrive without labels in a streaming manner, making label acquisition a crucial problem. Existing solutions typically rely on human experts or artificial intelligence (AI) models for annotation. Nevertheless, due to the subjectivity among human experts and the inherent limitations of AI models, the labels they provide are often unreliable. In this work, we propose CoFL, a novel consensus driven human-AI collaborative FL method, which utilizes the complementarity between humans and AI models to produce more reliable labels. Moreover, CoFL leverages cross-client consensus among human experts to further enhance the collaboration process. Experiments on three datasets demonstrate that CoFL consistently outperforms baseline methods under various settings. The code is available at https://github.com/huaiguang233/CoFL.
Large EEG Models (LEMs) are drawing increasing attention in EEG, as large-scale pretraining yields transferable representations that improve generalization. As EEG research moves to real-world deployment, objectives and paradigms diversify, yielding increasingly heterogeneous and unevenly scaled datasets across institutions. This evolution induces distribution drift and scale imbalance, making static pretrained LEMs brittle and requiring adaptation to newly collected data. Under the constraints of the high cost of retraining and the privacy sensitivity of newly collected data, federated learning offers a practical route to update LEMs via decentralized parameter updates. Yet standard federated fine-tuning is unstable for pretrained LEMs, suffering backbone drift, misaligned updates under non-IID heterogeneity, and scale imbalance. We propose FedGLoRA: we freeze the backbone and communicate only LoRA adapters via a dual-branch design, with a shared global branch and a private on-device branch. FedGLoRA aggregates global updates on a Grassmann subspace with a Grassmann-proximal constraint, and uses bidirectional distillation to mitigate imbalance. Experiments on motor imagery and emotion recognition show consistent gains and improved stability over strong baselines.
Traditional multi-view learning models are primarily designed for static datasets with fixed views. However, in dynamic incremental view environments, this approach inevitably leads to view forgetting, where the introduction of new views weakens previously acquired knowledge. In contrast, the human brain exhibits remarkable memory retention and knowledge transfer capabilities when receiving objects described from different perspectives, with past experience further supplementing new insights. Inspired by underlying neural processing mechanisms, we propose a novel view incremental learning framework named Hebbian View Orthogonal Projection (HVOP). HVOP constructs a knowledge transfer space, where gradient updates are projected onto the orthogonal complement of historical representations, thereby mitigating interference between old and new views. By further incorporating recursive lateral connections and Hebbian learning rules, the proposed model imparts brain-like dynamic adaptability to the learning process, enhancing knowledge transfer and integration, thereby enabling stable knowledge transfer under evolving views. We validate HVOP on node classification tasks, demonstrating its superior performance in both knowledge retention and transfer compared to traditional methods. The results highlight the efficacy of biologically inspired mechanisms in mitigating the view forgetting phenomenon.
Recent writing assistants are increasingly shifting from passive, prompt-driven interaction to proactive, suggestion-based completion, which integrates localized continuations into the writing flow and reduces coordination burden. However, existing evaluations simply focus on output quality, failing to capture how users accept, edit, or repair suggestions in real-time interaction, and thus obscuring the true usability of proactive co-writing systems. To address this gap, we adopt a sequential, behavior-centered view of interactive writing and formalize co-writing as a Human-in-the-Loop Markov Decision Process, modeling writing as an interaction shaped by user acceptance and editing decisions. Based on this formulation, we introduce the Co-Writing Fidelity Suite, an interaction-aware metric suite that captures both user–assistant alignment and cognitive editing effort, including Hierarchical Acceptance Rate and Knowledge-aware Editing Distance. We conduct a large-scale simulation study across 16 writing domains, using 1,688 controlled continuation queries sampled from different writing stages. Our analysis reveals systematic effects of interaction structure on acceptance behavior and editing cost. A follow-up user study with 30 participants confirms that these behavioral patterns align with real user experience. Together, our findings demonstrate that interaction-aware evaluation provides insights beyond output-only metrics and informs the design of more effective proactive writing assistants.
Cross-modal brain networks characterize the complex connections between different brain regions from both functional and structural perspectives, which is of significant importance for brain network analysis and the diagnosis of brain diseases. However, existing methods have failed to fully exploit the complementary information between functional and structural brain networks, neglecting the guiding role of topology in the transmission of functional modal information, as well as the potential associations of cross-modal high-order information. To address these challenges, this paper proposes a cross-modal dynamic hypergraph computing (CDHGC) framework for brain disease diagnosis and an in-depth analysis of coupled functional-structural brain networks. The CDHGC comprises two key modules: the topology-guided dynamic hypergraph generation module, which combines the topology-constrained KNN and topology-guided centroid sampling methods to uncover high-order correlations among brain regions across modalities based on topological structure. This module dynamically optimizes the hypergraph structure during training to explore potential associations of cross-modal information. The cross-modal hypergraph convolution module efficiently aggregates information from functional-structural brain networks through a cross-modal attention mechanism during message-passing, resulting in high-order joint representations. Experiments conducted on the ADNI and ABIDE datasets demonstrate that the proposed method outperforms current state-of-the-art approaches. Interpretability analysis reveals that the proposed method can discover multi-modal biomarkers associated with brain diseases.
High-fidelity reconstruction of mel-spectrograms from EEG signals remains a formidable challenge, primarily due to the inherent inter-subject variability of neural patterns and the semantic gap between heterogeneous feature representations of these two modalities. To alleviate both issues, this paper proposes a Hierarchical Feature Fusion Network with bi-phase subject IDentifier modulation (HFFN-ID) for reconstructing mel-spectrograms from EEG signals. The primary improvements of the HFFN-ID framework are from two aspects, a Binary-Phase Subject-ID Modulation (BiPSM) mechanism for explicit subject conditioning and a Condition-guided Hierarchical Fusion (CHF) component for dynamic multi-layer feature synthesis, which respectively aim to address the problems of inter-subject variability in neural patterns and heterogeneous cross-modality feature fusion. Moreover, a speech envelope feature-based pre-training strategy is incorporated to initialize the parameter space, inspired by the shared low-level representations across different speech features. On the SparrKULee dataset, HFFN-ID establishes a new benchmark with a pearson correlation coefficient of 0.0723, representing a substantial 44% relative improvement over existing baseline. Importantly, HFFN-ID is highly efficient, achieving a 38.7% parameter reduction and a 2.5× training speedup. These results highlight the effectiveness of subject-conditioned and hierarchically fused architectures for advancing high-fidelity neural speech decoding.
Human experts often struggle to select the best option from a large set of items with multiple competing objectives, a process bottlenecked by the difficulty of formalizing complex, implicit preferences. To address this, we introduce LISTEN (LLM-based Iterative Selection with Trade-off Evaluation from Natural-language), an agentic LLM-based framework that treats the LLM as a decision-making agent capable of iteratively refining its internal preference model and taking actions (e.g., proposing utilities or selecting candidates) to maximize alignment with a user's implicit goals. To operate within LLM constraints like context windows and inference costs, we propose two iterative algorithms: LISTEN-U, which uses the LLM to refine a parametric utility function, and LISTEN-T, a non-parametric method that performs tournament-style selections over small batches of solutions. Evaluated on diverse tasks including flight booking, shopping, and exam scheduling, our results show LISTEN-U excels when preferences are parametrically aligned (a property we measure with a novel concordance metric), while LISTEN-T offers more robust performance overall. This work explores a promising direction for steering complex multi-objective decisions directly with natural language, reducing the cognitive burden of traditional preference elicitation. Code is available at https://github.com/AdamJovine/LISTEN.
While deep learning has significantly enhanced robotic perception in agriculture, autonomous decision-making in dense and occluded environments remains a persistent challenge. This paper proposes a VR-based expert motion capture framework to bridge this gap by integrating high-fidelity virtual environments with human expertise. By reconstructing a realistic tomato greenhouse in Unity and utilizing 27-point motion capture suits, we captured multi-dimensional expert demonstration data encompassing trajectories, postures, and velocities. In the VR environment, ``hard cases" featuring frequent collisions can be selectively generated at a higher frequency, allowing experts to provide concentrated demonstrations of specialized postures that markedly accelerate algorithmic learning efficiency. Crucially, the virtual framework enables multiple experts to repetitively harvest identical targets to match optimal trajectories—a process precluded in reality by the destructive nature of harvesting. Moreover, it bypasses biological growth and seasonal constraints, facilitating continuous, high-frequency experimentation independent of crop maturation. These expert data were subsequently integrated into a multi-task reinforcement learning model as experience replay and reward templates. Experimental results demonstrate that our expert-driven algorithm significantly outperforms baseline methods in harvesting efficiency, collision avoidance, and stability, establishing VR-guided motion capture as a robust and scalable paradigm for advancing agricultural robotics.
Object detection in high-resolution wide (HRW) shots presents unique challenges due to the extreme sparsity of objects and the variability in sparsity ratios across images. Conventional detectors, designed for close-up settings like MS COCO, struggle to generalize to these scenarios, leading to inefficient computation and degraded performance. While prior work has focused on spatial sparsity, the potential of sparsity in the channel dimension remains largely underexplored. In this paper, we propose ThinFormer, a dynamic and efficient framework tailored for object detection in HRW shots. ThinFormer leverages sparsity in both the spatial domain and the channel domain, the latter enabled by a novel local feed-forward network (local FFN) that exploits the duality between spatial and channel representations. To adapt to varying levels of scene complexity, we introduce a dynamic sparsity ratio estimator, trained in a self-supervised manner, which guides selective activation of channels based on semantic richness. Extensive experiments on the PANDA benchmark demonstrate that ThinFormer significantly reduces computational cost while maintaining competitive detection performance compared to state-of-the-art methods. Our results highlight the effectiveness of embracing multi-domain sparsity for scalable and robust object detection in complex, real-world HRW imagery. Code and supplementary material are available at https://github.com/LiuKunpeng03/Thinformer.
Combining predictions from heterogeneous classifiers—such as in-house deep learning models, human experts, and large language models (LLMs)—is a key challenge, especially in data-scarce environments such as humanitarian operations. We propose a flexible Bayesian framework to effectively fuse these diverse inputs. By integrating classifier logits with auxiliary human feedback (e.g., confidence, image clarity) using an ordered probit process, our model generalizes prior work to accommodate the real-world properties of these classifiers. We validate our framework on a challenging product recognition task in food bank operations, an environment defined by data scarcity and an inexperienced volunteer workforce. Our combined model significantly outperforms standalone ResNet, human, and LLM-based approaches, demonstrating the practical benefits of fusing these heterogeneous signal sources.
Artificial intelligence models typically perform well on large-scale datasets, yet their effectiveness tends to degrade in real-world scenarios with scarce data, such as medical diagnostics. In contrast, humans can learn and reason effectively from few examples. Even when novel objects differ significantly from prior observations, humans can quickly infer category distributions accurately. Inspired by this, we explore how to leverage human-inspired cognitive mechanisms to improve the distribution calibration in few-shot learning models. Based on the ``fast-slow thinking'' dual-process theory, we propose a novel cognition-inspired few-shot learning framework. It mimics the human cognition when handling novel information: it first rapidly screens relevant knowledge through ``fast thinking'' and then infers inter-class relationships and calibrates distributions via ``slow thinking''. Specifically, the fast-thinking stage employs a gating mechanism to quickly match input samples with known base-class prototypes, activating relevant candidate knowledge. In the slow-thinking stage, the model aggregates inter-class edge embeddings into a summary relation graph and then applies divergent Gaussian sampling to generate multiple relation graphs representing different association strengths, thereby achieving distribution calibration for few-shot classes. Experiments on public few-shot benchmarks and medical image datasets show competitive performance. Visualization analyses further reveal that our framework exhibits meaningful interpretability.
Motor imagery (MI) electroencephalography (EEG) decoding has benefited from deep learning, yet methods operate in Euclidean space and behave as opaque black boxes, neglecting the intrinsic geometry of functional brain connectivity. EEG connectivity descriptors, such as phase synchrony and covariance matrices, naturally reside on the manifold of symmetric positive definite (SPD) matrices, where Euclidean operations are geometrically inconsistent and hinder interpretability. This work proposes the Multi-frequency and Multi-scale Riemannian Network (MFMSRNet), an interpretable end-to-end geometry-aware framework for MI EEG decoding on the SPD manifold. The method constructs kernelized phase-locking value (KPLV) functional connectivity (FC) matrices to capture nonlinear phase synchrony while ensuring positive definiteness. An attention-based Riemannian fusion mechanism adaptively integrates information across multiple frequency bands in the tangent space. Furthermore, a multi-scale Riemannian network extracts global, hemispheric, and local connectivity patterns via manifold-preserving bilinear mappings and smooth eigenvalue rectification. Extensive experiments indicate that MFMSRNet yields more expressive and interpretable representations for robust MI decoding, offering a promising solution for reliable brain–computer interface applications. The code is available at https://github.com/Raeno-Rao/MFMSRNet.
Safety-aware shared autonomy aims to enable an autonomous agent to collaborate with a human operator, completing tasks under safety constraints while maximally preserving human intent. However, existing methods often underperform in long-horizon tasks that require balancing task performance under safety constraints with the degree of intervention—particularly when accurately predicting future unsafe events is critical. This limitation largely stems from their reliance on the current state alone to decide when to intervene and how to modify actions. We propose World Model Assisted Safety Planning (WASP), a model-predictive shared autonomy framework for explicit safety constraint satisfaction. We first formulate a safety-aware world model, and leverage its online predictive reasoning to decide when to intervene by jointly assessing prospective safety violations and degradation in task performance, thereby intervening only when necessary while enforcing safety constraints at decision time. Once intervention is triggered, WASP plans a short-horizon residual correction using world model rollouts, filters out unsafe candidates, and executes the least-deviating correction among the remaining high-return options in a receding-horizon loop. Experiments across diverse vision-based safety-critical domains show that WASP substantially reduces safety violations while preserving task performance and reducing unnecessary interventions over prior shared autonomy baselines.
Spiking neural networks (SNNs), which are brain-inspired and spike-driven, achieve high energy efficiency. However, a performance gap between SNNs and artificial neural networks (ANNs) still remains. Knowledge distillation (KD) is commonly adopted to improve SNN performance, but existing methods typically enforce uniform alignment across all timesteps, either from a teacher network or through inter-temporal self-distillation, implicitly assuming that per-timestep predictions should be treated equally. In practice, SNN predictions vary and evolve over time, and intermediate timesteps need not all be individually correct even when the final aggregated output is correct. Under such conditions, effective distillation should not force every timestep toward the same supervision target, but instead provide corrective guidance to erroneous timesteps while preserving useful temporal dynamics. To address this issue, we propose Selective Alignment Knowledge Distillation (SeAl-KD), which selectively aligns class-level and temporal knowledge by equalizing competing logits at erroneous timesteps and reweighting temporal alignment based on confidence and inter-timestep similarity. Extensive experiments on static image and neuromorphic event-based datasets demonstrate consistent improvements over existing distillation methods. The code is available at https://github.com/KaiSUN1/SeAl.
Modeling emotional dynamics in multi-turn dialogues remains challenging due to emotion shifts, where emotional states evolve gradually rather than change abruptly. Existing methods often rely on one-hot supervision, which fails to reflect the progressive nature of human emotions, particularly for neutral-valence emotions with subtle trajectories. To address this limitation, we propose a pseudo soft-label guided Progressive Hybrid-Distributional (PHD) learning framework. PHD reconstructs discrete labels into hybrid distributional representations that encode inter-emotion relations and mixed emotional tendencies. Based on these representations, a progressive training strategy is introduced to guide the model from learning blended emotional states toward the standard one-to-one prediction objective. Furthermore, we design a Graded Contrastive Learning mechanism that replaces rigid binary allocation with graded supervision to alleviate label conflicts. Experiments on benchmark datasets demonstrate PHD consistently improves diverse baseline models, yielding average w-F1 gains of 1.29%. Our framework highlights the critical importance of modeling emotional dynamics as a gradual, distributional process.
Despite extensive research, computational creativity still lacks a conceptually grounded, practically relevant, and empirically applicable formal definition across contexts such as art, scientific research, and games. Existing approaches to creativity often focus on proposing informal conceptual definitions without providing formal computational concepts. Alternatively, they propose formal definitions without carefully examining their underlying assumptions or providing validation of these definitions by testing their performance in specific application settings. To address these problems, this paper proposes several formal postulates of creativity grounded in prior psychological work on creativity, especially as related to Boden’s three core criteria of creativity: newness, value, and surprise. Furthermore, we propose a conceptual mathematical definition of creativity that aligns with these postulates. We also validate our proposed creativity formula in two applications: determining the creativity of particular chess moves, and assessing the creativity of paintings by some artists, such as Picasso, Renoir, and other painters. Experimental results demonstrate that our formula of creativity successfully captures creative patterns across both the chess and the art domains. This suggests that creativity can be effectively characterized through a unified mathematical structure based on three fundamental criteria, namely newness, value, and surprise, whose specific measurements are adapted to the particular domains.
Domain-generalizable speech emotion recognition (DG-SER) aims to ensure the robustness of SER models across unknown domains, which is essential for real-world human-machine interaction systems. Most DG-SER approaches employ alignment or adversarial strategies with domain labels to promote generalization. However, these strategies often confine generalization to predefined domains, limiting robustness under diverse real-world speech variations. To address these challenges, this paper proposes an emotion-preserving conditional information bottleneck framework (EP-CIB) for domain-free DG-SER. Specifically, EP-CIB introduces a nuisance proxy representation learning module to learn a nuisance proxy as the broad non-emotional variability without requiring any domain annotations, covering both defined and previously unseen domains. It then extracts coarse-grained emotion features via the consistency-aware emotion representation learning module. EP-CIB further introduces an emotion-preserving conditional information bottleneck that, conditioned on the emotion label, disentangles the nuisance proxy from the coarse-grained emotion representation, improving domain-free generalization under open-ended domain shifts. EP-CIB departs from implicit domain surrogates in prior domain-free methods by explicitly learning a proxy for nuisance domains and disentangling it from emotion features, enabling domain-free emotion representation learning. The state-of-the-art performance in both speaker-independent and cross-corpus settings, including an 18% improvement on EmoDB-to-CASIA transfer, demonstrates the effectiveness of EP-CIB for DG-SER.
Knowledge Tracing (KT) is fundamental to intelligent education systems, yet relies on educational logs that are selectively observed. The non-random nature of exercise recommendations and student choices inevitably induces severe selection bias. Most existing KT methods neglect this issue, training on observed logs using standard empirical risk, which yields biased mastery estimates and accumulates errors in subsequent recommendations. To address this, we introduce a doubly robust (DR) formulation for KT that integrates a propensity model with an error imputation model, theoretically guaranteeing unbiasedness if either model is accurate. Beyond unbiasedness, in the sequential setting of KT, we identify that the estimator's performance is compromised by variance-dependent stochastic deviations that accumulate over time, thereby causing training instability and limiting performance. To mitigate this, we derive a generalization bound that explicitly characterizes the impact of estimator variance and identifies temporal smoothness as a key factor in controlling it. Building on these theoretical insights, we propose the Temporal Smoothness Doubly Robust (TSDR) framework. TSDR jointly optimizes the KT predictor and the imputation model with a smoothness regularizer, effectively reducing variance while preserving the unbiasedness guarantee of DR. Experiments on multiple real-world benchmarks demonstrate that TSDR consistently enhances various state-of-the-art KT backbones, underscoring the vital role of principled bias correction in KT.
Driver cognitive distraction is a major cause of road collisions and remains difficult to detect. Unlike manual or visual distraction, cognitive distraction is diverted by thoughts unrelated to driving, even when the driver appears visually attentive and exhibits no explicit physical movements. In this work, we propose EyeCue, a gaze-empowered egocentric video understanding framework, to detect driver cognitive distraction. A key insight is that cognitive distraction manifests in the interaction between eye gaze and visual context. To capture this interaction, EyeCue integrates eye gaze with egocentric video to enable context-aware modeling of the driver's attention over time. Furthermore, to tackle the limited scale and diversity of existing datasets, we introduce CogDrive, a comprehensive multi-scenario dataset that augments four existing driving datasets with cognitive distraction annotations. Through extensive evaluations on CogDrive, we show that EyeCue achieves the highest accuracy of 74.38%, outperforming 11 baselines from 6 model families by over 7%. Notably, EyeCue can achieve an accuracy of over 70% across various driving scenarios (different road types, times of day, and weather conditions) with strong generalizability. These results highlight the importance of modeling gaze-context interactions and the effectiveness of cross-modal interaction modeling for multimodal cognitive distraction detection. Our codes and CogDrive dataset resources are available at https://github.com/langzhang2000/EyeCue.
Learning to Defer operates by deferring AI-uncertain samples to humans, enabling the system to outperform either alone. Some works extend it by introducing human-AI combination to deferral. However, they remain limited to binary choices. Given that AI, humans, and human-AI combination are each indispensable, we extend the binary deferral paradigm to a ternary one. There are two challenges: i) how to effectively select among the three modes, especially distinguishing humans from combination? ii) how to design an enhanced combination without being degraded by unreliable model predictions? To address the challenges, we propose TriHAI, a tri-mode deferral method that routes samples to suitable modes based on normalized confidence scores obtained through a discriminative gating network, and enhances combination mode by fusing model predictions refined by human-guided Conformal Prediction and human predictions via Bayesian theory. Experiments on three datasets show that TriHAI surpasses other human-AI baselines by up to 9.57% in accuracy.
Online EEG decoding is pivotal for real-world Brain-Computer Interfaces (BCIs) but confronts significant challenges arising from continuous distribution shifts, including both inter- and intra-subject variations. Existing Unsupervised Continual Domain Adaptation (UCDA) methods typically rely on rigid hard boundaries such as subject switch labels or fixed batch sizes to trigger adaptation. Meanwhile, some online EEG decoding methods that utilize detected shifts as soft boundaries are ill-suited for unsupervised scenarios and lack effective distribution alignment strategies. To address these issues, we propose a novel Prototype-driven online EEG Decoding framework (PRED). PRED incorporates a Subject Shift Detector (SSD) based on policy stability to reliably identify latent domain shifts unsupervisedly, thereby constructing adaptive soft boundaries. Furthermore, we design a Prototype-guided Shift Correction (PSC) mechanism that leverages a multi-granular memory structure to guide distribution alignment while preserving semantic stability. Experiments on three public datasets confirm that PRED achieves superior plasticity and stability, and demonstrates significant futurity, i.e., the capability of forward transfer to a subject's future samples.