2025-01-13 | | Total: 20
Effective error detection is crucial to prevent task disruption and maintain user trust. Traditional methods often rely on task-specific models or user reporting, which can be inflexible or slow. Recent research suggests social signals, naturally exhibited by users in response to robot errors, can enable more flexible, timely error detection. However, most studies rely on post hoc analysis, leaving their real-time effectiveness uncertain and lacking user-centric evaluation. In this work, we developed a proactive error detection system that combines user behavioral signals (facial action units and speech), user feedback, and error context for automatic error detection. In a study (N = 28), we compared our proactive system to a status quo reactive approach. Results show our system 1) reliably and flexibly detects error, 2) detects errors faster than the reactive approach, and 3) is perceived more favorably by users than the reactive one. We discuss recommendations for enabling robot error awareness in future HRI systems.
Social Anxiety Disorder (SAD) significantly impacts individuals' daily lives and relationships. The conventional methods for SAD detection involve physical consultations and self-reported questionnaires, but they have limitations such as time consumption and bias. This paper introduces video analysis as a promising method for early SAD detection. Specifically, we present a new approach for detecting SAD in individuals from various bodily features extracted from the video data. We conducted a study to collect video data of 92 participants performing impromptu speech in a controlled environment. Using the video data, we studied the behavioral change in participants' head, body, eye gaze, and action units. By applying a range of machine learning and deep learning algorithms, we achieved an accuracy rate of up to 74\% in classifying participants as SAD or non-SAD. Video-based SAD detection offers a non-intrusive and scalable approach that can be deployed in real-time, potentially enhancing early detection and intervention capabilities.
The increasing complexity of machine learning models in computer vision, particularly in face verification, requires the development of explainable artificial intelligence (XAI) to enhance interpretability and transparency. This study extends previous work by integrating semantic concepts derived from human cognitive processes into XAI frameworks to bridge the comprehension gap between model outputs and human understanding. We propose a novel approach combining global and local explanations, using semantic features defined by user-selected facial landmarks to generate similarity maps and textual explanations via large language models (LLMs). The methodology was validated through quantitative experiments and user feedback, demonstrating improved interpretability. Results indicate that our semantic-based approach, particularly the most detailed set, offers a more nuanced understanding of model decisions than traditional methods. User studies highlight a preference for our semantic explanations over traditional pixelbased heatmaps, emphasizing the benefits of human-centric interpretability in AI. This work contributes to the ongoing efforts to create XAI frameworks that align AI models behaviour with human cognitive processes, fostering trust and acceptance in critical applications.
This entry provides an overview of Human-centered Geospatial Data Science, highlighting the gaps it aims to bridge, its significance, and its key topics and research. Geospatial Data Science, which derives geographic knowledge and insights from large volumes of geospatial big data using advanced Geospatial Artificial Intelligence (GeoAI), has been widely used to tackle a wide range of geographic problems. However, it often overlooks the subjective human experiences that fundamentally influence human-environment interactions, and few strategies have been developed to ensure that these technologies follow ethical guidelines and prioritize human values. Human-centered Geospatial Data Science advocates for two primary focuses. First, it advances our understanding of human-environment interactions by leveraging Geospatial Data Science to measure and analyze human subjective experiences at place including emotion, perception, cognition, and creativity. Second, it advocates for the development of responsible and ethical Geospatial Data Science methods that protect geoprivacy, enhance fairness and reduce bias, and improve the explainability and transparency of geospatial technologies. With these two missions, Human-centered Geospatial Data Sciences brings a fresh perspective to develop and utilize geospatial technologies that positively impact society and benefit human well-being and the humanities.
Assistive mobile robots are a transformative technology that helps persons with disabilities regain the ability to move freely. Although autonomous wheelchairs significantly reduce user effort, they still require human input to allow users to maintain control and adapt to changing environments. Brain Computer Interface (BCI) stands out as a highly user-friendly option that does not require physical movement. Current BCI systems can understand whether users want to accelerate or decelerate, but they implement these changes in discrete speed steps rather than allowing for smooth, continuous velocity adjustments. This limitation prevents the systems from mimicking the natural, fluid speed changes seen in human self-paced motion. The authors aim to address this limitation by redesigning the perception-action cycle in a BCI controlled robotic system: improving how the robotic agent interprets the user's motion intentions (world state) and implementing these actions in a way that better reflects natural physical properties of motion, such as inertia and damping. The scope of this paper focuses on the perception aspect. We asked and answered a normative question "what computation should the robotic agent carry out to optimally perceive incomplete or noisy sensory observations?" Empirical EEG data were collected, and probabilistic representation that served as world state distributions were learned and evaluated in a Generative Adversarial Network framework. The ROS framework was established that connected with a Gazebo environment containing a digital twin of an indoor space and a virtual model of a robotic wheelchair. Signal processing and statistical analyses were implemented to identity the most discriminative features in the spatial-spectral-temporal dimensions, which are then used to construct the world model for the robotic agent to interpret user motion intentions as a Bayesian observer.
As social media platforms are increasingly adopted, the data the data people leave behind is shining new light into our understanding of phenomena, ranging from socio-economic-political events to the spread of infectious diseases. This chapter presents research conducted in the past decade that has harnessed social media data in the service of mental health and well-being. The discussion is organized along three thrusts: a first that highlights how social media data has been utilized to detect and predict risk to varied mental health concerns; a second thrust that focuses on translation paradigms that can enable to use of such social media based algorithms in the real-world; and the final thrust that brings to the fore the ethical considerations and challenges that engender the conduct of this research as well as its translation. The chapter concludes by noting open questions and problems in this emergent area, emphasizing the need for deeper interdisciplinary collaborations and participatory research design, incorporating and centering on human agency, and attention to societal inequities and harms that may result from or be exacerbated in this line of computational social science research.
Robots, as AI with physical instantiation, inhabit our social and physical world, where their actions have both social and physical consequences, posing challenges for researchers when designing social robots. This study starts with a scoping review to identify discussions and potential concerns arising from interactions with robotic systems. Two focus groups of technology ethics experts then validated a comprehensive list of key topics and values in human-robot interaction (HRI) literature. These insights were integrated into the HRI Value Compass web tool, to help HRI researchers identify ethical values in robot design. The tool was evaluated in a pilot study. This work benefits the HRI community by highlighting key concerns in human-robot interactions and providing an instrument to help researchers design robots that align with human values, ensuring future robotic systems adhere to these values in social applications.
Fabric has been a fundamental part of human life for thousands of years, providing comfort, protection, and aesthetic expression. While modern advancements have enhanced fabric's functionality, it remains static and unchangeable, failing to adapt to our evolving body shapes and preferences. This lack of adaptability can lead to unsustainable practices, as consumers often buy more items to meet their changing needs. In this paper, we propose ExoFabric, a re-moldable fabric system for customized soft goods applications. We created ExoFabric by embedding thermoplastic threads into fabric through computerized embroidery to allow for tunability between rigid plastic and conformable fabric. We defined a library of design primitives to enable geometric formability, stiffness, and stretchability by identifying suitable fabrics, threads, embroidery parameters, and machine limitations. To facilitate practical applications, we demonstrated practical methods for linking parameters to application requirements, showcasing form-fitting wearables, structural support, and shape-changeable furniture for repeatable or one-time customization.
Making errors is part of the programming process -- even for the most seasoned professionals. Novices in particular are bound to make many errors while learning. It is well known that traditional (compiler/interpreter) programming error messages have been less than helpful for many novices and can have effects such as being frustrating, containing confusing jargon, and being downright misleading. Recent work has found that large language models (LLMs) can generate excellent error explanations, but that the effectiveness of these error messages heavily depends on whether the LLM has been provided with context -- typically the original source code where the problem occurred. Knowing that programming error messages can be misleading and/or contain that serves little-to-no use (particularly for novices) we explore the reverse: what happens when GPT-3.5 is prompted for error explanations on just the erroneous source code itself -- original compiler/interpreter produced error message excluded. We utilized various strategies to make more effective error explanations, including one-shot prompting and fine-tuning. We report the baseline results of how effective the error explanations are at providing feedback, as well as how various prompting strategies might improve the explanations' effectiveness. Our results can help educators by understanding how LLMs respond to such prompts that novices are bound to make, and hopefully lead to more effective use of Generative AI in the classroom.
With the advancement of large language models (LLMs), intelligent models have evolved from mere tools to autonomous agents with their own goals and strategies for cooperating with humans. This evolution has birthed a novel paradigm in NLP, i.e., human-model cooperation, that has yielded remarkable progress in numerous NLP tasks in recent years. In this paper, we take the first step to present a thorough review of human-model cooperation, exploring its principles, formalizations, and open challenges. In particular, we introduce a new taxonomy that provides a unified perspective to summarize existing approaches. Also, we discuss potential frontier areas and their corresponding challenges. We regard our work as an entry point, paving the way for more breakthrough research in this regard.
In Reinforcement Learning from Human Feedback (RLHF), it is crucial to learn suitable reward models from human feedback to align large language models (LLMs) with human intentions. However, human feedback can often be noisy, inconsistent, or biased, especially when evaluating complex responses. Such feedback can lead to misaligned reward signals, potentially causing unintended side effects during the RLHF process. To address these challenges, we explore the use of influence functions to measure the impact of human feedback on the performance of reward models. We propose a compute-efficient approximation method that enables the application of influence functions to LLM-based reward models and large-scale preference datasets. In our experiments, we demonstrate two key applications of influence functions: (1) detecting common forms of labeler bias in human feedback datasets and (2) guiding labelers to refine their strategies to align more closely with expert feedback. By quantifying the impact of human feedback on reward models, we believe that influence functions can enhance feedback interpretability and contribute to scalable oversight in RLHF, helping labelers provide more accurate and consistent feedback. Source code is available at https://github.com/mintaywon/IF_RLHF
Micromobility vehicles, such as e-scooters, are increasingly popular in urban communities but present significant challenges in terms of road safety, user privacy, infrastructure planning, and civil engineering. Addressing these critical issues requires a large-scale and easily accessible research infrastructure to collect diverse mobility and contextual data from micromobility users in realistic settings. To this end, we present ScooterLab, a community research testbed comprising a fleet of customizable battery-powered micromobility vehicles retrofitted with advanced sensing, communication, and control capabilities. ScooterLab enables interdisciplinary research at the intersection of computing, mobility, and urban planning by providing researchers with tools to design and deploy customized sensing experiments and access curated datasets. The testbed will enable advances in machine learning, privacy, and urban transportation research while promoting sustainable mobility.
Motor execution, a fundamental aspect of human behavior, has been extensively studied using BCI technologies. EEG and fNIRS have been utilized to provide valuable insights, but their individual limitations have hindered performance. This study investigates the effectiveness of fusing electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) data for classifying rest versus task states in a motor execution paradigm. Using the SMR Hybrid BCI dataset, this work compares unimodal (EEG and fNIRS) classifiers with a multimodal fusion approach. It proposes Motor Execution using Convolutional Additive Self-Attention Mechanisms (MECASA), a novel architecture leveraging convolutional operations and self-attention to capture complex patterns in multimodal data. MECASA, built upon the CAS-ViT architecture, employs a computationally efficient, convolutional-based self-attention module (CASA), a hybrid block design, and a dedicated fusion network to combine features from separate EEG and fNIRS processing streams. Experimental results demonstrate that MECASA consistently outperforms established methods across all modalities (EEG, fNIRS, and fused), with fusion consistently improving accuracy compared to single-modality approaches. fNIRS generally achieved higher accuracy than EEG alone. Ablation studies revealed optimal configurations for MECASA, with embedding dimensions of 64-128 providing the best performance for EEG data and OD128 (upsampled optical density) yielding superior results for fNIRS data. This work highlights the potential of deep learning, specifically MECASA, to enhance EEG-fNIRS fusion for BCI applications.
Brain--computer interfaces are groundbreaking technology whereby brain signals are used to control external devices. Despite some advances in recent years, electroencephalogram (EEG)-based motor-imagery tasks face challenges, such as amplitude and phase variability and complex spatial correlations, with a need for smaller models and faster inference. In this study, we develop a prototype, called the Lightweight Geometric Learning Brain--Computer Interface (LGL-BCI), which uses our customized geometric deep learning architecture for swift model inference without sacrificing accuracy. LGL-BCI contains an EEG channel selection module via a feature decomposition algorithm to reduce the dimensionality of a symmetric positive definite matrix, providing adaptiveness among the continuously changing EEG signal. Meanwhile, a built-in lossless transformation helps boost the inference speed. The performance of our solution was evaluated using two real-world EEG devices and two public EEG datasets. LGL-BCI demonstrated significant improvements, achieving an accuracy of 82.54% compared to 62.22% for the state-of-the-art approach. Furthermore, LGL-BCI uses fewer parameters (64.9K vs. 183.7K), highlighting its computational efficiency. These findings underscore both the superior accuracy and computational efficiency of LGL-BCI, demonstrating the feasibility and robustness of geometric deep learning in motor-imagery brain--computer interface applications.
Mentorship in open source software (OSS) is a vital, multifaceted process that includes onboarding newcomers, fostering skill development, and enhancing community building. This study examines task-focused mentoring strategies that help mentees complete their tasks and the ideal personal qualities and outcomes of good mentorship in OSS communities. We conducted two surveys to gather contributor perceptions: the first survey, with 70 mentors, mapped 17 mentoring challenges to 21 strategies that help support mentees. The second survey, with 85 contributors, assessed the importance of personal qualities and ideal mentorship outcomes. Our findings not only provide actionable strategies to help mentees overcome challenges and become successful contributors but also guide current and future mentors and OSS communities in understanding the personal qualities that are the cornerstone of good mentorship and the outcomes that mentor-mentee pairs should aspire to achieve.
This study examines the intersection of academic pressure and sleep within Taiwanese families, revealing how cultural norms and expectations shape sleep practices. Through interviews and two-week diaries from eleven families, we found that academic demands significantly influence children's sleep patterns, leading to reduced sleep duration and varied sleep schedules. Our research highlights the importance of integrating care and attuning into the design of sleep-tracking technologies, advocating for a family informatics approach that considers both health needs and social expectations. By exploring these dynamics, we contribute to a broader understanding of family contexts in diverse cultural settings and offer insights for more inclusive technology design.
The ability to effectively visualize data is crucial in the contemporary world where information is often voluminous and complex. Visualizations, such as charts, graphs, and maps, provide an intuitive and easily understandable means to interpret, analyze, and communicate patterns, trends, and insights hidden within large datasets. These graphical representations can help researchers, policymakers, and the public to better comprehend and respond to a multitude of issues. In this study, we explore a visualization tool to interpret and understand various data of COVID-19 pandemic. While others have shown COVID-19 visualization methods/tools, our tool provides a mean to analyze COVID-19 data in a more comprehensive way. We have used the public data from NY Times and CDC, and various COVID-19 data (e.g., core places, patterns, foot traffic) from Safegraph. Figure 1 shows the basic view of our visualization view. In addition to providing visualizations of these data, our visualization also considered the Surprising Map. The Surprising Map is a type of choropleth map that can avoid misleading of producing visual prominence to known base rates or to artifacts of sample size and normalization in visualizing the density of events in spatial data. It is based on Bayesian surprise-it creates a space of equi-plausible models and uses Bayesian updating to re-estimate their plausibility based on individual events.
Think-alouds are a common HCI usability method where participants verbalize their thoughts while using interfaces. However, their utility in cross-cultural settings, particularly in the Global South, is unclear, where cultural differences impact user interactions. This paper investigates the usability challenges teachers in rural Côte d'Ivoire faced when using a chatbot designed to support an educational program. We conducted think-aloud sessions with 20 teachers two weeks after a chatbot deployment, analyzing their navigation, errors, and time spent on tasks. We discuss our approach and findings that helped us identify usability issues and challenging features for improving the chatbot designs. Our note summarizes our reflections on using think-aloud and contributes to discussions on its culturally sensitive adaptation in the Global South.
Designing culturally relevant questionnaires for ICTD research is challenging, particularly when adapting surveys for populations to non-western contexts. Prior work adapted questionnaires through expert reviews and pilot studies, which are resource-intensive and time-consuming. To address these challenges, we propose using large language models (LLMs) to automate the questionnaire pretesting process in cross-cultural settings. Our study used LLMs to adapt a U.S.-focused climate opinion survey for a South African audience. We then tested the adapted questionnaire with 116 South African participants via Prolific, asking them to provide feedback on both versions. Participants perceived the LLM-adapted questions as slightly more favorable than the traditional version. Our note opens discussions on the potential role of LLMs in adapting surveys and facilitating cross-cultural questionnaire design.
In this study, we investigated gaze-based interaction methods within a virtual reality game with a visual search task with 52 participants. We compared four different interaction techniques: Selection by dwell time or confirmation of selection by head orientation, nodding or smooth pursuit eye movements. We evaluated both subjective and objective performance metrics, including NASA-TLX for subjective task load as well as time to find the correct targets and points achieved for objective analysis. The results showed significant differences between the interaction methods in terms of NASA TLX dimensions, time to find the right targets, and overall performance scores, suggesting differential effectiveness of gaze-based approaches in improving intuitive system communication. Interestingly, the results revealed gender-specific differences, suggesting interesting implications for the design of gaze-based interaction paradigms that are optimized for different user needs and preferences. These findings could help to develop more customized and effective gaze interaction systems that can improve accessibility and user satisfaction.