2025-02-07 | | Total: 16
The proliferation of NLP-powered language technologies, AI-based natural language generation models, and English as a mainstream means of communication among both native and non-native speakers make the output of AI-powered tools especially intriguing to linguists. This paper investigates how Grammarly and ChatGPT affect the English language regarding wordiness vs. conciseness. A case study focusing on the purpose subordinator in order to is presented to illustrate the way in which Grammarly and ChatGPT recommend shorter grammatical structures instead of longer and more elaborate ones. Although the analysed sentences were produced by native speakers, are perfectly correct, and were extracted from a language corpus of contemporary English, both Grammarly and ChatGPT suggest more conciseness and less verbosity, even for relatively short sentences. The present article argues that technologies such as Grammarly not only mirror language change but also have the potential to facilitate or accelerate it.
Despite extensive safety alignment efforts, large language models (LLMs) remain vulnerable to jailbreak attacks that elicit harmful behavior. While existing studies predominantly focus on attack methods that require technical expertise, two critical questions remain underexplored: (1) Are jailbroken responses truly useful in enabling average users to carry out harmful actions? (2) Do safety vulnerabilities exist in more common, simple human-LLM interactions? In this paper, we demonstrate that LLM responses most effectively facilitate harmful actions when they are both actionable and informative--two attributes easily elicited in multi-step, multilingual interactions. Using this insight, we propose HarmScore, a jailbreak metric that measures how effectively an LLM response enables harmful actions, and Speak Easy, a simple multi-step, multilingual attack framework. Notably, by incorporating Speak Easy into direct request and jailbreak baselines, we see an average absolute increase of 0.319 in Attack Success Rate and 0.426 in HarmScore in both open-source and proprietary LLMs across four safety benchmarks. Our work reveals a critical yet often overlooked vulnerability: Malicious users can easily exploit common interaction patterns for harmful intentions.
A wide variety of resource allocation problems operate under resource constraints that are managed by a central arbitrator, with agents who evaluate and communicate preferences over these resources. We formulate this broad class of problems as Distributed Evaluation, Centralized Allocation (DECA) problems and propose methods to learn fair and efficient policies in centralized resource allocation. Our methods are applied to learning long-term fairness in a novel and general framework for fairness in multi-agent systems. We show three different methods based on Double Deep Q-Learning: (1) A joint weighted optimization of fairness and utility, (2) a split optimization, learning two separate Q-estimators for utility and fairness, and (3) an online policy perturbation to guide existing black-box utility functions toward fair solutions. Our methods outperform existing fair MARL approaches on multiple resource allocation domains, even when evaluated using diverse fairness functions, and allow for flexible online trade-offs between utility and fairness.
Despite the growing interest in human-AI decision making, experimental studies with domain experts remain rare, largely due to the complexity of working with domain experts and the challenges in setting up realistic experiments. In this work, we conduct an in-depth collaboration with radiologists in prostate cancer diagnosis based on MRI images. Building on existing tools for teaching prostate cancer diagnosis, we develop an interface and conduct two experiments to study how AI assistance and performance feedback shape the decision making of domain experts. In Study 1, clinicians were asked to provide an initial diagnosis (human), then view the AI's prediction, and subsequently finalize their decision (human-AI team). In Study 2 (after a memory wash-out period), the same participants first received aggregated performance statistics from Study 1, specifically their own performance, the AI's performance, and their human-AI team performance, and then directly viewed the AI's prediction before making their diagnosis (i.e., no independent initial diagnosis). These two workflows represent realistic ways that clinical AI tools might be used in practice, where the second study simulates a scenario where doctors can adjust their reliance and trust on AI based on prior performance feedback. Our findings show that, while human-AI teams consistently outperform humans alone, they still underperform the AI due to under-reliance, similar to prior studies with crowdworkers. Providing clinicians with performance feedback did not significantly improve the performance of human-AI teams, although showing AI decisions in advance nudges people to follow AI more. Meanwhile, we observe that the ensemble of human-AI teams can outperform AI alone, suggesting promising directions for human-AI collaboration.
The Human Cognitive Simulation Framework represents a significant advancement in integrating human cognitive capabilities into artificial intelligence systems. By merging short-term memory (conversation context), long-term memory (interaction context), advanced cognitive processing, and efficient knowledge management, it ensures contextual coherence and persistent data storage, enhancing personalization and continuity in human-AI interactions. The framework employs a unified database that synchronizes these contexts while incorporating logical, creative, and analog processing modules inspired by human brain hemispheric functions to perform structured tasks and complex inferences. Dynamic knowledge updates enable real-time integration, improving adaptability and fostering applications in education, behavior analysis, and knowledge management. Despite its potential to process vast data volumes and enhance user experience, challenges remain in scalability, cognitive bias mitigation, and ethical compliance. This framework lays the foundation for future research in continuous learning algorithms, sustainability, and multimodal adaptability, positioning Cognitive AI as a transformative model in emerging fields.
We draw on our experience working on system and software assurance and evaluation for systems important to society to summarise how safety engineering is performed in traditional critical systems, such as aircraft flight control. We analyse how this critical systems perspective might support the development and implementation of AI Safety Frameworks. We present the analysis in terms of: system engineering, safety and risk analysis, and decision analysis and support. We consider four key questions: What is the system? How good does it have to be? What is the impact of criticality on system development? and How much should we trust it? We identify topics worthy of further discussion. In particular, we are concerned that system boundaries are not broad enough, that the tolerability and nature of the risks are not sufficiently elaborated, and that the assurance methods lack theories that would allow behaviours to be adequately assured. We advocate the use of assurance cases based on Assurance 2.0 to support decision making in which the criticality of the decision as well as the criticality of the system are evaluated. We point out the orders of magnitude difference in confidence needed in critical rather than everyday systems and how everyday techniques do not scale in rigour. Finally we map our findings in detail to two of the questions posed by the FAISC organisers and we note that the engineering of critical systems has evolved through open and diverse discussion. We hope that topics identified here will support the post-FAISC dialogues.
The rapid advancement of artificial intelligence (AI) has brought about significant societal changes, necessitating robust AI governance frameworks. This study analyzed the research trends in AI governance within the framework of the EU AI Act. This study conducted a bibliometric analysis to examine the publications indexed in the Web of Science database. Our findings reveal that research on AI governance, particularly concerning AI systems regulated by the EU AI Act, remains relatively limited compared to the broader AI research landscape. Nonetheless, a growing interdisciplinary interest in AI governance is evident, with notable contributions from multi-disciplinary journals and open-access publications. Dominant research themes include ethical considerations, privacy concerns, and the growing impact of generative AI, such as ChatGPT. Notably, education, healthcare, and worker management are prominent application domains. Keyword network analysis highlights education, ethics, and ChatGPT as central keywords, underscoring the importance of these areas in current AI governance research. Subsequently, a comprehensive literature review was undertaken based on the bibliometric analysis findings to identify research trends, challenges, and insights within the categories of the EU AI Act. The findings provide valuable insights for researchers and policymakers, informing future research directions and contributing to developing comprehensive AI governance frameworks beyond the EU AI Act.
We propose a conceptualization and implementation of AI ethics via the capability approach. We aim to show that conceptualizing AI ethics through the capability approach has two main advantages for AI ethics as a discipline. First, it helps clarify the ethical dimension of AI tools. Second, it provides guidance to implementing ethical considerations within the design of AI tools. We illustrate these advantages in the context of AI tools in medicine, by showing how ethics-based auditing of AI tools in medicine can greatly benefit from our capability-based approach.
Artificial intelligence (AI) and machine learning (ML) have made tremendous advancements in the past decades. From simple recommendation systems to more complex tumor identification systems, AI/ML systems have been utilized in a plethora of applications. This rapid growth of AI/ML and its proliferation in numerous private and public sector applications, while successful, has also opened new challenges and obstacles for regulators. With almost little to no human involvement required for some of the new decision-making AI/ML systems, there is now a pressing need to ensure the responsible use of these systems. Particularly in federal government use-cases, the use of AI technologies must be carefully governed by appropriate transparency and accountability mechanisms. This has given rise to new interdisciplinary fields of AI research such as \textit{Responsible AI (RAI)}. In this position paper we provide a brief overview of development in RAI and discuss some of the motivating principles commonly explored in the field. An overview of the current regulatory landscape relating to AI is also discussed with analysis of different Executive Orders, policies and frameworks. We then present examples of how federal agencies are aiming for the responsible use of AI, specifically we present use-case examples of different projects and research from the Census Bureau on implementing the responsible use of AI. We also provide a brief overview for a Responsible AI Assessment Toolkit currently under-development aimed at helping federal agencies operationalize RAI principles. Finally, a robust discussion on how different policies/regulations map to RAI principles, along with challenges and opportunities for regulation/governance of responsible AI within the federal government is presented.
The rapid advancement of Large Language Models (LLMs) has created a critical gap in consumer protection due to the lack of standardized certification processes for LLM-powered Artificial Intelligence (AI) systems. This paper argues that current regulatory approaches, which focus on compute-level thresholds and generalized model evaluations, are insufficient to ensure the safety and effectiveness of specific LLM-based user experiences. We propose a shift towards a certification process centered on actual user-facing experiences and the curation of high-quality datasets for evaluation. This approach offers several benefits: it drives consumer confidence in AI system performance, enables businesses to demonstrate the credibility of their products, and allows regulators to focus on direct consumer protection. The paper outlines a potential certification workflow, emphasizing the importance of domain-specific datasets and expert evaluation. By repositioning data as the strategic center of regulatory efforts, this framework aims to address the challenges posed by the probabilistic nature of AI systems and the rapid pace of technological advancement. This shift in regulatory focus has the potential to foster innovation while ensuring responsible AI development, ultimately benefiting consumers, businesses, and government entities alike.
This article reports the results of a study examining the ability of legal and non-legal Large Language Models to perform legal analysis using the Issue-Rule-Application-Conclusion framework. LLMs were tested on legal reasoning tasks involving rule analysis and analogical reasoning. The results show that LLMs can conduct basic IRAC analysis, but are limited by brief responses lacking detail, an inability to commit to answers, false confidence, and hallucinations. The study compares legal and nonlegal LLMs, identifies shortcomings, and explores traits that may hinder their ability to think like a lawyer. It also discusses the implications for legal education and practice, highlighting the need for critical thinking skills in future lawyers and the potential pitfalls of overreliance on artificial intelligence AI resulting in a loss of logic, reasoning, and critical thinking skills.
The integration of Large Language Models (LLMs) into healthcare settings has gained significant attention, particularly for question-answering tasks. Given the high-stakes nature of healthcare, it is essential to ensure that LLM-generated content is accurate and reliable to prevent adverse outcomes. However, the development of robust evaluation metrics and methodologies remains a matter of much debate. We examine the performance of publicly available LLM-based chatbots for menopause-related queries, using a mixed-methods approach to evaluate safety, consensus, objectivity, reproducibility, and explainability. Our findings highlight the promise and limitations of traditional evaluation metrics for sensitive health topics. We propose the need for customized and ethically grounded evaluation frameworks to assess LLMs to advance safe and effective use in healthcare.
The AI research community plays a vital role in shaping the scientific, engineering, and societal goals of AI research. In this position paper, we argue that focusing on the highly contested topic of `artificial general intelligence' (`AGI') undermines our ability to choose effective goals. We identify six key traps -- obstacles to productive goal setting -- that are aggravated by AGI discourse: Illusion of Consensus, Supercharging Bad Science, Presuming Value-Neutrality, Goal Lottery, Generality Debt, and Normalized Exclusion. To avoid these traps, we argue that the AI research community needs to (1) prioritize specificity in engineering and societal goals, (2) center pluralism about multiple worthwhile approaches to multiple valuable goals, and (3) foster innovation through greater inclusion of disciplines and communities. Therefore, the AI research community needs to stop treating `AGI' as the north-star goal of AI research.
Generative AI has gained a significant foothold in the creative and artistic sectors. In this context, the concept of creative work is influenced by discourses originating from technological stakeholders and mainstream media. The framing of narratives surrounding creativity and artistic production not only reflects a particular vision of culture but also actively contributes to shaping it. In this article, we review online media outlets and analyze the dominant narratives around AI's impact on creative work that they convey. We found that the discourse promotes creativity freed from its material realisation through human labor. The separation of the idea from its material conditions is achieved by automation, which is the driving force behind productive efficiency assessed as the reduction of time taken to produce. And the withdrawal of the skills typically required in the execution of the creative process is seen as a means for democratising creativity. This discourse tends to correspond to the dominant techno-positivist vision and to assert power over the creative economy and culture.
The evolution of existing transportation systems, mainly driven by urbanization and increased availability of mobility options, such as private, profit-maximizing ride-hailing companies, calls for tools to reason about their design and regulation. To study this complex socio-technical problem, one needs to account for the strategic interactions of the stakeholders involved in the mobility ecosystem. In this paper, we present a game-theoretic framework to model multi-modal mobility systems, focusing on municipalities, service providers, and travelers. Through a user-friendly, Graphical User Interface, one can visualize system dynamics and compute equilibria for various scenarios. The framework enables stakeholders to assess the impact of local decisions (e.g., fleet size for services or taxes for private companies) on the full mobility system. Furthermore, this project aims to foster STEM interest among high school students (e.g., in the context of prior activities in Switzerland, and planned activities with the MIT museum). This initiative combines theoretical advancements, practical applications, and educational outreach to improve mobility system design.
Search engines, often viewed as reliable gateways to information, tailor search results using customization algorithms based on user preferences, location, and more. While this can be useful for routine queries, it raises concerns when the topics are sensitive or contentious, possibly limiting exposure to diverse viewpoints and increasing polarization. To examine the extent of this tailoring, we focused on the Israel-Palestine conflict and developed a privacy-protecting tool to audit the behavior of three search engines: DuckDuckGo, Google and Yahoo. Our study focused on two main questions: (1) How do search results for the same query about the conflict vary among different users? and (2) Are these results influenced by the user's location and browsing history? Our findings revealed significant customization based on location and browsing preferences, unlike previous studies that found only mild personalization for general topics. Moreover, queries related to the conflict were more customized than unrelated queries, and the results were not neutral concerning the conflict's portrayal.