arXiv subjects

Aditya Majumdar

Publications and source records attributed to Aditya Majumdar.

5 recordsLinked to original sources

arXiv2026

CHAI for LLMs: Improving Code-Mixed Translation in Large Language Models through Reinforcement Learning with AI Feedback

Large language models (LLMs) show strong performance across many tasks but remain weak at understanding code-mixed (CM) language. Despite this limitation, improving LLMs for CM tasks has received little attention. To address this gap, we propose CHAI, a general-purpose framework for enhancing LLM performance on CM tasks, focusing on CM translation. CHAI leverages four key ideas. First, we investigate the use of LLMs as annotators to address the scarcity of high-quality CM datasets. Second, we leverage these LLM annotations to generate large-scale preference data and apply reinforcement learning from AI feedback (RLAIF) to improve CM translation. Third, we incorporate LLM-generated domain knowledge as a constitution, enabling iterative response refinement. Fourth, we perform extensive evaluations on real-world datasets and settings. Results show that CHAI-powered LLMs outperform state-of-the-art open-source models by 68.45% on average in human-adjudicated win rate on CM translation tasks. This work is a step toward more inclusive open-source code-mixed LLMs.

cs.CL↗

arXiv2026

The Hardness of Achieving Impact in AI for Social Impact Research: A Ground-Level View of Challenges & Opportunities

AI for Social Impact (AI4SI) is an emergent field harnessing interdisciplinarities between the fields of artificial intelligence (AI), machine learning (ML), and the social sciences to address societal issues aligned with the United Nations Sustainable Development Goals (UN SDGs), such as universal healthcare, climate action, etc. Despite AI4SI's rising popularity, achieving tangible, on-the-ground impact remains a significant challenge. In particular, identifying collaborators open to co-designing and deploying AI4SI-based solutions in real-world settings is often difficult. Thus, many projects stall at the proof-of-concept stage, unable to scale to production-level deployment. Drawing on twenty-six AI4SI researchers' interviews, primarily from academic institutions though also including some industry researchers and practitioners, and the authors' own lived experiences, this paper employs thematic analysis to highlight structural, organizational, communication, collaboration, and operational challenges hindering socially impactful AI4SI deployments. While there are no easy fixes, the authors synthesize best practices and actionable strategies from interviews and personal experiences, positioning this paper as a practical guide for AI4SI researchers and organizations pursuing socially impactful collaborations$^1$. $^1$We note that our findings are most directly applicable to academic research groups in the global north, as governmental, startup, and global south researchers' perspectives are underrepresented in our sample.

cs.HC↗

arXiv2026

Demonstrably Informed Consent in Privacy Policy Flows: Evidence from a Randomized Experiment

Privacy policies govern how personal data is collected, used, and shared. Yet, in most privacy-policy consent flows, agreement is operationalized as a single click at the end of a long, opaque policy document. Recent privacy-law scholarship has argued for a standard of demonstrably informed consent. That is, the party drafting and designing privacy-policy consent mechanisms must generate reliable evidence that a person demonstrates comprehension of the consequential terms to which they agree. To this end, we study pedagogical friction as a design framing: minimal interventions embedded within a privacy-policy consent flow that aim to support demonstrated comprehension while keeping burden on the user low. In a randomized experiment, we tested pedagogical friction for demonstrably informed consent in the context of a privacy policy for an edtech app for young children. We recruited 293 parents of kids ages 3-8 to review the app's privacy policy under one of six conditions that varied presentation format and pacing, then complete a six-question comprehension quiz. Three conditions offered a second policy review and quiz retake for participants who did not pass this quiz on their first attempt. We find that the slide-based condition (G3) achieved the highest first-attempt threshold attainment (>=80%) (41.7%), followed by the paced, sectioned condition (G4) (30.6%). In the retake conditions, 64.9% of participants who completed a second attempt improved their score. Notably, in conditions that did not gate consent on demonstrated comprehension, 97.3% of participants who scored below the threshold still chose to consent, suggesting that ungated consent flows can record agreement without demonstrated comprehension. Our results suggest that pedagogical friction can strengthen the evidentiary basis of consent and clarify what it costs in time and burden.

cs.HC↗

arXiv2026

Learning Password Best Practices Through In-Task Instruction

Users often make security- and privacy-relevant decisions without a clear understanding of the rules that govern safe behavior. We introduce pedagogical friction, a design approach that inserts brief, instructional interactions at the moment of action. We evaluate this approach in the context of password creation, a familiar task with clear quality criteria. We conducted a randomized study with 128 participants across four interface conditions that varied the depth and interactivity of guidance. We assessed three outcomes: (1) rule compliance in a subsequent password task without guidance, (2) accuracy on survey questions tied to password rules, and (3) behavior-knowledge alignment, which captures whether participants who correctly followed a rule also recognized it on the survey. Across the guided conditions, participants corrected most rule violations in the follow-up task and showed high behavior-knowledge alignment. Survey results suggested clearer advantages for some rule types, especially symbol related questions. These results position pedagogical friction as a lightweight intervention for security- and privacy-critical interfaces.

cs.HC↗

arXiv2025

Dr. GPT Will See You Now, but Should It? Exploring the Benefits and Harms of Large Language Models in Medical Diagnosis using Crowdsourced Clinical Cases

The proliferation of Large Language Models (LLMs) in high-stakes applications such as medical (self-)diagnosis and preliminary triage raises significant ethical and practical concerns about the effectiveness, appropriateness, and possible harmfulness of the use of these technologies for health-related concerns and queries. Some prior work has considered the effectiveness of LLMs in answering expert-written health queries/prompts, questions from medical examination banks, or queries based on pre-existing clinical cases. Unfortunately, these existing studies completely ignore an in-the-wild evaluation of the effectiveness of LLMs in answering everyday health concerns and queries typically asked by general users, which corresponds to the more prevalent use case for LLMs. To address this research gap, this paper presents the findings from a university-level competition that leveraged a novel, crowdsourced approach for evaluating the effectiveness of LLMs in answering everyday health queries. Over the course of a week, a total of 34 participants prompted four publicly accessible LLMs with 212 real (or imagined) health concerns, and the LLM generated responses were evaluated by a team of nine board-certified physicians. At a high level, our findings indicate that on average, 76% of the 212 LLM responses were deemed to be accurate by physicians. Further, with the help of medical professionals, we investigated whether RAG versions of these LLMs (powered with a comprehensive medical knowledge base) can improve the quality of responses generated by LLMs. Finally, we also derive qualitative insights to explain our quantitative findings by conducting interviews with seven medical professionals who were shown all the prompts in our competition. This paper aims to provide a more grounded understanding of how LLMs perform in real-world everyday health communication.

cs.CY↗