arXiv subjects

Manaal Basha

Publications and source records attributed to Manaal Basha.

4 recordsLinked to original sources

arXiv2026

Toward Inclusive AI-Driven Development: Exploring Gender Differences in Code Generation Tool Interactions

The increasing reliance on Code Generation Tools (CGTs), such as Claude Code and GitHub Copilot, is revamping programming workflows and raising critical questions about fairness and inclusivity in human-AI collaboration. While CGTs offer potential productivity enhancements, their effectiveness across diverse user groups have not been sufficiently investigated. We hypothesized that developers' interactions with CGTs vary based on gender, influencing task outcomes and cognitive load, as prior research suggests that gender differences can affect technology use and cognitive processing. This study employed a mixed-subjects design with 39 participants, evenly divided by gender for a counterbalanced design. Participants completed two programming tasks of medium to high difficulty using two distinct treatments: only CGT assistance and only internet access. Task orders and conditions were counterbalanced to mitigate order effects. We collected cognitive load surveys, screen recordings, and task performance metrics such as completion time, code correctness, and CGT interaction behaviors. Our results indicate no statistically significant gender differences in cognitive load or performance outcomes when using CGTs compared to Internet-based workflows. CGTs reduce intrinsic and extraneous cognitive load compared to Internet based workflows, but the reduction was not statistically significantly. However, CGTs improved advanced code correctness. Our results suggest that CGTs can lower cognitive load and enhance performance on complex coding tasks without significantly affecting core correctness or completion time. These findings highlight how CGT usage can reduce cognitive burden and support more equitable programming experiences across users.

cs.SE↗

arXiv2026

Do Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code Generation

Large Language Models (LLMs) are increasingly integrated into software engineering workflows, helping developers write, debug, test, and maintain code. While prompt wording and structure are known to influence model performance, the impact of psychologically inspired prompt framings remains unexplored. This study investigates whether different psychology-based communication strategies that humans use to persuade or motivate others can lead to more effective prompt framing, which may, in turn, affect LLM behaviour in coding tasks. Drawing on Yukl & Falbe's well-known taxonomy, we operationalized eight influence tactics (like rational persuasion, ingratiation, and exchange) into reproducible prompt templates. These prompt templates were evaluated across five leading open-weight LLMs using two widely adopted benchmarks: LiveCodeBench and SWE-bench Verified. We assessed the resulting code output on four key software quality dimensions: functional correctness, quality, maintainability, and security. Our results show that certain influence-induced prompt framings, particularly those emphasizing urgency, were associated with reduced correctness and security. This work presents the first large-scale empirical study of influence-induced prompt framing in software engineering tasks, offering insights into how linguistic cues may shape LLM outputs. We conclude with practical insights for designing transparent and interpretable human-AI interactions in code generation.

cs.SE↗

arXiv2025

Cracking CodeWhisperer: Analyzing Developers' Interactions and Patterns During Programming Tasks

The use of AI code-generation tools is becoming increasingly common, making it important to understand how software developers are adopting these tools. In this study, we investigate how developers engage with Amazon's CodeWhisperer, an LLM-based code-generation tool. We conducted two user studies with two groups of 10 participants each, interacting with CodeWhisperer - the first to understand which interactions were critical to capture and the second to collect low-level interaction data using a custom telemetry plugin. Our mixed-methods analysis identified four behavioral patterns: 1) incremental code refinement, 2) explicit instruction using natural language comments, 3) baseline structuring with model suggestions, and 4) integrative use with external sources. We provide a comprehensive analysis of these patterns .

cs.SE↗

arXiv2025

CodeWatcher: IDE Telemetry Data Extraction Tool for Understanding Coding Interactions with LLMs

Understanding how developers interact with code generation tools (CGTs) requires detailed, real-time data on programming behavior which is often difficult to collect without disrupting workflow. We present \textit{CodeWatcher}, a lightweight, unobtrusive client-server system designed to capture fine-grained interaction events from within the Visual Studio Code (VS Code) editor. \textit{CodeWatcher} logs semantically meaningful events such as insertions made by CGTs, deletions, copy-paste actions, and focus shifts, enabling continuous monitoring of developer activity without modifying user workflows. The system comprises a VS Code plugin, a Python-based RESTful API, and a MongoDB backend, all containerized for scalability and ease of deployment. By structuring and timestamping each event, \textit{CodeWatcher} enables post-hoc reconstruction of coding sessions and facilitates rich behavioral analyses, including how and when CGTs are used during development. This infrastructure is crucial for supporting research on responsible AI, developer productivity, and the human-centered evaluation of CGTs. Please find the demo, diagrams, and tool here: https://osf.io/j2kru/overview.

cs.SE↗