SearcharxivSearch

arXiv subjects

Ambar Murillo

Publications and source records attributed to Ambar Murillo.

3 recordsLinked to original sources

Structuring Human-AI Productive Interdependence by Strategic Level of Automation Selection for Qualitative Inquiry

While Large Language Models (LLMs) offer a solution to the scale-versus-depth dilemma in qualitative analysis, the paradigm of maximizing automation is fundamentally at odds with the interpretive nature of qualitative inquiry. We argue that effective Human-AI collaboration is not an automation problem, but an interdependence problem. This paper reframes the design of "co-data" systems through the lens of Interdependence Theory, proposing a formal framework to structure human-AI productive interdependence. The framework guides the selection of an appropriate Level of Automation (LoA) for different stages of the qualitative analysis process by assessing task risk and the cost of validation. We present a case study where this framework led to a deliberately interdependent workflow, fostering the calibrated trust necessary for rigorous analysis. We conclude by presenting three design principles that instantiate this framework, demonstrating how to leverage AI as a powerful partner while preserving the human researcher's irreplaceable role in the transformation process of meaning-making.

cs.HC

Understanding and supporting how developers prompt for LLM-powered code editing in practice

Large Language Models (LLMs) are rapidly transforming software engineering, with coding assistants embedded in an IDE becoming increasingly prevalent. While research has focused on improving the tools and understanding developer perceptions, a critical gap exists in understanding how developers actually use these tools in their daily workflows, and, crucially, where they struggle. This paper addresses part of this gap through a multi-phased investigation of developer interactions with an LLM-powered code editing feature, Transform Code, in an IDE widely used at Google. First, we analyze telemetry logs of the feature usage, revealing that frequent re-prompting can be an indicator of developer struggles with using Transform Code. Second, we conduct a qualitative analysis of unsatisfactory requests, identifying five key categories of information often missing from developer prompts. Finally, based on these findings, we propose and evaluate a tool, AutoPrompter, for automatically improving prompts by inferring missing information from the surrounding code context, leading to a 27% improvement in edit correctness on our test set.

cs.SE

Creating benchmarkable components to measure the quality ofAI-enhanced developer tools

In the AI community, benchmarks to evaluate model quality are well established, but an equivalent approach to benchmarking products built upon generative AI models is still missing. This has had two consequences. First, it has made teams focus on model quality over the developer experience, while successful products combine both. Second, product team have struggled to answer questions about their products in relation to their competitors. In this case study, we share: (1) our process to create robust, enterprise-grade and modular components to support the benchmarking of the developer experience (DX) dimensions of our team's AI for code offerings, and (2) the components we have created to do so, including demographics and attitudes towards AI surveys, a benchmarkable task, and task and feature surveys. By doing so, we hope to lower the barrier to the DX benchmarking of genAI-enhanced code products.

cs.SE