arXiv subjects

Sophia Chen

Publications and source records attributed to Sophia Chen.

4 recordsLinked to original sources

arXiv2026

Towards Apples to Apples for AI Evaluations: From Real-World Use Cases to Evaluation Scenarios

AI measurement science has a wide variety of methodologies and measurements for comparing AI systems, resulting in what often appear to be "apples-to-oranges" comparisons across AI evaluations. To move toward "apples-to-apples" comparisons in real-world AI evaluations, this work advocates for methodological transparency in evaluation scenarios, operational grounding, and human-centered design (HCD) principles. We propose a repeatable process for transforming high-level use cases to detailed scenarios by eliciting use cases from subject matter experts (SMEs) via a structured AI Use Case Worksheet with six key elements: use case, sector, user (direct and indirect), intended outcomes, expected impacts (positive and negative), and KPIs and metrics. We demonstrate utility of the worksheet and process in the U.S. financial services sector. This paper reports on example high-level AI use cases identified by financial services sector SMEs: cyber defense enablement, developer productivity, financial crime aggregation, suspicious activity report (SAR) filing, credit memo generation, and internal call center support. These AI use cases provided are illustrative of the process and not exhaustive. Central to our work is a three-stage expansion pipeline combining LLM prompting with human reviews to generate 107 scenarios from those use cases elicited from SMEs. This process integrates iterative human reviews at every juncture to ensure operational grounding: for scenario titles and descriptions; for core scenario elements like users, benefits and risks, and metrics; and for scenario narratives and evaluation objectives. Human checkpoints ensure scenarios remain reflective of real-world usage and human needs. We describe a validation rubric to assess scenario quality. By defining key scenario components, this work supports a more consistent and meaningful paradigm for human-centered AI evaluations.

cs.HC↗

arXiv2025

It's About Time: The Temporal and Modal Dynamics of Copilot Usage

We analyze 37.5 million deidentified conversations with Microsoft's Copilot between January and September 2025. Unlike prior analyses of AI usage, we focus not just on what people do with AI, but on how and when they do it. We find that how people use AI depends fundamentally on context and device type. On mobile, health is the dominant topic, which is consistent across every hour and every month we observed - with users seeking not just information but also advice. On desktop, the pattern is strikingly different: work and technology dominate during business hours, with "Work and Career" overtaking "Technology" as the top topic precisely between 8 a.m. and 5 p.m. These differences extend to temporal rhythms: programming queries spike on weekdays while gaming rises on weekends, philosophical questions climb during late-night hours, and relationship conversations surge on Valentine's Day. These patterns suggest that users have rapidly integrated AI into the full texture of their lives, as a work aid at their desks and a companion on their phones.

cs.CY↗

arXiv2024

Design and Characterization of High Efficiency Single-stage Electromagnetic Coil Guns

This study presents several novel approaches to improve the efficiency of a single-stage coil gun. Conventional designs typically feature a uniformly wound solenoid and a ferrite projectile. For our research, we constructed a microcontroller-based prototype to test several new enhancements, including the use of a bipolar current pulse, a stepped multilayer coil with non-uniform winding densities, and the replacement of conventional ferrite projectiles with a neodymium permanent magnet. These modifications were designed to reduce energy loss and improve projectile acceleration by changing magnetic field strength and effectively controlling the magnetic flux. The experimental results show that the proposed methods resulted in significant efficiency improvements, with the varying current pulse and stepped coil design providing enhanced magnetic force at key points in the projectile's path, and the permanent magnet projectile contributing to higher velocities and efficiencies by leveraging the current pulses. Our findings suggest that combining these enhancements significantly improves coil gun performance, achieving higher velocities and efficiencies. These findings can be applied to future coil gun developments, such as multi-stage coil gun systems.

eess.SY↗

arXiv2018

Young Stars and their Variability with LSST

Young stars exhibit short-term photometric variability caused by mass accretion events from circumstellar disks, the presence of dusty warps within the inner disks, starspots that rotate across the stellar surfaces, and flares. Long-term variability also occurs owing to starspot longevity and cycles, and from changes in stellar angular momenta and activity as the stars age. We propose to observe the Carina star-forming region in different bands with a cadence of 30 minutes every night for one week per year to clarify the nature of both the short-term and long-term variability of the thousands of young stars in this region. By obtaining well-sampled multicolor lightcurves of this dense young cluster, LSST would acquire the first statistically significant data on how these objects vary on both short and long timescales. This information will allow us to relate the observed variability to stellar properties such as mass, age, binarity, and to environmental properties such as location within or exterior to the H II region, and to the presence or absence of a circumstellar disk.

astro-ph.SR↗