arXiv subjects

Paul Schmiedmayer

Publications and source records attributed to Paul Schmiedmayer.

18 recordsLinked to original sources

arXiv2026

Beyond Screen Time: Demonstrating the Value of App Activity Logs to Understand User Behavior Further

Research on smartphone use remains fragmented, with studies differing in measures, methods, and platforms, limiting what we know about everyday behavior. We argue that event-level app activity logs should form the backbone of research on actual rather than recalled smartphone use. We conduct a secondary analysis of 4,571,252 app events from 1,972 participants across three longitudinal cohorts differing in age, country, recruitment, and platform, complemented by a published reference cohort. We reproduce established aggregate and micro-usage measures and examine temporal structure, application composition, transitions, and individual distinctiveness. Aggregate usage varies less than the organization of activity: adolescent use, for example, is structured around school schedules, while other cohorts show weaker within-day patterns. Application and transition patterns reveal behavioral differences obscured by screen time; an intervention reduced daily usage while increasing mean session duration. Activity traces also enabled 15-22% top-1 participant re-identification. We discuss methodological, reproducibility, and privacy implications for HCI.

cs.HC↗

arXiv2026

How do people plan digitally: An in-the-wild investigation of task planning through a smartphone app

Daily planning supports goal attainment and productivity, and people increasingly delegate it to digital tools. Plans are postponed, revised, and left unresolved rather than executed as intended. Understanding these changes requires following tasks from creation through later updates and recorded outcomes. We report an in-the-wild study of task planning using time-stamped logs of 24,265 tasks from 957 users of a widely used daily planner app, observed over six weeks alongside self-reported surveys. Following each task across its lifecycle, only 26.4% moved directly from creation to completion and 32.0% were abandoned; all-day and longer tasks were abandoned disproportionately, and 77.6% of timed tasks were marked complete later than intended. Latent profile analysis of 909 users indicated High Engagement (11.1%), Low Engagement (19.7%), and Passive (69.2%) profiles; active days on app distinguished the profiles and were associated with post-survey completion. These findings support evaluating planning tools across the task lifecycle.

cs.HC↗

arXiv2026

The CAST-framework: Measure and model social media use as a multi-level phenomenon through real-world applications

Designing social media experiences that support well-being requires understanding when, how, and for whom use matters. Screen-time totals omit content and context, and connecting these with behavior and experience requires coordinating measurements across timescales. We introduce the CAST framework to connect measurement choices with person-specific models of exposure, behavior, physiology, and experience. Its dimensions specify where observations occur, how they are obtained, what they measure, and at what temporal resolution. Responses to interventions, such as whether to proceed after an app-opening pause, enter as behavioral measurements. We propose four synchronized measurement modules linking mobile and wearable data with self-reports and intervention responses. A synthetic demonstration with 120 simulated participants over 28 days illustrates how daily aggregation can obscure opposing effects of different activities under specified generating assumptions. The framework guides selection of measures and outcomes for evaluating social media interfaces and interventions.

cs.HC↗

arXiv2026

TimeRLM: Recursive Language Models Enable Precise Anomaly Localization in Long-Context Time-Series

Precise anomaly localization over long-context time series is a crucial task in monitoring applications across clinical care, industrial operations, financial services, and logistics, where brief evidence may hide inside long spans of high-frequency data. Time-Series Language Models (TSLMs) are able to ingest time series data and verbalize findings on anomalies in natural language; however, recent benchmarks report a decrease in retrieval performance at long contexts, mirroring failure modes in text, vision, and audio. In the text domain, Recursive Language Models (RLMs) can recover much of this lost performance by keeping context external to the large language model (LLM), allowing the model to query it through code. We present TimeRLM, an RLM formulation for time-series that sequentially manipulates the signal using code and vision capabilities. We further introduce AnomalyXL, a synthetic long-context anomaly localization benchmark with programmatically injected anomalies that require precise retrieval. We implement five different task categories and two variants: AnomalyXL-MCQ and AnomalyXL-Localize. TimeRLM outperforms every evaluated TSLM and single-pass baseline on four of the five AnomalyXL-Localize tasks, reaching 0.682 IoU on localization and 0.745 on classify-with-evidence, versus at most 0.329 and 0.072 across all baselines. We post-train TimeRLM using reinforcement learning. The resulting model further improves performance and requires approximately one-third as many agent interaction turns as its untrained base model to produce a final answer. On unseen real-world ECG, sleep and software observability recordings, the post-trained TimeRLM retains or improves performance, surpassing TSLMs despite being trained exclusively on synthetic data. Our findings suggest recursive interaction with time-series is an effective approach for long-horizon retrieval.

cs.LG↗

arXiv2026

OpenMHC: Accelerating the Science of Wearable Foundation Models

Mobile and wearable devices offer an unprecedented opportunity for continuous, passive health monitoring and active health coaching. However, the largest wearable datasets are not publicly available for research, and leading wearable foundation models trained on such datasets are rarely open-weight or come with reproducible training code. To accelerate open science in wearable health, we release OpenMyHeartCounts (OpenMHC), the largest and most comprehensive broadly accessible wearable health dataset to date, released to qualified researchers, alongside open-source implementations of recent wearable foundation models. OpenMHC, derived from over a decade of data collected through the My Heart Counts study app, includes >60 million hours of wearable data across 19 sensor channels (e.g., step count, heart rate, sleep, workouts) and up to 169 linked variables, including health, lifestyle, mood, and behavior from 11,894 consenting participants. Furthermore, we introduce a unified, open benchmark that enables standardized comparison of wearable health models across three tracks: health and behavior downstream prediction, multivariate data imputation, and time-series forecasting. We benchmark classical methods alongside recent wearable and multivariate time series foundation models. By releasing data under broad research access, alongside open-source code and model weights, at this unprecedented scale, we aim to democratize wearable health AI research and enable the community to drive open progress in this domain.

cs.LG↗

arXiv2026

The ABC of digital health: A framework for translating digital health interventions into real-world applications

Research-based digital health interventions are often presented as potential solutions for extending health care in the real world. Yet the vast majority of these interventions fails to move beyond controlled studies. Existing frameworks offer valuable guidance for intervention development and testing, but provide less concrete support for translating these evidenced intervention mechanisms into sustained real-world applications. This paper introduces the ABC framework, referring to Accessibility, Buildability, and Continuity, as a practical model for a successful translation. Accessibility captures whether diverse users can find, understand, and begin using an application with minimal friction. Buildability refers to the development of an app that supports the iteration, integration, and personalization of features. Continuity describes both sustained user engagement and the operational capacity to maintain an application over time without disproportionate increases in cost, infrastructure, or human support. Different combinations of the ABC-dimensions make an application scalable (AB), automated (BC), and adherent (AC). By linking design decisions to these features, ABC offers a shared language for researchers, designers, and policymakers seeking to build or evaluate digital health interventions that work beyond trials and are viable applications in everyday life.

cs.HC↗

arXiv2026

TS-Haystack: A Multi-Task Retrieval Benchmark for Long-Context Time-Series Reasoning

Time Series Language Models (TSLMs) promise reasoning over real-world temporal data, but their ability to retrieve and reason over long time-series remains largely untested. We introduce TS-Haystack, a multi-domain retrieval benchmark with ten event-grounded question-answering tasks over contexts from 100 seconds to 24 hours, spanning direct retrieval, temporal reasoning, multi-step reasoning, and contextual anomaly detection. Existing TSLMs exhibit severe long-context degradation: accuracy declines with context length, direct-tokenization models run out of memory beyond 100 seconds on high-rate signals, and time-interval-grounded tasks collapse toward near-zero accuracy when increasing the time-series lengths, aligning with existing literature on text and multi-modal long context retrieval. An agentic retrieval framework using specialized time-series classifier tools matches or outperforms SoTA TSLMs on 9 of 10 tasks, highlighting agentic retrieval as a promising approach for long-context TSLMs.

cs.LG↗

arXiv2026

Patient-Level Multimodal Question Answering from Multi-Site Auscultation Recordings

Auscultation is a vital diagnostic tool, yet its utility is often limited by subjective interpretation. While general-purpose Audio-Language Models (ALMs) excel in general domains, they struggle with the nuances of physiological signals. We propose a framework that aligns multi-site auscultation recordings directly with a frozen Large Language Model (LLM) embedding space via gated cross-attention. By leveraging the LLM's latent world knowledge, our approach moves beyond isolated classification toward holistic, patient-level assessment. On the CaReSound benchmark, our model achieves a state-of-the-art 0.865 F1-macro and 0.952 BERTScore. We demonstrate that lightweight, domain-specific encoders rival large-scale ALMs and that multi-site aggregation provides spatial redundancy that mitigates temporal truncation. This alignment of medical acoustics with text foundations offers a scalable path for bridging signal processing and clinical assessment.

cs.SD↗

arXiv2026

How Well Do Multimodal Models Reason on ECG Signals?

While multimodal large language models offer a promising solution to the "black box" nature of health AI by generating interpretable reasoning traces, verifying the validity of these traces remains a critical challenge. Existing evaluation methods are either unscalable, relying on manual clinician review, or superficial, utilizing proxy metrics (e.g. QA) that fail to capture the semantic correctness of clinical logic. In this work, we introduce a reproducible framework for evaluating reasoning in ECG signals. We propose decomposing reasoning into two distinct, components: (i) Perception, the accurate identification of patterns within the raw signal, and (ii) Deduction, the logical application of domain knowledge to those patterns. To evaluate Perception, we employ an agentic framework that generates code to empirically verify the temporal structures described in the reasoning trace. To evaluate Deduction, we measure the alignment of the model's logic against a structured database of established clinical criteria in a retrieval-based approach. This dual-verification method enables the scalable assessment of "true" reasoning capabilities.

cs.AI↗

arXiv2026

Bloom: Designing for LLM-Augmented Behavior Change Interactions

Large language models (LLMs) offer novel opportunities to support health behavior change, yet existing work has narrowly focused on text-only interactions. Building on decades of HCI research on effective behavior change interactions, we present Bloom, an application for physical activity promotion that integrates an LLM-based health coaching chatbot with existing design strategies and UI elements. As part of Bloom's development, we conducted a redteaming evaluation and contribute a safety benchmark dataset. In a four-week randomized field study (N=54) comparing Bloom to a no-LLM control, we observed important shifts in psychological outcomes: participants in the LLM condition reported stronger beliefs that activity was beneficial, greater enjoyment, and more self-compassion. Both conditions significantly increased physical activity levels, doubling the proportion of participants meeting recommended weekly guidelines, though descriptively, we observed no advantage for the LLM condition in short-term physical activity levels. Instead, our findings suggest that LLMs may be more effective at shifting mindsets that precede longer-term behavior change.

cs.HC↗

arXiv2026

OpenTSLM: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Data

LLMs have emerged as powerful tools for interpreting multimodal data. In medicine, they hold particular promise for synthesizing large volumes of clinical information into actionable insights and digital health applications. Yet, a major limitation remains their inability to handle time series. To overcome this gap, we present OpenTSLM, a family of Time Series Language Models (TSLMs) created by integrating time series as a native modality to pretrained LLMs, enabling reasoning over multiple time series of any length. We investigate two architectures for OpenTSLM. The first, OpenTSLM-SoftPrompt, models time series implicitly by concatenating learnable time series tokens with text tokens via soft prompting. Although parameter-efficient, we hypothesize that explicit time series modeling scales better and outperforms implicit approaches. We thus introduce OpenTSLM-Flamingo, which integrates time series with text via cross-attention. We benchmark both variants against baselines that treat time series as text tokens or plots, across a suite of text-time-series Chain-of-Thought (CoT) reasoning tasks. We introduce three datasets: HAR-CoT, Sleep-CoT, and ECG-QA-CoT. Across all, OpenTSLM models outperform baselines, reaching 69.9 F1 in sleep staging and 65.4 in HAR, compared to 9.05 and 52.2 for finetuned text-only models. Notably, even 1B-parameter OpenTSLM models surpass GPT-4o (15.47 and 2.95). OpenTSLM-Flamingo matches OpenTSLM-SoftPrompt in performance and outperforms on longer sequences, while maintaining stable memory requirements. By contrast, SoftPrompt grows exponentially in memory with sequence length, requiring around 110 GB compared to 40 GB VRAM when training on ECG-QA with LLaMA-3B. Expert reviews by clinicians find strong reasoning capabilities exhibited by OpenTSLMs on ECG-QA. To facilitate further research, we provide all code, datasets, and models open-source.

cs.LG↗

arXiv2025

Spezi Data Pipeline: Streamlining FHIR-based Interoperable Digital Health Data Workflows

The increasing adoption of digital health technologies has amplified the need for robust, interoperable solutions to manage complex healthcare data. We present the Spezi Data Pipeline, an open-source Python toolkit designed to streamline the analysis of digital health data, from secure access and retrieval to processing, visualization, and export. The Pipeline is integrated into the larger Stanford Spezi open-source ecosystem for developing research and translational digital health software systems. Leveraging HL7 FHIR-based data representations, the pipeline enables standardized handling of diverse data types--including sensor-derived observations, ECG recordings, and clinical questionnaires--across research and clinical environments. We detail the modular system architecture and demonstrate its application using real-world data from the PAWS at Stanford University, in which the pipeline facilitated efficient extraction, transformation, and clinician-driven review of Apple Watch ECG data, supporting annotation and comparative analysis alongside traditional monitors. By reducing the need for bespoke development and enhancing workflow efficiency, the Spezi Data Pipeline advances the scalability and interoperability of digital health research, ultimately supporting improved care delivery and patient outcomes.

cs.DB↗

arXiv2025

GPTCoach: Towards LLM-Based Physical Activity Coaching

Mobile health applications show promise for scalable physical activity promotion but are often insufficiently personalized. In contrast, health coaching offers highly personalized support but can be prohibitively expensive and inaccessible. This study draws inspiration from health coaching to explore how large language models (LLMs) might address personalization challenges in mobile health. We conduct formative interviews with 12 health professionals and 10 potential coaching recipients to develop design principles for an LLM-based health coach. We then built GPTCoach, a chatbot that implements the onboarding conversation from an evidence-based coaching program, uses conversational strategies from motivational interviewing, and incorporates wearable data to create personalized physical activity plans. In a lab study with 16 participants using three months of historical data, we find promising evidence that GPTCoach gathers rich qualitative information to offer personalized support, with users feeling comfortable sharing concerns. We conclude with implications for future research on LLM-based physical activity support.

cs.HC↗

arXiv2025

Toward Scalable Access to Neurodevelopmental Screening: Insights, Implementation, and Challenges

Children with neurodevelopmental disorders require timely intervention to improve long-term outcomes, yet early screening remains inaccessible in many regions. A scalable solution integrating standardized assessments with physiological data collection, such as electroencephalogram (EEG) recordings, could enable early detection in routine settings by non-specialists. To address this, we introduce NeuroNest, a mobile and cloud-based platform for large-scale EEG data collection, neurodevelopmental screening, and research. We provide a comprehensive review of existing behavioral and biomarker-based approaches, consumer-grade EEG devices, and emerging machine learning techniques. NeuroNest integrates low-cost EEG devices with digital screening tools, establishing a scalable, open-source infrastructure for non-invasive data collection, automated analysis, and interoperability across diverse hardware. Beyond the system architecture and reference implementation, we highlight key challenges in EEG data standardization, device interoperability, and bridging behavioral and physiological assessments. Our findings emphasize the need for future research on standardized data exchange, algorithm validation, and ecosystem development to expand screening accessibility. By providing an extensible, open-source system, NeuroNest advances machine learning-based early detection while fostering collaboration in screening technologies, clinical applications, and public health.

eess.SP↗

arXiv2025

Medicine on the Edge: Comparative Performance Analysis of On-Device LLMs for Clinical Reasoning

The deployment of Large Language Models (LLM) on mobile devices offers significant potential for medical applications, enhancing privacy, security, and cost-efficiency by eliminating reliance on cloud-based services and keeping sensitive health data local. However, the performance and accuracy of on-device LLMs in real-world medical contexts remain underexplored. In this study, we benchmark publicly available on-device LLMs using the AMEGA dataset, evaluating accuracy, computational efficiency, and thermal limitation across various mobile devices. Our results indicate that compact general-purpose models like Phi-3 Mini achieve a strong balance between speed and accuracy, while medically fine-tuned models such as Med42 and Aloe attain the highest accuracy. Notably, deploying LLMs on older devices remains feasible, with memory constraints posing a greater challenge than raw processing power. Our study underscores the potential of on-device LLMs for healthcare while emphasizing the need for more efficient inference and models tailored to real-world clinical reasoning.

cs.CL↗

arXiv2024

Dynamic Fog Computing for Enhanced LLM Execution in Medical Applications

The ability of large language models (LLMs) to transform, interpret, and comprehend vast quantities of heterogeneous data presents a significant opportunity to enhance data-driven care delivery. However, the sensitive nature of protected health information (PHI) raises valid concerns about data privacy and trust in remote LLM platforms. In addition, the cost associated with cloud-based artificial intelligence (AI) services continues to impede widespread adoption. To address these challenges, we propose a shift in the LLM execution environment from opaque, centralized cloud providers to a decentralized and dynamic fog computing architecture. By executing open-weight LLMs in more trusted environments, such as the user's edge device or a fog layer within a local network, we aim to mitigate the privacy, trust, and financial challenges associated with cloud-based LLMs. We further present SpeziLLM, an open-source framework designed to facilitate rapid and seamless leveraging of different LLM execution layers and lowering barriers to LLM integration in digital health applications. We demonstrate SpeziLLM's broad applicability across six digital health applications, showcasing its versatility in various healthcare settings.

cs.CL↗

arXiv2024

LLM on FHIR -- Demystifying Health Records

Objective: To enhance health literacy and accessibility of health information for a diverse patient population by developing a patient-centered artificial intelligence (AI) solution using large language models (LLMs) and Fast Healthcare Interoperability Resources (FHIR) application programming interfaces (APIs). Materials and Methods: The research involved developing LLM on FHIR, an open-source mobile application allowing users to interact with their health records using LLMs. The app is built on Stanford's Spezi ecosystem and uses OpenAI's GPT-4. A pilot study was conducted with the SyntheticMass patient dataset and evaluated by medical experts to assess the app's effectiveness in increasing health literacy. The evaluation focused on the accuracy, relevance, and understandability of the LLM's responses to common patient questions. Results: LLM on FHIR demonstrated varying but generally high degrees of accuracy and relevance in providing understandable health information to patients. The app effectively translated medical data into patient-friendly language and was able to adapt its responses to different patient profiles. However, challenges included variability in LLM responses and the need for precise filtering of health data. Discussion and Conclusion: LLMs offer significant potential in improving health literacy and making health records more accessible. LLM on FHIR, as a pioneering application in this field, demonstrates the feasibility and challenges of integrating LLMs into patient care. While promising, the implementation and pilot also highlight risks such as inconsistent responses and the importance of replicable output. Future directions include better resource identification mechanisms and executing LLMs on-device to enhance privacy and reduce costs.

cs.CY↗

arXiv2023

The Path to a Modular and Standards-based Digital Health Ecosystem

Software engineering for digital health applications entails several challenges, including heterogeneous data acquisition, data standardization, software reuse, security, and privacy considerations. We explore these challenges and how our Stanford Spezi ecosystem addresses these challenges by providing a modular and standards-based open-source digital health ecosystem. Spezi enables developers to select and integrate modules according to their needs and facilitates an open-source community to democratize access to building digital health innovations.

cs.SE↗