arXiv subjects

Mathias Funk

Publications and source records attributed to Mathias Funk.

11 recordsLinked to original sources

arXiv2026

Zero-Shot Respiratory Sound Classification through LLM-Augmented Audio-Text Alignment

Self-supervised respiratory encoders lack semantic grounding in clinical domain needed for zero-shot inference, limiting their utility without task-specific labeled data. We propose a framework that aligns these encoders with medical terminology in a shared latent space turning them into a zero-shot-capable foundation model. To address paired data scarcity, we use a medical LLM to synthesize structured reports from metadata, creating dense semantic anchors for contrastive learning. Our training combines a sigmoid-based contrastive loss with encoder's native SSL objective and similarity-aware negative sampling to sharpen pathological boundaries. Across 9 tasks on 6 datasets, our method achieves a 61.3% mean zero-shot AUC, surpassing CLAP (51.4%) and Qwen2-Audio (54.9%) while reaching the highest linear probing AUC (71.6%) with only 43% of data used by full-scale baselines, showing that structured semantic alignment outperforms large-scale, general-purpose models in clinical diagnostics.

cs.CL↗

arXiv2026

StethoLM: Audio Language Model for Cardiopulmonary Analysis Across Clinical Tasks

Listening to heart and lung sounds - auscultation - is one of the first and most fundamental steps in a clinical examination. Despite being fast and non-invasive, it demands years of experience to interpret subtle audio cues. Recent deep learning methods have made progress in automating cardiopulmonary sound analysis, yet most are restricted to simple classification and offer little clinical interpretability or decision support. We present StethoLM, the first audio-language model specialized for cardiopulmonary auscultation, capable of performing instruction-driven clinical tasks across the full spectrum of auscultation analysis. StethoLM integrates audio encoding with a medical language model backbone and is trained on StethoBench, a comprehensive benchmark comprising 77,027 instruction-response pairs synthesized from 16,125 labeled cardiopulmonary recordings spanning seven clinical task categories: binary classification, detection, reporting, reasoning, differential diagnosis, comparison, and location-based analysis. Through multi-stage training that combines supervised fine-tuning and direct preference optimization, StethoLM achieves substantial gains in performance and robustness on out-of-distribution data. Our work establishes a foundation for instruction-following AI systems in clinical auscultation.

cs.LG↗

arXiv2026

Conversing with Objects toward Fluid Human and Artificial Identities during Life Transitions

People's identities change during life transitions, e.g., studying abroad. They bring everyday objects that embody memories and reflect their identities during such moves. To assist in these transitions, we ask how people's human identities could be influenced by their objects through an artificial agent. This paper presents an exploratory research-through-design study around how people undergoing life transitions experience conversing with their everyday objects through a chatbot. Drawing on a two-week field deployment and interviews with 12 participants, we contribute (1) a conceptualization of 'trans-embodiment' describing the asynchronous imagination of object and human identities on the chatbot, (2) empirical evidence of the resulting emotional and reflective experiences, and (3) three types of object identities for designing conversational agents that role-play objects. Our contributions sum up to triangulating human-agent-object identity as trans-embodiment in supporting life transitions.

cs.HC↗

arXiv2025

Helmsman: Autonomous Synthesis of Federated Learning Systems via Collaborative LLM Agents

Federated Learning (FL) offers a powerful paradigm for training models on decentralized data, but its promise is often undermined by the immense complexity of designing and deploying robust systems. The need to select, combine, and tune strategies for multifaceted challenges like data heterogeneity and system constraints has become a critical bottleneck, resulting in brittle, bespoke solutions. To address this, we introduce Helmsman, a novel multi-agent system that automates the end-to-end synthesis of federated learning systems from high-level user specifications. It emulates a principled research and development workflow through three collaborative phases: (1) interactive human-in-the-loop planning to formulate a sound research plan, (2) modular code generation by supervised agent teams, and (3) a closed-loop of autonomous evaluation and refinement in a sandboxed simulation environment. To facilitate rigorous evaluation, we also introduce AgentFL-Bench, a new benchmark comprising 16 diverse tasks designed to assess the system-level generation capabilities of agentic systems in FL. Extensive experiments demonstrate that our approach generates solutions competitive with, and often superior to, established hand-crafted baselines. Our work represents a significant step towards the automated engineering of complex decentralized AI systems.

cs.AI↗

arXiv2025

Sensemaking Through Making: Developing Clinical Domain Knowledge by Crafting Synthetic Datasets and Prototyping System Architectures

Designers have ample opportunities to impact the healthcare domain. However, hospitals are often closed ecosystems that pose challenges in engaging clinical stakeholders, developing domain knowledge, and accessing relevant systems and data. In this paper, we introduce a making-oriented approach to help designers understand the intricacies of their target healthcare context. Using Remote Patient Monitoring (RPM) as a case study, we explore how manually crafting synthetic datasets based on real-world observations enables designers to learn about complex data-driven healthcare systems. Our process involves observing and modeling the real-world RPM context, crafting synthetic datasets, and iteratively prototyping a simplified RPM system that balances contextual richness and intentional abstraction. Through this iterative process of sensemaking through making, designers can still develop context familiarity when direct access to the actual healthcare system is limited. Our approach emphasizes the value of hands-on interaction with data structures to support designers in understanding opaque healthcare systems.

cs.HC↗

arXiv2025

FAST: Federated Active Learning with Foundation Models for Communication-efficient Sampling and Training

Federated Active Learning (FAL) has emerged as a promising framework to leverage large quantities of unlabeled data across distributed clients while preserving data privacy. However, real-world deployments remain limited by high annotation costs and communication-intensive sampling processes, particularly in a cross-silo setting, when clients possess substantial local datasets. This paper addresses the crucial question: What is the best practice to reduce communication costs in human-in-the-loop learning with minimal annotator effort? Existing FAL methods typically rely on iterative annotation processes that separate active sampling from federated updates, leading to multiple rounds of expensive communication and annotation. In response, we introduce FAST, a two-pass FAL framework that harnesses foundation models for weak labeling in a preliminary pass, followed by a refinement pass focused exclusively on the most uncertain samples. By leveraging representation knowledge from foundation models and integrating refinement steps into a streamlined workflow, FAST substantially reduces the overhead incurred by iterative active sampling. Extensive experiments on diverse medical and natural image benchmarks demonstrate that FAST outperforms existing FAL methods by an average of 4.36% while reducing communication rounds eightfold under a limited 5% labeling budget.

cs.LG↗

arXiv2024

Assembling the Puzzle: Exploring Collaboration and Data Sensemaking in Nursing Practices for Remote Patient Monitoring

Remote patient monitoring (RPM) involves the remote collection and transmission of patient health data, serving as a notable application of data-driven healthcare. This technology facilitates clinical monitoring and decision-making, offering benefits like reduced healthcare costs and improved patient outcomes. However, RPM also introduces challenges common to data-driven healthcare, such as additional data work that can disrupt clinician's workflow. This study explores the daily practices, collaboration mechanisms, and sensemaking processes of nurses in RPM through field observations and interviews with six stakeholders. Preliminary results indicate that RPM's scale-up pushes clinicians toward asynchronous collaboration. Data sensemaking is crucial for this type of collaboration, but existing technologies often create friction rather than support. This work provides empirical insights into clinical workflow in nursing practice, especially RPM. We suggest recognizing data sensemaking as a distinct nursing practice within data work and recommend further investigation into its role in the workflow of nurses in RPM.

cs.HC↗

arXiv2024

Collaboratively Learning Federated Models from Noisy Decentralized Data

Federated learning (FL) has emerged as a prominent method for collaboratively training machine learning models using local data from edge devices, all while keeping data decentralized. However, accounting for the quality of data contributed by local clients remains a critical challenge in FL, as local data are often susceptible to corruption by various forms of noise and perturbations, which compromise the aggregation process and lead to a subpar global model. In this work, we focus on addressing the problem of noisy data in the input space, an under-explored area compared to the label noise. We propose a comprehensive assessment of client input in the gradient space, inspired by the distinct disparity observed between the density of gradient norm distributions of models trained on noisy and clean input data. Based on this observation, we introduce a straightforward yet effective approach to identify clients with low-quality data at the initial stage of FL. Furthermore, we propose a noise-aware FL aggregation method, namely Federated Noise-Sifting (FedNS), which can be used as a plug-in approach in conjunction with widely used FL strategies. Our extensive evaluation on diverse benchmark datasets under different federated settings demonstrates the efficacy of FedNS. Our method effortlessly integrates with existing FL strategies, enhancing the global model's performance by up to 13.68% in IID and 15.85% in non-IID settings when learning from noisy decentralized data.

cs.LG↗

arXiv2022

Seeing through Things: Exploring the Design Space of Privacy-Aware Data-Enabled Objects

Increasing amounts of sensor-augmented research objects have been used in design research. We call these objects Data-Enabled Objects, which can be integrated into daily activities capturing data about people's detailed whereabouts, behaviours and routines. These objects provide data perspectives on everyday life for contextual design research. However, data-enabled objects are still computational devices with limited privacy awareness and nuanced data sharing. To better design data-enabled objects, we explore privacy design spaces by inviting 18 teams of undergraduate design students to re-design the same type of sensor-enabled home research camera. We developed the Connected Peekaboo Toolkit (CPT) to support the design teams in designing, building, and directly deploying their prototypes in real home studies. We conducted Thematic Analysis to analyse their outcomes which led us to interpret that privacy is not just an obstacle but can be a driver by unfolding an exploration of possible design spaces for data-enabled objects.

cs.HC↗

arXiv2020

Sonification of Facial Actions for Musical Expression

The central role of the face in social interaction and non-verbal communication suggests we explore facial action as a means of musical expression. This paper presents the design, implementation, and preliminary studies of a novel system utilizing face detection and optic flow algorithms to associate facial movements with sound synthesis in a topographically specific fashion. We report on our experience with various gesture-to-sound mappings and applications, and describe our preliminary experiments at musical performance using the system.

cs.HC↗

arXiv2020

Visual Exploration of Movement Relatedness for Multi-species Ecology Analysis

Advances in GPS telemetry technology have enabled analysis of animal movement in open areas. Ecologists today are utilizing modern analytic tools to study animal behaviors from large quantity of GPS coordinates. Analytic tools with automatic event extraction functionality can be used to investigate potential interactions between animals by locating relevant segments in movement trajectories. However, such automation can easily overlook the spatial, temporal, social context as well as potential data problems. To this end, this paper explores the visual presentations that also clarify the spatial-temporal contexts, social surroudings, as well as underlying data uncertainties of multi-species animal interactions. The outcome system presents the proximity-based, time-varying relatedness between animal entities through pairwise (PW) or individual-to-group (i-G) perspectives. Focusing on the relational aspects, we employ both static depictions and animations to communicate the travelling of individuals. Our contributions are a novel visualization system that helps investigate the subtle variations of long term spatial-temporal relatedness while considering small group patterns. Our evaluation with movement ecologists shows that the system gives them quick access to valuable clues in discovering insights into multi-species movements and signs of potential interactions.

cs.HC↗