arXiv subjects

Hanna Drimalla

Publications and source records attributed to Hanna Drimalla.

8 recordsLinked to original sources

arXiv2026

Beyond Contact Sensors: Deep learning with Pseudo-Labeling for remote Photoplethysmography

Heart rate is a critical biomarker of health, and remote photoplethysmography (rPPG) enables its contactless estimation from video data for telemedicine applications. Recent advancements in deep learning based rPPG methods achieve state-of-the-art results, outperforming classical signal-processing methods in complex scenarios. However, deep learning methods depend on datasets with precise synchronization between videos and ground truth signals collected via contact sensors, whereas signal-processing-based methods do not. To address this dependence on labeled datasets, which are labor-intensive to collect, we investigate under which circumstances pseudo-labels extracted using unsupervised signal-processing methods can replace contact sensors labels for training deep learning methods. Our systematic evaluations found that for datasets with imperfect synchronization, the pseudo-label approach outperforms supervised training on contact sensors. For datasets with good synchronization, results are mixed: within-dataset evaluation shows no significant difference between training methods, while cross-dataset evaluation favors supervised training. However, removing a single outlier participant significantly improves the pseudo-label approach's cross-dataset performance, highlighting the importance of label quality. These results demonstrate that signal-processing methods can generate valid training signals for deep learning models, reducing dependency on labor-intensive dataset collection while maintaining competitive performance.

cs.LG↗

arXiv2026

What Clinicians Need: Designing, Developing and Evaluating an AI-Based Decision Support System for Autism Assessment

AI methods promise to support autism spectrum condition (ASC) diagnostics in adults, a complex and time-consuming process, that is characterized by a shortage of specialized clinicians. To date, clinicians' needs and their interaction with such AI-based support remain underexplored. Our work aims to develop and evaluate an AI-based clinical decision support system (CDSS) for ASC assessment, and to investigate how it impacts clinicians' decision-making. By interviewing clinicians of varying experience levels, we identified five challenges and derived design strategies. Based on that, we developed SIT-CARE, a CDSS, which provides AI-based recommendations and data visualizations of clinically relevant nonverbal behavior. Through an evaluation study with newly recruited clinicians, we found that SIT-CARE led to different decision paths in regard to the ASC assessment, which are reflected in clinicians' mental models and decision changes. Overall, SIT-CARE demonstrated potential in improving initial diagnostic assessments, supporting in-depth diagnosis and empowering less experienced clinicians.

cs.HC↗

arXiv2026

Video-based Social Interaction Behavior Analysis with the Simulated Interaction Task for Children (Kids-SIT)

Accurately quantifying children's social interaction behavior is part of understanding their cognitive and emotional development, as well as mental health conditions. Kids-SIT is a web-based tool designed to computationally analyse children's behaviors by engaging them in a standardized video conversation while their responses are video recorded. In a pre-registered study with 21 healthy children and 12 children diagnosed with social anxiety disorder (SAD), aged 9-14 years, we assess its potential as an accessible paradigm for automated analysis of children's social interaction behavior. We evaluate whether the Kids-SIT can elicit naturalistic interaction patterns in healthy children, and how well automatic feature extraction methods can detect these patterns. We analyse children's subjective impressions, verbal responses, and non-verbal behaviors. Non-verbal behaviors were manually annotated and, independently, automatically extracted using state-of-the-art methods. In an exploratory analysis, we further assess whether automatically extracted features can distinguish between children with and without SAD. Verbal responses and post-hoc impressions indicate that the Kids-SIT elicits natural social interaction behavior. Non-verbal behavior aligned with this pattern: children looked at their interaction partner most of the time, particularly while listening rather than speaking. Smiling and gazing toward the partner occurred more frequently during the person-directed liked and disliked parts of the conversation than during the picture-description phase. These patterns were captured by both annotations and computational methods. Automatically extracted features enabled above-chance differentiation between children with and without SAD. Our results underscore the potential of the Kids-SIT for analysing children's social interaction behavior, with applicability extending to clinical contexts.

cs.HC↗

arXiv2026

Automatic Detection of Stress from Speech in the Trier Social Stress Test

Automatically detecting stress in speech provides an unobtrusive way to gain insights relevant to behavioral research or clinical assessment. This study investigates the automatic differentiation between a stressful and non-stressful situation, and the prediction of physiological and affective stress responses. Speech data was collected from 50 participants who either completed the Trier Social Stress Test (TSST) or a non-stressful control condition. With a processing pipeline that included speaker diarization and machine learning models, we achieved stress detection performance significantly above a mean baseline. Moreover, relevant physiological and affective stress responses were partially predictable from acoustic-prosodic features. Feature-importance analyses identified the most informative predictors contributing to model performance. The findings demonstrate that speech can serve as a meaningful and unobtrusive indicator of multiple dimensions of the human stress response.

cs.LG↗

arXiv2026

Sorry for the late reply: Response times and reciprocity in WhatsApp and Instagram chats

Chat communication is often fast-paced, creating the expectation of quick replies. While the timing of exchanges is known to foster closeness and enjoyment, it remains largely unexplored whether chat partners with strong ties reciprocate each other's response times. Using 3.4 million messages from 889 chats across 97 donations of anonymous WhatsApp and Instagram chats, we analyzed response times, their balance between chat partners, and its stability over time. To our knowledge, this is the first study to examine response speed as an expression of reciprocity, bridging a key aspect of online communication with a fundamental principle of social interactions. We found that around 70% of WhatsApp and 44% of Instagram messages were answered within five minutes, confirming the fast pace of instant messaging. Overall, the response speed between chat partners was similar. The response speed similarity was evident both in the overall response-time distributions of chat partners assessed with Jensen-Shannon distance and in the steep regression slopes (0.786 for WhatsApp and 0.796 for Instagram) linking one person's probability of responding within five minutes to the partner's corresponding probability. Importantly, the dispersion of response time similarity over months showed that this balance persists over time. Our results position response time balance as a marker of reciprocity in computer-mediated communication, offering a new way to quantitatively study this fundamental principle of social interaction. We suggest using response speed balance as a complementary metric in the analysis of relationship dynamics, such as the strengthening or weakening of social ties.

cs.SI↗

arXiv2025

Improving Autism Detection with Multimodal Behavioral Analysis

Due to the complex and resource-intensive nature of diagnosing Autism Spectrum Condition (ASC), several computer-aided diagnostic support methods have been proposed to detect autism by analyzing behavioral cues in patient video data. While these models show promising results on some datasets, they struggle with poor gaze feature performance and lack of real-world generalizability. To tackle these challenges, we analyze a standardized video dataset comprising 168 participants with ASC (46% female) and 157 non-autistic participants (46% female), making it, to our knowledge, the largest and most balanced dataset available. We conduct a multimodal analysis of facial expressions, voice prosody, head motion, heart rate variability (HRV), and gaze behavior. To address the limitations of prior gaze models, we introduce novel statistical descriptors that quantify variability in eye gaze angles, improving gaze-based classification accuracy from 64% to 69% and aligning computational findings with clinical research on gaze aversion in ASC. Using late fusion, we achieve a classification accuracy of 74%, demonstrating the effectiveness of integrating behavioral markers across multiple modalities. Our findings highlight the potential for scalable, video-based screening tools to support autism assessment.

cs.CV↗

arXiv2025

Generalization of Video-Based Heart Rate Estimation Methods To Low Illumination and Elevated Heart Rates

Heart rate is a physiological signal that provides information about an individual's health and affective state. Remote photoplethysmography (rPPG) allows the estimation of this signal from video recordings of a person's face. Classical rPPG methods make use of signal processing techniques, while recent rPPG methods utilize deep learning networks. Methods are typically evaluated on datasets collected in well-lit environments with participants at resting heart rates. However, little investigation has been done on how well these methods adapt to variations in illumination and heart rate. In this work, we systematically evaluate representative state-of-the-art methods for remote heart rate estimation. Specifically, we evaluate four classical methods and four deep learning-based rPPG estimation methods in terms of their generalization ability to changing scenarios, including low lighting conditions and elevated heart rates. For a thorough evaluation of existing approaches, we collected a novel dataset called CHILL, which systematically varies heart rate and lighting conditions. The dataset consists of recordings from 45 participants in four different scenarios. The video data was collected under two different lighting conditions (high and low) and normal and elevated heart rates. In addition, we selected two public datasets to conduct within- and cross-dataset evaluations of the rPPG methods. Our experimental results indicate that classical methods are not significantly impacted by low-light conditions. Meanwhile, some deep learning methods were found to be more robust to changes in lighting conditions but encountered challenges in estimating high heart rates. The cross-dataset evaluation revealed that the selected deep learning methods underperformed when influencing factors such as elevated heart rates and low lighting conditions were not present in the training set.

cs.LG↗

arXiv2023

Towards Interpretability in Audio and Visual Affective Machine Learning: A Review

Machine learning is frequently used in affective computing, but presents challenges due the opacity of state-of-the-art machine learning methods. Because of the impact affective machine learning systems may have on an individual's life, it is important that models be made transparent to detect and mitigate biased decision making. In this regard, affective machine learning could benefit from the recent advancements in explainable artificial intelligence (XAI) research. We perform a structured literature review to examine the use of interpretability in the context of affective machine learning. We focus on studies using audio, visual, or audiovisual data for model training and identified 29 research articles. Our findings show an emergence of the use of interpretability methods in the last five years. However, their use is currently limited regarding the range of methods used, the depth of evaluations, and the consideration of use-cases. We outline the main gaps in the research and provide recommendations for researchers that aim to implement interpretable methods for affective machine learning.

cs.LG↗