SearcharxivSearch

arXiv subjects

Jefferson Ortega

Publications and source records attributed to Jefferson Ortega.

2 recordsLinked to original sources

Human-computer interactions predict mental health

Scalable assessments of mental illness remain a critical roadblock toward accessible and equitable care. Here, we show that everyday human-computer interactions encode high-dimensional information about self-reported psychological distress and wellbeing. We introduce MAILA, a MAchine-learning framework for Inferring Latent mental states from digital Activity. We trained MAILA on 18,200 cursor and touchscreen recordings labeled with 1.3 million mental-health self-reports collected from 9,500 participants. MAILA predicts dynamic mental states along 13 dimensions of distress and wellbeing, detects within-person changes over time, and resolves experimentally induced fluctuations in experienced arousal and valence. At the group level, MAILA recovers demographic and time-of-day patterns in self-reported mental health with high fidelity. MAILA also captures information only partially reflected in verbal self-report and, in a synthetic proof of concept, improves the ability of a frontier large language model to infer user mental health. By extracting signatures of psychological function that have so far remained untapped, MAILA provides the first large-scale, systematic proof of principle that human-computer interactions encode multidimensional and transferable information about mental health.

q-bio.NC

VEATIC: Video-based Emotion and Affect Tracking in Context Dataset

Human affect recognition has been a significant topic in psychophysics and computer vision. However, the currently published datasets have many limitations. For example, most datasets contain frames that contain only information about facial expressions. Due to the limitations of previous datasets, it is very hard to either understand the mechanisms for affect recognition of humans or generalize well on common cases for computer vision models trained on those datasets. In this work, we introduce a brand new large dataset, the Video-based Emotion and Affect Tracking in Context Dataset (VEATIC), that can conquer the limitations of the previous datasets. VEATIC has 124 video clips from Hollywood movies, documentaries, and home videos with continuous valence and arousal ratings of each frame via real-time annotation. Along with the dataset, we propose a new computer vision task to infer the affect of the selected character via both context and character information in each video frame. Additionally, we propose a simple model to benchmark this new computer vision task. We also compare the performance of the pretrained model using our dataset with other similar datasets. Experiments show the competing results of our pretrained model via VEATIC, indicating the generalizability of VEATIC. Our dataset is available at https://veatic.github.io.

cs.CV