Representation Matters in Longitudinal Affective Computing

Abstract

Longitudinal, in-the-wild, wearable sensing yields day-level physiology, sleep, activity, and environmental streams, whereas affect and cognition are labeled only episodically (per waves). We recast this cadence mismatch as a temporal representation problem and compare three wave-level mappings from dense histories to sparse labels: levels (within-wave summaries), absolute drift (change across waves), and proportional drift. Using almost a year of data from 82 adults in the Providemus alz study, we model 21 affect and cognition outcomes. Day-scale signals are reduced to compact wave-level descriptors (central tendency, dispersion, and distributional shape) and learned with four regressors under two orthogonal evaluation axes: leave-one-subject-out and leave-one-wave-out. Performance is reported as scaled MAE using both mean and median across folds. Differences emerge: affective states are best predicted by wave-to-wave absolute drift, whereas cognitive performance aligns with within-wave levels, reflecting emotion dynamic theories. Across windowing features, shape descriptors (e.g., minima, kurtosis) carry more signal than simple means/medians. We contribute a representation triad for sparse-label modelling, a wave-level feature schema applicable on-device, and a dual-axis reporting practice that separates cross-participant generalization from temporal robustness. These results convert temporal representation from an implicit preprocessing step into an explicit, testable design choice for real-world affective-computing applications in brain health.