arXiv subjects

Ivan Liu

Publications and source records attributed to Ivan Liu.

3 recordsLinked to original sources

arXiv2024

Generative Technology for Human Emotion Recognition: A Scope Review

Affective computing stands at the forefront of artificial intelligence (AI), seeking to imbue machines with the ability to comprehend and respond to human emotions. Central to this field is emotion recognition, which endeavors to identify and interpret human emotional states from different modalities, such as speech, facial images, text, and physiological signals. In recent years, important progress has been made in generative models, including Autoencoder, Generative Adversarial Network, Diffusion Model, and Large Language Model. These models, with their powerful data generation capabilities, emerge as pivotal tools in advancing emotion recognition. However, up to now, there remains a paucity of systematic efforts that review generative technology for emotion recognition. This survey aims to bridge the gaps in the existing literature by conducting a comprehensive analysis of over 320 research papers until June 2024. Specifically, this survey will firstly introduce the mathematical principles of different generative models and the commonly used datasets. Subsequently, through a taxonomy, it will provide an in-depth analysis of how generative techniques address emotion recognition based on different modalities in several aspects, including data augmentation, feature extraction, semi-supervised learning, cross-domain, etc. Finally, the review will outline future research directions, emphasizing the potential of generative models to advance the field of emotion recognition and enhance the emotional intelligence of AI systems.

cs.LG↗

arXiv2024

A Finger on the Pulse of Cardiovascular Health: Estimating Blood Pressure with Smartphone Photoplethysmography-Based Pulse Waveform Analysis

Utilizing mobile phone cameras for continuous blood pressure (BP) monitoring presents a cost-effective and accessible approach, yet it is challenged by limitations in accuracy and interpretability. This study introduces four innovative strategies to enhance smartphone-based photoplethysmography for BP estimation (SPW-BP), addressing the interpretability-accuracy dilemma. First, we employ often-neglected data-quality improvement techniques, such as height normalization, corrupt data removal, and boundary signal reconstruction. Second, we conduct a comprehensive analysis of thirty waveform indicators across three categories to identify the most predictive features. Third, we use SHapley Additive exPlanations (SHAP) analysis to ensure the transparency and explainability of machine learning outcomes. Fourth, we utilize Bland-Altman analysis alongside AAMI and BHS standards for comparative evaluation. Data from 127 participants demonstrated a significant correlation between smartphone-captured waveform features and those from standard BP monitoring devices. Employing multiple linear regression within a cross-validation framework, waveform variables predicted systolic blood pressure (SBP) with a mean absolute error (MAE) of 3.08-16.64 mmHg and diastolic blood pressure (DBP) with an MAE of 2.86-13.16 mmHg. Further application of Random Forest models significantly improved the prediction MAE for SBP to 2.61-15.21 mmHg and for DBP to 2.14-11.22 mmHg, indicating enhanced predictive accuracy. Correlation and SHAP analysis identified key features for improving BP estimation. However, Bland-Altman analysis revealed systematic biases, and MAE analysis showed that the results did not meet AAMI and BHS accuracy standards. Our findings highlight the potential of SPW-BP, yet suggest that smartphone PPG technology is not yet a viable alternative to traditional medical devices for BP measurement.

eess.SP↗

arXiv2024

Your blush gives you away: detecting hidden mental states with remote photoplethysmography and thermal imaging

Multimodal emotion recognition techniques are increasingly essential for assessing mental states. Image-based methods, however, tend to focus predominantly on overt visual cues and often overlook subtler mental state changes. Psychophysiological research has demonstrated that HR and skin temperature are effective in detecting ANS activities, thereby revealing these subtle changes. However, traditional HR tools are generally more costly and less portable, while skin temperature analysis usually necessitates extensive manual processing. Advances in remote-PPG and automatic thermal ROI detection algorithms have been developed to address these issues, yet their accuracy in practical applications remains limited. This study aims to bridge this gap by integrating r-PPG with thermal imaging to enhance prediction performance. Ninety participants completed a 20-minute questionnaire to induce cognitive stress, followed by watching a film aimed at eliciting moral elevation. The results demonstrate that the combination of r-PPG and thermal imaging effectively detects emotional shifts. Using r-PPG alone, the prediction accuracy was 77% for cognitive stress and 61% for moral elevation, as determined by SVM. Thermal imaging alone achieved 79% accuracy for cognitive stress and 78% for moral elevation, utilizing a RF algorithm. An early fusion strategy of these modalities significantly improved accuracies, achieving 87% for cognitive stress and 83% for moral elevation using RF. Further analysis, which utilized statistical metrics and explainable machine learning methods including SHAP, highlighted key features and clarified the relationship between cardiac responses and facial temperature variations. Notably, it was observed that cardiovascular features derived from r-PPG models had a more pronounced influence in data fusion, despite thermal imaging's higher predictive accuracy in unimodal analysis.

cs.CY↗