
Multimodal emotion recognition Multimodal physiological and behavioral emotion recognition is of critical importance in affective computing, human-computer interaction (HCI), and mental-health analytics. However, current deep learning models generally do not take modalities into account (disregarding their causal and temporal inter-dependencies) and ignore personality-based variability that is essential to realistic affect modelling. To address these shortcomings, the given paper proposes CausalGraph-EmotionNet, a personality-conscious causal graph transformer, which combines Neural-ODE-based temporal evolution with causal attention-assisted multimodal fusion. The AFFECT data of each modality (EEG, electrodermal activity, facial activity, eye gaze, pupil dilation, and cursor movement) is modeled as a dynamic causal graph the time-varying connectivity of which reflects time-varying functional and directional interactions. The merged embeddings are optimized by personality-conditioned causal attention systems, which allows making individualized and interpretable inferences about emotions. Large-scale experiments on the AFFET dataset indicate that CausalGraph-EmotionNet has 84.6% accuracy and 80.8% macro-F1, outperforming CNN, RNN, GCN, Transformer and PhysioGraph-Transformer. The model significantly enhances the identification of more complex affective conditions like fear and disgust, it is resilient to a 40% loss in modality and has interpretable causal maps that bridge personality dimensions and modality salience. The findings make CausalGraph-EmotionNet a state-of-the-art, explainable and causally motivated architecture of multimodal emotion recognition - a unification of data-driven learning with psychologically relevant causal inferences.
Affective Computing, High-Level Resources, Emotion Recognition, Causal Graph Transformer, Neural ODE, Personality-Sensitive Fusion, Multimodal Learning, Explainable AI Physiological Signals AFFEC Dataset.
Affective Computing, High-Level Resources, Emotion Recognition, Causal Graph Transformer, Neural ODE, Personality-Sensitive Fusion, Multimodal Learning, Explainable AI Physiological Signals AFFEC Dataset.
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
