KANFeel: A Novel Kolmogorov-Arnold Network-Based Multimodal Emotion Recognition Framework

Multimodal emotion recognition (MER) has become a significant interdisciplinary research area in HCI by analyzing various data modalities. Recent studies have shown promising results in MER by fusing features across modalities. However, efficiency in real-world scenarios remains a challenge. To improve the model efficiency, Kolmogorov-Arnold Networks (KANs) have been proposed as efficient alternatives to Multi-Layer Perceptrons (MLPs). However, employing KANs in MER is still underexplored. This paper introduces KANFeel, a novel KAN-based MER framework that processes multimodal inputs to predict emotional states. Furthermore, we adopt the KAN-based model to replace the attention mechanism in the transformer, referred to as KANFeel-Attent, to achieve enhanced recognition performance. Comprehensive evaluations across three public datasets are conducted to analyze the efficiency improvements of KANFeel by comparing model parameters and training speeds. Finally, emotion recognition enhancements, measured through accuracy and F1-score, are validated using the KANFeel and KANFeel-Attent models against baseline and state-of-the-art methods.