Unimodal to Multimodal Emotion Recognition: A Systematic Review

Abstract

As Artificial intelligence is the fastest growing field for interdisciplinary research field in which computer science techniques can be implemented to solve the problems of social science. Emotion recognition is very challenging work as every person will express his/her emotion in different ways. The primary goal of this paper is to present the detailed study and critical analysis of existing unimodal recognition techniques like expression recognition from still images or video, audio expression recognition from available speech data , text expression from available social media post and psychological signals expression from EEG,EDA and ECG data. As unimodal emotion analysis is suffering from lower accuracy and lack of reliability, the multimodal emotion fusion is essential. This paper summarized the background of the multi modal fusion with existing data sets, feature extraction methods, different deep learning models they used and various fusion techniques like feature level or decision level.