부산대학교 도서관

The purpose of human multimodal sentiment analysis is to use data from multiple modalities to more accurately identify sentiment and emotion. Since the data between different modalities is usually heterogeneous. The main point of this research is to efficiently extract and fuse the data in each modality that is relevant to the other modalities. To better achieve this, this paper proposes the Tri-modalilties Fusion Network(TFN), a novel fusion network to fuse the tri-modalities representations. The model takes three pairs of modality groups as input, and in order to fully use the information of all modalities, each group has three modalities, one modality as the primary modal and the rest as auxiliary modalityies to enhance the primary modal. Experimental results on CMU-MOSI and CMU-MOSEI datasets for multimodal sentiment analysis demonstrates the superiority of our approach.