학술논문
다중 모달리티 의미론적 분할에서의 무작위 마스킹과 가중치 공유 기법
Random masking and weight sharing techniques in multimodal semantic segmentation
Random masking and weight sharing techniques in multimodal semantic segmentation
Document Type
Article
Text
Text
Author
유용현; 박효빈; 이상욱; 김유현; 김대한; 이규만; 최동걸; Yonghyeon Yu; Hyobin Park; Sangwook Lee; Yuhyun Kim; Daehan Kim; Kyuman Lee; Dong-Geol Choi
Source
한국차세대컴퓨팅학회 논문지, 04/30/2024, Vol. 20, Issue 2, p. 66-78
Subject
Language
Korean
ISSN
1975-681X
Abstract
Utilizing the multi-modality of RGB and thermal images is valuable in various situations where visibility is reduced due to bad weather, fog, or lighting conditions. In previous semantic segmentation work, high recognition performance has been achieved by combining features extracted from RGB images with features from thermal images, but the performance is limited because the output features are combined without considering the model weights according to the characteristics of the modalities. In this paper, we configure the networks to have common weights during the learning process of RGB and Thermal images, and configure the modality features with more information to lead the other modalities in the loss calculation step. We also configure complementary masking modules and residual networks to achieve a performance improvement of 12.25 on the FMB dataset.