학술논문

Refining the Unseen: Self-supervised Two-stream Feature Extraction for Image Quality Assessment
Document Type
Conference
Source
2023 IEEE International Conference on Data Mining (ICDM) ICDM Data Mining (ICDM), 2023 IEEE International Conference on. :1193-1198 Dec, 2023
Subject
Communication, Networking and Broadcast Technologies
Computing and Processing
Image quality
Transfer learning
Self-supervised learning
Streaming media
Feature extraction
Distortion
Quality assessment
Image quality assessment
Contrastive learning
Representation learning
Image fidelity and structure
Language
ISSN
2374-8486
Abstract
The inadequacy of labeled datasets for image quality assessment has led to the development and popularity of self-supervised approaches. However, most existing self-supervised methods primarily focus on content and fidelity features extracted with convolutional neural networks, overlooking the crucial importance of structural features in quality assessment. To address this problem, we present a novel self-supervised two-stream feature extraction and representation approach. In our approach, the first stream leverages a contrastive learning framework to extract image fidelity features, while the second stream emphasizes structural features by incorporating an attention mechanism. This innovative combination results in a comprehensive feature representation for quality assessment. Moreover, our proposed method facilitates transfer learning, allowing the pre-trained two-stream model in the source domain to be seamlessly applied to target domains for quality regression. This compatibility with transfer learning enhances the adaptability and generalization of the model. Extensive experiments are carried out on three synthetic distortion datasets to validate the effectiveness of our approach. The results demonstrate that our work not only competes with state-of-the-art self-supervised methods but also outperforms some supervised approaches.