KOR

e-Article

A Dual-Branch Deep Learning Architecture for Multisensor and Multitemporal Remote Sensing Semantic Segmentation
Document Type
Periodical
Source
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing Selected Topics in Applied Earth Observations and Remote Sensing, IEEE Journal of. 16:2147-2162 2023
Subject
Geoscience
Signal Processing and Analysis
Power, Energy and Industry Applications
Spatial resolution
Feature extraction
Time series analysis
Analytical models
Image sensors
Data mining
Training
Deep learning
Deep learning (DL) classification
multiresolution
multisensor data
multitemporal images
remote sensing (RS)
very-high-resolution (VHR) images
Language
ISSN
1939-1404
2151-1535
Abstract
Multisensor data analysis allows exploiting heterogeneous data regularly acquired by the many available remote sensing (RS) systems. Machine- and deep-learning methods use the information of heterogeneous sources to improve the results obtained by using single-source data. However, the state-of-the-art methods analyze either the multiscale information of multisensor multiresolution images or the time component of image time series. We propose a supervised deep-learning classification method that jointly performs a multiscale and multitemporal analysis of RS multitemporal images acquired by different sensors. The proposed method processes very-high-resolution (VHR) images using a residual network with a wide receptive field that handles geometrical details and multitemporal high-resolution (HR) image using a 3-D convolutional neural network that analyzes both the spatial and temporal information. The multiscale and multitemporal features are processed together in a decoder to retrieve a land-cover map. We tested the proposed method on two multisensor and multitemporal datasets. One is composed of VHR orthophotos and Sentinel-2 multitemporal images for pasture classification, and another is composed of VHR orthophotos and Sentinel-1 multitemporal images. Results proved the effectiveness of the proposed classification method.