학술논문

Marine Snow Simulation and Elimination in Video
Document Type
Conference
Source
2023 IEEE Underwater Technology (UT) Underwater Technology (UT), 2023 IEEE. :1-10 Mar, 2023
Subject
Fields, Waves and Electromagnetics
Power, Energy and Industry Applications
Robotics and Control Systems
Training
Autonomous underwater vehicles
Recurrent neural networks
Snow
Aggregates
Training data
Data models
marine snow
underwater robotics
sim2real
deep learning
computer vision
Language
Abstract
Marine snow is formed by the aggregation of small particles of organic debris and inorganic sediments that settle through the water column and give the appearance of snow falling through the sky. Although critical to aquatic ecosystems, marine snow can potentially compromise underwater computer vision tasks. This works begins with a survey of techniques for overcoming such issues. Since many recent techniques rely on machine learning, which in turn relies on large amounts of training data, we also survey current approaches at generating such data. After discussing opportunities in existing approaches, we present novel solutions for simulation, isolation, and removal of marine snow in both images and video. Unlike other approaches, we focus on simulating marine snow in industrial scenarios with moving cameras and photorealistic discoloration, lens distortion, and motion blur typical in imagery collected by underwater robots. To remove the marine snow, we use only synthetic data, including a new dataset that we share online, to train a convolutional autoencoder extended to consider temporal information in consecutive video frames. Experiments are then conducted on two real datasets – one collected by an autonomous underwater vehicle in a shallow lake and the other by a remotely-operated vehicle in the deep sea.