학술논문

Reliability Estimation of ML for Image Perception: A Lightweight Nonlinear Transformation Approach Based on Full Reference Image Quality Metrics
Document Type
Conference
Source
2023 IEEE 16th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC) MCSOC Embedded Multicore/Many-core Systems-on-Chip (MCSoC), 2023 IEEE 16th International Symposium on. :186-193 Dec, 2023
Subject
Components, Circuits, Devices and Systems
Computing and Processing
Robotics and Control Systems
Image quality
Measurement
Machine learning algorithms
Multicore processing
Perturbation methods
Brightness
Machine learning
AI
image quality metrics
nonlinear modeling
Language
ISSN
2771-3075
Abstract
As machine learning (ML) models for image perception continue to advance, ensuring their robustness and reliability under various real-world scenarios remains a significant challenge. Image quality factors, such as blur, brightness, and other environmental conditions, can significantly affect the performance of these algorithms, leading to inaccurate detection and potential failures in critical applications. In this paper, we propose a comprehensive diagnosis framework that leverages image quality metrics to assess and enhance the performance of these algorithms. To accomplish this goal, we deliberately introduce disturbances in parameters such as brightness, saturation, and other relevant factors. Subsequently, we compute a set of full-reference image quality metrics to evaluate the image quality after the perturbations. Once we have obtained the metrics, we apply a nonlinear transformation to these values. Based on the transformed metrics, we create a regression model that predicts the detection Intersection over Union (IOU). To validate our framework, we conducted experiments using three state-of-the-art machine learning models for object detection and instance segmentation. The models were subjected to various scenarios with different levels of image quality perturbations. Our experimental results clearly demonstrate the possibility of establishing a strong correlation between image quality metrics and the performance of the algorithms.