학술논문
Evaluation Tool to Diagnose Faults and Discrepancy in Semi-Automated or Manual Annotations in Ultrasound Cine Loops (Videos)
Document Type
Conference
Author
Source
2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) Engineering in Medicine & Biology Society (EMBC), 2022 44th Annual International Conference of the IEEE. :525-528 Jul, 2022
Subject
Language
ISSN
2694-0604
Abstract
Good quality (annotated) data is one of the most important aspects of supervised deep learning. Tasks such as semantic segmentation have a huge data requirement in exchange for only satisfactory performance. Large-scale annotations spread across multiple annotators tends to create inconsistencies, as there are various manual and semi-automated techniques involved. This mandates an external evaluator or expert to check and narrow down the problematic annotations. Studies have shown that even marking a few instances wrong in classification can lead to a significant performance drop in the model (mislabeling only 10% of one class can degrade the total performance of all classes by up to 10%). It has been noticed that fault localization by a medical expert is one of the most expensive and time-consuming processes. In this paper, we propose a novel framework for detecting the inconsistencies in the annotation of every object/anatomy in a specific image. We leverage the power of semi-supervised deep learning models (STCN) to help produce high-quality data for AI segmentation algorithms. Evaluation using this algorithm has been shown to reduce annotation review time by at least 5 hours for just 1000 images, and the quality of ground truth data improved thereby increasing the performance of the model by almost 3%.