학술논문
Assessment of Radiology Artificial Intelligence Software: A Validation and Evaluation Framework.
Document Type
Academic Journal
Author
Tanguay W; 60352Centre hospitalier de l'Université de Montréal, Montréal, QC, Canada.; Department of Radiology, Radiation Oncology and Nuclear Medicine, 25443Université de Montréal, Montréal, QC, Canada.; Acar P; 60352Centre hospitalier de l'Université de Montréal, Montréal, QC, Canada.; Department of Radiology, Radiation Oncology and Nuclear Medicine, 25443Université de Montréal, Montréal, QC, Canada.; Fine B; Department of Diagnostic Imaging, 5543Trillium Health Partners, Mississauga, ON, Canada.; Department of Medical Imaging, University of Toronto, Toronto, ON, Canada.; Abdolell M; Department of Radiology, Dalhousie University, Halifax, NS, Canada.; Gong B; Department of Radiology, Vancouver General Hospital, University of British Columbia, Vancouver, BC, Canada.; Cadrin-Chênevert A; Department of Medical Imaging, CISSS Lanaudière, Université Laval, Joliette, QC, Canada.; Chartrand-Lefebvre C; 60352Centre hospitalier de l'Université de Montréal, Montréal, QC, Canada.; Department of Radiology, Radiation Oncology and Nuclear Medicine, 25443Université de Montréal, Montréal, QC, Canada.; Chalaoui J; 60352Centre hospitalier de l'Université de Montréal, Montréal, QC, Canada.; Department of Radiology, Radiation Oncology and Nuclear Medicine, 25443Université de Montréal, Montréal, QC, Canada.; Gorgos A; 60352Centre hospitalier de l'Université de Montréal, Montréal, QC, Canada.; Department of Radiology, Radiation Oncology and Nuclear Medicine, 25443Université de Montréal, Montréal, QC, Canada.; Chin AS; 60352Centre hospitalier de l'Université de Montréal, Montréal, QC, Canada.; Department of Radiology, Radiation Oncology and Nuclear Medicine, 25443Université de Montréal, Montréal, QC, Canada.; Prénovault J; 60352Centre hospitalier de l'Université de Montréal, Montréal, QC, Canada.; Department of Radiology, Radiation Oncology and Nuclear Medicine, 25443Université de Montréal, Montréal, QC, Canada.; Guilbert F; 60352Centre hospitalier de l'Université de Montréal, Montréal, QC, Canada.; Department of Radiology, Radiation Oncology and Nuclear Medicine, 25443Université de Montréal, Montréal, QC, Canada.; Létourneau-Guillon L; 60352Centre hospitalier de l'Université de Montréal, Montréal, QC, Canada.; Department of Radiology, Radiation Oncology and Nuclear Medicine, 25443Université de Montréal, Montréal, QC, Canada.; Chong J; Department of Medical Imaging, Western University, London, ON, Canada.; Tang A; 60352Centre hospitalier de l'Université de Montréal, Montréal, QC, Canada.; Department of Radiology, Radiation Oncology and Nuclear Medicine, 25443Université de Montréal, Montréal, QC, Canada.
Source
Publisher: SAGE Publications Country of Publication: United States NLM ID: 8812910 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1488-2361 (Electronic) Linking ISSN: 08465371 NLM ISO Abbreviation: Can Assoc Radiol J Subsets: MEDLINE
Subject
Language
English
Abstract
Artificial intelligence (AI) software in radiology is becoming increasingly prevalent and performance is improving rapidly with new applications for given use cases being developed continuously, oftentimes with development and validation occurring in parallel. Several guidelines have provided reporting standards for publications of AI-based research in medicine and radiology. Yet, there is an unmet need for recommendations on the assessment of AI software before adoption and after commercialization. As the radiology AI ecosystem continues to grow and mature, a formalization of system assessment and evaluation is paramount to ensure patient safety, relevance and support to clinical workflows, and optimal allocation of limited AI development and validation resources before broader implementation into clinical practice. To fulfil these needs, we provide a glossary for AI software types, use cases and roles within the clinical workflow; list healthcare needs, key performance indicators and required information about software prior to assessment; and lay out examples of software performance metrics per software category. This conceptual framework is intended to streamline communication with the AI software industry and provide healthcare decision makers and radiologists with tools to assess the potential use of these software. The proposed software evaluation framework lays the foundation for a radiologist-led prospective validation network of radiology AI software. Learning Points: The rapid expansion of AI applications in radiology requires standardization of AI software specification, classification, and evaluation. The Canadian Association of Radiologists' AI Tech & Apps Working Group Proposes an AI Specification document format and supports the implementation of a clinical expert evaluation process for Radiology AI software.