학술논문
Exploring the role of an artificial intelligence chatbot on appendicitis management: an experimental study on ChatGPT.
Document Type
Academic Journal
Author
Gracias D; Department of Surgery, Townsville Hospital, Townsville, Queensland, Australia.; Siu A; Faculty of Medicine and Health, Central Clinical School, The University of Sydney, Sydney, New South Wales, Australia.; Surgical Outcomes Research Centre (SOuRCe), Royal Prince Alfred Hospital, Camperdown, New South Wales, Australia.; Concord Institute of Academic Surgery, Concord Hospital, Concord, New South Wales, Australia.; Seth I; Department of Surgery, Peninsula Health, Melbourne, Victoria, Australia.; Department of Surgery, Bendigo Health, Bendigo, Victoria, Australia.; Dooreemeah D; Department of Surgery, Bendigo Health, Bendigo, Victoria, Australia.; Lee A; Department of Surgery, Bendigo Health, Bendigo, Victoria, Australia.
Source
Publisher: Wiley-Blackwell Publishing Asia Country of Publication: Australia NLM ID: 101086634 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1445-2197 (Electronic) Linking ISSN: 14451433 NLM ISO Abbreviation: ANZ J Surg Subsets: MEDLINE
Subject
Language
English
Abstract
Background: Appendicitis is a common surgical condition that requires urgent medical attention. Recent advancements in artificial intelligence and large language processing, such as ChatGPT, have demonstrated potential in supporting healthcare management and scientific research. This study aims to evaluate the accuracy and comprehensiveness of ChatGPT's knowledge on appendicitis management.
Methods: Six questions related to appendicitis management were created by experienced RACS qualified general surgeons to assess ChatGPT's ability to provide accurate information. The criteria of ChatGPT answers' accuracy were compared with current healthcare guidelines for appendicitis and subjective evaluation by two RACS qualified General Surgeons. Additionally, ChatGPT was then asked to provide five high level evidence references to support its responses.
Results: ChatGPT provided clinically relevant information on appendicitis management, however, was inconsistent in doing so and often provided superficial information. Further to this, ChatGPT encountered difficulties in generating relevant references, with some being either non-existent or incorrect.
Conclusion: ChatGPT has the potential to provide timely and comprehensible medical information on appendicitis management to laypersons. However, its issue of inaccuracy in information and production of non-existent or erroneous references presents a challenge for researchers and clinicians who may inadvertently employ such information in their research or healthcare. Therefore, clinicians should exercise caution when using ChatGPT for these purposes.
(© 2023 Royal Australasian College of Surgeons.)
Methods: Six questions related to appendicitis management were created by experienced RACS qualified general surgeons to assess ChatGPT's ability to provide accurate information. The criteria of ChatGPT answers' accuracy were compared with current healthcare guidelines for appendicitis and subjective evaluation by two RACS qualified General Surgeons. Additionally, ChatGPT was then asked to provide five high level evidence references to support its responses.
Results: ChatGPT provided clinically relevant information on appendicitis management, however, was inconsistent in doing so and often provided superficial information. Further to this, ChatGPT encountered difficulties in generating relevant references, with some being either non-existent or incorrect.
Conclusion: ChatGPT has the potential to provide timely and comprehensible medical information on appendicitis management to laypersons. However, its issue of inaccuracy in information and production of non-existent or erroneous references presents a challenge for researchers and clinicians who may inadvertently employ such information in their research or healthcare. Therefore, clinicians should exercise caution when using ChatGPT for these purposes.
(© 2023 Royal Australasian College of Surgeons.)