학술논문

A Framework for Evaluating MRC Approaches with Unanswerable Questions
Document Type
Conference
Source
2022 IEEE 18th International Conference on e-Science (e-Science) ESCIENCE e-Science (e-Science), 2022 IEEE 18th International Conference on. :435-436 Oct, 2022
Subject
Communication, Networking and Broadcast Technologies
Computing and Processing
Semantics
Neural networks
Robustness
Data models
Question answering (information retrieval)
Task analysis
Data Augmentation
Machine Reading Comprehension
Question Answering
Robustness Evaluation
Unanswerable Questions
Language
Abstract
Machine reading comprehension (MRC) is a challenging task in natural language processing that demonstrates the language understanding of the machine. An approach to tackle this challenge requires the machine to answer the question about the given context when needed and abstain from answering when there is no answer. Recent works attempted to solve this challenge with various comprehensive neural network architectures for sequences such as SAN, U-Net, EQuANt, and others that were trained on the SQuAD 2.0 dataset containing unanswerable questions. However, the robustness of these approaches has not been evaluated. In this paper, we propose a data augmentation approach that converts answerable questions to unanswerable questions in the SQuAD 2.0 dataset by altering the entities in the question to its antonym from ConceptNet which is a semantic network. The augmented data is, then, fitted into the U-Net question answering model to evaluate the robustness of the model.