KOR

e-Article

Explainable Human-Machine Teaming using Model Checking and Interpretable Machine Learning
Document Type
Conference
Source
2023 IEEE/ACM 11th International Conference on Formal Methods in Software Engineering (FormaliSE) FORMALISE Formal Methods in Software Engineering (FormaliSE), 2023 IEEE/ACM 11th International Conference on. :18-28 May, 2023
Subject
Computing and Processing
Closed box
Machine learning
Medical services
Model checking
Physiology
Hazards
Teamwork
Human-machine teaming
formal analysis
statistical model checking
interpretable machine learning
Language
ISSN
2575-5099
Abstract
The human-machine teaming paradigm promotes tight teamwork between humans and autonomous machines that collaborate in the same physical space. This paradigm is increasingly widespread in critical domains, such as healthcare and domestic assistance. These systems are expected to build a certain level of trust by enforcing dependability and exhibiting interpretable behavior. However, trustworthiness is negatively affected by the black-box nature of these systems, which typically make fully autonomous decisions that may be confusing for humans or cause hazards in critical domains. We present the EASE approach, whose goal is to build better trust in human-machine teaming leveraging statistical model checking and model-agnostic interpretable machine learning. We illustrate EASE through an example in healthcare featuring an infinite (dense) space of human-machine uncertain factors, such as diverse physical and physiological characteristics of the agents involved in the teamwork. Our evaluation demonstrates the suitability and cost-effectiveness of EASE in explaining dependability properties in human-machine teaming.