TY - JOUR
T1 - Explainable machine learning in image classification models: An uncertainty quantification perspective
AU - Zhang, Xiaoge
AU - Chan, Felix T.S.
AU - Mahadevan, Sankaran
N1 - Funding Information:
The work presented in this paper is supported by Centre for Advances in Reliability and Safety (CAiRS), an InnoHK Research Cluster of HKSAR Government, and the Research Committee of The Hong Kong Polytechnic University under project code 1-BE6V.
Publisher Copyright:
© 2022 Elsevier B.V.
PY - 2022/5/11
Y1 - 2022/5/11
N2 - The poor explainability of deep learning models has hindered their adoption in safety and quality-critical applications. This paper focuses on image classification models and aims to enhance the explainability of deep learning models through the development of an uncertainty quantification-based framework. The proposed methodology consists of three major steps. In the first step, we adopt dropout-based Bayesian neural network to characterize the structure and parameter uncertainty inherent in deep learning models, propagate and represent such uncertainties to the model prediction as a distribution. Next, we employ entropy as a quantitative indicator to measure the uncertainty in model prediction, and develop an Empirical Cumulative Distribution Function (ECDF)-based approach to determine an appropriate threshold value for the purpose of deciding when to accept or reject the model prediction. Secondly, in the cases with high model prediction uncertainty, we combine the prediction difference analysis (PDA) approach with dropout-based Bayesian neural network to quantify the uncertainty in pixel-wise feature importance, and identify the locations in the input image that highly correlate with the model prediction uncertainty. In the third step, we develop a robustness-based design optimization formulation to enhance the relevance between input features and model prediction, and leverage a differential evolution approach to optimize the pixels in the input image with high uncertainty in feature importance. Experimental studies in MNIST and CIFAR-10 image classifications are included to demonstrate the effectiveness of the proposed approach in increasing the explainability of deep learning models.
AB - The poor explainability of deep learning models has hindered their adoption in safety and quality-critical applications. This paper focuses on image classification models and aims to enhance the explainability of deep learning models through the development of an uncertainty quantification-based framework. The proposed methodology consists of three major steps. In the first step, we adopt dropout-based Bayesian neural network to characterize the structure and parameter uncertainty inherent in deep learning models, propagate and represent such uncertainties to the model prediction as a distribution. Next, we employ entropy as a quantitative indicator to measure the uncertainty in model prediction, and develop an Empirical Cumulative Distribution Function (ECDF)-based approach to determine an appropriate threshold value for the purpose of deciding when to accept or reject the model prediction. Secondly, in the cases with high model prediction uncertainty, we combine the prediction difference analysis (PDA) approach with dropout-based Bayesian neural network to quantify the uncertainty in pixel-wise feature importance, and identify the locations in the input image that highly correlate with the model prediction uncertainty. In the third step, we develop a robustness-based design optimization formulation to enhance the relevance between input features and model prediction, and leverage a differential evolution approach to optimize the pixels in the input image with high uncertainty in feature importance. Experimental studies in MNIST and CIFAR-10 image classifications are included to demonstrate the effectiveness of the proposed approach in increasing the explainability of deep learning models.
KW - Bayesian neural network
KW - Deep learning
KW - Model explainability
KW - Prediction difference analysis
KW - Uncertainty quantification
UR - http://www.scopus.com/inward/record.url?scp=85125656682&partnerID=8YFLogxK
U2 - 10.1016/j.knosys.2022.108418
DO - 10.1016/j.knosys.2022.108418
M3 - Journal article
AN - SCOPUS:85125656682
SN - 0950-7051
VL - 243
JO - Knowledge-Based Systems
JF - Knowledge-Based Systems
M1 - 108418
ER -