TY - JOUR
T1 - EvalAttAI
T2 - A Holistic Approach to Evaluating Attribution Maps in Robust and Non-Robust Models
AU - Nielsen, Ian E.
AU - Ramachandran, Ravi P.
AU - Bouaynaya, Nidhal
AU - Fathallah-Shaykh, Hassan M.
AU - Rasool, Ghulam
N1 - Publisher Copyright: © 2013 IEEE.
PY - 2023
Y1 - 2023
N2 - The expansion of explainable artificial intelligence as a field of research has generated numerous methods of visualizing and understanding the black box of a machine learning model. Attribution maps are commonly used to highlight the parts of the input image that influence the model to make a specific decision. At the same time, numerous recent research papers in the fields of machine learning and explainable artificial intelligence have demonstrated the essential role of robustness to natural noise and adversarial attacks in determining the features learned by a model. This paper focuses on evaluating methods of attribution mapping to find whether robust neural networks are more explainable, particularly within the application of classification for medical imaging. However, there is no consensus on how to evaluate attribution maps. To solve this, we propose a new explainability faithfulness metric, EvalAttAI, that addresses the limitations of prior metrics. We evaluate various attribution methods on multiple datasets and find that Bayesian deep neural networks using the Variational Density Propagation technique are consistently more explainable when used with the best performing attribution method, the Vanilla Gradient. Our results suggest that robust neural networks may not always be more explainable, despite producing more visually plausible attribution maps.
AB - The expansion of explainable artificial intelligence as a field of research has generated numerous methods of visualizing and understanding the black box of a machine learning model. Attribution maps are commonly used to highlight the parts of the input image that influence the model to make a specific decision. At the same time, numerous recent research papers in the fields of machine learning and explainable artificial intelligence have demonstrated the essential role of robustness to natural noise and adversarial attacks in determining the features learned by a model. This paper focuses on evaluating methods of attribution mapping to find whether robust neural networks are more explainable, particularly within the application of classification for medical imaging. However, there is no consensus on how to evaluate attribution maps. To solve this, we propose a new explainability faithfulness metric, EvalAttAI, that addresses the limitations of prior metrics. We evaluate various attribution methods on multiple datasets and find that Bayesian deep neural networks using the Variational Density Propagation technique are consistently more explainable when used with the best performing attribution method, the Vanilla Gradient. Our results suggest that robust neural networks may not always be more explainable, despite producing more visually plausible attribution maps.
UR - http://www.scopus.com/inward/record.url?scp=85166776811&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85166776811&partnerID=8YFLogxK
U2 - https://doi.org/10.1109/ACCESS.2023.3300242
DO - https://doi.org/10.1109/ACCESS.2023.3300242
M3 - Article
SN - 2169-3536
VL - 11
SP - 82556
EP - 82569
JO - IEEE Access
JF - IEEE Access
ER -