Explainable AI

Current research on explainability and interpretability of machine learning algorithms

This site is a community effort by the Explainable AI members. Please join the group and reach out to the administrators if you have any questions.

Made with Jekyll and Hyde. Idea and base code for the website adapted from ml4code.

Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples

Yinpeng Dong, Hang Su, Jun Zhu, Fan Bao. 2017

[ArXiv]    

Deep neural networks (DNNs) have demonstrated impressive performance on a wide array of tasks, but they are usually considered opaque since internal structure and learned parameters are not interpretable. In this paper, we re-examine the internal representations of DNNs using adversarial images, which are generated by an ensemble-optimization algorithm. We find that: (1) the neurons in DNNs do not truly detect semantic objects/parts, but respond to objects/parts only as recurrent discriminative patches; (2) deep visual representations are not robust distributed codes of visual concepts because the representations of adversarial images are largely not consistent with those of real images, although they have similar visual appearance, both of which are different from previous findings. To further improve the interpretability of DNNs, we propose an adversarial training scheme with a consistent loss such that the neurons are endowed with human-interpretable concepts. The induced interpretable representations enable us to trace eventual outcomes back to influential neurons. Therefore, human users can know how the models make predictions, as well as when and why they make errors.