Alert button
Picture for Pedram Khorsandi

Pedram Khorsandi

Alert button

Do Explanations Explain? Model Knows Best

Add code
Bookmark button
Alert button
Mar 04, 2022
Ashkan Khakzar, Pedram Khorsandi, Rozhin Nobahari, Nassir Navab

Figure 1 for Do Explanations Explain? Model Knows Best
Figure 2 for Do Explanations Explain? Model Knows Best
Figure 3 for Do Explanations Explain? Model Knows Best
Figure 4 for Do Explanations Explain? Model Knows Best

It is a mystery which input features contribute to a neural network's output. Various explanation (feature attribution) methods are proposed in the literature to shed light on the problem. One peculiar observation is that these explanations (attributions) point to different features as being important. The phenomenon raises the question, which explanation to trust? We propose a framework for evaluating the explanations using the neural network model itself. The framework leverages the network to generate input features that impose a particular behavior on the output. Using the generated features, we devise controlled experimental setups to evaluate whether an explanation method conforms to an axiom. Thus we propose an empirical framework for axiomatic evaluation of explanation methods. We evaluate well-known and promising explanation solutions using the proposed framework. The framework provides a toolset to reveal properties and drawbacks within existing and future explanation solutions.

* CVPR 2022 (IEEE/CVF Computer Vision and Pattern Recognition) 
Viaarxiv icon
Github IconAccess Paper or Ask Questions
Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon

Are socially-aware trajectory prediction models really socially-aware?

Add code
Bookmark button
Alert button
Aug 24, 2021
Saeed Saadatnejad, Mohammadhossein Bahari, Pedram Khorsandi, Mohammad Saneian, Seyed-Mohsen Moosavi-Dezfooli, Alexandre Alahi

Figure 1 for Are socially-aware trajectory prediction models really socially-aware?
Figure 2 for Are socially-aware trajectory prediction models really socially-aware?
Figure 3 for Are socially-aware trajectory prediction models really socially-aware?
Figure 4 for Are socially-aware trajectory prediction models really socially-aware?

Our field has recently witnessed an arms race of neural network-based trajectory predictors. While these predictors are at the core of many applications such as autonomous navigation or pedestrian flow simulations, their adversarial robustness has not been carefully studied. In this paper, we introduce a socially-attended attack to assess the social understanding of prediction models in terms of collision avoidance. An attack is a small yet carefully-crafted perturbations to fail predictors. Technically, we define collision as a failure mode of the output, and propose hard- and soft-attention mechanisms to guide our attack. Thanks to our attack, we shed light on the limitations of the current models in terms of their social understanding. We demonstrate the strengths of our method on the recent trajectory prediction models. Finally, we show that our attack can be employed to increase the social understanding of state-of-the-art models. The code is available online: https://s-attack.github.io/

Viaarxiv icon
Github IconAccess Paper or Ask Questions
Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon