Add to Chrome
Add to Firefox
Add to EdgeSatyam Dwivedi
Semi-supervised counterfactual explanations
Mar 22, 2023
Counterfactual explanations for machine learning models are used to find minimal interventions to the feature values such that the model changes the prediction to a different output or a target output. A valid counterfactual explanation should have likely feature values. Here, we address the challenge of generating counterfactual explanations that lie in the same data distribution as that of the training data and more importantly, they belong to the target class distribution. This requirement has been addressed through the incorporation of auto-encoder reconstruction loss in the counterfactual search process. Connecting the output behavior of the classifier to the latent space of the auto-encoder has further improved the speed of the counterfactual search process and the interpretability of the resulting counterfactual explanations. Continuing this line of research, we show further improvement in the interpretability of counterfactual explanations when the auto-encoder is trained in a semi-supervised fashion with class tagged input data. We empirically evaluate our approach on several datasets and show considerable improvement in-terms of several metrics.
Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems
Jun 15, 2022
We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9.3B, their subsequent distillation into smaller models ranging from 17M-170M parameters, and their application to the Natural Language Understanding (NLU) component of a virtual assistant system. Though we train using 70% spoken-form data, our teacher models perform comparably to XLM-R and mT5 when evaluated on the written-form Cross-lingual Natural Language Inference (XNLI) corpus. We perform a second stage of pretraining on our teacher models using in-domain data from our system, improving error rates by 3.86% relative for intent classification and 7.01% relative for slot filling. We find that even a 170M-parameter model distilled from our Stage 2 teacher model has 2.88% better intent classification and 7.69% better slot filling error rates when compared to the 2.3B-parameter teacher trained only on public data (Stage 1), emphasizing the importance of in-domain data for pretraining. When evaluated offline using labeled NLU data, our 17M-parameter Stage 2 distilled model outperforms both XLM-R Base (85M params) and DistillBERT (42M params) by 4.23% to 6.14%, respectively. Finally, we present results from a full virtual assistant experimentation platform, where we find that models trained using our pretraining and distillation pipeline outperform models distilled from 85M-parameter teachers by 3.74%-4.91% on an automatic measurement of full-system user dissatisfaction.
* KDD 2022
Semi-Passive 3D Positioning of Multiple RIS-Enabled Users
Apr 25, 2021
Reconfigurable intelligent surfaces (RISs) are set to be a revolutionary technology in the 6th generation of wireless systems. In this work, we study the application of RIS in a multi-user passive localization scenario, where we have one transmitter (Tx) and multiple asynchronous receivers (Rxs) with known locations. We aim to estimate the locations of multiple users equipped with RISs. The RISs only reflect the signal from the Tx to the Rxs and are not used as active transceivers themselves. Each Rx receives the signal from the Tx (LOS path) and the reflected signal from the RISs (NLOS path). We show that users' 3D position can be estimated with submeter accuracy in a large area around the transmitter, using the LOS and NLOS time-of-arrival measurements at the Rxs. We do so, by developing the signal model, deriving the Cramer-Rao bounds, and devising an estimator that attains these bounds. Furthermore, by orthogonalizing the RIS phase profiles across different users, we circumvent inter-path interference.