Add to Chrome
Add to Firefox
Add to EdgeAnna Sotnikova
"Flex Tape Can't Fix That": Bias and Misinformation in Edited Language Models
Feb 29, 2024
Model editing has emerged as a cost-effective strategy to update knowledge stored in language models. However, model editing can have unintended consequences after edits are applied: information unrelated to the edits can also be changed, and other general behaviors of the model can be wrongly altered. In this work, we investigate how model editing methods unexpectedly amplify model biases post-edit. We introduce a novel benchmark dataset, Seesaw-CF, for measuring bias-related harms of model editing and conduct the first in-depth investigation of how different weight-editing methods impact model bias. Specifically, we focus on biases with respect to demographic attributes such as race, geographic origin, and gender, as well as qualitative flaws in long-form texts generated by edited language models. We find that edited models exhibit, to various degrees, more biased behavior as they become less confident in attributes for Asian, African, and South American subjects. Furthermore, edited models amplify sexism and xenophobia in text generations while remaining seemingly coherent and logical. Finally, editing facts about place of birth, country of citizenship, or gender have particularly negative effects on the model's knowledge about unrelated features like field of work.
Multilingual large language models leak human stereotypes across language boundaries
Dec 12, 2023Multilingual large language models have been increasingly popular for their proficiency in comprehending and generating text across various languages. Previous research has shown that the presence of stereotypes and biases in monolingual large language models can be attributed to the nature of their training data, which is collected from humans and reflects societal biases. Multilingual language models undergo the same training procedure as monolingual ones, albeit with training data sourced from various languages. This raises the question: do stereotypes present in one social context leak across languages within the model? In our work, we first define the term ``stereotype leakage'' and propose a framework for its measurement. With this framework, we investigate how stereotypical associations leak across four languages: English, Russian, Chinese, and Hindi. To quantify the stereotype leakage, we employ an approach from social psychology, measuring stereotypes via group-trait associations. We evaluate human stereotypes and stereotypical associations manifested in multilingual large language models such as mBERT, mT5, and ChatGPT. Our findings show a noticeable leakage of positive, negative, and non-polar associations across all languages. Notably, Hindi within multilingual models appears to be the most susceptible to influence from other languages, while Chinese is the least. Additionally, ChatGPT exhibits a better alignment with human scores than other models.
Analyzing and modeling network travel patterns during the Ukraine invasion using crowd-sourced pervasive traffic data
Aug 08, 2022
In 2022, Ukraine is suffering an invasion which has resulted in acute impacts playing out over time and geography. This paper examines the impact of the ongoing disruption on traffic behavior using analytics as well as zonal-based network models. The methodology is a data-driven approach that utilizes obtained travel-time conditions within an evolutionary algorithm framework which infers origin-destination demand values in an automated process based on traffic assignment. Because of the automation of the implementation, numerous daily models can be approximated for multiple cities. The novelty of this paper versus the previously published core methodology includes an analysis to ensure the obtained data is appropriate since some data sources were disabled due to the ongoing disruption. Further, novelty includes a direct linkage of the analysis to the timeline of disruptions to examine the interaction in a new way. Finally, specific network metrics are identified which are particularly suited for conceptualizing the impact of conflict disruptions on traffic network conditions. The ultimate aim is to establish processes, concepts and analysis to advance the broader activity of rapidly quantifying the traffic impacts of conflict scenarios.
Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models
Jun 23, 2022
NLP models trained on text have been shown to reproduce human stereotypes, which can magnify harms to marginalized groups when systems are deployed at scale. We adapt the Agency-Belief-Communion (ABC) stereotype model of Koch et al. (2016) from social psychology as a framework for the systematic study and discovery of stereotypic group-trait associations in language models (LMs). We introduce the sensitivity test (SeT) for measuring stereotypical associations from language models. To evaluate SeT and other measures using the ABC model, we collect group-trait judgments from U.S.-based subjects to compare with English LM stereotypes. Finally, we extend this framework to measure LM stereotyping of intersectional identities.