Swarnadeep Bhar

Swarnadeep Bhar

NLP Researcher

Sitemap

A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.

Pages

Posts

articles

Limits for Learning with Language Models

Published in Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), 2023

Nevertheless, several recent papers provide empirical evidence that LLMs fail to capture important aspects of linguistic meaning. Focusing on universal quantification, we provide a theoretical foundation for these empirical findings by proving that LLMs cannot learn certain fundamental semantic properties including semantic entailment and consistency as they are defined in formal semantics. More generally, we show that LLMs are unable to learn concepts beyond the first level of the Borel Hierarchy, which imposes severe limits on the ability of LMs, both large and small, to capture many aspects of linguistic meaning. This means that LLMs will continue to operate without formal guarantees on tasks that require entailments and deep linguistic understanding. Read more

Recommended citation: Asher, N., Bhar, S., Chaturvedi, A., Hunter, J., & Paul, S. (2023). Limits for Learning with Language Models. In Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023) (pp. 236–248).

Analyzing semantic faithfulness of language models via input intervention on conversational question answering

Published in Computational Linguistics, 2024

Transformer-based language models have been shown to be highly effective for several NLP tasks. In this article, we consider three transformer models, BERT, RoBERTa, and XLNet, in both small and large versions, and investigate how faithful their representations are with respect to the semantic content of texts. We formalize a notion of semantic faithfulness, in which the semantic content of a text should causally figure in a model’s inferences in question answering Read more

Recommended citation: Chaturvedi, A., Bhar, S., Saha, S., Garain, U., & Asher, N. (2024). Analyzing Semantic Faithfulness of Language Models via Input Intervention on Question Answering. Computational Linguistics, 50(1), 119-155.

Strong hallucinations from negation and how to fix them

Published in Findings of ACL 2024, 2024

Despite great performance on many tasks, language models (LMs) still struggle with reasoning, sometimes providing responses that cannot possibly be true because they stem from logical incoherence. We call such responses strong hallucinations and prove that they follow from its computation of its internal representations for logical operators and outputs from those representations Read more

Recommended citation: Asher, Nicholas, and Swarnadeep Bhar. "Strong hallucinations from negation and how to fix them." arXiv preprint arXiv:2402.10543 (2024).

expressions

IPoXP: Internet Protocol over Xylophone Players

We introduce IP over Xylophone Players (IPoXP), a novel Internet protocol between two computers using xylophone-based Arduino interfaces. In our implementation, human operators are situated within the lowest layer of the network, transmitting data between computers by striking designated keys. We discuss how IPoXP inverts the traditional mode of human-computer interaction, with a computer using the human as an interface to communicate with another computer Read more

0 (the game)

One of the many forks of the popular game 1024 by Veewo Studio (which is conceptually similar to Threes by Asher Vollmer). Try to combine all the 0 tiles until they add up to 1. Read more

robots.txt.php

An algorithmically-generated robots.txt, which disallows all bots with one exception: the bot requesting the file is allowed full access. Read more

dystopedia

A Markov chain Twitter bot trained on titles of Wikipedia articles that have been deleted. Read more

AcademicPages

AcademicPages is a ready-to-fork GitHub Pages template for academic personal websites, based on structured data in markdown files. I created it for this website, then released it so others can make their own, which are hosted for free by GitHub. Over 500 people have! Read more

talks

Actor-Network Theory

Published in Social Aspects of Information Systems course, 2013

An introduction to Actor Network Theory for students in the Masters of Information Management and Systems (MIMS) course Read more

Governing the Commons

Published in History of Information, 2014

A lecture on the history of Wikipedia, in the broader context of the history of reference works. Read more

Moderating Online Conversation Spaces

Published in Social Aspects of Information Systems course, 2015

An overview of how various online platforms moderate content, discussing issues that link up to the theories discussed in the Social Aspects of Information Systems class. Read more

Peer Production and Wikipedia

Published in Social Aspects of Information Systems course, 2015

An overview of Wikipedia and other peer production platforms, discussing issues that link up to the theories discussed in the Social Aspects of Information Systems class. Read more

The Bot Multiple: Unpacking the Materialities of Automated Software Agents

Published in Annual Meeting of the Society for the Social Study of Science (4S), 2015

I examine the roles that automated software agents (or bots) play in the governance and moderation of Wikipedia, Twitter, and reddit – three online platforms that differently uphold a related set of commitments to ‘open’ and ‘public’ online participation. Read more

Scraping Wikipedia Data

Published in The Hacker Within, BIDS, 2016

A tutorial (with Jupyter notebooks) about how to use APIs to query structured data from Wikipedia articles and the Wikidata project. Read more

Community Sustainability in Wikipedia: A Review of Research and Initiatives

Published in PyData SF, 2016

Wikipedia relies on one of the world’s largest open collaboration communities. Since 2001, the community has grown substantially and faced many challenges. This presentation reviews research and initiatives around community sustainability in Wikipedia that are relevant for many open source projects, including issues of newcomer retention, governance, automated moderation, and marginalized groups. Read more

“The Wisdom of Bots:” An ethnographic study of the delegation of governance work to information infrastructures in Wikipedia

Published in Annual Meeting of the Society for the Social Study of Science (4S), 2016

Wikipedians rely on software agents to govern the ‘anyone can edit’ encyclopedia project, in the absence of more formal and traditional organizational structures. Lessons from Wikipedia’s bots speak to debates about how algorithms are being delegated governance work in sites of cultural production. Read more

Jupyter and the Changing Rituals around Computation

Published in JupyterCon, 2017

We (Stuart Geiger, Brittany Fiore-Gartland, and Charlotte Cabasse-Mazel) share ethnographic findings made observing and working with Jupyter notebooks, focusing on how people use Jupyter to create and deliver computational narratives in particular local contexts, like classrooms, hackathons, research collaborations, and more. Read more

Computational Ethnography and the Ethnography of Computation

Published in Berkeley Institute for Data Science, 2017

Ethnography is traditionally a qualitative and inductive methodology – with its origins in cultural anthropology – that is now widely used to holistically investigate people’s lived experiences in and across cultures. In this talk, I define and discuss two ways of thinking about the role of ethnographic methods around computation, then discuss how my research relates to both. Read more

Are the bots really fighting? Behind the scenes of a reproducible replication

Published in UC-Berkeley Department of Statistics: Reproducible and Collaborative Data Science, 2017

A guest lecture for Fernando Perez’s STAT 159/259 course on Reproducible and Collaborative Data Science, in which I discuss issues of open science and reproducibility around our recent paper Operationalizing conflict and cooperation between automated software agents in Wikipedia: A replication and expansion of ‘Even Good Bots Fight’ Read more

“But it wouldn’t be an encyclopedia; it would be a wiki”: The changing imagined affordances of wikis, 1995-2002

Published in 2017 Annual Meeting of the Association of Internet Researchers, 2017

This paper examines the early history of “anyone can edit” wiki software – originally developed in 1995, six years before Wikipedia’s origin. While today, the idea of a wiki is associated with large-scale, massively-distributed encyclopedic knowledge production, this was not always the case. Articles on pre-Wikipedia wikis were often closer to a Joycean stream of consciousness than Wikipedia’s Britannica-inspired texts that speak in single voice, and the underlying wiki platform lacked many of the affordances that are now taken for granted in wiki platforms. In fact, the creator of the first wiki advised Wikipedia’s co-founders that the goals of creating a general-purpose encyclopedia and a wiki were inherently contradictory. Read more

The Humanity of Artificial Intelligence

Published in Bay Area Science Festival, 2017

Today, “artificial intelligence” seems to be everywhere – in our phones, vacuums, hospitals, and inboxes – but it can be hard to separate science fiction from science fact. Many discussions about AI imagine a fully autonomous superintelligence that designs itself with little to no human intervention, making decisions in ways that humans cannot possibly understand. Yet the work of designing, developing, engineering, training, and testing such systems requires a massive amount of human labor, which is typically erased when such systems are released as products. In this talk, I give a human-centered, behind-the-scenes introduction to machine learning, illustrating the creative, interpretive, and often messy work humans do to make autonomous agents work. Understanding the humanity behind artificial intelligence is important if we want to think constructively about issues of bias, fairness, accountability, and transparency in AI. Read more

Computational Ethnography and the Ethnography of Computation: The Case for Context

Published in School of Information and Library Science, University of North Carolina at Chapel Hill, 2018

Ethnography is traditionally a qualitative and inductive methodology that is now widely used to holistically investigate people’s lived experiences in and across cultures. In this talk, I define and discuss two ways of thinking about the role of ethnographic methods around computation, then discuss how my research relates to both. Read more

Computational Ethnography and the Ethnography of Computation: The Case for Context

Published in School of Information Sciences, University of Illinois at Urbana-Champaign, 2018

Ethnography is traditionally a qualitative and inductive methodology that is now widely used to holistically investigate people’s lived experiences in and across cultures. In this talk, I define and discuss two ways of thinking about the role of ethnographic methods around computation, then discuss how my research relates to both. Read more

Computational Ethnography and the Ethnography of Computation: The Case for Context

Published in College of Information Studies, University of Maryland at College Park, 2018

Ethnography is traditionally a qualitative and inductive methodology that is now widely used to holistically investigate people’s lived experiences in and across cultures. In this talk, I define and discuss two ways of thinking about the role of ethnographic methods around computation, then discuss how my research relates to both. Read more

Publics: Witnessing and Measuring

Published in UC-Berkeley: Human Contexts and Ethics of Data course, 2018

A guest lecture for Cathryn Carson and Margo Boenig-Liptsin’s course on Human Contexts and Ethics of Data (HIST 182C, STS 100C), focusing on how various publics generate, analyze, and interpret data. Read more

The Human Contexts of Data: Infrastructures, Institutions, and Interpretations

Published in University of Manchester, Data Science Institute, 2018

In this talk, I discuss the role of qualitative and ethnographic methods in relation to computer, information, and data science. These holistic, reflexive, and meta-level approaches to studying data and computation in context help us better understand how to both support and practice data analytics at various scales. Read more

Computational Ethnography and the Ethnography of Computation: The Case for Context

Published in IT University of Copenhagen, ETHOSlab, 2018

Ethnography is traditionally a qualitative and inductive methodology that is now widely used to holistically investigate people’s lived experiences in and across cultures. In this talk, I define and discuss two ways of thinking about the role of ethnographic methods around computation, then discuss how my research relates to both. Read more

Key Values: What We Talk About When We Talk About ‘Open Science’

Published in Open Science Symposium, Department of Second Language Studies, University of Hawaiʻi at Mānoa, 2018

Openness in science is hard to disagree with as an abstract principle, but what exactly do we mean when we call for science to be made open – or more open than before? In this talk, I introduce and unpack the many different goals, strategies, products, values, and assumptions of the broad open science movement. Read more

Knowing User Populations at Scale: From the Science of the State to Platform Governmentality

Published in 2018 Annual Conference of the International Communication Association, 2018

How can institutions that own and operate large-scale social media platforms come to know “their users” at scale? In this talk, I discuss ways of knowing user populations at scale, drawing on Foucault’s account of governmentality, particularly the role of statistics in the formation of the modern nation state. Read more

The Types, Roles, and Practices of Documentation in Data Analytics Open Source Software Libraries: A Collaborative Ethnography of Documentation Work

Published in 2018 European Conference on Computer-Supported Cooperative Work, 2018

Data analytics increasingly relies on open source software (OSS) libraries that extend scripted languages like python and R. Software documentation for these libraries is crucial for people across all experience levels, but documentation work raises many challenges, particularly in open source communities. In this collaboration between ethnographers and data scientists, we discuss the types, roles, practices, and motivations around documentation in data analytics OSS libraries. Read more

Designing and Using Data Science Ethically

Published in Machine Learning and User Experience San Francisco (MLUXSF), 2018

With the rise of Machine Learning and AI to solve human-focused needs, how do we design and use data science ethically to help empower and support people? Read more

teaching

Software Carpentry Instructor

Published:

Software Carpentry is a global non-profit organization that provides free, short workshops on scientific computing and data science. I have been a certified instructor with SWC since May 2016. Read more