Talk at Technolinguistics in Practice conference

ElPaCo team members Andreas Liesenfeld and Mark Dingemanse will travel to Siegen for the Technolinguistics in Practice conference, May 24-26. They will give a talk titled How human interaction can inspire convivial language technology. Abstract:

As interactive language technologies increasingly become part of our everyday lives, one of the biggest frustrations remains their rigidity: they are designed to avoid turbulence by funneling people into pre-set dialogue flows, cannot gracefully repair breakdowns, and are devoid of social accountability. This has people adapting to tools rather than the other way around. Here we critically assess notions of language and technology that lie at the base of current developments, and propose a redirection inspired by Ivan Illich’s (1973) notion of convivial tools. A general challenge in this area is that while critical and radical deconstruction is necessary, the search is on for constructive ways of applying cumulative insights from decades of empirical work on human interaction. In this contribution, we aim to address this challenge in two ways.
First, we expose the narrow linguistic and cultural roots of much of today’s NLP (Joshi et al. 2020) and contrast it with the wealth of data and knowledge available in cross-linguistic corpora of everyday informal conversation (Dingemanse and Liesenfeld 2022). Looking at everyday language use can bring to light unseen diversity in semiotic practices, and can uncover ways in which interactional resources can serve human empowerment and inspire the design of language technology (Liesenfeld and Buschmeier 2023). Second, we build on new empirical work as well as prior critical and ethnomethodological work into social interaction (McIlvenny 1993; Button et al. 1995) to develop constructive ways of evaluating artificial conversational agents. Current evaluation methods in NLP/AI focus on the putative “humanness” and “fluidity” of “language generation” (Finch and Choi 2020) — all of these unexamined notions in need of technolinguistic scrutiny. Using insights from conversation analysis, we instead make a case for an action-oriented approach to evaluation (cf. Housley, Albert, and Stokoe 2019). We report on a set of heuristics for human evaluation that can help to systematically probe the interactive capabilities of dialogue systems. Such ‘counterfoil research’ (Illich 1973) is of critical importance to arrive at tools that better support human flourishing. This work provides a stepping stone for engineers and conversation designers who seek to ground their work in observable orientations to social action, rather than automated metrics and scoreboards divorced from interactional practices.
Combining critical and practical perspectives will contribute to respecifying taken-for-granted notions of language and technology, will deepen our understanding of the irreducibly social and interactive aspects of human tool use, and will allow the translation of some of these insights into technology design and engineering practices.

ElPaCo goes to Cologne for DGfS

The Annual Conference of the Deutsche Gesellschaft für Sprachforschung is one of the larger linguistics conferences in Germany. This year, ElPaCo will be represented in the form of a plenary talk by PI Mark Dingemanse and a contribution to the special session on creativity and routine in interaction (AG4) by postdocs Andreas Liesenfeld and Marlou Rasenberg.

Update

We made many new connections, saw some old friends, and took some pictures.

ElPaCo at Dutch Speech Tech Day

The ElPaCo project is represented at the first installment of Dutch Speech Tech Day, a new event that “connects students, researchers, developers and users of speech technology in the Netherlands.”

We’ll be presenting a poster How we talk: lessons for interactive speech technology. In keeping with the goal of the Speech Tech Day to share the latest insights, our poster provides a broad-ranging overview of recent work in the Elementary Particles of Conversation project. It summarizes work from the following papers :

  • Dingemanse, M., & Liesenfeld, A. (2022). From text to talk: Harnessing conversational corpora for humane and diversity-aware language technology. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 5614–5633. Dublin: Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.385
  • Liesenfeld, A., & Dingemanse, M. (2022). Building and curating conversational corpora for diversity-aware language science and technology. Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), 1178–1192. Marseille. doi: 10.48550/arXiv.2203.03399
  • Liesenfeld, A., & Dingemanse, M. (2022). Bottom-up discovery of structure and variation in response tokens (‘backchannels’) across diverse languages. Proceeding of Interspeech 2022. 2022. doi: 10.21437/Interspeech.2022-11288
  • Lopez, A., Liesenfeld, A., & Dingemanse, M. (2022). Evaluation of Automatic Speech Recognition for Conversational Speech in Dutch, English and German: What Goes Missing? Proceedings of the 18th Conference on Natural Language Processing (KONVENS 2022). Potsdam.

New position paper: Beyond Single-Mindedness

The ElPaCo project has spearheaded an ambitious position paper in the Progress and Puzzles in Cognitive Science series. Assembling a broad collective of 28 authors from across the cognitive sciences and beyond, we argue for a figure-ground reversal that puts interaction at the heart of cognition. Read the paper (doi, pdf) or check out the companion website for more information.

“Key concerns of cognitive science may be illuminated by a change of perspective that locates cognition not in isolated but in interacting minds.” (source)

Digital Humanities grant to support our work

Some excellent news to end the year on: we were awarded a grant from the Digital Approaches to the Humanities programme of the Netherlands eScience Center. The eScience center is the Netherlands’ national centre for academic research software.

One of our longer term goals is to make an impact on work in NLP and linguistics by building digital tools that support qualitative and quantitative approaches to conversational structure. Andreas Liesenfeld, postdoc in the ElPaCo project: “Support from the eScience center will help us to improve our existing codebase and work towards broader impact.”

L: New ways of visualizing conversational flow (osf.io/zd34r). R: Clustering Dutch conversational words using methods from bioacoustics (osf.io/7t9pn).

The funding means that the project can host a professional software engineer for a year. PI Mark Dingemanse: “Our work is code-heavy and we place special emphasis on new and compelling ways to analyse and visualize conversational structure. To do this well we need to pioneer new computional tools. We’re happy that the eScience center offers this opportunity to contribute to the strength and longevity of our project.”

Invited talk at Dept of Computational Linguistics, Düsseldorf

ElPaCo team members Andreas Liesenfeld and Mark Dingemanse visit Düsseldorf for an invited talk at the Department of Computational Linguistics. Abstract:

What social interaction and linguistic diversity can tell us about language (and technology)

The primary ecology of natural language is in real-life episodes of human interaction. This is where people learn language and where they use it to coordinate joint actions, build social relations, and exchange information. In contrast, when machines encounter language, it tends to be radically divorced from this habitat and reduced to large amounts of decontextualised non-interactive text. Natural languages are also characterized by diversity at many levels, from sound and sign systems to syntax and semantics. In contrast, the language samples that inform language technology tend to be limited to a handful of well-resourced languages, representing only a tiny sliver of the world’s linguistic diversity.

In this talk we show how a view of language rooted in social interaction sheds new light on turn-taking, pragmatic reasoning, and joint action coordination, with implications for linguistic typology, dialogue modelling, speech recognition, and the design of conversational interfaces. We will provide an overview of current work in our research project Elementary Particles of Conversation and cover a range of methods and results, from computational modelling to comparative linguistics and from distributional pragmatics to dialogue model evaluation.

3 conferences in September

It’s a busy month for ElPaCo project as we present papers, lectures and tutorials at the Joint Conference on Language Evolution in Kanazawa, KONVENS 2022 in Potsdam, and Interspeech 2022 in Incheon, Korea.

At the Joint Conference on Language evolution we presented new comparative and computational work on the cross-linguistic shapes and cultural evolution of response tokens (work by Marieke Woensdregt, Andreas Liesenfeld and Mark Dingemanse).

At KONVENS in Potsdam, team members Andreas Liesenfeld, Ada Lopez and Mark Dingemanse presented a tutorial covering work on conversational corpora, conversational AI, and ASR, building on the research programme set out in our ACL paper.

Also at KONVENS, Ada Lopez presented her first first author paper coming out of her Research MA lab rotation in the ElPaCo project. Pointing ASR at conversational corpora in 3 languages, we go beyond WER and trace what goes missing. Check out the short paper here.

At Interspeech 2022 in Incheon, Andreas Liesenfeld and Mark Dingemanse presented a first foray into language-agnostic approaches to identifying and comparing response tokens (aka backchannels) in conversational audio corpora across 16 languages (8 phyla). Find the paper here.

ElPaCo tutorial at KONVENS in Potsdam

ElPaCo members Ada Lopez and Andreas Liesenfeld have travelled to Potsdam to present their work at KONVENS, one of the largest computational linguistics conferences in Germany. Among other things, they presented a half-day tutorial.

The tutorial helps implement one of the ‘valorization’ goals of the Elementary Particles of Conversation project: to help engineers and practitioners get a handle on the subtleties of human interaction.