Ada Lopez starts as a PhD candidate

30 August 2023

We’re very happy to announce the arrival of Ada Lopez as a new PhD candidate in the ElPaCo project. Ada’s PhD is generously funded by a PhD fellowship from the Centre for Language Studies at Radboud University. She will work on various topics at the intersection of social interaction and language technology, probably focusing on speech recognition (ASR).

As it happens, Ada’s arrival is also a return: she joined our project for a lab rotation in 2021 as part of her ResearchMA programme, which resulted in her involvement in two peer-reviewed conference papers (see below). We’re very happy to be able to work with her over the next four years!

Lopez, A., Liesenfeld, A., & Dingemanse, M. (2022). Evaluation of Automatic Speech Recognition for Conversational Speech in Dutch, English and German: What Goes Missing? Proceedings of the 18th Conference on Natural Language Processing (KONVENS 2022). PDF
Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023). The timing bottleneck: Why timing and overlap are mission-critical for conversational user interfaces, speech recognition and dialogue systems. Proceedings of the 24th Annual SIGdial Meeting on Discourse and Dialogue. doi: 10.48550/arXiv.2307.15493 PDF

Veni grant for ElPaCo alum Woensdregt 🍾

22 August 2023

Marieke Woensdregt (ElPaco postdoc 2019-2021) has received a competitive Veni grant to carry out fundamental research on language and social interaction. Woensdregt will use computational modelling and experiments to develop an explanation of how such social reasoning and social interaction work together to enable fast and flexible language use. We’re very happy to continue our collaboration and looking forward to see the work coming out of this project!

ElPaCo at CUI’23

21 July 2023

The ElPaCo team is represented at the 5th International Conference on Conversational Interfaces (CUI’23) at Eindhoven University of Technology, July 19-21 2023.

Andreas Liesenfeld presents a poster on behalf of the team (co-authored with Ada Lopez and Mark Dingemanse) on Opening up ChatGPT. The poster comes with a short paper in the CUI proceedings and with a live tracker of degrees of openness in current large language models of the instruction-following kind: opening-up-chatgpt.github.io.

Liesenfeld, A., Lopez, A. & Dingemanse, M. 2023. “Opening up ChatGPT: Tracking Openness, Transparency, and Accountability in Instruction-Tuned Text Generators.” In CUI ’23: Proceedings of the 5th International Conference on Conversational User Interfaces. July 19-21, Eindhoven. doi: 10.1145/3571884.3604316 (arxiv).

Mark Dingemanse was keynote speaker alongside Manuel Giuliani and Verena Rieser at the 2nd installment of the Working with Troubles and Failure in Human-Robot Interaction workshop (WTF 2023), co-located with CUI’23.

While today’s language models may allow us to paper over deficits in understanding by generating coherent and confident prose, conversational user interfaces usually need to go beyond simulating conversation, and would ideally support modes of interaction that approach the resilience and flexibility of human language use. My main goal in this talk is to review some of the ways in which the empirical study of human interaction can contribute towards this goal.

Mark Dingemanse

ElPaCo goes to Brussels

10 July 2023

We’re in Brussels for the 18th conference of the International Pragmatics Assocation. With over 1500 registered participants and over a 1000 talks in 24 parallel sessions over 5 days, this is one of the biggest conferences in linguistics.

ElPaCO PI Mark Dingemanse gives the opening plenary on Sunday: The social shape of language.

Panel on robust and flexible language technology

On Tuesday, Andreas Liesenfeld organises a panel with Hendrik Buschmeier (Bielefeld University): Robust and flexible interactive language technology for human empowerment. Contributions include:

  • Chloe Clavel: The machine/deep learning perspective for conversational AI and the modeling of language specifics
  • Dimosthenis Kontogiorgios and David Schlangen, Explainable embodied intelligence for collaborative robots: an interactive approach
  • Justine Cassell (INRIA), Culturally-aware educational language technologies
  • Karola Pitsch: How do users with mild cognitive impairment deal with conversational user interfaces? Investigating situations of signaling trouble
  • Rosalyn Langedijk & Kerstin Fischer: Field testing an interactive language technology in an elderly care facility
  • Sviatlana Höhn: Conversational AI platform for digital robot-assisted autism therapy

Andreas Liesenfeld promoted to Assistant Professor 🍾

1 July 2023

Less than two years after joining the ElPaCo project from Hong Kong, postdoctoral researcher Andreas Liesenfeld has secured a tenure track position as Assistant Professor in the Language & Communication department at Radboud University. This promotion is not only highly deserved, it will also enable him to continue the fruitful research directions he has started exploring while also contributing to the department’s teaching programme in the areas of language technology and human interaction. In a happy coincidence, Andreas also recently received the CLS Outstanding Research Award for the first paper he co-wrote within ElPaCo. Some of Andreas’ recent papers include:

Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023). The timing bottleneck: Why timing and overlap are mission-critical for conversational user interfaces, speech recognition and dialogue systems. Proceedings of the 24th Annual SIGdial Meeting on Discourse and Dialogue. doi: 10.48550/arXiv.2307.15493 PDF
Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023). Opening up ChatGPT: tracking openness, transparency, and accountability in instruction-tuned text generators. CUI ’23: Proceedings of the 5th International Conference on Conversational User Interfaces, 1–6. doi: 10.1145/3571884.3604316 PDF
Liesenfeld, A., & Dingemanse, M. (2022). Bottom-up discovery of structure and variation in response tokens (‘backchannels’) across diverse languages. Proceedings of Interspeech 2022, 1126–1130. doi: 10.21437/Interspeech.2022-11288 PDF
Dingemanse, M., & Liesenfeld, A. (2022). From text to talk: Harnessing conversational corpora for humane and diversity-aware language technology. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 5614–5633. doi: 10.18653/v1/2022.acl-long.385 PDF

Outstanding Research Award for ‘From text to talk’ 🍾

30 June 2023

We won an Outstanding Research Award 2021-2022 for our paper From text to talk! The Outstanding Research Award is awarded every two years by the Centre for Language Studies to “a research project or publication with a major scholarly impact on the field.”

Fun fact: this is the very first paper we wrote together. Here’s to many more! 🥂

Talk at Technolinguistics in Practice conference

4 May 2023

ElPaCo team members Andreas Liesenfeld and Mark Dingemanse will travel to Siegen for the Technolinguistics in Practice conference, May 24-26. They will give a talk titled How human interaction can inspire convivial language technology. Abstract:

As interactive language technologies increasingly become part of our everyday lives, one of the biggest frustrations remains their rigidity: they are designed to avoid turbulence by funneling people into pre-set dialogue flows, cannot gracefully repair breakdowns, and are devoid of social accountability. This has people adapting to tools rather than the other way around. Here we critically assess notions of language and technology that lie at the base of current developments, and propose a redirection inspired by Ivan Illich’s (1973) notion of convivial tools. A general challenge in this area is that while critical and radical deconstruction is necessary, the search is on for constructive ways of applying cumulative insights from decades of empirical work on human interaction. In this contribution, we aim to address this challenge in two ways.
First, we expose the narrow linguistic and cultural roots of much of today’s NLP (Joshi et al. 2020) and contrast it with the wealth of data and knowledge available in cross-linguistic corpora of everyday informal conversation (Dingemanse and Liesenfeld 2022). Looking at everyday language use can bring to light unseen diversity in semiotic practices, and can uncover ways in which interactional resources can serve human empowerment and inspire the design of language technology (Liesenfeld and Buschmeier 2023). Second, we build on new empirical work as well as prior critical and ethnomethodological work into social interaction (McIlvenny 1993; Button et al. 1995) to develop constructive ways of evaluating artificial conversational agents. Current evaluation methods in NLP/AI focus on the putative “humanness” and “fluidity” of “language generation” (Finch and Choi 2020) — all of these unexamined notions in need of technolinguistic scrutiny. Using insights from conversation analysis, we instead make a case for an action-oriented approach to evaluation (cf. Housley, Albert, and Stokoe 2019). We report on a set of heuristics for human evaluation that can help to systematically probe the interactive capabilities of dialogue systems. Such ‘counterfoil research’ (Illich 1973) is of critical importance to arrive at tools that better support human flourishing. This work provides a stepping stone for engineers and conversation designers who seek to ground their work in observable orientations to social action, rather than automated metrics and scoreboards divorced from interactional practices.
Combining critical and practical perspectives will contribute to respecifying taken-for-granted notions of language and technology, will deepen our understanding of the irreducibly social and interactive aspects of human tool use, and will allow the translation of some of these insights into technology design and engineering practices.

ElPaCo goes to Cologne for DGfS

1 March 2023

The Annual Conference of the Deutsche Gesellschaft für Sprachforschung is one of the larger linguistics conferences in Germany. This year, ElPaCo will be represented in the form of a plenary talk by PI Mark Dingemanse and a contribution to the special session on creativity and routine in interaction (AG4) by postdocs Andreas Liesenfeld and Marlou Rasenberg.

Abstract of Dingemanse’s plenary lecture:

When we laud the supreme generativity and expressivity of language, we rarely stop to think about the full range of resources that underpin it. Received views tend to focus on our remarkable mental capacities for composition and decomposition, a cognitivist perspective that makes it easy to overlook the contributions of public interaction. In this talk I presents some results from a research programme that tries to round out the picture by exploring how language is shaped by and for social interaction. I sketch how interjections —long overlooked as mere performance— are streamlined and conventionalized tools that help streamline linguistic interaction and scaffold language development. I also show how interactional resources like repair and continuers provide us with an infrastructure to balance creativity and convention in turn-by-turn interaction. The research I report on is part of the NWO-funded project Elementary Particles of Conversation and represents joint work with colleagues including Marieke Woensdregt, Andreas Liesenfeld, Marlou Rasenberg and Ada Lopez.

Update

We made many new connections, saw some old friends, and took some pictures.

DGfS plenary lecture by ElPaCO PI MD
Andreas Liesenfeld and Marlou Rasenberg present in the AG4 'Linguistic feedback' workshop
ElPaCo team members Mark (left) and Marlou and Andreas (right) present at the DGfS conference

ElPaCo at Dutch Speech Tech Day

15 February 2023

The ElPaCo project is represented at the first installment of Dutch Speech Tech Day, a new event that “connects students, researchers, developers and users of speech technology in the Netherlands.”

We’ll be presenting a poster How we talk: lessons for interactive speech technology. In keeping with the goal of the Speech Tech Day to share the latest insights, our poster provides a broad-ranging overview of recent work in the Elementary Particles of Conversation project. It summarizes work from the following papers :

  • Dingemanse, M., & Liesenfeld, A. (2022). From text to talk: Harnessing conversational corpora for humane and diversity-aware language technology. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 5614–5633. Dublin: Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.385
  • Liesenfeld, A., & Dingemanse, M. (2022). Building and curating conversational corpora for diversity-aware language science and technology. Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), 1178–1192. Marseille. doi: 10.48550/arXiv.2203.03399
  • Liesenfeld, A., & Dingemanse, M. (2022). Bottom-up discovery of structure and variation in response tokens (‘backchannels’) across diverse languages. Proceeding of Interspeech 2022. 2022. doi: 10.21437/Interspeech.2022-11288
  • Lopez, A., Liesenfeld, A., & Dingemanse, M. (2022). Evaluation of Automatic Speech Recognition for Conversational Speech in Dutch, English and German: What Goes Missing? Proceedings of the 18th Conference on Natural Language Processing (KONVENS 2022). Potsdam.

New position paper: Beyond Single-Mindedness

8 February 2023

The ElPaCo project has spearheaded an ambitious position paper in the Progress and Puzzles in Cognitive Science series. Assembling a broad collective of 28 authors from across the cognitive sciences and beyond, we argue for a figure-ground reversal that puts interaction at the heart of cognition. Read the paper (doi, pdf) or check out the companion website for more information.

“Key concerns of cognitive science may be illuminated by a change of perspective that locates cognition not in isolated but in interacting minds.” (source)