Anatomy and frequency of interactive repair

A With interactive repair, another participant initiates repair, inviting a repair solution by the first; the repair initiation is a pivot, pointing both back and forward. B While a fitted response is preferred, initiating repair is always a possible next move; likewise, within repair, while a restricted format is preferred, an open format is always an option. C Across diverse languages, formats for interactive repair range fall into three types, depending on how they target the trouble in prior turn and the kind of response they typically invite; these can be ranked from less to more specific in terms of the grasp of the trouble source they display. D Empirical cumulative distribution of independent repair sequences (black curve) as they occur over time in informal conversation in a global sample of 12 languages (grey curves). Across languages, the steepest part of the slope is around 17 s, the average 84 s, and nearly all sequences occur within a 4-min window from the last.

Dingemanse, M., & Enfield, N. J. (2024). Interactive repair and the foundations of language. Trends in Cognitive Sciences, 28(1), 30–42. doi: 10.1016/j.tics.2023.09.003 PDF

How conversational data challenges speech recognition (ASR)

A Word error rates (WER) for five speech-to-text systems in six languages. B One minute of English conversation as annotated by human transcribers (top) and by five speech-to-text systems, showing that while most do some diarization, all underestimate the number of transitions and none represent overlapping turns (Whisper offers no diarization). C Speaker transitions and distribution of floor transfer offset times (all languages), showing that even ASR systems that support diarization do not represent overlapping annotations in their output.

Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023). The timing bottleneck: Why timing and overlap are mission-critical for conversational user interfaces, speech recognition and dialogue systems. Proceedings of the 24th Annual SIGdial Meeting on Discourse and Dialogue, 482–495. doi: 10.18653/v1/2023.sigdial-1.45 PDF

How speech recognition warps dialog act classification

How different speech recognition engines warp dialog act classification in the same dataset of conversational English. For 8 frequent dialog acts, coloured lines show dialog acts based on ASR output deviate from those based on human transcripts of the same data (baseline). Dot size scales to number of times a tag is assigned. Only the most frequently assigned dialog acts (with at least 25 tokens in at least one dataset) are shown here. Mean absolute percentage deviations by ASR system: nemo 27.8%, amazon 31.4%, whisper 33.8%, rev 47.4%.

Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023). The timing bottleneck: Why timing and overlap are mission-critical for conversational user interfaces, speech recognition and dialogue systems. Proceedings of the 24th Annual SIGdial Meeting on Discourse and Dialogue, 482–495. doi: 10.18653/v1/2023.sigdial-1.45 PDF

Vowel space and the colour circle

Illustration of how colour space is mapped onto vowel space based on the findings for >1100 participants in Cuskley, Dingemanse et al. 2019. Red usually goes with back vowels like /a/, while light hues like yellow and green go with front vowels like /i/ and darker hues go with /u/ and /o/. None of this is deterministic: associations vary across people and this just represents one of the most common solutions on average. Made by MD for the classroom materials in Van Leeuwen & Dingemanse 2022.

van Leeuwen, T., & Dingemanse, M. (2022). Samenwerkende zintuigen. In S. Dekker & H. Kause (Eds.), Wetenschappelijke doorbraken de klas in! (pp. 85–116). Wetenschapsknooppunt Radboud Universiteit. PDF
Cuskley, C., Dingemanse, M., Kirby, S., & van Leeuwen, T. M. (2019). Cross-modal associations and synesthesia: Categorical perception and structure in vowel–color mappings in a large online sample. Behavior Research Methods, 51(4), 1651–1675. doi: 10.3758/s13428-019-01203-7 PDF

Repair across species

Types of redoings of communicative behaviour and their interactional contingency. This diagram sums up the species-agnostic framework for studying communicative repair we introduce in a wide-ranging review of animal communication systems.

Heesen, R., Fröhlich, M., Sievers, C., Woensdregt, M., & Dingemanse, M. (2022). Coordinating social action:  A primer for the cross-species investigation of communicative repair. Philosophical Transactions of the Royal Society B: Biological Sciences, 377(1859), 20210110. doi: 10.1098/rstb.2021.0110 PDF

Vowel-colour associations

L: The vowel space with colour associations by a synaesthete. R: The same vowels displayed according to tongue position when produced. Visualization: Christine Cuskley & Mark Dingemanse. For an interactive version of this visual, see here.

van Leeuwen, T., & Dingemanse, M. (2022). Samenwerkende zintuigen. In S. Dekker & H. Kause (Eds.), Wetenschappelijke doorbraken de klas in! (pp. 85–116). Wetenschapsknooppunt Radboud Universiteit. PDF
Cuskley, C., Dingemanse, M., Kirby, S., & van Leeuwen, T. M. (2019). Cross-modal associations and synesthesia: Categorical perception and structure in vowel–color mappings in a large online sample. Behavior Research Methods, 51(4), 1651–1675. doi: 10.3758/s13428-019-01203-7 PDF

Five dimensions of alignment

The relationship between the two parts of a behavior pair can vary on five dimensions, as outlined in this table. For each dimension, we visualize two different relationships between instances of behavior—one with a solid arrow and one with a dashed arrow. For meaning, we use tangram figures to visualize the referent of speech and/or gestures

Rasenberg, M., Özyürek, A., & Dingemanse, M. (2020). Alignment in Multimodal Interaction: An Integrative Framework. Cognitive Science, 44(11). doi: 10.1111/cogs.12911 PDF

Locations and settings

There are a myriad ways to refer to places, but one useful way to think about their affordances in interaction is in terms of a distinction between locations and settings. Locations tell you where something is; settings invoke activities and actors. Many place references usefully combine the two: setting a story in the graveyard area not only localizes it for the audience in the know, but also provides a setting for ominous encounters.

Dingemanse, M., Rossi, G., & Floyd, S. (2017). Place reference in story beginnings: a cross-linguistic study of narrative and interactional affordances. Language in Society, 46(2), 129–158. doi: 10.1017/S0047404516001019 PDF

Ideophone constructions in Siwu

The canonical syntactic home of ideophones in Siwu is toward the end of the clause. A finer analysis of patterns of occurrence in the corpus reveals a number of constructions in which ideophones can occur. The five most common constructions, together accounting for 95 % of ideophone tokens, are shown here.

This type of visualization —a table with horizontal bar plot— has no name as of yet. It uses the same logic as E.J. Tufte’s sparklines, which also display numerical information inline.

Dingemanse, M. (2017). Expressiveness and system integration. On the typology of ideophones, with special reference to Siwu. STUF - Language Typology and Universals, 70(2), 363–384. doi: 10.1515/stuf-2017-0018 PDF

Two dimensions of interactive repair

Two dimensions of formats for repair initiation. The distinction between open and restricted type formats is retrospective: it is about the nature and location of the trouble in prior turn. The distinction between request and offer type formats is prospective: it is about the nature of the response that is relevant in next turn. The two dimensions together define three basic types of formats for repair initiation: (1) open request, (2) restricted request, and (3) restricted offer.

Dingemanse, M., & Enfield, N. J. (2015). Other-initiated repair across languages: towards a typology of conversational structures. Open Linguistics, 1, 98–118. doi: 10.2478/opli-2014-0007 PDF

Properties and formats of repair

Using elementary properties of interactional resources, we can capture commonalities and differences between repair formats in principled and precise ways. For instance, to capture the distinctions between four repair initiation formats in English (as presented in Sidnell 2010), we can use the following three properties: Question (is there a content question word?), Repetition (does the repair initiator repeat some material from the prior turn?) and Confirmation (does the repair initiator make confirmation relevant in next turn?).

Dingemanse, M., & Enfield, N. J. (2015). Other-initiated repair across languages: towards a typology of conversational structures. Open Linguistics, 1, 98–118. doi: 10.2478/opli-2014-0007 PDF

Elements of other-initiated repair

A repair sequence consists of a repair initiation that points back to a prior turn (identifying it as a trouble source) and points forward to a next turn (the repair solution). The visual style of this schematic was adapted in a broader account of repair in conversation by Albert & De Ruiter.

Dingemanse, M., Roberts, S. G., Baranova, J., Blythe, J., Drew, P., Floyd, S., Gisladottir, R. S., Kendrick, K. H., Levinson, S. C., Manrique, E., Rossi, G., & Enfield, N. J. (2015). Universal Principles in the Repair of Communication Problems. PLOS ONE, 10(9), e0136100. doi: 10.1371/journal.pone.0136100

Arbitrariness, iconicity and systematicity

(A, B) Words show arbitrariness when there are conventional associations between forms and meanings. Words show iconicity when there are perceptuomotor analogies between forms and meanings, here indicated by shape, size and proximity (inset). (B, C) Words show systematicity when statistical regularities in phonological form, here indicated by color, serve as cues to abstract categories such as word classes. (D) The cues involved in systematicity differ across languages and may be arbitrary. (E) The perceptual analogies involved in iconicity transcend languages and may be universal.

Dingemanse, M., Blasi, D. E., Lupyan, G., Christiansen, M. H., & Monaghan, P. (2015). Arbitrariness, iconicity and systematicity in language. Trends in Cognitive Sciences, 19(10), 603–615. doi: 10.1016/j.tics.2015.07.013 PDF

Known cross-modal associations to vowels

Diagram of attested cross-modal mappings to linguistic sound represented on typical vowel space. (Figure by first author Gwilym Lockwood.)

Lockwood, G., & Dingemanse, M. (2015). Iconicity in the lab: a review of behavioural, developmental, and neuroimaging research into sound-symbolism. Frontiers in Psychology, 6(1246), 1–14. doi: 10.3389/fpsyg.2015.01246 PDF

Deideophonisation and ideophonisation

Deideophonization turns depictive signs into descriptive ones by decreasing expressiveness and increasing morphosyntactic integration; ideophonization turns descriptive signs into depictive ones by increasing expressiveness and decreasing morphosyntactic integration.

This simple diagram was created in 2012, in a style that evokes typical Langackerian cognitive linguistics diagrams. Published (due to editorial delays) only in 2017.

The paper has a further variation on the theme, displaying the two types of ideophone constructions in Siwu as “Bound” versus “Free” and placing them on opposite ends of this continuum:

Dingemanse, M. (2017). Expressiveness and system integration. On the typology of ideophones, with special reference to Siwu. STUF - Language Typology and Universals, 70(2), 363–384. doi: 10.1515/stuf-2017-0018 PDF

Noun classification in Siwu

Nouns in Siwu come with noun class prefixes that also mark number (singular, plural, or mass). Most grammars present such classes as simple SG/PL class pairings, making it hard to see underlying regularities. In this diagram, line thickness shows relative frequency. This kind of visualization is helpful for learners but also for linguists, who may be able to use it in work on grammaticalization and change.

Dingemanse, M. (2011). The Meaning and Use of Ideophones in Siwu [PhD dissertation, Radboud University]. http://thesis.ideophone.org/