A dive into my work from the perspective of figures — diagrams, photographs, plots, and other visually supportive material. For other views, see publications by theme or the full list.
-
How frequent are interjections?
The occurrence of interjections in 10-min excerpts of informal dyadic conversations in six spoken languages. Every panel shows the turns of a dyadic exchange; colored dots indicate turns that belong to the top 10 most common one-word standalone turn formats in the language. These excerpts cannot support strong comparative or typological inferences; they are only meant to give an impression of the prevalence of interjections across unrelated languages.
Dingemanse, M. (2024). Interjections at the heart of language. Annual Review of Linguistics. doi: 10.1146/annurev-linguistics-031422-124743 PDF -
Anatomy and frequency of interactive repair
A With interactive repair, another participant initiates repair, inviting a repair solution by the first; the repair initiation is a pivot, pointing both back and forward. B While a fitted response is preferred, initiating repair is always a possible next move; likewise, within repair, while a restricted format is preferred, an open format is always an option. C Across diverse languages, formats for interactive repair range fall into three types, depending on how they target the trouble in prior turn and the kind of response they typically invite; these can be ranked from less to more specific in terms of the grasp of the trouble source they display. D Empirical cumulative distribution of independent repair sequences (black curve) as they occur over time in informal conversation in a global sample of 12 languages (grey curves). Across languages, the steepest part of the slope is around 17 s, the average 84 s, and nearly all sequences occur within a 4-min window from the last.
Dingemanse, M., & Enfield, N. J. (2023). Interactive repair and the foundations of language. Trends in Cognitive Sciences. doi: 10.1016/j.tics.2023.09.003 PDF -
How conversational data challenges speech recognition (ASR)
A Word error rates (WER) for five speech-to-text systems in six languages. B One minute of English conversation as annotated by human transcribers (top) and by five speech-to-text systems, showing that while most do some diarization, all underestimate the number of transitions and none represent overlapping turns (Whisper offers no diarization). C Speaker transitions and distribution of floor transfer offset times (all languages), showing that even ASR systems that support diarization do not represent overlapping annotations in their output.
Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023). The timing bottleneck: Why timing and overlap are mission-critical for conversational user interfaces, speech recognition and dialogue systems. Proceedings of the 24th Annual SIGdial Meeting on Discourse and Dialogue. doi: 10.1145/3571884.3604316 PDF -
How speech recognition warps dialog act classification
How different speech recognition engines warp dialog act classification in the same dataset of conversational English. For 8 frequent dialog acts, coloured lines show dialog acts based on ASR output deviate from those based on human transcripts of the same data (baseline). Dot size scales to number of times a tag is assigned. Only the most frequently assigned dialog acts (with at least 25 tokens in at least one dataset) are shown here. Mean absolute percentage deviations by ASR system: nemo 27.8%, amazon 31.4%, whisper 33.8%, rev 47.4%.
Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023). The timing bottleneck: Why timing and overlap are mission-critical for conversational user interfaces, speech recognition and dialogue systems. Proceedings of the 24th Annual SIGdial Meeting on Discourse and Dialogue. doi: 10.1145/3571884.3604316 PDF -
Opening up ChatGPT
ChatGPT is sufficiently well known to warrant critical scrutiny, and for this project we wrote a paper, developed a website where we track open-source instruction-tuned large language models, designed a poster for presentation at the ACM conference on Conversational User Interfaces (CUI’23) and, yes, even designed a logo that combines a key image of the open source movement with a variation on ChatGPT’s corporate logo.
Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023). Opening up ChatGPT: tracking openness, transparency, and accountability in instruction-tuned text generators. ACM Conference on Conversational User Interfaces (CUI ’23), July 19-21, Eindhoven. doi: 10.1145/3571884.3604316 PDF -
Acts of kindness around the world
A global comparative study of 8 languages on 5 continents finds that people overwhelmingly like to help one another, independent of differences in language, culture or environment. This is a surprising finding from the perspective of anthropological and economic research, which has tended to foreground differences in how people work together and share resources.
Rossi, G., Dingemanse, M., Floyd, S., Baranova, J., Blythe, J., Kendrick, K. H., Zinken, J., & Enfield, N. J. (2023). Shared cross-cultural principles underlie human prosocial behavior at the smallest scale. Scientific Reports, 13(1), 6057. doi: 10.1038/s41598-023-30580-5 PDFTags: map -
Iconicity ratings
Iconicity ratings are a key tool in psycholinguistic studies of vocabulary. This figure shows the distribution of ratings for 14,000 English words in two ways: (a) A kernel density plot of the distribution of average ratings; the dashed line indicates a normal distribution with the same mean and standard deviation; (b) standard deviations across raters (y-axis) as a function of average rating (x-axis). Extreme values are rarer, but people agree more strongly on them. (Figure by first author Bodo Winter, open data here.)
Winter, B., Lupyan, G., Perry, L. K., Dingemanse, M., & Perlman, M. (2023). Iconicity ratings for 14,000+ English words. Behavior Research Methods. doi: 10.3758/s13428-023-02112-6 PDF -
Iconicity measures across tasks
Discriminability of iconicity measures from different tasks. Iconicity ratings have been transformed so that they vary between 0 and 1 (to compare with guessing accuracies). Guesses —where people try to guess the meaning of an iconic word, or the word form belonging to a given meaning— appear to be somewhat more evenly spread than ratings. Iconicity ratings by native speakers (rightmost, showing data from Thompson et al. 2020) are on average higher than iconicity ratings by people who don’t speak the language whose words they rate, confirming the notion that native speakers will generally feel that words of their own language are more iconic. (Figure by Bonnie McLean, open data here.)
McLean, B., Dunn, M., & Dingemanse, M. (2023). Two measures are better than one: combining iconicity ratings and guessing experiments for a more nuanced picture of iconicity in the lexicon. Language and Cognition, 1–24. doi: 10.1017/langcog.2023.9 PDF -
Beyond Single-Mindedness
Seen from Earth, the movements of celestial bodies display near-intractable complexity. When taking not a single vantage point but multiple (here, Sun and Earth), suddenly the picture changes, and new forms of order become visible (Sousanis, 2015). Likewise, key concerns of cognitive science may be illuminated by a change of perspective that locates cognition not in isolated but in interacting minds.
Image from Dingemanse et al. (2023). Sources: Left: Encyclopaedia Brittanica (1771), after a similar engraving by Cassini (via); Right: Copernicus (1543) De revolutionibus orbium cœlestium.
Dingemanse, M., Liesenfeld, A., Rasenberg, M., Albert, S., Ameka, F. K., Birhane, A., Bolis, D., Cassell, J., Clift, R., Cuffari, E., De Jaegher, H., Dutilh Novaes, C., Enfield, N. J., Fusaroli, R., Gregoromichelaki, E., Hutchins, E., Konvalinka, I., Milton, D., Rączaszek-Leonardi, J., … Wiltschko, M. (2023). Beyond Single-Mindedness: A Figure-Ground Reversal for the Cognitive Sciences. Cognitive Science, 47. doi: 10.1111/cogs.13230 PDFTags: illustration -
Multimodal effort in repair sequences
Boxplots showing the joint amount of multimodal effort invested by both participants to resolve the interactional trouble. The boxes represent the interquartile range; the middle line the median; the whiskers the minimum and maximum scores (outliers excluded). Every dot represents a repair sequence, i.e., repair initiation and repair solution together. As the specificity of repair formats goes up, joint multimodal effort invested goes down.
Rasenberg, M., Pouw, W., Özyürek, A., & Dingemanse, M. (2022). The multimodal nature of communicative efficiency in social interaction. Scientific Reports, 12(1), 19111. doi: 10.1038/s41598-022-22883-w PDF -
Vowel space and the colour circle
Illustration of how colour space is mapped onto vowel space based on the findings for >1100 participants in Cuskley, Dingemanse et al. 2019. Red usually goes with back vowels like /a/, while light hues like yellow and green go with front vowels like /i/ and darker hues go with /u/ and /o/. None of this is deterministic: associations vary across people and this just represents one of the most common solutions on average. Made by MD for the classroom materials in Van Leeuwen & Dingemanse 2022.
van Leeuwen, T., & Dingemanse, M. (2022). Samenwerkende zintuigen. In S. Dekker & H. Kause (Eds.), Wetenschappelijke doorbraken de klas in! (pp. 85–116). Wetenschapsknooppunt Radboud Universiteit. PDFCuskley, C., Dingemanse, M., Kirby, S., & van Leeuwen, T. M. (2019). Cross-modal associations and synesthesia: Categorical perception and structure in vowel–color mappings in a large online sample. Behavior Research Methods, 51(4), 1651–1675. doi: 10.3758/s13428-019-01203-7 PDF -
Simulating phonetic evolution
Plots of where in a phonetic possibility space different words end up after 10,000 rounds of interaction, across 20 independent simulation runs (each cloud of 100 exemplar dots/triangles represents a single word at round 10,000 of a single simulation run). Blue, yellow, green and orange are regular words; purple is the continuer word. On each independent simulation run, all words are initialised at randomly selected positions in the space. A shows a selection of 6 separate simulation runs chosen for illustrative purposes (showing how regular words end up in different positions); B shows the end-state of all 20 simulation runs overlaid. Parameter settings: (i) minimal effort bias 3 times as strong for continuer word (G=1250) than for regular vocabulary words (G=5000), and (ii) the bias for reuse of features (i.e. segment-similarity bias) is not applied to the continuer category.
Dingemanse, M., Liesenfeld, A., & Woensdregt, M. (2022). Convergent cultural evolution of continuers (mmhm). The Evolution of Language: Proceedings of the Joint Conference on Language Evolution (JCoLE), 61–67. PDF -
Sequential context of continuers
A Candidate continuer forms in 10 unrelated languages, B shown in their natural sequential ecology (annotations as in the original data), C with spectrograms and pitch traces of representative tokens made using the Parselmouth interface to Praat (Jadoul et al., 2018; Boersma & Weenink, 2013).
Dingemanse, M., Liesenfeld, A., & Woensdregt, M. (2022). Convergent cultural evolution of continuers (mmhm). The Evolution of Language: Proceedings of the Joint Conference on Language Evolution (JCoLE), 61–67. PDF -
Sampling response tokens
A. Overview of included languages with dataset size in hours and top 3 sequentially identified response tokens as transcribed in the corpus. B. Location of largest speech community. C. Assessing the impact of sparse data on UMAP projections using three samples of Dutch response tokens. A look at the full dataset (a) and random-sampled subsets of decreasing size (b, c) suggests isomorphism across scales and interpretability of clustering solutions as small as 150 tokens.
Liesenfeld, A., & Dingemanse, M. (2022). Bottom-up discovery of structure and variation in response tokens (‘backchannels’) across diverse languages. Proceedings of Interspeech 2022, 1126–1130. doi: 10.21437/Interspeech.2022-11288 PDF -
Cultural evolution of continuers
Continuers (frequent standalone utterances like mm-hm that people often use in succession) differ in interesting ways from other elements that are common, like top tokens (the most common words in a corpus) and discontinuers (frequent standalone utterances that people do not produce in successive streaks). A. Length of tokens for continuers, discontinuers and top tokens in 32 languages. B. Frequencies of major sound classes across types. Vowel nuclei occur across types, but continuers stand out for their preferences for nasals. C. Random forest analysis of 118 continuer forms in 32 spoken languages showing the top 10 most predictive phonemes (out of 29 attested).
Dingemanse, M., Liesenfeld, A., & Woensdregt, M. (2022). Convergent cultural evolution of continuers (mmhm). The Evolution of Language: Proceedings of the Joint Conference on Language Evolution (JCoLE), 61–67. PDF -
Clustering response tokens
Response tokens like English mhmm, uhuhh, yeah or Catalan mm, sí, vale are tricky to study in the wild: their phonetic realizations can be quite different from how they are transcribed. Here we use UMAP, a method for dimensionality reduction used in bioacoustics and other fields, to explore the shape of inventories of response tokens in 16 languages. Every point represents a single response token; the closer two points are the more similar they are acoustically. Spectrograms drawn around the rim of the plots provide a direct view of the acoustic structure of tokens and enable quick sanity checks.
Liesenfeld, A., & Dingemanse, M. (2022). Bottom-up discovery of structure and variation in response tokens (‘backchannels’) across diverse languages. Proceedings of Interspeech 2022, 1126–1130. doi: 10.21437/Interspeech.2022-11288 PDF -
Quality control for conversational corpora
Conversational data can be transcribed in many ways. This panel provides a quick way to gauge the quality of transcriptions, here illustrated with data from Ambel (Arnold, 2017). A. Distribution of the timing of dyadic turn-transitions with positive values representing gaps between turns and negative values representing overlaps.
This kind of normal distribution centered around 0 ms is typical; when corpora starkly diverge from this it usually indicates noninteractive data, or segmentation methods that do not represent the actual timing of utterances. B. Distribution of transition time by duration, allowing the spotting of outliers and artefacts of automation (e.g. many turns of similar durations). C. A frequency/rank plot allows a quick sanity check of expected power law distributions and a look at the most frequent tokens in the corpus. D. Three randomly selected 10 second stretches of dyadic conversation give an impression of the timing and content of annotations in the corpus.Liesenfeld, A., & Dingemanse, M. (2022). Building and curating conversational corpora for diversity-aware language science and technology. Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), 1178–1192. https://aclanthology.org/2022.lrec-1.126 PDF -
How ASR training data differs from real conversation
L: Distributions of durations of utterances and sentences (in ms) in corpora of informal conversation (blue) and CommonVoice ASR training sets (red) in Hungarian, Dutch, and Catalan. Modal duration and annotation content differ dramatically by data type: 496ms (6 words, 27 characters) for conversational turns and 4642ms (10 words, 58 characters) for ASR training items. R: Visualization of tokens that feature more prominently in conversational data (blue) and ASR training data (red) in Dutch. Source data: 80k randomsampled items from the Corpus of Spoken Dutch (Taalunie, 2014) and the Common Voice corpus for automatic speech recognition in Dutch (Ardila et al., 2020), based on Scaled F score metric, plotted using scattertext (Kessler, 2017)
Liesenfeld, A., & Dingemanse, M. (2022). Building and curating conversational corpora for diversity-aware language science and technology. Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), 1178–1192. https://aclanthology.org/2022.lrec-1.126 PDF -
A survey of conversational corpora
Under the auspices of various language documentation projects, language resources have been collected in more and more communities across the world, and these often include at least some conversational data. Such corpora harbour important insights for language science and technology. This map plots >60 corpora found to offer at least some conversational data.
Liesenfeld, A., & Dingemanse, M. (2022). Building and curating conversational corpora for diversity-aware language science and technology. Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), 1178–1192. https://aclanthology.org/2022.lrec-1.126 PDFTags: map -
Repair across species
Types of redoings of communicative behaviour and their interactional contingency. This diagram sums up the species-agnostic framework for studying communicative repair we introduce in a wide-ranging review of animal communication systems.
Heesen, R., Fröhlich, M., Sievers, C., Woensdregt, M., & Dingemanse, M. (2022). Coordinating social action: A primer for the cross-species investigation of communicative repair. Philosophical Transactions of the Royal Society B: Biological Sciences, 377(1859), 20210110. doi: 10.1098/rstb.2021.0110 PDF -
From text to talk
Most NLP methods and models focus on text rather than talk. What are they missing? Scattertext plot of words and phrases characteristic of spoken interaction (green) versus written text (purple) in English, with words most characteristic of conversational interaction in the upper left (and shown in a separate inset on the right). High-frequency metacommunicative interjections like uhhuh, hm, wow, um are most typical of talk, and most often underrepresented in text.
Dingemanse, M., & Liesenfeld, A. (2022). From text to talk: Harnessing conversational corpora for humane and diversity-aware language technology. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 5614–5633. doi: 10.18653/v1/2022.acl-long.385 PDF -
Mhmm over time
Even apparently universal patterns (like the use of ‘mhm’ during tellings) can show important cross-cultural differences. A. Continuers (marked ○) are among the most frequent recipient behaviours in both English and Korean, shown here in four 80 second stretches of tellings. B. However, the relative frequency of continuers is about twice as high in Korean based on 100 random samples of 80 second segments in both languages: on average, 21% of turns are continuers in Korean, against 9% of turns in English (measures expressed this way to control for speech rate differences).
Dingemanse, M., & Liesenfeld, A. (2022). From text to talk: Harnessing conversational corpora for humane and diversity-aware language technology. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 5614–5633. doi: 10.18653/v1/2022.acl-long.385 PDF -
Timing of yes/no sequences
Assessing the timing of turn-taking requires careful operationalisation. The largest comparative study so far (Stivers et al., 2009) looked at polar questions and their answers in order to have a directly comparable sequential context.
In our paper on conversational corpora, we use this same sequential context, and compare it to the larger set of dyadic speaker transitions in interaction. Given the broad-scale comparability of the overall timing distributions (in grey) and the more controlled subset of at least 250 question-answer sequences per language (in black), we conclude that QA sequences can act as a useful proxy for timing in general (supporting Stivers et al. 2009), but also that QA-sequences are not necessary for a relatively robust impression of overall timing.
Dingemanse, M., & Liesenfeld, A. (2022). From text to talk: Harnessing conversational corpora for humane and diversity-aware language technology. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 5614–5633. doi: 10.18653/v1/2022.acl-long.385 PDF -
Vowel-colour associations
L: The vowel space with colour associations by a synaesthete. R: The same vowels displayed according to tongue position when produced. Visualization: Christine Cuskley & Mark Dingemanse. For an interactive version of this visual, see here.
van Leeuwen, T., & Dingemanse, M. (2022). Samenwerkende zintuigen. In S. Dekker & H. Kause (Eds.), Wetenschappelijke doorbraken de klas in! (pp. 85–116). Wetenschapsknooppunt Radboud Universiteit. PDFCuskley, C., Dingemanse, M., Kirby, S., & van Leeuwen, T. M. (2019). Cross-modal associations and synesthesia: Categorical perception and structure in vowel–color mappings in a large online sample. Behavior Research Methods, 51(4), 1651–1675. doi: 10.3758/s13428-019-01203-7 PDF -
Zipf in conversation
Frequency/rank distributions of tokenized items (‘words’) and recurring turn formats in conversational corpora with at least 20 such turn formats, representing 22 languages (8 phyla). Tokenized items (blue) show a linear frequency/rank relation in log/log space. Recurring turn formats (whether one-word ○ or multi-word ＋) appear to obey a similar frequency/rank distribution for the 20% of turns that occur >20 times (purple), tapering off towards lower frequencies and unique turns (grey). Fit fluctuates with corpus size and the parallelism of distributions is most apparent in larger corpora.
Dingemanse, M., & Liesenfeld, A. (2022). From text to talk: Harnessing conversational corpora for humane and diversity-aware language technology. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 5614–5633. doi: 10.18653/v1/2022.acl-long.385 PDF -
/r/ for rough in Indo-European
A Across the Indo-European language family, the proportion of rough words with /r/ is much higher than the proportion of smooth words with /r/; B Each dot represents a language (size of the circle = number of words); whiskers show 95% Bayesian credible intervals corresponding to the mixed-effects Bayesian logistic regression analysis indicating that rough words have a much higher proportion of /r/ (posterior mean = 63%) than smooth words (posterior mean= 35%).
Winter, B., Sóskuthy, M., Perlman, M., & Dingemanse, M. (2022). Trilled /r/ is associated with roughness, linking sound and touch across spoken languages. Scientific Reports, 12(1), 1035. doi: 10.1038/s41598-021-04311-7 PDF -
/r/ for rough
How phonemes relate to roughness ratings. Left: The ten most predictive phonemes in a random forest analysis for A English and B Hungarian; the vertical black line corresponds to the absolute value of the least predictive phoneme, which is a heuristic cut-off rule for predictors that do not contribute. Right: Boxplots (whiskers = smallest, largest value within 1.5× IQR) for words with and without /r/ in English and Hungarian.
Winter, B., Sóskuthy, M., Perlman, M., & Dingemanse, M. (2022). Trilled /r/ is associated with roughness, linking sound and touch across spoken languages. Scientific Reports, 12(1), 1035. doi: 10.1038/s41598-021-04311-7 PDF -
Rolling /r/ around the world
Map accompanying news coverage of our study of the link between /r/ and rough textures. The red data points represent languages that often feature /r/ in words with words for rough textures but not words for smooth textures. Blue data points, much rarer, are cases where the pattern is the reverse. The map shows that overwhelmingly, languages prefer to express rough meanings with /r/ sounds (if they have them).
Winter, B., Sóskuthy, M., Perlman, M., & Dingemanse, M. (2022). Trilled /r/ is associated with roughness, linking sound and touch across spoken languages. Scientific Reports, 12(1), 1035. doi: 10.1038/s41598-021-04311-7 PDF -
Gesture kinematics
Setup of a study using motion tracking to investigate continuous properties of evolving manual signals. Panel a: Seed gestures for a fixed set of meanings are learned by next generations in an iterative learning experiment. Panel b: Using motion tracking, we derive automatic kinematic measures of entropy, temporal variability and intermittency over time and over generations.
Pouw, W., Dingemanse, M., Motamedi, Y., & Özyürek, A. (2021). A Systematic Investigation of Gesture Kinematics in Evolving Manual Languages in the Lab. Cognitive Science, 45(7), e13014. doi: 10.1111/cogs.13014 PDF -
Bootstraps, bridges and scaffolds
Graphic I made for a talk about our paper on roles of iconicity in words learning. As part of this paper we briefly review the role of metaphors in theories about language & development.
Nielsen, A. K. S., & Dingemanse, M. (2021). Iconicity in Word Learning and Beyond: A Critical Review. Language and Speech, 64(1), 52–72. doi: 10.1177/0023830920914339 PDFTags: illustration -
The iconicity boom
Proportional number of publications cataloged in Web of Science (1900–2017), showing concurrent upsurges in six topics related to iconicity (corrected for overall publication volume).
Nielsen, A. K. S., & Dingemanse, M. (2021). Iconicity in Word Learning and Beyond: A Critical Review. Language and Speech, 64(1), 52–72. doi: 10.1177/0023830920914339 PDF -
Five dimensions of alignment
The relationship between the two parts of a behavior pair can vary on five dimensions, as outlined in this table. For each dimension, we visualize two different relationships between instances of behavior—one with a solid arrow and one with a dashed arrow. For meaning, we use tangram figures to visualize the referent of speech and/or gestures
Rasenberg, M., Özyürek, A., & Dingemanse, M. (2020). Alignment in Multimodal Interaction: An Integrative Framework. Cognitive Science, 44(11). doi: 10.1111/cogs.12911 PDF -
Computational complexity of repair and pragmatic reasoning
A comparison of computational complexity (in basic computation steps) by agent type and lexicon size. The main take-away from this figure is that complexity increases exponentially with lexicon size for pragmatic agents, but only linearly for interactional agents. Three types of agent are compared, each with three lexicon sizes. The Interactional agent is a model equipped with a simple form of repair and no pragmatic reasoning. The other two agents cannot initiate repair, but instead feature pragmatic reasoning. The Frugally Pragmatic agent is a model that only uses complex pragmatic reasoning above a certain uncertainty threshold; the Fully Pragmatic agent always uses it. For interactional agents with a 6 × 4 lexicon no data is visible as the computation cost is very small (48) relative to the range of the y-axis.
Arkel, J. van, Woensdregt, M., Dingemanse, M., & Blokpoel, M. (2020). A simple repair mechanism can alleviate computational demands of pragmatic reasoning: simulations and complexity analysis. Proceedings of the 24th Conference on Computational Natural Language Learning. doi: 10.18653/v1/2020.conll-1.14 PDF -
Shooing words
Shooing words —words that people use to chase away chickens— turn out to be highly similar across unrelated languages. These illustrations by Josje van Koppen accompanied a write-up about my serendipitous finding in popular science magazine Onze Taal.
The actual table from my paper looks a lot less exciting, but it does contain additional information about language families and about words for ‘chicken’ in the same set of languages. The basic conclusions is that words for ‘shoo’, but not ‘chicken’, show strong convergence towards sibilant sounds in 17 languages from 11 unrelated language families.
Illustrations from: Renckens, Erica. “‘Ksst!’ Het Lokken En Wegjagen van Dieren.” Onze Taal, 2020.
Dingemanse, M. (2020). Recruiting assistance and collaboration: a West-African corpus study. In S. Floyd, G. Rossi, & N. J. Enfield (Eds.), Getting others to do things: A pragmatic typology of recruitments (pp. 369–421). Language Science Press. PDF -
Three barreled request
After a check of starting conditions (‘take uh:. is there water there?’, line 1), a sequence of pointing gestures accompanies a three-barreled request: ‘take this gallon’, ‘pour some water’, ‘put it on the fire’.
Dingemanse, M. (2020). Recruiting assistance and collaboration: a West-African corpus study. In S. Floyd, G. Rossi, & N. J. Enfield (Eds.), Getting others to do things: A pragmatic typology of recruitments (pp. 369–421). Language Science Press. PDF -
Getting others to do things
Interactional challenges to be negotiated in recruitment sequences, along with some of the interactional practices mobilized to address them.
Not a figure, I know, but sometimes tables are the only way to bring multidimensional problem space into view. In this case, the table is also a map to the resources discussed in the paper.
Dingemanse, M. (2020). Recruiting assistance and collaboration: a West-African corpus study. In S. Floyd, G. Rossi, & N. J. Enfield (Eds.), Getting others to do things: A pragmatic typology of recruitments (pp. 369–421). Language Science Press. https://doi.org/10.5281/zenodo.4018387 PDF -
Iconicity and funniness ratings
The intersection of iconicity and funniness ratings for 1419 words. A: Scatterplot of iconicity and funniness ratings in which each dot corresponds to a word. A loess function generates the smoothed conditional mean with 0.95 confidence interval. Panels B and C show the distribution of iconicity and funniness ratings in this dataset.
Dingemanse, M., & Thompson, B. (2020). Playful iconicity: structural markedness underlies the relation between funniness and iconicity. Language and Cognition, 12(1), 203–224. doi: 10.1017/langcog.2019.49 PDF -
“Hand me that phone”
Bella is holding Aku’s phone and taking a call Aku asked her to pick up. Speaking into the phone, she notes she is ‘not sister Aku’. When it becomes clear the caller wants Aku, Aku asks Bella to give the phone back, adding a gesture of reaching out to receive the phone. These kinds of events (‘recruitments’) are frequent in everyday interaction and show how people weave together talk and action to get others to do things.
Dingemanse, M. (2020). Recruiting assistance and collaboration: a West-African corpus study. In S. Floyd, G. Rossi, & N. J. Enfield (Eds.), Getting others to do things: A pragmatic typology of recruitments (pp. 369–421). Language Science Press. https://doi.org/10.5281/zenodo.4018387 PDF -
The space between our heads
Not strictly a scientific visualization, and not by me. Still included here because it is a compelling illustration of the central point of this essay on brain-to-brain interfaces, which deals with naïve ideas about a cyberpunk future in which we’d be connected by wires instead of words. (Source of the image is Technology Review, who got it from shutterstock.)
-
Playful iconicity
Illustration accompanying news coverage in NRC of our paper on playful iconicity: when words sound like what they mean. By Jet Peters.
Dingemanse, M., & Thompson, B. (2020). Playful iconicity: structural markedness underlies the relation between funniness and iconicity. Language and Cognition, 12(1), 203–224. doi: 10.1017/langcog.2019.49 PDF -
Structural markedness
The relation between structural markedness and funniness ratings (A), iconicity ratings (B), and funniness and iconicity together (C), in a set of 1.419 English words. Each dot represents 14 or 15 words. Solid line with smoothed mean shows cumulative markedness. Other lines show relative prevalence of complex onsets (flap), codas (clunk), and verbal diminutives (drizzle). Higher structural markedness goes together with higher iconicity and funniness ratings. This supports the theory of structural markedness as a metacommunicative cue.
Dingemanse, M., & Thompson, B. (2020). Playful iconicity: structural markedness underlies the relation between funniness and iconicity. Language and Cognition, 12(1), 203–224. doi: 10.1017/langcog.2019.49 PDF -
Vowel-colour associations
Vowel-colour associations for 1164 participants (central panel), showing, clockwise from bottom left, (a) a participant with very low structure yet high consistency across trials, probably a false positive synaesthete, (b) a typical nonsynaesthete with mappings that are both inconsistent and unstructured; (c) a middling participant with significant structure but inconistent choices across trials; (d) a highly structured but inconsistent participant; and (e) a typical vowel-colour synaesthete, with highly structured, consistent and categorical mappings.
Cuskley, C., Dingemanse, M., Kirby, S., & van Leeuwen, T. M. (2019). Cross-modal associations and synesthesia: Categorical perception and structure in vowel–color mappings in a large online sample. Behavior Research Methods, 51(4), 1651–1675. doi: 10.3758/s13428-019-01203-7 PDF -
Codability of sensory domains
The hierarchy of the senses across languages according to the mean codability of each domain, with the presumed universal Aristotelian hierarchy on Top. There is no universal hierarchy of the senses across diverse languages worldwide. (Figure by coauthor Sean G. Roberts, open data here.)
Majid, A., Roberts, S. G., Cilissen, L., Emmorey, K., Nicodemus, B., O’Grady, L., Woll, B., LeLan, B., de Sousa, H., Cansler, B. L., Shayan, S., de Vos, C., Senft, G., Enfield, N. J., Razak, R. A., Fedden, S., Tufvesson, S., Dingemanse, M., Ozturk, O., … Levinson, S. C. (2018). Differential coding of perception in the world’s languages. Proceedings of the National Academy of Sciences, 115(45), 11369–11376. doi: 10.1073/pnas.1720419115 PDFTags: graph -
Language of perception
Languages (and researchers) contributing to a large comparative study of the differential coding of perception across cultures. Locations indicate field sites where data were collected.
Majid, A., Roberts, S. G., Cilissen, L., Emmorey, K., Nicodemus, B., O’Grady, L., Woll, B., LeLan, B., de Sousa, H., Cansler, B. L., Shayan, S., de Vos, C., Senft, G., Enfield, N. J., Razak, R. A., Fedden, S., Tufvesson, S., Dingemanse, M., Ozturk, O., … Levinson, S. C. (2018). Differential coding of perception in the world’s languages. Proceedings of the National Academy of Sciences, 115(45), 11369–11376. doi: 10.1073/pnas.1720419115 PDFTags: map -
Locations and settings
There are a myriad ways to refer to places, but one useful way to think about their affordances in interaction is in terms of a distinction between locations and settings. Locations tell you where something is; settings invoke activities and actors. Many place references usefully combine the two: setting a story in the graveyard area not only localizes it for the audience in the know, but also provides a setting for ominous encounters.
Dingemanse, M., Rossi, G., & Floyd, S. (2017). Place reference in story beginnings: a cross-linguistic study of narrative and interactional affordances. Language in Society, 46(2), 129–158. doi: 10.1017/S0047404516001019 PDF -
Ideophone constructions in Siwu
The canonical syntactic home of ideophones in Siwu is toward the end of the clause. A finer analysis of patterns of occurrence in the corpus reveals a number of constructions in which ideophones can occur. The five most common constructions, together accounting for 95 % of ideophone tokens, are shown here.
This type of visualization —a table with horizontal bar plot— has no name as of yet. It uses the same logic as E.J. Tufte’s sparklines, which also display numerical information inline.
Dingemanse, M. (2017). Expressiveness and system integration. On the typology of ideophones, with special reference to Siwu. STUF – Language Typology and Universals, 70(2), 363–384. doi: 10.1515/stuf-2017-0018 PDF -
How ideophones stand out
Pitch trace of a Japanese utterance starting with two tokens of the ideophone zabɯ:n ‘splash’, showing how they are produced in the upper part of the speaker’s pitch range, and how their articulation is drawn out relative to other non-ideophonic elements in the utterance. This illustrates the special treatment that ideophones often get in everyday speech, which makes them stand out from the surrounding material.
Dingemanse, M., & Akita, K. (2017). An inverse relation between expressiveness and grammatical integration: on the morphosyntactic typology of ideophones, with special reference to Japanese. Journal of Linguistics, 53(3), 501–532. doi: 10.1017/S002222671600030X PDF -
Two dimensions of interactive repair
Two dimensions of formats for repair initiation. The distinction between open and restricted type formats is retrospective: it is about the nature and location of the trouble in prior turn. The distinction between request and offer type formats is prospective: it is about the nature of the response that is relevant in next turn. The two dimensions together define three basic types of formats for repair initiation: (1) open request, (2) restricted request, and (3) restricted offer.
Dingemanse, M., & Enfield, N. J. (2015). Other-initiated repair across languages: towards a typology of conversational structures. Open Linguistics, 1, 98–118. doi: 10.2478/opli-2014-0007 PDF -
Properties and formats of repair
Using elementary properties of interactional resources, we can capture commonalities and differences between repair formats in principled and precise ways. For instance, to capture the distinctions between four repair initiation formats in English (as presented in Sidnell 2010), we can use the following three properties: Question (is there a content question word?), Repetition (does the repair initiator repeat some material from the prior turn?) and Confirmation (does the repair initiator make confirmation relevant in next turn?).
Dingemanse, M., & Enfield, N. J. (2015). Other-initiated repair across languages: towards a typology of conversational structures. Open Linguistics, 1, 98–118. doi: 10.2478/opli-2014-0007 PDF -
Probability of encountering repair
Interactive repair —when people work together to fix trouble in conversation— is quite common. In these 12 languages from around the world, it takes only 84 seconds on average between one repair sequence and the next. The sheer frequency shows how important repair is as a system that keeps conversation on track and helps us negotiate common understanding in a world full of noise. We are united in asking questions.
Dingemanse, M., Roberts, S. G., Baranova, J., Blythe, J., Drew, P., Floyd, S., Gisladottir, R. S., Kendrick, K. H., Levinson, S. C., Manrique, E., Rossi, G., & Enfield, N. J. (2015). Universal Principles in the Repair of Communication Problems. PLOS ONE, 10(9), e0136100. doi: 10.1371/journal.pone.0136100 -
‘Seeing as’
My dear friend Ruben Owiafe was one of the most colourful Siwu teachers I had. His explanation of what it means to provide folk definitions is insightful in terms of both its content and form. As he explained, they enable you to see one thing in terms of another: “If you see this here” (points to his right), “you see how it is here” (points to his left). Figures and diagrams in academic publications serve the same kind of purpose. They provide us with different ways of seeing, and help us understand by analogy.
Dingemanse, M. (2015). Folk definitions in linguistic fieldwork. In J. Essegbey, B. Henderson, & F. McLaughlin (Eds.), Language Documentation and Endangerment in Africa (pp. 215–238). John Benjamins. PDF -
Elements of other-initiated repair
A repair sequence consists of a repair initiation that points back to a prior turn (identifying it as a trouble source) and points forward to a next turn (the repair solution). The visual style of this schematic was adapted in a broader account of repair in conversation by Albert & De Ruiter.
Dingemanse, M., Roberts, S. G., Baranova, J., Blythe, J., Drew, P., Floyd, S., Gisladottir, R. S., Kendrick, K. H., Levinson, S. C., Manrique, E., Rossi, G., & Enfield, N. J. (2015). Universal Principles in the Repair of Communication Problems. PLOS ONE, 10(9), e0136100. doi: 10.1371/journal.pone.0136100 -
Arbitrariness, iconicity and systematicity
(A, B) Words show arbitrariness when there are conventional associations between forms and meanings. Words show iconicity when there are perceptuomotor analogies between forms and meanings, here indicated by shape, size and proximity (inset). (B, C) Words show systematicity when statistical regularities in phonological form, here indicated by color, serve as cues to abstract categories such as word classes. (D) The cues involved in systematicity differ across languages and may be arbitrary. (E) The perceptual analogies involved in iconicity transcend languages and may be universal.
Dingemanse, M., Blasi, D. E., Lupyan, G., Christiansen, M. H., & Monaghan, P. (2015). Arbitrariness, iconicity and systematicity in language. Trends in Cognitive Sciences, 19(10), 603–615. doi: 10.1016/j.tics.2015.07.013 PDF -
Folk definitions of ideophones
Ideophones have rich imagistic meanings that can be hard to describe. In explaining the ideophone minimini, four speakers of Siwu independently use gestures that are both similar (in depicting a spherical shape) and different (in size, handshape, and method of representation). Collectively, the gestures illustrate an elusive aspect of the ideophone’s meaning while also showing that its linguistic form as spoken word is more conventionalized than the gestures it comes with.
Dingemanse, M. (2015). Folk definitions in linguistic fieldwork. In J. Essegbey, B. Henderson, & F. McLaughlin (Eds.), Language Documentation and Endangerment in Africa (pp. 215–238). John Benjamins. PDF -
Magritte on depiction
Dingemanse, M. (2015). Ideophones and reduplication: Depiction, description, and the interpretation of repeated talk in discourse. Studies in Language, 39(4), 946–970. doi: 10.1075/sl.39.4.05din PDF -
Known cross-modal associations to vowels
Diagram of attested cross-modal mappings to linguistic sound represented on typical vowel space. (Figure by first author Gwilym Lockwood.)
Lockwood, G., & Dingemanse, M. (2015). Iconicity in the lab: a review of behavioural, developmental, and neuroimaging research into sound-symbolism. Frontiers in Psychology, 6(1246), 1–14. doi: 10.3389/fpsyg.2015.01246 PDF -
Which words are the same across languages?
Illustration made by Frank Landsbergen for a piece on universal words I wrote for a popular science book. It covers three types of words that, each for their own reason, come out similarly across languages. The three types are: (i) interactional tools (huh? for repair, oh! for a news receipt); (ii) expressive interjections (au for ‘ouch’); and (iii) onomatopoeia (bam ‘BAM’).
Simplifying somewhat, interactional tools are similar across languages because the ecology they live in (the rapid-fire turn-taking of conversation) provides the same selective pressures across languages; a case of convergent cultural evolution. Expressive interjections may go back to ancestral vocalizations also found in our close evolutionary relatives. And onomatopoeia come out similarly to the extent that they imitate the same kinds of sounds.
Dingemanse, M. (2014). Welk woord is in elke taal hetzelfde? In S. Deurloo (Ed.), Waarom drinken we zoveel koffie? 101 slimme vragen (pp. 159–161). Kennislink. PDFDingemanse, M. (2023). Interjections. In E. van Lier (Ed.), The Oxford Handbook of Word Classes. Oxford University Press. https://doi.org/10.31234/osf.io/ngcrs PDFDingemanse, M. (2017). On the margins of language: Ideophones, interjections and dependencies in linguistic theory. In N. J. Enfield (Ed.), Dependencies in language (pp. 195–202). Language Science Press. PDF -
The Austin/Clark action ladder
Herb Clark, building on Austin’s (1962) distinctions of levels of speech acts, notes that successful communication is grounded in joint actions by speaker and addressee at at least four distinct levels. In the Austin/Clark action ladder, higher levels depend on lower levels in terms of causality (higher levels are implemented by means of lower ones) and entailment (completion of a higher level entails completion of the ones below it). As a corollary, the action ladder exhibits the property of “downward evidence”: evidence that B recognized A’s intended action (level 4) is also evidence that B succeeded in interpreting A’s words (level 3), that B correctly identified the words (level 2), and that B attended to A’s vocalisation (level 1). All four levels are involved in building mutual understanding, and each of them can be a locus of trouble.
Dingemanse, M., Blythe, J., & Dirksmeyer, T. (2014). Formats for other-initiation of repair across languages: An exercise in pragmatic typology. Studies in Language, 38(1), 5–43. doi: 10.1075/sl.38.1.01din PDF -
Cultural evolution of continuous signals
The cultural evolution of continuous signals over 4 generations in a single experimental chain of iterated communication. Colour represents communicative success. Through trial and error, participants in consecutive trials narrow down to a set of signals that is both iconic (in mirroring aspects of form) and systematic (in using slope direction to signal the way animals are facing). This represents in miniature form how iconicity can provide the building blocks for systematicity in linguistic systems.
Dingemanse, M., Verhoef, T., & Roberts, S. G. (2014). The role of iconicity in the cultural evolution of communicative signals. In B. de Boer & T. Verhoef (Eds.), Proceedings of Evolang X Workshop on Signals, Speech and Signs (pp. 11–15). PDF -
Universal and specific aspects of social interaction
“So she was weird today,” Kofi says. In response to Aku’s “What?”, Kofi closes his eyes and moves jerkily from side to side. All present turn to him to watch. Aku checks: “The spirit- the spirit’s been coming again?” Kofi confirms and tells a story of spirit possession. The segment, only a few seconds long, illustrates both universal and culture-specific aspects of social interaction.
Dingemanse, M., & Floyd, S. (2014). Conversation across cultures. In N. J. Enfield, P. Kockelman, & J. Sidnell (Eds.), Cambridge Handbook of Linguistic Anthropology (pp. 434–464). Cambridge University Press. PDFTags: photo -
‘Huh?’ around the world
A word like huh? —used to initiate repair when, for example, one has not clearly heard what someone just said— is found in roughly the same form and function in conversational corpora from 31 spoken languages from across the globe. The ten in bold are examined in phonetic detail and found to be about as similar to each other as variants of the word dog across English varieties. Languages 11–20 are from [14], 21–31 from sources cited. Locations are approximate. 1. Cha‘palaa ʔa:↘ 2. Icelandic ha
3. Spanish e↗ 4. Siwu ã:↗ 5. Dutch h
↗ 6. Italian ε:↗ 7. Russian a:↗ 8. Lao hã:↗ 9. Mandarin Chinese ã:↗ 10. Murrinh-Patha a:↗ 11. ‡Âkhoe Hai//om hε↗ 12. Chintang hã↗ 13. Duna ɛ̃:↗ 14. English hã↗ 15. French ɛ̃:↗ 16. Hungarian hm↗/ha↗ 17. Kri ha:↗ 18. Tzeltal hai↗ 19. Yélî Dnye ɛ̃:↗ 20. Yurakaré æ↗ 21. Lahu hãi
[38] 22. Tai/Lue há↗ [92] 23. Japanese e↗ [93] 24. Korean e↗ [94] 25. German hɛ̃ [95] 26. Norwegian hæ↗ [96] 27. Herero e↗ [97] 28. Kikongo e↗ [98] 29. Tzotzil e↗ [99] 30. Bequia Creole ha:↗ [100] 31. Zapotec aj↗ [101].
Dingemanse, M., Torreira, F., & Enfield, N. J. (2013). Is “Huh?” a universal word? Conversational infrastructure and the convergent evolution of linguistic items. PLOS ONE, 8(11), e78273. doi: 10.1371/journal.pone.0078273 -
Depiction in speech and gesture
On the ground is a plate of metal on which two small amounts of gunpowder have been laid to dry in the sun; besides it stands the speaker, explaining why one needs to be careful when igniting the gunpowder to test its quality: it may flare up “SHÛ, SHÛ”, a vocal depiction that is produced in precise synchrony with the two hands moving symmetrically in a quick upward motion. (The right hand holds an object.)
Dingemanse, M. (2013). Ideophones and gesture in everyday speech. Gesture, 13(2), 143–165. doi: 10.1075/gest.13.2.02din PDF -
Deideophonisation and ideophonisation
Deideophonization turns depictive signs into descriptive ones by decreasing expressiveness and increasing morphosyntactic integration; ideophonization turns descriptive signs into depictive ones by increasing expressiveness and decreasing morphosyntactic integration.
This simple diagram was created in 2012, in a style that evokes typical Langackerian cognitive linguistics diagrams. Published (due to editorial delays) only in 2017.
The paper has a further variation on the theme, displaying the two types of ideophone constructions in Siwu as “Bound” versus “Free” and placing them on opposite ends of this continuum:
Dingemanse, M. (2017). Expressiveness and system integration. On the typology of ideophones, with special reference to Siwu. STUF – Language Typology and Universals, 70(2), 363–384. doi: 10.1515/stuf-2017-0018 PDF -
Clustering ideophones
MDS plot of similarity ratings for ideophones derived from a pile-sorting field task. Interpretable clusters are circled and indicated in the plot. One group, with saaa ‘cool sensation’, nyagbalaa ‘pungent’, buàà ‘tasteless’, nyɛ̃kɛ̃nyɛ̃kɛ̃ ‘intensely sweet’ and mɛ̃rɛ̃mɛ̃rɛ̃ ‘sweet’, can be characterised as TASTE. Another cluster includes dɔbɔrɔɔ ‘soft’, safaraa ‘coarse-grained’, wòsòròò ‘rough’, fũɛ̃ fũɛ̃ ‘malleable’, wùrùfùù ‘fluffy’, pɔlɔpɔlɔ ‘smooth’, fiɛfiɛ ‘silky’, kpɔlɔkpɔlɔ ‘slippery’ and pɔtɔpɔtɔ ‘soggy’. These ideophones seem to form a domain of HAPTIC TOUCH. Another group is comprised of gelegele ‘shiny’, fututu ‘pure white’, kpinakpina ‘black’ and wɔ̃̀rã̀wɔ̃̀rã̀ ‘spotted’. This domain we may summarise as SURFACE APPEARANCE. A further cluster is formed by minimini ‘spherical’ and gìlìgìlì ‘circular’ (these two tightly together) and sɔ̀dzɔ̀lɔ̀ɔ̀ ‘oblong’, miɔmiɔ ‘pointed’ and tagbaraa ‘long’, suggesting a broader domain of SHAPE.
Dingemanse, M. (2011). The Meaning and Use of Ideophones in Siwu [PhD dissertation, Radboud University]. http://thesis.ideophone.org/ -
Noun classification in Siwu
Nouns in Siwu come with noun class prefixes that also mark number (singular, plural, or mass). Most grammars present such classes as simple SG/PL class pairings, making it hard to see underlying regularities. In this diagram, line thickness shows relative frequency. This kind of visualization is helpful for learners but also for linguists, who may be able to use it in work on grammaticalization and change.
Dingemanse, M. (2011). The Meaning and Use of Ideophones in Siwu [PhD dissertation, Radboud University]. http://thesis.ideophone.org/
barplot boxplot clustering conversation density depiction diagram duration ecdf frequency gesture grammar graph iconicity illustration interaction linguistics logo map MDS panel phonetics phonology photo popsci random forest repair scatterplot sequence simulation sparkline spectrogram speech synaesthesia table time series timing transcript turn-taking typology UMAP