The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation
Sound-symbolic word classes are found in different cultures and languages worldwide. These words are continuously produced to code complex information about events. Here we explore the capacity of creative language to transport complex multisensory information in a controlled experiment, where our p...
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | JOUR |
| Materias: | |
| Acceso en línea: | http://hdl.handle.net/20.500.12110/paper_19326203_v13_n3_p_Taitz |
| Aporte de: |
| id |
todo:paper_19326203_v13_n3_p_Taitz |
|---|---|
| record_format |
dspace |
| spelling |
todo:paper_19326203_v13_n3_p_Taitz2023-10-03T16:34:57Z The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation Taitz, A. Florencia Assaneo, M. Elisei, N. Trípodi, M. Cohen, L. Sitt, J.D. Trevisan, M.A. adult article classifier consonant controlled study female human human experiment lip male physics sound stimulus tongue vowel hearing language middle aged phonetics physiology sound speech perception theoretical model vision voice young adult Adult Auditory Perception Female Humans Language Male Middle Aged Models, Theoretical Phonetics Physics Sound Speech Perception Visual Perception Voice Young Adult Sound-symbolic word classes are found in different cultures and languages worldwide. These words are continuously produced to code complex information about events. Here we explore the capacity of creative language to transport complex multisensory information in a controlled experiment, where our participants improvised onomatopoeias from noisy moving objects in audio, visual and audiovisual formats. We found that consonants communicate movement types (slide, hit or ring) mainly through the manner of articulation in the vocal tract. Vowels communicate shapes in visual stimuli (spiky or rounded) and sound frequencies in auditory stimuli through the configuration of the lips and tongue. A machine learning model was trained to classify movement types and used to validate generalizations of our results across formats. We implemented the classifier with a list of cross-linguistic onomatopoeias simple actions were correctly classified, while different aspects were selected to build onomatopoeias of complex actions. These results show how the different aspects of complex sensory information are coded and how they interact in the creation of novel onomatopoeias. © 2018 Taitz et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. JOUR info:eu-repo/semantics/openAccess http://creativecommons.org/licenses/by/2.5/ar http://hdl.handle.net/20.500.12110/paper_19326203_v13_n3_p_Taitz |
| institution |
Universidad de Buenos Aires |
| institution_str |
I-28 |
| repository_str |
R-134 |
| collection |
Biblioteca Digital - Facultad de Ciencias Exactas y Naturales (UBA) |
| topic |
adult article classifier consonant controlled study female human human experiment lip male physics sound stimulus tongue vowel hearing language middle aged phonetics physiology sound speech perception theoretical model vision voice young adult Adult Auditory Perception Female Humans Language Male Middle Aged Models, Theoretical Phonetics Physics Sound Speech Perception Visual Perception Voice Young Adult |
| spellingShingle |
adult article classifier consonant controlled study female human human experiment lip male physics sound stimulus tongue vowel hearing language middle aged phonetics physiology sound speech perception theoretical model vision voice young adult Adult Auditory Perception Female Humans Language Male Middle Aged Models, Theoretical Phonetics Physics Sound Speech Perception Visual Perception Voice Young Adult Taitz, A. Florencia Assaneo, M. Elisei, N. Trípodi, M. Cohen, L. Sitt, J.D. Trevisan, M.A. The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation |
| topic_facet |
adult article classifier consonant controlled study female human human experiment lip male physics sound stimulus tongue vowel hearing language middle aged phonetics physiology sound speech perception theoretical model vision voice young adult Adult Auditory Perception Female Humans Language Male Middle Aged Models, Theoretical Phonetics Physics Sound Speech Perception Visual Perception Voice Young Adult |
| description |
Sound-symbolic word classes are found in different cultures and languages worldwide. These words are continuously produced to code complex information about events. Here we explore the capacity of creative language to transport complex multisensory information in a controlled experiment, where our participants improvised onomatopoeias from noisy moving objects in audio, visual and audiovisual formats. We found that consonants communicate movement types (slide, hit or ring) mainly through the manner of articulation in the vocal tract. Vowels communicate shapes in visual stimuli (spiky or rounded) and sound frequencies in auditory stimuli through the configuration of the lips and tongue. A machine learning model was trained to classify movement types and used to validate generalizations of our results across formats. We implemented the classifier with a list of cross-linguistic onomatopoeias simple actions were correctly classified, while different aspects were selected to build onomatopoeias of complex actions. These results show how the different aspects of complex sensory information are coded and how they interact in the creation of novel onomatopoeias. © 2018 Taitz et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
| format |
JOUR |
| author |
Taitz, A. Florencia Assaneo, M. Elisei, N. Trípodi, M. Cohen, L. Sitt, J.D. Trevisan, M.A. |
| author_facet |
Taitz, A. Florencia Assaneo, M. Elisei, N. Trípodi, M. Cohen, L. Sitt, J.D. Trevisan, M.A. |
| author_sort |
Taitz, A. |
| title |
The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation |
| title_short |
The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation |
| title_full |
The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation |
| title_fullStr |
The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation |
| title_full_unstemmed |
The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation |
| title_sort |
audiovisual structure of onomatopoeias: an intrusion of real-world physics in lexical creation |
| url |
http://hdl.handle.net/20.500.12110/paper_19326203_v13_n3_p_Taitz |
| work_keys_str_mv |
AT taitza theaudiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation AT florenciaassaneom theaudiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation AT elisein theaudiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation AT tripodim theaudiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation AT cohenl theaudiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation AT sittjd theaudiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation AT trevisanma theaudiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation AT taitza audiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation AT florenciaassaneom audiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation AT elisein audiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation AT tripodim audiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation AT cohenl audiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation AT sittjd audiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation AT trevisanma audiovisualstructureofonomatopoeiasanintrusionofrealworldphysicsinlexicalcreation |
| _version_ |
1807323314911182848 |