The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation

Sound-symbolic word classes are found in different cultures and languages worldwide. These words are continuously produced to code complex information about events. Here we explore the capacity of creative language to transport complex multisensory information in a controlled experiment, where our p...

Descripción completa

Guardado en:
Detalles Bibliográficos
Autores principales: Taitz, A., Florencia Assaneo, M., Elisei, N., Trípodi, M., Cohen, L., Sitt, J.D., Trevisan, M.A.
Formato: JOUR
Materias:
adult
article
classifier
consonant
controlled study
female
human
human experiment
lip
male
physics
sound
stimulus
tongue
vowel
hearing
language
middle aged
phonetics
physiology
speech perception
theoretical model
vision
voice
young adult
Adult
Auditory Perception
Female
Humans
Language
Male
Middle Aged
Models, Theoretical
Phonetics
Physics
Sound
Speech Perception
Visual Perception
Voice
Young Adult
Acceso en línea:http://hdl.handle.net/20.500.12110/paper_19326203_v13_n3_p_Taitz
Aporte de:
Biblioteca Digital - Facultad de Ciencias Exactas y Naturales (UBA) de Universidad de Buenos Aires
Descripción
Sumario:Sound-symbolic word classes are found in different cultures and languages worldwide. These words are continuously produced to code complex information about events. Here we explore the capacity of creative language to transport complex multisensory information in a controlled experiment, where our participants improvised onomatopoeias from noisy moving objects in audio, visual and audiovisual formats. We found that consonants communicate movement types (slide, hit or ring) mainly through the manner of articulation in the vocal tract. Vowels communicate shapes in visual stimuli (spiky or rounded) and sound frequencies in auditory stimuli through the configuration of the lips and tongue. A machine learning model was trained to classify movement types and used to validate generalizations of our results across formats. We implemented the classifier with a list of cross-linguistic onomatopoeias simple actions were correctly classified, while different aspects were selected to build onomatopoeias of complex actions. These results show how the different aspects of complex sensory information are coded and how they interact in the creation of novel onomatopoeias. © 2018 Taitz et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Ejemplares similares