TY  - JOUR
TI  - The Impact of Spatial Cognition on Cross-Modal Interaction between Emotional Semantics and Auditory Perception
T2  - Psychology. Journal of Higher School of Economics
IS  - Psychology. Journal of Higher School of Economics
AB  - According to the embodied cognition view, abstract concepts, including emotional ones, are grounded in our sensorimotor experience of the physical environment. For example, emotionally positive words (happiness) are associated with the upper part of vertical space, whereas the negative ones (desperation) are linked with the lower parts of physical space. Interactions between different representational domains are expressed, among others, in the so-called cross-modal correspondence effect — facilitation of stimulus processing in one modality by congruent aspects of information presented in another modality (e.g., high-pitch sound — high vertical location). The present study attempted, for the first time, to induce this effect using auditory stimuli of varied pitch and emotional words with defined spatial connotations. Thirty-six volunteers (26 women, 18-34 years old) were simultaneously presented with 1000 or 2000 Hz tones and words of varied emotional valence that had been rated for their spatial associations in vertically oriented space. The participants’ task was to identify the pitch of the presented tone as high or low. The analysis of reaction times using a linear mixed-effects model demonstrated statistically significant differences between congruent (e.g., emotionally positive word — high tone) and noncongruent conditions, as well as between congruent and neutral conditions. The effect of cross-modal correspondence was also evaluated for each stimulus word as well as for high and low tones separately, showing its specificity to high-pitch sounds and a subset of words. In sum, the results showed a cross-modal correspondence effect that spans across different levels of cognitive processing: from perceptual analysis of auditory signals to verbal semantic processing of complex abstract ideas. Importantly, we also present a set of Russian-language nouns rated for their spatial and emotional properties that can be used in future psycho- and neurolinguistic studies on embodied semantics.
AU  - Ekaterina Andriushchenko
AU  - Ekaterina Blinova
AU  - Yury Shtyrov
AU  - Kirill Miroshnik
AU  - Viktor Timokhov
AU  - Armina Janyan
AU  - Olga Shcherbakova
UR  - https://psy-journal.hse.ru/en/2022-19-4/801279011.html
PY  - 2022
SP  - 736-756
VL  - 19