MID Researchers at SMC, HRI, and ISMIR conferences

Alessandro IOP and Sandra PAULETTO. PERCEPTION OF EMOTIONS IN MULTIMODAL STIMULI: THE CASE OF KNOCKING ON A DOOR, Sound and Music Computing Conference, 2021

Knocking sounds are highly expressive. In our previous research we have shown that from the sound of knocking actions alone a person can differentiate between different basic emotional states. In media productions, such as film and games, knocks can be very important storytelling de- vices as they allow the story to transition from one part to another. Research has shown that colours can affect our perception of emotions. However the relationship between colours and emotions is complex and dependent on mul- tiple factors. In this study we investigate how the visual characteristics of a door, more specifically its colour, tex- ture and material, presented together with emotionally ex- pressive knocking actions, can affect the perception of the overall emotion evoked in the audience. Results show that the door's visual characteristics have little effect on the overall perception of emotions, which remains dominated by the emotions expressed by the knocking sounds.

Roberto Bresin, Emma Frid, Adrian Benigno Latupeirissa and Claudio Panariello. Robust Non-Verbal Expression in Humanoid Robots: New Methods for Augmenting Expressive Movements with Sound. Workshop on Sound in Human-Robot Interaction at HRI 2021

The aim of the SONAO project is to establish new methods based on sonification of expressive movements for achieving a robust interaction between users and humanoid robots. We want to achievethis by combining
competences of the research team members inthe fields of social robotics, sound and music computing, affective computing, and body motion analysis. We want to engineersound models for implementing
effective mappings between stylized body movements and sound parameters that will enable anagent to express high-level body motion qualities through sound.These mappings are paramount for supporting feedback to andunderstanding robot body motion. The project will result in thedevelopment of new theories, guidelines, models, and tools for the sonic representation of high-level body motion qualities in interactive
applications. This work is part of the growing research field known as data sonification, in which we combine methodsand knowledge from the fields of interactive sonification, embodied cognition, multisensory perception, non-verbal and gesturalcommunication in robots.

Olof Misgeld, Torbjörn Gulz, Andre Holzapfel and Jura Miniotaitė. A CASE STUDY OF DEEP ENCULTURATION AND SENSORIMOTOR SYNCHRONIZATION TO REAL MUSIC. International Society for Music Information Retrieval Conference

Synchronization of movement to music is a behavioural capacity that separates humans from most other species. Whereas such movements have been studied using a wide range of methods, only few studies have investigated synchronisation to real music stimuli in a cross-culturally comparative setting. The present study employs beat tracking evaluation metrics and accent histograms to analyze the differences in the ways
participants from two cultural groups synchronize their tapping with either familiar or unfamiliar music stimuli. Instead of choosing two apparently remote cultural groups, we selected two groups of musicians
that share cultural backgrounds, but that differ regarding the music style they specialize in. The employed method to record tapping responses in audio format facilitates a fine-grained analysis of metrical accents that emerge from the responses. The identified differences between groups are related to the metrical structures
inherent to the two musical styles, such as non-isochronicity of the beat, and differences between the groups document the influence of the deep enculturation of participants to their style of expertise. Besides these findings, our study sheds light on a conceptual weakness of a common beat tracking evaluation metric, when applied to human tapping instead of machine generated beat estimations.