Perception of musical notes in the visual modality: using synaesthesia to explore auditory-visual associations

Abstract

Can our understanding of auditory and visual senses be enriched to enhance human-computer interactions? This research focuses on understanding auditory-visual cross-modal associations through tests conducted using an ad-hoc program called SoundStrokes and more immersive environments, such as VR. Using research from a rare condition called “synaesthesia”, we investigate musical pitch-colour associations, as well as the influence of various factors such as emotion, personality traits, musical ability, and age on the perception between these two senses. Using the knowledge of existing auditory-visual association tests, the research contemplates the potential of Virtual Reality (VR) as an exploratory tool. Rather than asserting VR's efficacy, the research aims to investigate how VR environments might offer a unique and immersive platform for adapting and possibly strengthening these complex sensory and emotional associations.

Links

Patrick O’ Toole, PhD Student, University College Cork

Publications by Patrick O'Toole

About the seminar series

Sound and Music Interaction Seminars is a seminar series organized by the Sound and Music Computing group  at the division of Media Technology and Interaction Design , KTH. The list of past and upcoming seminars is available here .