Playing (with) Sound

Of the Animation of Digitized Sounds and their Reenactment by Playful Scenarios in the Design of Interactive Audio Applications

Doctoral Dissertation (Doktor der Philosophie)
Norbert Schnell, 2013
University of Music and Performing Arts Graz

Abstract

Investigating sound and interaction, this dissertation has its foundations in over a decade of practice in the design of interactive audio applications and the development of software tools supporting this design practice. The concerned applications are sound installations, digital instruments, games, and simulations. However, the principal contribution of this dissertation lies in the conceptualization of fundamental aspects in sound and interactions design with recorded sound and music. The first part of the dissertation introduces two key concepts, animation and reenactment, that inform the design of interactive audio applications. While the concept of animation allows for laying out a comprehensive cultural background that draws on influences from philosophy, science, and technology, reenactment is investigated as a concept in interaction design based on recorded sound materials. Even if rarely applied in design or engineering – or in the creative work with sound – the notion of animation connects sound and interaction design to a larger context of artistic practices, audio and music technologies, engineering, and philosophy. Starting from Aristotle’s idea of the soul, the investigation of animation follows the parallel development of philosophical concepts (i.e. soul, mind, spirit, agency) and technical concepts (i.e. mechanics, automation, cybernetics) over many centuries. Particular attention is hereby paid to sound reproduction and music technologies. The concept of reenactment is rooted in the ideas of enaction and embodied cognition. From this perspective, listening is examined in relationship with further interactions that support or accompany listening. An investigation of sound reproduction technologies insists in particular on the concept of congruency, as a central factor in the understanding of interaction. Sound and interaction design here appears as the composition of – technical and semantic – congruencies that define the users’ engagement into interaction with an application. After an exploration of epistemological and hermeneutical aspects of interactive technologies, the reenactment concept is further enriched by the investigation of related concepts such as metaphor, affordance, and play. Finally, playing technique and scenarios are proposed as fundamental concepts in sound and interaction design that, beyond affordance, allows for taking into account cultural factors as well as the notion of practice. The second part of the dissertation presents nine interactive audio applications. All of the applications have been realized in the past few years in collaborations on artistic and pedagogical projects or as demo applications showcasing design ideas and technologies. The descriptions of the applications show how the concept of reenactment applies to the scenarios of interactive audio applications and how the concept allows for efficiently highlighting important aspects of interaction design.