Perceptually Informed Organization of Textural Sounds

Doctoral Dissertation (Doktor der Philosophie)
Thomas Grill, 2012
University of Music and Performing Arts Graz

Abstract

This thesis researches practice-motivated organization of digital audio in the context of sound-based music, including modeling, structuring, characterization, and visualization. It is based on two main keystones: first, the perceptual orientation of those techniques, and second, the focus on textural sound material. I propose a model for the compact representation of textural sound, incorporating psychoacoustically informed timbral and micro-temporal aspects. A simple distance function supports the application of algorithms for search and retrieval, and clustering and classification. This technique is demonstrated in the context of an artistic audiovisual installation. Utilizing this model, I explore structure discovery in digital audio, covering segmentation into slices of coherent content and semi-automatic identification of representative sound materials. These techniques are useful for quickly surveying audio data, or for preprocessing musical material. The method is evaluated against examples of music annotation for two well-known works of electroacoustic music. A central topic of the thesis is the inquiry into perceptually meaningful descriptions of textural sounds. The repertory grid interview technique has been used to elicit bipolar personal constructs representing relevant characteristics. A subsequent large-scale online survey has been conducted to refine and evaluate those descriptions. Building on these findings, I have developed computable audio descriptors capable of measuring the most significant of the previously elicited perceptual qualities. The descriptors have been tuned using results of the previous survey to match human perception targeting either for individual accuracy or for mutual independence. Finally, I propose a visualization strategy for textural sounds described by the same perceptual qualities. The intuitive, map-like appearance is appropriate for screen-based sound browsing interfaces. Its suitability has been verified through another large-scale online survey. A prototype online sound browser application has been developed for the purpose of illustration.