Till innehåll på sidan
Till KTH:s startsida

Active Listening with the Music Puzzle

The Music Puzzle is a game where the aim is to reconstruct a music performance from small segments that are shifted in pitch and filtered. The game targets hearing impaired persons in order to train listening skills.

We conducted an experiment using a purposefully designed audio-based game called the Music Puzzle with Japanese university students with different levels of hearing acuity and experience with music in order to determine the effects of these factors on solving such games. The game was played with three sound sets or modes; speech, music, and a mix of the two.

The results showed that people with hearing loss had longer processing times for sounds when playing the game. Solving the game task in the speech mode was found particularly difficult for the group with hearing loss, and while they found the game difficult in general, they expressed a fondness for the game and a preference for music. Participants with less musical experience showed difficulties in playing the game with musical material. We were able to explain the impacts of hearing acuity and musical experience; furthermore, we can promote this kind of tool as a viable way to train hearing by focused listening to sound, particularly with music.

Team

Kjetil Falkenberg
Kjetil Falkenberg universitetslektor
Rumi Hiraga
Rumi Hiraga NTUT Japan

Funding

JSPS, NVidia, The Scandinavia-Japan Sasakawa Foundation

Project Duration

2012 - 2017

Publications

[1]
K. F. Hansen, Z. Li och H. Wang, "A music puzzle game application for engaging in active listening," i Proceedings of 97th Information Science and Music (SIGMUS) Research Conference, 2012.
[2]
M. Matsubara et al., "An Inquiry into Hearing-Impaired Student's Musical Activities : How Do They Listen to the Music?," i ICMPC-ASCOM, 2014.
[3]
R. Hiraga et al., "Music perception of hearing-impaired persons with focus on one test subject," i Proceedings of 2015 IEEE International Conference on Systems, Man, and Cybernetics, 2015.
[4]
Z. Li, H. Wang och K. F. Hansen, "Music Puzzle: An Application for Hearing Training on Android Smart Devices," TMH-QPSR special issue: Proceedings of SMC Sweden 2012 Sound and Music Computing, Understanding and Practicing in Sweden, vol. 52, no. 1, s. 44-45, 2012.
[5]
K. F. Hansen et al., "Music Puzzle : An audio-based computer game that inspires to train listening abilities," i Advances in Computer Entertainment : 10th International Conference, ACE 2013, Boekelo, The Netherlands, November 12-15, 2013. Proceedings, 2013, s. 540-543.
[6]
R. Hiraga et al., "Promote active listening with music games," i Proceedings of the first international Music & CI Symposium, 2016.
[7]
R. Hiraga och K. F. Hansen, "Sound Preferences of Persons with Hearing Loss Playing an Audio-Based Computer Game," i Proceedings of the 3rd ACM international workshop on Interactive multimedia on mobile & portable devices, 2013, s. 25-30.
[8]
K. F. Hansen och R. Hiraga, "The Effects of Musical Experience and Hearing Loss on Solving an Audio-Based Gaming Task," Applied Sciences, vol. 7, no. 12, 2017.
[9]
M. Matsubara et al., "聴覚障害学生を対象とした聴能向上のための音楽トレーニングプロジェクト利用," , 2014.