Designing a Multimodal Viewer for Piano Performance Analysis - a Pedagogy-First Approach

Joonhyung Bae, Hyeyoon Cho, Kirak Kim, Dawon Park, Taegyun Kwon, Yoon-Seok Choi, Hyeon Hur, Shigeru Kai, Yohei Wada, Satoshi Obata, Akira Maezawa, Jaebum Park, Jonghwa Park, Juhan Nam

Primary Subject: Software/Library Demo

Some of the required materials for this paper do not exist: Video

Abstract:

Abstract instructions in piano education, such as "raise your wrist" and "relax your tension," lead to varying interpretations among learners, preventing instructors from effectively conveying their intended pedagogical guidance. To address this problem, this study conducted systematic interviews with a piano professor with 18 years teaching experience, and two researchers derived seven core need groups through cross-validation. Based on these findings, we developed a web-based dashboard prototype integrating video, motion capture, and musical scores, enabling instructors to provide concrete, visual feedback instead of relying solely on abstract verbal instructions. Technical feasibility was validated through 109 performance datasets.