Dagstuhl Seminar 24302
Learning with Music Signals: Technology Meets Education
( Jul 21 – Jul 26, 2024 )
Permalink
Organizers
- Cynthia Liem (TU Delft, NL)
- Brian McFee (New York University, US)
- Meinard Müller (Universität Erlangen-Nürnberg, DE)
Contact
- Marsha Kleinbauer (for scientific matters)
- Simone Schilke (for administrative matters)
Shared Documents
- Dagstuhl Materials Page (Use personal credentials as created in DOOR to log in)
Schedule
This executive summary provides an overview of our discussions on advancing technology and education in music information retrieval (MIR) and related fields, summarizing the main topics covered in the seminar. We also describe the seminar's group composition, overall organization, and activities. Finally, we reflect on the most important aspects of the seminar and conclude with future implications and acknowledgments.
Overview
In the last twenty years, the field of music information retrieval (MIR) has undergone rapid developments in terms of the problems considered, the methodology, and its applications. Using conceptually simple tasks and methods evaluated on small and idealized datasets in its beginnings, MIR now contributes to a wide range of concepts, models, and algorithms that extend our capabilities of accessing, analyzing, understanding, and creating music. Given the complexity and diversity of music, MIR research considers various aspects such as genre, instrumentation, musical form, melodic and harmonic properties, dynamics, tempo, rhythm, and timbre, to name a few. Furthermore, music is inherently multimodal, incorporating speech-like signals (e.g., singing), videos (e.g., of live performances), static images (e.g., scanned music scores), and text (e.g., lyrics and reviews). This wealth of data makes MIR an interdisciplinary and challenging field of research, which closely connects to technical disciplines such as signal processing, machine learning, and information retrieval, as well as mathematics, musicology, psychology, and the digital humanities.
Having the Dagstuhl Seminar 24302 titled "Learning with Music Signals: Technology Meets Education," our objective was to advance technology and education in MIR and related disciplines using music as a challenging and instructive multimedia domain. Thinking of data-driven machine learning approaches, we discussed recent deep learning (DL) approaches and their ability to learn from training examples to make accurate predictions for previously unseen data. Furthermore, by learning from the experience of traditional engineering approaches, our aim was to better understand existing and build more interpretable DL-based systems (e.g., through integrating prior knowledge).
Beyond these technically oriented perspectives, an essential focus of our seminar was to approach the concept of learning from other angles, including pedagogical, educational, psychological, and user-centered viewpoints. We argue that music is an essential part of our lives that most people feel connected to. Therefore, music yields an intuitive entry point to support education in technical disciplines. In particular, we explored how music may serve as a vehicle to make learning and teaching signal processing and machine learning an interactive pursuit.

In all perspectives on learning, the question of reproducible research, including open access to data and software, is becoming increasingly important (so future insights can be built upon existing ones). As an overarching topic of our seminar, we discussed questions about open science and good scientific practice. This is a key issue, especially in higher education, where the open exchange of best practices and teaching materials significantly impacts the interdisciplinary and transnational education of the next generation of researchers.
In summary, in our Dagstuhl Seminar we approached and explored the concept of learning from different angles, see also Figure 1. Besides considering technological developments, the seminar equally addressed aspects of data and model understanding, transdisciplinary methodology and applications, science communication, and education.
Participants and Group Composition
In our seminar, we had $28$ participants from various locations around the world, including North America (eight participants from the United States), Asia (four participants from China, India, Japan, and South Korea), and Europe (16 participants from France, Germany, the Netherlands, Spain, Sweden, and the United Kingdom). Beyond geographic diversity, many participants had cross-cultural backgrounds and experiences. As naturally happens in international research fields, part of this comes from participants' work experiences in other countries and cultures than those of their country of birth. At the same time, several participants were first-generation for being in research/higher education, had second-generation migration backgrounds (meaning their parents were born in different countries, and often cultures, than themselves) - or, because of the nature of their affiliation, brought extensive experience in teaching with students from such backgrounds.
The seminar was not only international, but also highly interdisciplinary. While most of the researchers specialized in music information retrieval with a technical focus on signal processing and machine learning, we also had participants with backgrounds in musicology, human-computer interaction, science education, mathematics, computer vision, and other fields. This diversity stimulated cross-disciplinary discussions, bringing together experts from both technical and non-technical disciplines and highlighting opportunities for new collaborations. Many participants had strong musical backgrounds, with some even having dual careers in engineering and music, leading to numerous social activities, including playing music together.
We also aimed to foster variety in terms of seniority levels, with four Ph.D. students and three postdoctoral participants, as well as gender diversity, with 10 out of 28 participants identifying as female. More than half of the participants (16 out of 28) were attending Dagstuhl for the first time and expressed enthusiasm about the open and retreat-like atmosphere.
In conclusion, by bringing together internationally renowned scientists and promising early-career researchers from different fields, our seminar provided support and encouragement for emerging talents on their academic paths.
Overall Organization and Schedule
Dagstuhl Seminars are known for their flexibility and interactivity, encouraging participants to discuss ideas and raise questions rather than merely presenting research results. In keeping with this tradition, we set the schedule during the seminar, inviting spontaneous contributions focused on future-oriented content. This approach helped us avoid a conference-like atmosphere, where the emphasis is often on past research achievements. Furthermore, instead of sitting in rows, we removed all tables and arranged the seating in a half-circle of chairs, significantly enhancing eye contact and interaction among all participants.
After the organizers provided an overview of the Dagstuhl concept, we began the first day with self-introductions, where each participant shared their background, expectations, and wishes for the seminar. We then proceeded with brief stimulus talks, lasting 15 to 20 minutes, in which selected participants addressed critical questions related to the seminar's overall theme in a non-technical manner. Each talk smoothly transitioned into an open discussion among all participants, with the presenter acting as the moderator. These discussions were well-received and often extended for more than half an hour. The first day concluded with a brainstorming session on central topics reflecting the participants' interests, helping to shape the schedule and format for the following day.
On the subsequent days, we continued with short stimulus talks followed by long and intensive discussion rounds. We also incorporated group discussions, splitting into smaller groups to delve into specific topics in greater depth. The results and conclusions of these parallel group sessions, which lasted between 60 to 90 minutes, were then presented and discussed with the entire group. Additionally, we included panel-like elements featuring moderators, panelists, interviews, surveys, and game-like group activities. On the last day, we concluded the seminar with a session we called "self-outroductions," where each participant presented their personal view on the seminar's results. In summary, thanks to excellent group dynamics and a fair distribution of speaking time, all participants had the opportunity to express their thoughts, effectively avoiding a monotonous conference-like presentation format.
In addition to scientific questions, our seminar also addressed the various challenges that younger colleagues typically face when establishing their research groups and academic curriculum at the beginning of their careers. As previously mentioned, many of our participants had cross-cultural backgrounds, either being born in Asian countries or as second-generation individuals raised in Western cultures. One of the highlights of our Dagstuhl Seminar was a panel discussion on the cross-cultural challenges in academia, especially for individuals with Asian roots living and working in Europe or the US. This deeply personal and enlightening event was facilitated by Dagstuhl's unique environment, which fosters trust and mutual understanding.
While working in technical engineering disciplines, most participants also had a strong background and interest in music. This versatility significantly enriched the seminar's atmosphere, fostering cross-disciplinary interactions and sparking thought-provoking discussions. It also led to intensive joint music-making during breaks and evenings. A particular highlight was the Thursday evening concert organized by Cynthia Liem and Christof Weiß, where various ensembles formed by participants performed a wide variety of music, including classical, Irish folk, and jazz.
Conclusions and Acknowledgment
At the Dagstuhl Seminar 24302, we used music as a motivating and tangible domain to explore different perspectives on learning. These perspectives stimulated conceptual discussions, laying the groundwork for future projects and academic curricula. We focused on how to teach and pass on new technologies to students, using music as a challenging application domain. With experts in MIR, signal processing, machine learning, software development, science education, and music sciences, our interdisciplinary seminar generated vibrant discussions and highlighted opportunities for new collaborations. Immediate outcomes, such as plans to share research data and software, also emerged from the discussions. We aimed to expose attendees, especially early-career researchers, to new ideas for designing academic curricula in computer science and beyond. Specific areas and topics addressed in this seminar included:
- Contextualized education
- Inclusive education
- Educational software systems
- Interactive software frameworks
- Science communication
- Transdisciplinary methodology and collaborative research
- Computational musicology
- Human-in-the-loop systems for music processing
- Data-driven machine learning for MIR
- Explainable deep learning for MIR
- Integration of musical knowledge
- Hybrid models for MIR
- Differentiable models for MIR
- Data mining, acquisition, measurement, and annotation
- Data/annotation quality
- Data accessibility and copyright issues
- Open science
- Reproducible and sustainable research
- Academic integrity and good scientific practice
Besides the scientific aspect, the social aspect of our seminar was equally important. We hosted an interdisciplinary, international, and interactive group of researchers, consisting of current and future leaders in our field. Many participants were visiting Dagstuhl for the first time and praised the open and inspiring setting. The group dynamics were excellent, with many personal exchanges and shared activities. Some scientists expressed their appreciation for the opportunity to engage in prolonged discussions with researchers from neighboring fields, something often impossible at typical conferences. A standout feature of our seminar was the interaction between younger researchers at the beginning of their academic careers and established researchers and educators. This facilitated a deeply enriching exchange between different generations, promoting mutual trust and understanding. The intensive dialogue between these groups was truly outstanding and highlighted the unique value of our seminar.
In conclusion, our expectations for the seminar were not only met but exceeded, particularly in terms of networking and community building. We want to express our gratitude to the Dagstuhl board for giving us the opportunity to organize this seminar, the Dagstuhl office for their exceptional support throughout the organization process, and the entire Dagstuhl staff for their excellent service during the seminar. In particular, we want to thank Heike Clemens, Andreas Dolzmann, Marsha Kleinbauer, Simone Schilke, and Christina Schwarz for their invaluable assistance in the preparation and organization of the seminar.

In the last twenty years, the field of music information retrieval (MIR) has undergone rapid developments in terms of the problems considered, the methodology, and its applications. Using conceptually simple tasks and methods evaluated on small and idealized datasets in its beginnings, MIR now contributes to a wide range of concepts, models, and algorithms that extend our capabilities of accessing, analyzing, understanding, and creating music. Given the complexity and diversity of music, MIR research considers various aspects such as genre, instrumentation, musical form, melodic and harmonic properties, dynamics, tempo, rhythm, and timbre, to name a few. Furthermore, music is inherently multimodal, incorporating speech-like signals (e.g., singing), videos (e.g., of live performances), static images (e.g., scanned music scores), and text (e.g., lyrics and reviews). This wealth of data makes MIR an interdisciplinary and challenging field of research, which closely connects to technical disciplines such as signal processing, machine learning, and information retrieval, as well as mathematics, musicology, psychology, and the digital humanities.
In this Dagstuhl Seminar, we aim to advance technology and education in these disciplines using music as a challenging and instructive multimedia domain. Thinking of data-driven machine learning approaches, we will discuss recent deep learning (DL) approaches and their ability to learn from training examples to make accurate predictions for previously unseen data. Furthermore, by learning from the experience of traditional engineering approaches, our objective is to better understand existing and build more interpretable DL-based systems (e.g., through integrating prior knowledge).
Beyond these technically oriented perspectives, an essential focus of our seminar is to approach the concept of learning from other perspectives, including a pedagogical, educational, psychological, and user-centered one. We argue that music is an essential part of our lives that most people feel connected to. Therefore, music yields an intuitive entry point to support education in technical disciplines. In particular, we will explore how music may serve as a vehicle to make learning and teaching signal processing and machine learning an interactive pursuit.
From an application perspective, we want to learn with and from domain experts about musical works and their recorded performances while exploring the potential of computational tools by considering complex music scenarios of musicological relevance. Finally, through a dialogue with social scientists, we want to gain a deeper understanding of human perception and interpretation of music and technology.
In all perspectives on learning, the question of reproducible research, including open access to data and software, is becoming increasingly important (so future insights can be built upon existing ones). As an overarching topic of our seminar, we will discuss questions about open science and good scientific practice. This is a key issue, especially in higher education, where the open exchange of best practices and teaching materials significantly impact the interdisciplinary and transnational education of the next generation of researchers.
In summary, in our Dagstuhl Seminar we want to approach and explore the concept of learning from different angles. Besides considering technological developments, the seminar equally addresses aspects of data and model understanding, transdisciplinary methodology and applications, science communication, and education.

Please log in to DOOR to see more details.
- Vipul Arora (Indian Institute of Technology Kanpur, IN) [dblp]
- Ching-Yu Chiu (Universität Erlangen-Nürnberg, DE)
- Roger B. Dannenberg (Carnegie Mellon University - Pittsburgh, US) [dblp]
- Christian Dittmar (Fraunhofer IIS - Erlangen, DE) [dblp]
- Zhiyao Duan (University of Rochester, US) [dblp]
- Mark Gotham (Durham University, GB)
- Masataka Goto (AIST - Ibaraki, JP) [dblp]
- Patricia Hu (Johannes Kepler Universität Linz, AT)
- Jaehun Kim (SiriusXM/Pandora - Oakland, US)
- Katherine M. Kinnaird (Smith College - Northampton, US) [dblp]
- Cynthia Liem (TU Delft, NL) [dblp]
- Lele Liu (Universität Würzburg, DE)
- Hanna Lukashevich (Fraunhofer IDMT - IIlmenau, DE) [dblp]
- Brian McFee (New York University, US) [dblp]
- Peter Meier (Universität Erlangen-Nürnberg, DE)
- Alia Morsi (UPF - Barcelona, ES) [dblp]
- Meinard Müller (Universität Erlangen-Nürnberg, DE) [dblp]
- Juhan Nam (KAIST - Daejeon, KR) [dblp]
- Alex Ruthmann (New York University, US) [dblp]
- Simon Schwär (Universität Erlangen-Nürnberg, DE) [dblp]
- Sebastian Stober (Otto-von-Guericke-Universität Magdeburg, DE) [dblp]
- Bob Sturm (KTH Royal Institute of Technology - Stockholm, SE) [dblp]
- Christopher J. Tralie (Ursinus College - Collegeville, US) [dblp]
- Timothy Tsai (Harvey Mudd College - Claremont, US) [dblp]
- Anja Volk (Utrecht University, NL) [dblp]
- Changhong Wang (Télécom Paris, FR)
- Christof Weiß (Universität Würzburg, DE) [dblp]
- Jordan Wirfs-Brock (Whitman College - Walla Walla, US) [dblp]
Classification
- Machine Learning
- Multimedia
- Sound
Keywords
- Music information retrieval
- education
- signal processing
- deep learning
- user interaction