In our daily lives, we as humans perceive our surroundings in an inherently multimodal way. Nevertheless, the vast majority of data analysis idioms are exclusively visual, not using the apparent potential of combined designs. While the visualization field studies visual data analysis solutions, the sonification field investigates comparable solutions that convey data over non-speech audio. There are several similarities between the methods and design theories of both approaches, such as the use of perceptual variables to encode data attributes, the use of marks and substrates for conveying items, and the role of interaction in manipulating the data representations.

Over the recent decades, both fields have established research communities, theoretical frameworks, and toolkit support. Although extensive research has been carried out both on the auditory and visual representation of data, comparatively little is known about their systematic and complementary combination for data exploration, analysis, and presentation. There are potential powerful synergies in combining both modalities to address the individual limitations of each other. Nevertheless, existing research on combinations has often focused only on one of the modalities.

To bridge this gap, the seminar will bring together researchers from the sonification and visualization communities. The seminar will deal with topics that arise from integrating visualization and sonification, as well as topics that are of concern for each field individually and can potentially contribute to both areas. In particular, the following questions will be discussed:

• What are the strengths of visualization and sonification respectively and how can they best be used in unison, complementing and supporting where necessary, considering human perception as well as various data properties?
• Which theoretical frameworks can be transferred from sonification to visualization and vice versa?
• Visualization has progressed much in the past decades towards shared design methods and tools: What can sonification learn from this lesson? Are shared design methods and processes envisageable, or even desired? Can we imagine integrated tools for visualization and sonification?
• In what way can audio-visual representations support accessibility of information and data?
• Can audio-visual representations increase the engagement of their users during a data analysis process?
• How can integrated sonification and visualization solutions be evaluated?
• Should integrated contributions be presented at existing visualization venues, sonification venues, or do we need new integrated venues?

The outcomes from these discussions will foster cross-fertilization and synergy of efforts by both fields, advance bimodal research on audio-visual analytics, and lay groundworks for multimodal approaches in the future.

Creative Commons BY 4.0

Wolfgang Aigner, Sara Lenzi, and Niklas Rönnberg