Dagstuhl Seminar 17411
Hyperspectral, Multispectral, and Multimodal (HMM) Imaging: Acquisition, Algorithms, and Applications
( Oct 08 – Oct 13, 2017 )
(Click in the middle of the image to enlarge)
Permalink
Please use the following short url to reference this page:
https://www.dagstuhl.de/17411
Organizers
- Gonzalo R. Arce (University of Delaware, US)
- Richard Bamler (DLR - Oberpfaffenhofen, DE)
- Shida Beigpour (MPI für Informatik - Saarbrücken, DE)
- Jon Yngve Hardeberg (Norwegian Univ. of Science & Technology - Gjøvik, NO)
- Andreas Kolb (Universität Siegen, DE)
Contact
- Andreas Dolzmann (for scientific matters)
- Annette Beyer (for administrative matters)
In the last couple of decades, Hyperspectral, Multispectral, and Multimodal (HMM) imaging has emerged as an essential tool in various fields of science, medicine, and technology. Compared to integrated broad-band information as, e.g., present in RGB images, HMM imaging strives to acquire a multitude of specific narrow bands of the electro-magnetic spectrum in order to solve specific detection or analysis tasks. HMM research is interested in studying light-matter interaction in a wide range of wavelengths from the high energy radiation down to Terahertz radiation (submillimeter waves). Furthermore, combining spectral data captured using different imaging modalities can unveil additional information of the scene that is not revealed solely by each of the individual imaging modalities.
HMM imaging has a wide range of applications including automotive industry, automatic material sorting, quality control, security, computer vision, medicine, cultural heritage, and biology. Researchers from many different disciplines are involved in HMM imaging and analysis. Each of these disciplines deals with this topic in a different way according to their own methodologies. Furthermore, the scientific literature related to HMM imaging is scattered over various conferences and journals from different disciplines which makes the communication and knowledge transfer very difficult. However, there are common interests and challenges faced by all of these fields, e.g., reducing costs, handling dimensionality, increasing resolution, visualization, and cross-channel information transferring and mapping which needs to be addressed. The various disciplines exploring HMM imaging can thus benefit from an out-of-the-box thinking and a broader vision of the principal concepts and challenges in HMM. This seminar aims to bring researchers from different scientific communities together, uniting them in tackling the challenges of HMM imaging and analysis.
Creative Commons BY 3.0 DE
Gonzalo R. Arce, Richard Bamler, Shida Beigpour, Jon Yngve Hardeberg, and Andreas Kolb
On the last day of the seminar, the attendees had a very intense discussion about the usefulness of the seminar itself, the grand challenges related to the highly interdisciplinary field of research, and the next steps that should be taken in order to further improve on the cross-fertilizing effects in hyperspectral, multispectral, and multimodal imaging.
Take Home Messages
All attendees agreed on the high quality and open mindedness of the discussions at both, the group level, e.g., in the plenary sessions and the working groups, and also on personal level. All participants assess this Dagstuhl seminar as a great success, especially due to the interdisciplinary discussion and the new insights resulting from this. Despite differences of the individual fields present in the seminar, e.g., remote sensing, color reproduction, and material classification, and despite the wide variety of applications such as medical, environmental monitoring, and arts, a large set of common questions and problems could be identified. All attendees highly appreciated the fact, that unlike in conferences, which usually have a rather narrow perspective on HMM challenges and solutions, as they usually address a single community with a very similar perspective on the field, this seminar brought together people with very different points of view.
This Dagstuhl seminar was a starting point of a number of connections that could be established directly, and several mid-, and maybe even long-term collaborations and joint research actions. There have been several highlights related to the full pipeline from data acquisition, via data processing to applications.
Grand Challenges
On the basis of common and interdisciplinary ground setup in this seminar, several challenges have been identified, which the seminar's attendees see as important to be addressed in further research and engineering work.
Data Acquisition Independent of the specific range of the addressed electromagnetic spectrum, the seminar participants see a severe restriction in the usage of HMM sensors due to their inflexibility, e.g., in selecting spectral bands, bulkiness, high calibration efforts, and acquisition speed (see also the working group report on this topic). Enhancing on these limiting factors has the potential to bring about fundamentally new spins in various application domains. Some approaches presented at the workshop have the potential to push back these limits to some degree. On the other hand, most likely there will be no general purpose HMM acquisition device available in the next decades that covers the majority of application requirements. Still, the seminar attendees agree on the importance of enhancing the applicability of existing and future acquisition devices towards more flexible band selection, fast and efficient, (semi-)automated calibration, and, for some applications, compactness. Ideally, future research provides means for an abstract definition of application specific characteristics from which a specific selection and/ or instantiation of an acquisition device can be deduced.
Data Processing and Validation Regarding data processing and validation, three main topics have been discussed: The usefulness and limitations of machine learning and, especially, deep learning (see working group report), the importance of verified and metric data, and the need for a proper reference and benchmarking data set. Even though there are and have been ongoing activities in spectral normalization and validation, e.g., on the level of CIE or other standardization institutions, or in the field of metrology, there still is the lack of widely existing and accepted methods and data even if restricted to specific fields of application.
The seminar participants see a huge potential in all three areas. Still, major obstacles have to be overcome in order to leverage these potentials. In machine learning/ deep learning, one main issue is the lack of guarantees that the results obey specific constraints to, e.g., physical limits or relations. The lack of verified, metric data, and proper reference and benchmarking data, on the other hand, can only be overcome if there is a stronger common basis for best practice within and, even more important, between the HMM sub-disciplines.
Information Exchange The existence of common information bases is tightly linked to the prior point regarding data processing and validation. So far, there are only few options and pseudo-standard for sharing data and algorithms. While there are good examples, e.g., Open CV library in computer vision, setting up this kind of "standard" is, and will be, much harder in the diverse and partially fragmented HMM research domain. Apparently, this chicken-egg problem can only be solved from within the involved research domains themselves by the normative power of fact of the actions taken by the researchers themselves.
Next Steps
Participants discussed various options for future activity as a follow-up on this Dagstuhl seminar. As one essential restriction of the discussion, attendees became aware of their own limitations in knowing all relevant work and requirements existing in the HMM research subfields. Therefore, the obvious approach to enhance the fields’ convergence by publications, e.g., a special issue or book, and/ or workshops has not the highest priority, even though an introductory workshop or piece of literature for 1st year PhD students would be highly appreciated.
However, participants of Dagstuhl seminar see two main options to proceed in order to keep the initiated process of convergence going and to improve on at least two of the three main challenges identified, i.e., regarding data processing and validation and the exchange of information.
HMM Webpage: As there is a severe lack in common information widely used and recognized, the group of researchers who attended Dagstuhl seminar see the potential of a common, web-based information platform.
In this respect, Masahiro Yamaguchi is open to provide the already established web-link multispectral.org and Andreas Kolb will investigate options for setting up and hosting this kind of platform. In any case, this kind of activity needs to rest on several shoulders, thus the attendees are called to follow through with the activities, on the operative level.
Follow-up Dagstuhl Seminar: As Dagstuhl supports follow-up seminars, the attendees agree on the usefulness of having this kind of seminar in order to evaluate the common, interdisciplinary activities that arose from the first Dagstuhl seminar. In case of a new edition of the workshop, participants agree on having more industrial partners involved.
Gonzalo R. Arce, Richard Bamler, Shida Beigpour, Jon Yngve Hardeberg, and Andreas Kolb
- Gonzalo R. Arce (University of Delaware, US) [dblp]
- Richard Bamler (DLR - Oberpfaffenhofen, DE) [dblp]
- Gladimir V. G. Baranoski (University of Waterloo, CA) [dblp]
- Shida Beigpour (MPI für Informatik - Saarbrücken, DE) [dblp]
- Ela Claridge (University of Birmingham, GB) [dblp]
- Donald G. Dansereau (Stanford University, US) [dblp]
- Hilda Deborah (Norwegian Univ. of Science & Technology - Gjøvik, NO) [dblp]
- Nicolas Dobigeon (University of Toulouse, FR) [dblp]
- Paul D. Gader (University of Florida - Gainesville, US) [dblp]
- Peter Gege (DLR - Oberpfaffenhofen, DE) [dblp]
- Sony George (Norwegian Univ. of Science & Technology - Gjøvik, NO) [dblp]
- Jon Yngve Hardeberg (Norwegian Univ. of Science & Technology - Gjøvik, NO) [dblp]
- Peter Haring Bolivar (Universität Siegen, DE) [dblp]
- Andreas Kolb (Universität Siegen, DE) [dblp]
- Dietrich Paulus (Universität Koblenz-Landau, DE) [dblp]
- Christoph Pomrehn (Hochschule Bonn-Rhein-Sieg - St. Augustin, DE)
- Noël Richard (University of Poitiers, FR) [dblp]
- Antonio Robles-Kelly (CSIRO - Canberra, AU) [dblp]
- Christoph Markus Schikora (Universität Siegen, DE) [dblp]
- Jean-Baptiste Thomas (Norwegian Univ. of Science & Technology - Gjøvik, NO) [dblp]
- Shoji Tominaga (Chiba University, JP) [dblp]
- Devis Tuia (Wageningen University Research WUR, NL) [dblp]
- Eva M. Valero Benito (University of Granada, ES)
- Rudolf Verdaasdonk (VU Medical Center - Amsterdam, NL) [dblp]
- Anna Katharina Wigger (Universität Siegen, DE)
- Masahiro Yamaguchi (Tokyo Inst. of Technology - Yokohama, JP) [dblp]
- Naoto Yokoya (DLR - Oberpfaffenhofen, DE) [dblp]
- Xiaoxiang Zhu (DLR Oberpfaffenhofen & TU München) [dblp]
Classification
- computer graphics / computer vision
Keywords
- Hyperspectral and Multispectral imaging and analysis
- Multi-modal sensor Fusion
- Computer Vision
- Remote sensing and Geoscience
- Compressive Sensing
