Dagstuhl Seminar 23302
Software Architecture and Machine Learning
( Jul 23 – Jul 28, 2023 )
(Click in the middle of the image to enlarge)
Permalink
Please use the following short url to reference this page:
https://www.dagstuhl.de/23302
Organizers
- Grace A. Lewis (Carnegie Mellon University - Pittsburgh, US)
- Henry Muccini (University of L'Aquila, IT)
- Axel-Cyrille Ngonga Ngomo (Universität Paderborn, DE)
- Roland Weiss (ABB - Mannheim, DE)
- Liming Zhu (Data61, CSIRO - Sydney, AU)
Contact
- Michael Gerke (for scientific matters)
- Jutka Gasiorowski (for administrative matters)
Schedule
The pervasive and distributed nature of many of today's software systems requires making complex design decisions to guarantee important system qualities such as performance, reliability, safety and security. The practices within the field of software architecture guide the design and development of software systems from its high-level blueprint down to their implementation and operations. While the fundamentals of software architecture practices do not change, their execution evolves to address architecting with, and for, new system paradigms and emerging technologies. Incorporating machine learning (ML) elements into systems is advancing rapidly with the availability of more compute power, specialized infrastructure, and better and more efficient algorithms to process increasing amounts of data. This evolution has led many domains to leverage ML for automation, data analytics, decision support, and advanced user interfaces, among others. Experience published and shared by both software engineering research and practitioner communities shows that organizations struggle to move ML models and pilot projects into production. Reported software engineering challenges fall mostly outside data science expertise, which is focused on the development of ML models and not necessarily their production-readiness, and include areas such as testing, requirements management, software architecture, and configuration management. Among these, software architecture has a special role as it serves as an abstraction between requirements and implementation and drives the structure and behavior of systems.
Some of the reported challenges in developing ML-enabled systems, such as interface stability, data storage and access, and data transformation, are already addressed by existing software architecture techniques and practices. However, traditional software development and deployment practices are faced with and pose significant challenges when developing ML-enabled systems, which are systems that integrate ML components. These ML components include model training and updating components; model serving and inference components; infrastructure components to support data collection, processing, and servicing; and operations infrastructure, such as MLOps pipelines to support automated build and deployment. As these new ML-related components are introduced into systems, different architectural concerns and architecting challenges take higher priority, which include working with non-determinism, understanding and designing for new classes of dependencies, and co-architecting the system as well as the ML model development pipelines, among others. In addition, integration of ML components into software systems places greater emphasis on qualities that are not as common in non-ML systems, such as monitorability as a first-class citizen, designing for extensibility, and data-centricity.
This Dagstuhl Seminar culminated as a response to recognizing that software architecture research and practitioner communities have an opportunity, if not an obligation, to fill key software architecture principles and practices gaps that are particularly critical when incorporating ML components into software systems. The seminar focused on two key themes:
- software architecture principles and practices for ML-enabled systems (SA4ML) and
- application of ML techniques for improved architecting of software systems (ML4SA).
A key goal of the seminar was to enable technical exchange among otherwise scattered research and practitioner communities, such as software engineering, software architecture, self-adaptive systems, and machine learning around software architecture and ML (SA&ML). This Dagstuhl Seminar presented an opportunity to get these communities together to develop a common vocabulary and a coherent research agenda with a better understanding of problems faced in the industry. The goals of this seminar were to establish a common understanding of key concepts that are central to architecting ML-enabled systems, elicit challenges of meeting key quality attributes of ML-enabled systems, and build a research roadmap for future work to address these challenges. As such, this seminar marks an important milestone in accelerating research in SA4ML and ML4SA work by fostering better communication around key concepts and between diverse stakeholders. Hereafter, we will use the term SA&ML to include both SA4ML and ML4SA research directions.
Seminar Format
The seminar on software architecture and machine learning was structured to foster an interactive and productive idea exchange environment for all participants. Participants were initially asked to prepare a single-slide introduction about themselves and their work in software architecture and machine learning, which helped to establish a foundational understanding among attendees. Prior to the workshop, organizers developed an initial version of a key concepts map and a research challenges map using a grounded theory approach, which was represented visually on miro boards. During the seminar, participants delivered 10-minute lightning talks about their SA&ML work, which were instrumental in refining the research and concept maps with keywords and challenges mentioned by the speakers. At the end of the first day, organizers summarized the discussions and identified key emerging topics and research areas, leading to an initial set of key quality attributes critical to ML-enabled system development. The second day continued with discussions and a working group formation session, where participants prioritized five critical quality attributes for ML-enabled systems: Evolvability, Uncertainty and Observability, Trust and Trustworthiness, and Data Centricity. Four working groups were formed around these quality attributes, with appointed editors documenting the discussions. Each group was provided with a draft template to guide their discussions for consistency, and organizers rotated among groups to offer diverse insights. Plenary meetings on the second and third days allowed groups to share progress and discuss challenges, complemented by end-of-day meetings among organizers to consolidate learning and monitor progress. The seminar concluded on the fourth day with presentations from each group, open for feedback from other participants. This collaborative review process led to the integration of feedback into the final discussions, resulting in a well-defined set of challenges and research directions, marking the seminar's successful completion. In addition, this format ensured a thorough exploration of the seminar's themes and fostered active participation and collaborative learning among attendees.
Creative Commons BY 4.0
Grace A. Lewis, Henry Muccini, Ipek Ozkaya, Karthik Vaidhyanathan, Roland Weiss, and Liming Zhu
The pervasive nature of software systems results in architecture challenges to guarantee better system qualities including performance, reliability, and security. The field of software architecture has continuously evolved to adapt to these challenges and new computing trends. Similarly, the field of machine learning (ML) is advancing rapidly with the availability of processing increasing amounts of data, due to better and specialized computing infrastructure as well as algorithms of increasing efficiency. This revolution has led many domains to leverage ML techniques for data analytics, decision support, and advanced user interfaces, among others. However, traditional software development and deployment practices are faced with and pose significant challenges when developing ML systems.
The increasing usage of ML, coupled with the software architecture challenges of the modern era, has resulted in two broad research challenges and opportunities: (1) developing software architecture principles and practices for ML-enabled systems (SA4ML), and (2) application of ML techniques for (better) architecting software systems (ML4SA) and detecting and triggering runtime needs for self-adaptation, architecture reconfiguration, and even ML model maintenance and evolution. Contributions in these two areas are emerging from researchers and practitioners but are scattered across different communities such as software engineering, software architecture, self-adaptive systems, and machine learning.
This Dagstuhl Seminar presents an opportunity to bring these communities together to develop a common vocabulary and a coherent research agenda with a better understanding of problems faced in industry. First, the seminar will frame ML challenges as quality attributes to understand the architectural design approaches and analysis techniques that would ensure their successful realization in ML systems. Second, it will focus on the role of ML for improving the architecture of software systems that will meet their quality goals over time. The seminar will be then organized around the major topics SA4ML and ML4SA.
In SA4ML, the seminar will discuss the state-of-the art and practice for architecting ML systems. The seminar participants will be asked to identify challenges and opportunities, and to identify the research directions for better architecting of ML systems. In terms of quality aspects, we intend to focus on the following two major topics: (1) Building architecture for the sustainability and evolution of ML systems, and (2) architecture patterns, styles, and tactics for ML model development pipelines and ML-enabled systems. The overall research question will be how to architect quality ML systems to drive concrete discussions on existing techniques and practices that apply as well as new ones that need new research.
In the ML4SA portion of the seminar, the participants will explore the role of ML in the improvement of architectures for current and future generations of software systems. The audience will be asked to identify current challenges and opportunities, and to identify the research agenda for architecting future systems using ML. Some topics that will be specifically discussed are self-adaptation driven by ML and ML-driven architecture reconfiguration. The macrotopic of quality in SA&ML will be pervasive to the seminar. We will specialize on the quality needs and opportunities associated with SA&ML. In the context of this seminar, quality can be seen from the two different perspectives characterizing this seminar: the specific quality characteristics associated with the ML-enabled systems under design (SA4ML), and the quality improvements enabled by using ML techniques when architecting software systems (ML4SA). The quality attributes of specific interest for this seminar include (1) trustworthiness; (2) ethics, including societal and environmental wellbeing, human values, fairness, privacy, contestability, accountability, etc., to the extent that they can be abstracted and managed by architectural design decisions; (3) transparency and explainability; (4) security and safety; (5) deployability and operability (for MLOps); (6) monitorability and observability; and (7) sustainability. Associated with this topic, the seminar will also discuss verification and validation techniques necessary to monitor, assess and measure quality attribute characteristics for SA&ML (including self-healing, auditability, compliance, and conformance).
Grace A. Lewis, Henry Muccini, Axel-Cyrille Ngonga Ngomo, Roland Weiss, and Liming Zhu
- Nelly Bencomo (Durham University, GB) [dblp]
- Justus Bogner (Universität Stuttgart, DE) [dblp]
- Jan Bosch (Chalmers University of Technology - Göteborg, SE) [dblp]
- Tao Chen (University of Birmingham, GB) [dblp]
- Xavier Franch (UPC Barcelona Tech, ES) [dblp]
- Helena Holmström Olsson (Malmö University, SE) [dblp]
- Benjamin Klöpper (Capgemini - Stuttgart, DE) [dblp]
- Fabio Kon (University of Sao Paulo, BR) [dblp]
- Thomas Kropf (Robert Bosch GmbH - Renningen, DE) [dblp]
- Grace A. Lewis (Carnegie Mellon University - Pittsburgh, US) [dblp]
- Henry Muccini (University of L'Aquila, IT) [dblp]
- Ipek Ozkaya (Carnegie Mellon University - Pittsburgh, US) [dblp]
- Marie Platenius-Mohr (ABB - Ladenburg, DE) [dblp]
- Lena Pons (Carnegie Mellon University - Pittsburgh, US) [dblp]
- Raghu Reddy (IIIT - Hyderabad, IN) [dblp]
- Bradley Schmerl (Carnegie Mellon University - Pittsburgh, US) [dblp]
- Christoph Schröer (Universität Oldenburg, DE) [dblp]
- Alex Serban (Siemens Healthineers, Erlangen, DE & University Transilvania of Brasov, RO) [dblp]
- Romina Spalazzese (Malmö University, SE) [dblp]
- Anastas Stoyanovsky (Amazon - Pittsburgh, US)
- Karthik Vaidhyanathan (IIIT - Hyderabad, IN) [dblp]
- Ingo Weber (TU München - Garching, DE) [dblp]
- Roland Weiss (ABB - Mannheim, DE) [dblp]
- Marc Zeller (Siemens - München, DE) [dblp]
- Liming Zhu (Data61, CSIRO - Sydney, AU) [dblp]
Classification
- Machine Learning
- Software Engineering
Keywords
- Machine Learning
- Software Architecture
- ML for Software Architecture
- Architecting ML systems.
