Overview and Motivation

Modern software systems evolve rapidly and often need to exist in many variants. Consider the Linux kernel with its uncountable number of variants. Each variant addresses different requirements, such as runtime environments ranging from Android phones to large super-computers and server farms. At the same time, the Linux kernel frequently boasts new versions, managed by thousands of developers. Yet, software versions–resulting from evolution in time–and variants–resulting from evolution in space–are managed radically differently. Version management relies on a version control system (Git) and sophisticated workflows–concepts that have been developed for decades in the field of software configuration management (SCM) [12, 24, 23]. Variant management in the Linux kernel relies on techniques known from the field of software product line engineering (SPLE) [27, 11, 13], such as an integrated software platform, a variant-aware build system [7], an interactive configurator tool [30], and a model-based representation [9, 8, 17, 1] of all kernel features [4, 28]. The Linux kernel is exemplary for many large-scale, variant-rich, and rapidly evolving software systems in industry [5, 3, 32], especially in the domains of embedded, cyber-physical, automotive, and avionics control systems.

Despite decades of research in both fields, the effective evolution of variant-rich systems is still an open problem. Three main challenges exist. First, while version control systems are well-integrated into development processes, product-line engineering requires investment into additional tooling and different processes that are difficult to adopt. In fact, organizations rarely adopt product line engineering from scratch [6], but rather use readily available version control systems with their branching and forking facilities–a strategy known as clone&own [15, 10]. While this strategy is simple, it does not scale with the number of variants, and then requires evolving (i.e., re-engineering) cloned variants into a product-line platform [2]. Second, evolving product-line platforms is substantially more complex than evolving single variants, mainly since developers need to work on all variants at the same time [25]. Third, the granularity of tracking versions of variants is still unclear. While the whole platform can be versioned, ideally, versioning at the level of features should be supported.

In summary, SCM and SPLE are two widely established, yet actively researched software engineering disciplines offering a variety of concepts to deal with software versions and variants [16, 14, 18, 21]. Yet, despite various attempts [22, 34, 33, 20], none of the two disciplines has been successful in establishing unified solutions addressing both problems at the same time–mainly due to the isolation of both communities and due to the absence of realistic and widely accepted requirements on how to evaluate the effectiveness of techniques for managing both versions and variants.

Goals of the Seminar

This Dagstuhl Seminar aimed at establishing a body of knowledge on unified version and variant management. We invited leading practitioners and researchers from both disciplines to discuss each other’s challenges, solutions, and experiences. The seminar’s goals were to: (i) survey state-of-the-art SCM and SPLE concepts and map both areas’ terminologies and open problems, (ii) gather industrial and academic challenges and requirements on integrated version and variant management, (iii) survey and assess existing evaluation approaches, and (iv) stablish a research agenda, research infrastructure, and working groups. To guide future research, the participants also discussed the basis to work on improved evaluation approaches–as benchmarks for new version and variant management techniques. As such, the long-term goal of the seminar was to enable the development and evaluation of enhanced version and variant management techniques that will be adopted in practice.

Week Overview

Monday. After an introduction of all participants, the seminar started off with general talks on versioning and variability. Bernhard Westfechtel set the stage with an introduction into version management concepts and workflows, which already illustrated some overlap with variability management concepts. For instance, directed deltas are conceptually similar to compositional variation mechanisms (e.g., feature modules or delta modules), and the construction of versions in intensional versioning can be related to the configuration-based derivation of individual variants from a product-line platform. The seminar continued with a talk by Don Batory, who discussed the integration of version control systems, variability management techniques, and integrated development environments (IDEs) based on ideas centering around a better representation and execution of program refactorings in versioned and variant-rich software systems. The talk by Thorsten Berger (actually given on Tuesday, since the introduction round and discussions for the other talks took more time) followed up on the concepts introduced in the previous talks and presented a survey on variation control systems, which support developers managing variant-rich systems in terms of features. Such variation control systems go back to the end of the 1970s with concepts and prototypes developed in the SCM community, but never made it into the mainstream. The talk surveyed their concepts and discussed problems likely prohibiting their adoption. Thereafter, we enjoyed three talks on industrial perspectives given by our industrial practitioners: Henrik Lönn (Volvo), Danilo Beuche (pure::systems), and Ramesh S. (General Motors; talk also given on Tuesday for timing reasons), confirming and explaining the gaps between academia and industry.

Tuesday. The day started with an introduction into the prospective breakout groups for the afternoon, followed by the talk of Christoph Seidl on versioning of product lines relying on a representation of feature versions in a new dialect of feature models, called Hyper Feature Models. Thereafter, the breakout sessions on four relevant topics took place, specifically: on a conceptual model to map SPLE and SCM concepts, on operations for managing versions and variants, on analyses of versions and variants, on workflows for managing versions and variants, and on first-class support of variability and versioning in programming languages. A benchmarking group was discussed, but abandoned in favor of first working on the foundations before discussing benchmarking techniques to evaluate prospective unified techniques for versioning and variability. The breakout group discussions continued until the afternoon, before the remaining talks from Monday were given (Thorsten Berger and Ramesh Sethu), followed by lightning talks from Shurui Zhou and Sandro Schulze. Shurui discussed the relevance of version and variability management in the domain of engineering AI-based systems, where models and large dataset need to be managed. Sandro proposed a roundtrip- engineering process relying on unified management of versioning and variability, relying on automated extraction of variability information from cloned variants (which should be integrated into a platform in a round-trip-engineering manner).

Wednesday. We started the day with a talk by Daniel Strüber on benchmarking scenarios and a survey of existing benchmarks. In fact, it is a common consensus of the community that the lack of strong, landmark benchmarks hinders the progress in both communities (SCM and SPLE). Thereafter, Yi Li presented his work on slicing of the history of software codebases along features, where features are represented by test cases to help identifying the relevant code in a longitudinal manner. Thomas Thüm then presented a vision on the–ideally automated–synchronization of cloned variants as followed by the VariantSync research project which is led by Thomas and Timo Kehrer. Thomas also presented a very first prototypical implementation of the VariantSync tool. The approach shares, based on audience feedback, ideas with the Virtual Platform, proposed by researchers in 2014 [1]. In the afternoon, the majority of the participants continued their discussion on their group trip to the city of Trier and a dinner at a local winery.

Thursday. The day began with a talk by Gabriele Taentzer, presenting a generalizing framework for transformations of software product lines, relying on the formalism of category theory. Another talk was given by Julia Rubin on equivalence checking of variants based on behavior instead of structural characteristics of changes. Thereafter, the breakout groups continued their discussions until the later afternoon, where the results were presented to the other seminar participants. After dinner, two lightning talks were given by Paulo Borba and Iris Reinhartz-Berger. Paulo discussed the detection of semantic merge conflicts in the light of avoiding unwanted feature interactions, and Iris presented insights from two research projects on behavior-derived variability analysis and mechanisms recommendation.

Friday. The last day of the seminar started with a talk by Lukas Linsbauer on his work towards a feature-oriented and distributed version-control system, relying on the variantintegration tooling ECCO. We then had a closing discussion, re-iterating the main challenges we identified throughout the seminar, as well as discussing future work.

Outcome of the Seminar

The seminar established breakout groups who continued their discussion after the seminar and already published two papers [7, 1] at the VariVolution workshop, hosted at the Systems and Software Product Line Conference (SPLC). In addition, a paper accepted at the main track of SPLC on benchmarking, relying on input from the seminar participants via a survey [31], and providing an initial infrastructure for community-oriented benchmark creation, can be seen as a core outcome of the seminar.

A core topic of the final discussion was the teaching of SPLE and SCM concepts–an important means to eventually improve the handling of versions and variants in practice. One of the problems identified is that, while SCM is covered sufficiently, the relevant variability-management concepts are not taught at the Bachelor’s level in the majority of universities. However, the discussants believe that practicing feature-oriented analysis and design early in the curriculum would be beneficial, where currently object-oriented analysis and design is dominating. Interestingly, based on the experience of the discussants, SPLE is still seen as something rather futuristic by students, which is somewhat surprising, given that building highly configurable systems and software platforms are established practices, so perhaps there is a perception and awareness problem that teaching needs to address. Naturally, a course teaching SPLE at the Bachelor’s level should also teach the relevant SCM concepts. A closely related topic discussed is that of teaching architectures, especially those of product lines, which is not really in the focus of current software architecture courses. Of course, it is generally difficult to talk to students about software architecture, since, as a discussant explains, a relevant abstract concept that students do not immediately perceive as relevant in the course of the studies. In contrast, with compilers and databases, students obtain some hands-on experience, which allows them to relate more closely to, especially with respect to a future job in the industry. This calls for close collaboration with industry in SPLE courses.

Establishing benchmarks turned out to be a more difficult problem than expected. Benchmarking was prominently discussed, as well as input elicited for a set of 11 high-level benchmarking scenarios defined by some of the seminar participants and organizers before the seminar. The participants plan to follow-up on creating concrete benchmarks upon the infrastructure created. One idea is to build a web application to contribute specific benchmark data (e.g., code integration examples, comprising the original code variants and the final result as a ground truth) to establish a community benchmark. Another interesting comment was that the currently published case studies and experience reports about variability management and product lines are relatively old and do not provide sufficient technical details. Furthermore, they also do not highlight the problems associated with clone&own and the need for product-line migration techniques adequately. This discussion is a call to arms for improving the benchmarking situation in the SCM and SPLE community.

Last but not least, an important outcome of the final discussion session of the seminar is the need for a commonly agreed set of core concepts, mechanisms and practices–a welldocumented Body of Knowledge (BOK) of our discipline. Currently, only some aspects of versioning in time and space are partially covered by the Software Engineering BOK (SWEBOK). However, for promoting a consistent view of our discipline worldwide and beyond our discipline borders as well as for having a foundation for a consistent curriculum development, a dedicated BOK or an extension of the SWEBOK may be necessary as a community effort.

References

1. Sven Apel, Don Batory, Christian Kästner, and Gunter Saake. Feature-Oriented Software Product Lines. Springer, Berlin Heidelberg, 2013.
2. Wesley K. G. Assunção, Roberto E. Lopez-Herrejon, Lukas Linsbauer, Silvia R. Vergilio, and Alexander Egyed. Reengineering legacy applications into software product lines: a systematic mapping. Empirical Software Engineering, 22(6):2972–3016, 2017.
3. Jonatas Ferreira Bastos, Paulo Anselmo da Mota Silveira Neto, Padraig O’Leary, Eduardo Santana de Almeida, and Silvio Romero de Lemos Meira. Software product lines adoption in small organizations. Journal of Systems and Software, 131(Supplement C):112– 128, 2017.
4. Thorsten Berger, Daniela Lettner, Julia Rubin, Paul Grünbacher, Adeline Silva, Martin Becker, Marsha Chechik, and Krzysztof Czarnecki. What is a Feature? A Qualitative Study of Features in Industrial Software Product Lines. In SPLC, 2015.
5. Thorsten Berger, Divya Nair, Ralf Rublack, Joanne M. Atlee, Krzysztof Czarnecki, and Andrzej Wasowski. Three cases of feature-based variability modeling in industry. In MODELS, 2014.
6. Thorsten Berger, Ralf Rublack, Divya Nair, Joanne M. Atlee, Martin Becker, Krzysztof Czarnecki, and Andrzej Wasowski. A Survey of Variability Modeling in Industrial Practice. In VaMoS, 2013.
7. Thorsten Berger, Steven She, Krzysztof Czarnecki, and Andrzej Wasowski. Feature-to- Code mapping in two large product lines. In SPLC, 2010.
8. Thorsten Berger, Steven She, Rafael Lotufo, Andrzej Wasowski, and Krzysztof Czarnecki. Variability modeling in the real: A perspective from the operating systems domain. In ASE, 2010.
9. Thorsten Berger, Steven She, Rafael Lotufo, Andrzej Wasowski, and Krzysztof Czarnecki. A Study of Variability Models and Languages in the Systems Software Domain. IEEE Transactions of Software Engineering, 39(12):1611–1640, 2013.
10. John Businge, Openja Moses, Sarah Nadi, Engineer Bainomugisha, and Thorsten Berger. Clone-based variability management in the Android ecosystem. In ICSME, 2018.
11. Paul Clements and Linda Northrop. Software Product Lines: Practices and Patterns. Addison-Wesley, Boston, MA, 2001.
12. Reidar Conradi and Bernhard Westfechtel. Version models for software configuration management. ACM Comput. Surv., 30(2):232–282, 1998.
13. Krzysztof Czarnecki and Ulrich W. Eisenecker. Generative Programming: Methods, Tools, and Applications. Addison-Wesley, Boston, MA, 2000.
14. Danny Dig, Kashif Manzoor, Ralph Johnson, and Tien N Nguyen. Refactoring-aware configuration management for object-oriented programs. In ICSE, 2007.
15. Yael Dubinsky, Julia Rubin, Thorsten Berger, Slawomir Duszynski, Martin Becker, and Krzysztof Czarnecki. An exploratory study of cloning in industrial software product lines. In CSMR, 2013.
16. Jacky Estublier, David Leblang, André van der Hoek, Reidar Conradi, Geoffrey Clemm, Walter Tichy, and Darcy Wiborg-Weber. Impact of software engineering research on the practice of software configuration management. ACM Transactions on Software Engineering and Methodology, 14(4):383–430, 2005.
17. Kyo Kang, Sholom Cohen, James Hess, William Nowak, and Spencer Peterson. Featureoriented domain analysis (FODA) feasibility study. Technical Report SEI-90-TR-21, CMU, 1990.
18. Timo Kehrer, Udo Kelter, and Gabriele Taentzer. A rule-based approach to the semantic
19. 9 Jacob Krueger, Wanzi Gu, Hui Shen, Mukelabai Mukelabai, Regina Hebig, and Thorsten Berger. Towards a better understanding of software features and their characteristics: A case study of Marlin. In VaMoS, 2018.
20. Vincent J. Kruskal. Managing multi-version programs with an editor. IBM Journal of Research and Development, 28(1):74–81, 1984.
21. Philip Langer, Manuel Wimmer, Petra Brosch, Markus Herrmannsdörfer, Martina Seidl, Konrad Wieland, and Gerti Kappel. A posteriori operation detection in evolving software models. Journal of Systems and Software, 86(2):551–566, 2013.
22. Lukas Linsbauer, Thorsten Berger, and Paul Grünbacher. A classification of variation control systems. In GPCE, 2017.
23. Stephen A. MacKay. The state of the art in concurrent, distributed configuration management.
24. 4 Axel Mahler. Configuration management. Chapter Variants: Keeping Things Together and Telling Them Apart. Wiley, 1995.
25. Jean Melo, Claus Brabrand, and Andrzej Wasowski. How does the degree of variability affect bug finding? In ICSE, ACM.
26. Mukelabai Mukelabai, Damir Nešic, Salome Maro, Thorsten Berger, and Jan-Philipp Steghöfer. Tackling combinatorial explosion: A study of industrial needs and practices for analyzing highly configurable systems. In ASE, 2018.
27. David Parnas. On the design and development of program families. IEEE Transactions on Software Engineering, 2(1):1–9, 1976.
28. Leonardo Passos, Jesus Padilla, Thorsten Berger, Sven Apel, Krzysztof Czarnecki, and Marco Tulio Valente. Feature scattering in the large: A longitudinal study of Linux kernel device drivers. In MODULARITY, 2015.
29. Christopher Pietsch, Timo Kehrer, Udo Kelter, Dennis Reuling, and Manuel Ohrndorf. SiPL–A Delta-Based Modeling Framework for Software Product Line Engineering. In ASE, 2015.
30. Julio Sincero, Horst Schirmeier, Wolfgang Schröder-Preikschat, and Olaf Spinczyk. Is the Linux kernel a software product line. In Workshop on Open Source Software and Product Lines, 2007.
31. Daniel Strueber, Mukelabai Mukelabai, Jacob Krueger, Stefan Fischer, Lukas Linsbauer, Jabier Martinez, and Thorsten Berger. Facing the truth: Benchmarking the techniques for the evolution of variant-rich systems. In SPLC, 2019.
32. Christer Thörn. Current state and potential of variability management practices in softwareintensive SMEs: Results from a regional industrial survey. Information and Software Technology, 52(4):411–421, 2010.
33. Eric Walkingshaw and Klaus Ostermann. Projectional editing of variational software. In GPCE, 2014.
34. Bernhard Westfechtel, Bjørn P. Munch, and Reidar Conradi. A layered architecture for uniform version management. IEEE Transactions of Software Engineering, 27(12):1111–1133, 2001.

Creative Commons BY 3.0 Unported license

Thorsten Berger, Marsha Chechik, Timo Kehrer, and Manuel Wimmer