Dagstuhl Seminar 13091
Analysis, Test and Verification in The Presence of Variability
( Feb 24 – Mar 01, 2013 )
(Click in the middle of the image to enlarge)
Permalink
Please use the following short url to reference this page:
https://www.dagstuhl.de/13091
Organizers
- Paulo Borba (Federal University of Pernambuco - Recife, BR)
- Myra B. Cohen (University of Nebraska - Lincoln, US)
- Axel Legay (INRIA Rennes - Bretagne Atlantique, FR)
- Andrzej Wasowski (IT University of Copenhagen, DK)
Contact
Variability management is a set of practices that enable the development of software products together as a single product family by organizing] the common and variant parts of source code, and other related assets. Software developers adopt variability management in order to increase the flexibility of software, or to be able to produce variations of software economically, reducing cost through increased reuse.
Variability in software is a fact. It is supported by a number of strong economic trends: a personalization trend (catering to the needs of diverse users), an adaptability trend (the need to make software more flexible), and by business incentives – to make software less expensive to produce through systematic reuse.
The development processes around designing such software, and tools for deriving variants (generative programming) are now well established. Technologies exist that support programming and compilation of software with variability: preprocessors, polymorphism, object-oriented frameworks, component and extension frameworks, code generation and model driven development (domain specific languages.
Only recently have researchers begun working with core technologies that support the correctness of product families, turning their attention towards supporting (and leveraging) variability. Automatic analyses of software with variability is difficult, because it requires reasoning about all (or a number of) the variants simultaneously. Custom analysis techniques, for software with variability, however, allows for early detection of errors, because many (or all) variations of the software can be analyzed simultaneously, and there may be overlap between parts of the system.
While it is simple to derive a variant of a C program by applying a preprocessor to it for instance, reasoning about all variations that the preprocessor code allows, is complex. New tools and techniques have started to appear that are able to tell whether all variants of a C program with preprocessor conditional directives adhere to the C grammar, but this is still in its infancy. Standard tools are ineffective, since they require compiling all of the variants of such a program, which is clearly infeasible since systems grow combinatorially, and there may be thousands of options governing the compilation process for a single product family. Similar problems affect all elements of the software development tool-chain: type systems, program analyses, model checkers, model based testing tools, etc.
Our goal is to consolidate and stimulate research on analyses of software models with variability, enabling the design of variability-aware tool chains. We bring together key researchers working on quality assurance challenges that arise from introducing variability, and some who do not work with variability, but that are experts in their respective areas of broader software analysis research.
The seminar will include:
1. Sessions on challenges of analysis of variability, including models and software analysis, given by experts in variability research.
2. Presentation of state of the art research on software analysis that is not developed with variability in mind, but that can be applied to tackling the variability challenge.
3. Dynamically planned sessions on how to address these challenges, how to transfer knowledge, tools, and benchmarks between research areas.
The methods in focus will be program analysis, model checking, type checking, and testing. We believe that the seminar will fruitfully mix computer science and software engineering researchers, allowing the former to derive interesting basic research problems stemming from practical needs, and inspiring the latter to use the latest research advances in software analysis technology to advance variability management tools.
The seminar "Analysis, Test and Verification in The Presence of Variability" that took place at Schloss Dagstuhl from February 24 to March 1, 2013, had the goal of consolidating and stimulating research on analysis of software models with variability, enabling the design of variability-aware tool chains. We brought together 46 key researchers from three continents, working on quality assurance challenges that arise from introducing variability, and some who do not work with variability, but that are experts in their respective areas in the broader domain of software analysis or testing research. The participants ranged from those in senior academic positions to successful graduate students. We also enjoyed the presence of several relevant experts from the software development industry.
The seminar included:
1. Invited presentations on state of the art research in SPL testing and verificationThe presentations were delivered by experts in variability research. The topics included classifying and unifying product-line analyses, combinatorial interaction testing, model-based testing, analysis of programs with variability and model checking with variability.
Material relevant to the topic of this Dagstuhl was organized in a recent classification by Thüm and coauthors [4]. The Dagstuhl seminar opened with a presentation of this classification, which created a common ontology for later presentations and discussions. This was very helpful for participants who had different areas of expertise.
2. A keynote presentation on the Challenges and Science of VariabilityWe organized a special keynote shared with the German FOSD meeting, that took place in parallel at the Schloss Dagstuhl facilities. The keynote speaker, Professor Don Batory, called for creating a simple meta-theory identifying and relating the core concepts and properties of variability science, i.e. the body of knowledge created by the community of researchers studying engineering of highly configurable systems. During the workshop, several candidates for the starting point of such theory were mentioned, such as using simple models in constructive logic [2], choice calculus [3] or Clafer [1].
3. A series of presentations on recent results in Variability AnalysisThe bulk of the programme was filled with a mixture of research presentations about recent research advances in verification, analysis and test of software with variability. This function of the seminar was particularly important, as the usual dissemination outlets for these contributions are often disjoint -- much of the work is normally presented in domain specific publication channels devoted to only test, verification or programming languages. For many participants the seminar created an opportunity to learn about advances at addressing similar problems in the neighboring research communities -- an experience that is rarely possible outside of Dagstuhl.
4. A session of student presentationsIn order to enrich the presentations by senior researchers with a stream of fresh ideas, we organized a special session devoted to short student presentations. The presenters were selected from the participants of the German FOSD meeting. For many of the students it was a rare opportunity to share their ideas with international authorities in their work area. The topics of these lightning presentations were closely related to the seminar goals and included among others, discussions of experimental evaluation of product line analysis strategies, static analysis, type checking for variability, and performance prediction for configurable systems. The session enabled closer integration between the participants of the two events. Many discussions between the two groups continued throughout the week.
5. Dynamically planned sessions on how to address the challenges, how to transfer knowledge, tools, and benchmarks between research areasThe first session (run by Professor Krzysztof Czarnecki) was devoted to extracting challenges for variability analysis out of industrial requirements. Participants from industry and participants from academia involved in industrial projects provided background on requirements known from projects in avionics, automotive and risk assessment domains. These were further discussed to identify research challenges for future work. The discussions were continued in a breakout session on product lines of safety critical systems. Other breakout sessions included dynamic product lines, generic representation of variability, and testing and modeling variability.
Overall, a core set of techniques were discussed at this seminar which include program analysis, model checking, type checking, and testing. We believe that the seminar fruitfully mixed computer science and software engineering researchers from several research sub-domains, allowing them to derive interesting basic research problems stemming from practical needs all related to how variability impacts their respective domains, with the sub-goal of inspiring the use of the latest research advances in software analysis technology to advance variability management tools.
Results
The different kinds of interactions offered by the seminar helped the participants to relate work covering different aspects in a number of dimensions such as:
- An overall approach to thinking about variability, as defined by Thüm's classification [4] of analysis into product based, family based, feature based and hybrids;
- Core techniques: testing, verification, refactoring, model checking, static analysis;
- Mechanisms for representing variability: if-defs, deltas, generic representation, etc.;
- Application domains;
- The nature of variability: static product lines, dynamic product lines, configurable systems.
The seminar also produced a bibliography of core readings on the topic, that can enable new graduate students to engage more quickly in this area of research.
Trying to classify approaches with respect to these dimensions helped to identify similarities and differences among different techniques (static analysis, model checking, testing, and verification). This, in turn, might trigger new collaborations and research results. The presentations and the ad-hoc discussion sessions helped people to clarify differences and similarities among configurable systems and dynamic and static product lines, with similar consequences to the ones described above. More generally, of course, the Dagstuhl provided the benefit of mixing young and experienced researchers, from different countries and research areas.
An informal survey among a handful of participants has shown that each of them have started 2-3 new collaborations as a result of the seminar. These collaborations took the form of initiated research papers, mutual research visits, or student exchanges. In one anecdotal case, a researcher started a collaboration with a colleague sitting in the same corridor at his home university--- but apparently one had to meet in Dagstuhl to enable the exchange of ideas. We can thus expect a new wave of research results in this area to flourish about a year from the seminar time. Because of this success, we intend to organize a follow up event in several years, be it under the Schloss Dagstuhl programme or under some other appropriate venue.
References:
- Kacper Bak, Krzysztof Czarnecki, and Andrzej Wasowski. Feature and Meta-Models in Clafer: Mixed, Specialized, and Coupled. In Proc. of the 3rd Int'l Conf. on Software Language Engineering (SLE'10), LNCS, Vol.6563, pp.102-122, Springer, 2011. http://dx.doi.org/10.1007/978-3-642-19440-5_7}{DOI: 10.1007/978-3-642-19440-5\_7.
- Benjamin Delaware, William R. Cook, and Don S. Batory. Product lines of theorems. In Proc. of the 2011 ACM Int'l Conf. on Object-oriented Programming Systems, Languages, and Applications (OOPSLA'11), pp. 595-608, ACM, 2011. http://dx.doi.org/10.11452048066.2048113 DOI:10.1145/2048066.2048113.
- Martin Erwig and Eric Walkingshaw. The Choice Calculus: A Representation for Software Variation. ACM Trans. Softw. Eng. Methodol., Vol. 21, Issue 1, pp. 6:1-6:27, 2011. http://dx.doi.org/10.1145/2063239.2063245 DOI:10.1145/2063239.2063245.
- Thomas Thüm, Sven Apel, Christian Kästner, Martin Kuhlemann, Ina Schaefer, and Gunter Saake. Analysis Strategies for Software Product Lines. Technical Report FIN-004-2012, School of Computer Science, University of Magdeburg, April 2012. http://www.cs.uni-magdeburg.de/inf_media/downloads/forschung/technical_reports_und_preprints/2012/04_2012.pdf.
Creative Commons BY 3.0 Unported license
Paulo Borba, Myra B. Cohen, Axel Legay, and Andrzej Wasowski
- Vander Alves (University of Brasilia, BR) [dblp]
- Sven Apel (Universität Passau, DE) [dblp]
- Joanne M. Atlee (University of Waterloo, CA) [dblp]
- Kacper Bak (University of Waterloo, CA) [dblp]
- Don Batory (University of Texas - Austin, US) [dblp]
- Thorsten Berger (IT University of Copenhagen, DK) [dblp]
- Eric Bodden (TU Darmstadt, DE) [dblp]
- Paulo Borba (Federal University of Pernambuco - Recife, BR) [dblp]
- Claus Brabrand (IT University of Copenhagen, DK) [dblp]
- Dave Clarke (KU Leuven, BE) [dblp]
- Andreas Classen (University of Namur, BE) [dblp]
- Roberta Coelho (Federal University of Rio Grande do Norte, BR) [dblp]
- Myra B. Cohen (University of Nebraska - Lincoln, US) [dblp]
- Maxime Cordy (Facultés Universitaires Notre-Dame de la Paix, BE) [dblp]
- Krzysztof Czarnecki (University of Waterloo, CA) [dblp]
- Sabrina de Figueirêdo Souto (Federal University of Pernambuco - Recife, BR) [dblp]
- Martin Erwig (Oregon State University, US) [dblp]
- Alessandro Fantechi (University of Florence, IT) [dblp]
- Brady J. Garvin (University of Nebraska - Lincoln, US) [dblp]
- Rohit Gheyi (Universidade Federal - Campina Grande, BR) [dblp]
- Stefania Gnesi (CNR - Pisa, IT) [dblp]
- Reiner Hähnle (TU Darmstadt, DE) [dblp]
- Øystein Haugen (SINTEF - Oslo, NO) [dblp]
- Martin Fagereng Johansen (University of Oslo, NO) [dblp]
- Christian Kästner (Carnegie Mellon University - Pittsburgh, US) [dblp]
- Shriram Krishnamurthi (Brown University - Providence, US) [dblp]
- Kim Lauenroth (adesso AG - Dortmund, DE) [dblp]
- Axel Legay (INRIA Rennes - Bretagne Atlantique, FR) [dblp]
- Martin Leucker (Universität Lübeck, DE) [dblp]
- Tiziana Margaria (Universität Potsdam, DE) [dblp]
- Dusica Marijan (Simula Research Laboratory - Lysaker, NO) [dblp]
- Jean-Vivien Millo (INRIA Sophia Antipolis - Méditerranée, FR) [dblp]
- Gilles Perrouin (University of Namur, BE) [dblp]
- Márcio Ribeiro (Federal University of Pernambuco - Recife, BR) [dblp]
- Ina Schaefer (TU Braunschweig, DE) [dblp]
- Holger Schlingloff (HU Berlin, DE) [dblp]
- Sergio Segura (University of Sevilla, ES) [dblp]
- Vanessa Stricker (Universität Duisburg - Essen, DE) [dblp]
- Leopoldo Teixeira (Federal University of Pernambuco - Recife, BR) [dblp]
- Maurice H. ter Beek (CNR - Pisa, IT) [dblp]
- Thomas Thüm (Universität Magdeburg, DE) [dblp]
- Társis Tolêdo (Federal University of Pernambuco - Recife, BR) [dblp]
- Salvador Trujillo (Ikerlan Research Centre - Arrasate-Mondragón, ES) [dblp]
- Eric Walkingshaw (Oregon State University, US) [dblp]
- Andrzej Wasowski (IT University of Copenhagen, DK) [dblp]
- Cemal Yilmaz (Sabanci University - Istanbul, TR) [dblp]
Classification
- programming languages / compiler
- semantics / formal methods
- software engineering
Keywords
- Verification
- Program Analysis
- Testing
- Semantics of Programming Languages
- Software Engineering
