The increasing power of modern computing systems coupled with the increasing demand for automation in diverse application areas has led to software stacks of massive complexity. The code base for the Boeing 787 or the F-35 Joint Strike Fighter aircrafts amounts to several million lines of code (LOC) each. The software in Chevy’s Volt automobile is reported at 10 million LOC. In 2009, premium cars were reported to contain over 100 million LOC. Given that commodity software contains, on average, between 0.5 and 0.76 bugs per 1000 LOC, and even extremely critical and well-reviewed code contains a bug per 10,000 LOC according to a NASA study, several hundred to several thousand residual software bugs are optimistic estimates for such large software systems.

If triggered during execution, either by accident or by malicious intent, these residual bugs can result in software/system failures with severe consequences. In order to cope with this problem, researchers from the software engineering, security, and fault tolerance areas are working on mechanisms for detecting residual faults and for limiting their effects at runtime. For the evaluation of these mechanisms, researchers often rely on software bug simulations (referred to as “mutations”, “fault injections”, or “vulnerability additions”) to create arbitrary numbers of bugs by modifying correct code. Such simulations need to resemble the characteristics of actual residual bugs as closely as possible to not threaten the evaluations’ validity. From the discussion of bug simulation approaches in the literature, we observe that different bug models are used in different communities (mostly the software engineering, security, and fault tolerance communities) and that the technical details of their simulations differ.

The goal of this Dagstuhl Seminar is to create awareness of existing work on residual bug simulations in the different communities and to establish a common understanding of the state of the art and open research problems.

Seminar Topics

The seminar topics result from combinations along three dimensions:

1. State of the art/practice in residual bug simulation vs. emergent problems
2. Residual bug models (i.e., what are residual bugs) vs. bug simulation techniques (i.e., how are residual bugs simulated)
3. Approaches/Synergies across the software engineering, security, and fault tolerance communities

Seminar Goals

Beyond creating awareness of related work on residual bug simulations across the different communities and initiating cross-community research collaborations, the expected results of the seminar are:

1. Advocacy of the seminar results in a “Systematization of Knowledge (SoK)” article, including a cross-community research agenda, preferably to be published in IEEE Software/Computer or ACM’s CACM/Computing Surveys given the broad visibility of these publications across the communities
2. Establishing a forum for regular cross-community exchange, e.g., a workshop rotating among the communities’ flagship conferences.
3. To share research results and artifacts related to residual bug simulation, we aim to establish a common repository and mailing list. Other possible options to ensure a timely dissemination of results across the different communities are to be discussed in the seminar.

Creative Commons BY 3.0 DE

Domenico Cotroneo, Cristina Nita-Rotaru, Karthik Pattabiraman, and Neeraj Suri