Dagstuhl Seminar 10111
Practical Software Testing: Tool Automation and Human Factors
( Mar 14 – Mar 19, 2010 )
Permalink
Organizers
- Mark Harman (University College London, GB)
- Henry Muccini (University of L'Aquila, IT)
- Wolfram Schulte (Microsoft Corporation - Redmond, US)
- Tao Xie (North Carolina State University - Raleigh, US)
Contact
Sponsors
Schedule
The main goal of the seminar “Practical Software Testing: Tool Automation and Human Factors” was to bring together academics working on algorithms, methods, and techniques for practical software testing, with practitioners, interested in developing more soundly-based and well-understood testing processes and practices. The seminar’s purpose was to make researchers aware of industry’s problems, and practitioners aware of research approaches. The seminar focused in particular on testing automation and human factors:
Tool automation. Automation of testing is a crucial concern in industry. It is only with automation that testing becomes practical and scalable to the size of a typical system with which the industry has to deal. Test automation or tool support spans the spectrum from test planning, generation, minimization, execution, oracle checking, to management. Test automation can exploit not only knowledge from the code under test but also from available models or specifications.
Human factors. Human factors play important roles in software testing. Given the code under test, tools can try to automate the generation of test inputs as much as possible, but test oracles still need to come from testers, who specify them in the form of specifications, properties, or test assertions, or directly inspect the actual test outputs for correctness. In addition, tools are not always perfect to deal with software complexity; testers need to cooperate with tools to effectively carry out testing tasks, by giving guidance to tools and interpreting results produced by tools. Thus testers need to be well trained.
In the week of March 14-19, 2010, 40 researchers from 11 countries (Canada, France, Germany, Italy, Luxembourg, the Netherlands, Sweden, Switzerland, South Africa, United Kingdom, United States) discussed their recent work, and recent and future trends in software testing. The seminar consisted of five main types of presentations or activities:topic-oriented presentations, researchoriented presentations, short self-introduction presentations, tool demos, and working group meetings and presentations.
In summary, the seminar accomplished all the expected goals, generating a great deal of forward momentum. The discussion and working groups allowed participants to form better understanding of open challenges and future directions in software testing. During the seminar, academic researchers and industrial researchers fully exchanged ideas for attempting to bridge the gap between research and practice. A number of participants exploited the substantial interactions at the seminar to foster future collaborations. After the seminar, the seminar organizers and participants compiled a bibliography by collecting a list of papers discussed or mentioned during the seminar. Several of the groups indicated that they intended to continue the discussion process after the seminar. We hope that the ideas and collaborations initiated at this Dagstuhl seminar in March 2010 will find fruition in papers, funded research projects, and technical innovations in the years to come.
- Nadia Alshahwan (University of Luxembourg, LU)
- Jamie Andrews (University of Western Ontario - London, CA)
- Thomas Ball (Microsoft Corporation - Redmond, US) [dblp]
- Antonia Bertolino (CNR - Pisa, IT) [dblp]
- Laura Dillon (Michigan State University - East Lansing, US)
- Gordon Fraser (Universität des Saarlandes, DE)
- Patrice Godefroid (Microsoft Corporation - Redmond, US) [dblp]
- Mark Grechanik (Accenture Labs - Chicago, US) [dblp]
- Wolfgang Grieskamp (Google - Sammamish, US) [dblp]
- Mark Harman (University College London, GB) [dblp]
- Mary Jean Harrold (Georgia Institute of Technology - Atlanta, US) [dblp]
- Robert M. Hierons (Brunel University, GB) [dblp]
- Yue Jia (King's College London, GB)
- A. J. Ko (University of Washington - Seattle, US) [dblp]
- Eileen T. Kraemer (University of Georgia, US) [dblp]
- Kiran Lakhotia (King's College London, GB)
- Yves Le Traon (University of Luxembourg, LU) [dblp]
- Bruno Legeard (Smartesting - Besancon, FR)
- Leonardo Mariani (University of Milan-Bicocca, IT)
- Darko Marinov (University of Illinois - Urbana-Champaign, US) [dblp]
- Henry Muccini (University of L'Aquila, IT) [dblp]
- Stephen Oney (Carnegie Mellon University, US) [dblp]
- Amit Paradkar (IBM TJ Watson Research Center - Yorktown Heights, US)
- John Penix (Google Inc. - Mountain View, US)
- Mauro Pezzè (University of Lugano, CH) [dblp]
- Andrea Polini (Università di Camerino, IT)
- Adam Porter (University of Maryland - College Park, US)
- Brian P. Robinson (ABB - Raleigh, US)
- David Rosenblum (University College London, GB) [dblp]
- Per Runeson (Lund University, SE) [dblp]
- David Schuler (Universität des Saarlandes, DE)
- Wolfram Schulte (Microsoft Corporation - Redmond, US) [dblp]
- Nikolai Tillmann (Microsoft Corporation - Redmond, US) [dblp]
- Paolo Tonella (Bruno Kessler Foundation - Trento, IT)
- Jan Tretmans (Embedded Systems Institute - Eindhoven, NL) [dblp]
- Willem Visser (Stellenbosch University - Matieland, ZA) [dblp]
- Michael W. Whalen (University of Minnesota - Minneapolis, US) [dblp]
- Tao Xie (North Carolina State University - Raleigh, US) [dblp]
- Shin Yoo (University College London, GB) [dblp]
- Michal Young (University of Oregon - Eugene, US)
Classification
- sw-engineering
- society / HCI
- verification / logic
- semantics / formal methods
Keywords
- Software testing
- test generation
- test automation
- test oracles
- testing tools
- humancomputer interaction
- code-based testing
- specification-based testing
- model-based testing