Dagstuhl Seminar 23411
Accountable Software Systems
( Oct 08 – Oct 13, 2023 )
Permalink
Organizers
- Bettina Könighofer (TU Graz, AT)
- Joshua A. Kroll (Naval Postgraduate School - Monterey, US)
- Ruzica Piskac (Yale University - New Haven, US)
- Michael Veale (University College London, GB)
Contact
- Andreas Dolzmann (for scientific matters)
- Simone Schilke (for administrative matters)
Accountability in the context of software is an emerging area that has attracted interest in disparate fields from computing and information science to philosophy to political science to law and policy studies. Presently, there is an increasing number of autonomous agents shifting into safety-critical aspects of society and our everyday lives. Autonomously driven vehicles share the roads with us. Computerized reasoning aids healthcare providers in disease diagnostics, recruiters in hiring, and even adjudicators in analyzing pretrial flight risks. While a common goal of these autonomous agents is to assist and improve human lives, it is a harsh reality that the opposite occurs far too frequently. Car accidents involving autonomously driven vehicles have been fatal, and bias-vulnerable autonomous decision-making has led to discriminatory assessment and abuse of individuals. As a result, accountability is started to be investigated in areas including AI, machine learning, control, systems development, data management, software engineering, programming languages, human factors/human-computer interaction, and formal verification communities.
Understanding accountability and the place of formal computing tools in assigning it has been recognized by funding agencies as an important research topic. Significant reforms are currently occurring in the law to the liability of autonomous systems, such as the proposed EU Artificial Intelligence Act; reforms to the EU Machinery Directive and Product Liability Directive; standardization processes around software and accountability at the ISO, IEEE, CEN/CENELEC, US NIST, and ETSI, among others; a range of proposals around reforms to intermediary liability and recommender systems in the US Congress and proposals from the White House Office of Science and Technology Policy, among many other ongoing changes and revisions to existing policies and recommendations for new policies around the world.
Throughout the Dagstuh Seminar, participants engaged in a rich tapestry of discussions, presentations, and working groups, focusing on three main areas:
- Justifying Assurance: Opportunities and Limits of Formal Tools for Accountability in Sociotechnical Context
- The Role(s) of Computing and Infrastructures in Accountability for Software
- New Formal Specification Languages and Modeling Techniques for Accountability
Group discussions further enriched the seminar's discourse. For example, debates around the Software as a Service (SaaS) model and the potential for a public, democratically managed cloud infrastructure highlighted the evolving nature of software maintenance and the democratization of digital infrastructure. Another group delved into the intricacies of software complexity, distinguishing between accountability and responsibility, and discussing the challenges of regulatory adaptation to technological innovation.
By having short talks to inspire discussions followed by in-depth discussions in breakout groups, the seminar successfully forged new conversations between the communities with a focus on how to improve upon the state of the art and practice in designing software systems with respect to accountability. Participants collectively recognized the importance of socio-technical approaches and formal methods in developing robust accountability frameworks, advocating for synergies between different fields to refine the state of the art and practice.
The seminar concluded with a group exercise to map and group open questions, which are documented at the end of the full report. This exercise naturally provided more questions than answers, but these questions were coalescing on deep and specific interdisciplinary challenges that participants intended to take forwards in their future work.
In conclusion, the Dagstuhl Seminar on "Accountable Software Systems" served as a pivotal forum for cross-disciplinary exchange, catalyzing new conversations and potential collaborations.Accountability in the context of software is an emerging area that has attracted interest in disparate fields from computing and information science to philosophy to political science to law and policy studies. Presently, there is an increasing number of autonomous agents shifting into safety-critical aspects of society and our everyday lives. Autonomously driven vehicles share the roads with us. Computerized reasoning aids healthcare providers in disease diagnostics, recruiters in hiring, and even adjudicators in analyzing pretrial flight risks. While a common goal of these autonomous agents is to assist and improve human lives, it is a harsh reality that the opposite occurs far too frequently. Car accidents involving autonomously driven vehicles have been fatal, and bias-vulnerable autonomous decision-making has led to discriminatory assessment and abuse of individuals. As a result, accountability is started to be investigated in areas including AI, machine learning, control, systems development, data management, software engineering, programming languages, human factors/human-computer interaction, and formal verification communities.
Understanding accountability and the place of formal computing tools in assigning it has been recognized by funding agencies as an important research topic. Significant reforms are currently occurring in the law to the liability of autonomous systems, such as the proposed EU Artificial Intelligence Act; reforms to the EU Machinery Directive and Product Liability Directive; standardization processes around software and accountability at the ISO, IEEE, CEN/CENELEC, US NIST, and ETSI, among others; a range of proposals around reforms to intermediary liability and recommender systems in the US Congress and proposals from the White House Office of Science and Technology Policy, among many other ongoing changes and revisions to existing policies and recommendations for new policies around the world.
The goal of this Dagstuhl Seminar is to integrate the perspectives of many different communities, convening a disparate group of experts to discuss open problems and potential future directions across a variety of domains with a focus on accountability. We have highlighted three main areas for this discussion:
- Justifying Assurance: Opportunities and Limits of Formal Tools for Accountability in Sociotechnical Context
- The Role(s) of Computing and Infrastructures in Accountability for Software
- New Formal Specification Languages and Modeling Techniques for Accountability
In addition to scientific contributions, we expect that this seminar will serve as a forum for cross-disciplinary discussions that would not otherwise take place. The seminar will forge new conversations between the communities with a focus on socio-technical research and formal methods community. Another goal of the seminar is to enable synergies between different fields attacking the problem of accountability to discuss and improve upon the state of the art and practice. As a result of the seminar, we plan to produce a report, that can act as a body of knowledge for researchers joining this newly forming community.
- Thomas Arnold (Tufts University - Medford, US) [dblp]
- Filip Cano (TU Graz, AT) [dblp]
- Corinne Cath (TU Delft, NL & University of Cambridge, GB) [dblp]
- Hana Chockler (King's College London, GB) [dblp]
- Aloni Cohen (University of Chicago, US) [dblp]
- Misty Davies (NASA - Moffett Field, US) [dblp]
- Roel Dobbe (TU Delft, NL) [dblp]
- Joan Feigenbaum (Yale University - New Haven, US) [dblp]
- David Fuenmayor (Universität Bamberg, DE) [dblp]
- Ashish Gehani (SRI - Menlo Park, US) [dblp]
- Jake Goldenfein (The University of Melbourne, AU) [dblp]
- Thomas T. Hildebrandt (University of Copenhagen, DK) [dblp]
- Divij Joshi (University College London, GB)
- Samuel Judson (Yale University - New Haven, US) [dblp]
- Daan Kolkman (TU Eindhoven, NL) [dblp]
- Bettina Könighofer (TU Graz, AT) [dblp]
- Joshua A. Kroll (Naval Postgraduate School - Monterey, US) [dblp]
- Robert Künnemann (CISPA - Saarbrücken, DE) [dblp]
- Stefan Leue (Universität Konstanz, DE) [dblp]
- Wouter Lueks (CISPA - Saarbrücken, DE) [dblp]
- Rupak Majumdar (MPI-SWS - Kaiserslautern, DE) [dblp]
- Kira Matus (The Hong Kong Univ. of Science & Technology, HK)
- Denis Merigoux (INRIA - Paris, FR) [dblp]
- Ruzica Piskac (Yale University - New Haven, US) [dblp]
- Scott Shapiro (Yale University - New Haven, US) [dblp]
- Jatinder Singh (University of Cambridge, GB) [dblp]
- Matthias Spielkamp (AW AlgorithmWatch - Berlin, DE) [dblp]
- Michael Veale (University College London, GB) [dblp]
- Rüdiger Wilhelmi (Universität Konstanz, DE) [dblp]
- Beyazit Yalcinkaya (University of California - Berkeley, US) [dblp]
- Neil Yorke-Smith (TU Delft, NL) [dblp]
Classification
- Computers and Society
- Multiagent Systems
- Other Computer Science
Keywords
- accountability
- autonomous systems
- decision making