In May 2023, a Dagstuhl Seminar took place in which 22 researchers and academics from across the world gathered for a week in Schloss Dagstuhl, Saarland, Germany, to discuss the future of the changing relationship between humans and computational machines, appraised by the group as an emerging form of “human-machine symbiosis”. The full report documents said seminar and the efforts of its participant to investigate the underlying mechanisms of human-machine symbiosis, as well as knowledge guiding the design of such technologies, and the challenges the field of human-machine symbiosis faces moving forward.

In 1960 Licklider predicted that the future of computing would be one in which humans and machines would become tightly coupled and form a complimentary “symbiosis”, extending human capabilities [19]. Today, it appears we may be witnessing the advent of such a future. The traditional conceptual frameworks that attempted to describe the human-machine relationship over the last century of computing in terms of command and response, of master and slave, are becoming ever more antiquated. With the emergence of increasingly intelligent algorithms, complex computer architecture, and advanced sensor technologies, machines are now able to sense, understand, and act on the world as agents independent of human input and oversight. Describing this growing trend as “human-computer integration” in 2017 [12, 13], Farooq and Gruden deemed it to be a new paradigm within HCI, with “humancomputer integration” referring to a move in technology away from the “stimulus-response” paradigm we commonly think of when we talk about interaction, and toward a “symbiotic partnership” between humans and computers, in which both parties are integrated and must be considered holistically.

The notion of human-computer integration was further developed at a 2018 Dagstuhl Seminar [26], in which 29 leading experts from industry and academia came together over a five-day seminar to develop and discuss the future of human-computer integration (HInt). The discussions during this seminar ultimately produced a foundational work titled “Next Steps in Human-Computer Integration” [25] which was presented at CHI 2020. The work, articulating a synthesis of contributions made toward human-Computer integration, summarised the contemporary state of the emerging paradigm. The paper defined HInt as “a new paradigm with the key property that computers become closely integrated with the user”, which included examples in which “humans and digital technology work together, either towards a shared goal or towards complementary goals (symbiosis)”; and “integration in which devices extend the experienced human body or in which the human body extends devices (fusion)”. Through this updated rendition, it became clear that this new state of the human-machine relationship was not only marked by the newly emerging capacity for machines to exhibit agency and work in tightly coupled collaborative partnerships with humans [20, 5, 18, 21, 29]; but also by the tendency for emerging technologies to extend human capabilities by bidirectionally sensing and actuating human physiology and act as extensions of the human body [7, 15, 16, 22, 33, 34].

Moving forward, much research has since been made in contribution to furthering our understanding of human-computer integration theory and the design of integration systems, including a definitive book on human-computer integration titled: “Human–Computer Integration: Towards Integrating the Human Body with the Computational Machine” [28]. These contemporary conceptualizations and theoretical contributions to HInt have largely taken on a more “bodily” focus, centering on how machines can integrate with the human body, physiological processes, and experiences [31, 32]. Such theoretical contributions include: the bodily integration framework [24], which describes the user experience of integration systems that both sense and actuate the human body by considering the human’s sense of bodily agency and sense of bodily ownership; experiential integration [6, 7], which seeks to understand how machines can be integrated into an individual’s pre-reflective experience of “self” as opposed two “other”; and the brain-computer integration framework [30], which provides a design space for brain-computer interfaces that bidirectionally sense and actuate the human brain, as well at the user experiences they create. Taken together, these works provide the conceptual tools to understand and design technologies that are physically close to or conform to the body [16, 33, 34, 2], extend the body’s sensory and motor capabilities[15, 22], augment cognitive abilities [4, 14, 10], and provide novel modalities for human expression and play [23, 3, 9, 17, 11, 21], facilitating empowerment and self-actualization.

More recently, contributions to HInt theory are increasingly concerned with the interagential dynamics and relationships that arise from systems that are closely coupled with the human body but are able to act with their own agency [20, 11], often sharing agency with, or borrowing agency from their human counterpart. Such theoretical works include: the intertwined integration framework [27], which maps the possible user experiences that arise from a combination of the alignment of the system with the user, and the user’s awareness of the agency of the system; integrated embodiment [18], which explores how human-ai partnerships can be embodied within a single physical body; and the integrated exertion framework [1], which describes the different experiences of human-machine partnerships that may arise in an exertion context. Considering these newer contributions, it becomes increasingly evident that the original conceptualization of symbiosis and fusion as two opposing forms of integration, one social and one bodily, no longer completely describes the new and evolving relationships we are beginning to see emerge between humans and computers, highlighting a significant gap in our understanding of human-machine symbiosis theory and design.

Considering the recent acceleration in technological advances that enable autonomous agents and artificial intelligence to form new and previously unseen relationships with humans, both socially and physiologically, the importance of having a strongly developed and fully articulated definitive theory of human-machine symbiosis and their design becomes extremely pertinent. While the growing presence of human-machine symbiosis in our daily lives can greatly extend human capacity, capability, experience, and culture, recent research has also highlighted that human-computer symbiosis holds the strong potential to develop into parasitic relationships which can be detrimental to humanity [8]. Thus, considering the growing relevance of symbiosis, in contrast to the relatively nascent state of the field in terms of theoretical understanding and design knowledge, the present seminar sought to lead a concerted effort in developing the theory of human-machine symbiosis, articulating its underlying mechanism, and outlining the grand challenges facing the field moving forward. In doing so, the seminar asked: What symbiosis is, the mechanisms through which it operates, and what kinds of symbioses are possible? Then with this knowledge, the seminar aimed to build an understanding of how to guide the design of symbiosis. Finally, through acknowledging the limitations to our current understanding of symbiosis and our abilities in designing symbiotic systems, the seminar articulated a set of grand challenges that we can work toward to move the field forward. he full report documents the activities undertaken by the participants of the seminar in an effort to actualize these aims

References

1. Josh Andres, Nathan Semertzidis, Zhuying Li, Yan Wang, and Florian Mueller. Integrated exertion–understanding the design of human–computer integration in an exertion context. ACM Transactions on Computer-Human Interaction, 29(6):1–28, 2023.
2. Andrea Bianchi, Steve Hodges, David J Cuartielles, Hyunjoo Oh, Mannu Lambrichts, and Anne Roudaut. Beyond prototyping boards: future paradigms for electronics toolkits. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–6, 2023.
3. Oğuz ‘Oz’ Buruk, Louise Petersen Matjeka, and Florian Mueller. Towards designing playful bodily extensions: Learning from expert interviews. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–20, 2023.
4. Yi Fei Cheng, Hang Yin, Yukang Yan, Jan Gugenheimer, and David Lindlbauer. Towards understanding diminished reality. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1–16, 2022.
5. S Chien, Seungyeon Choo, Marc Aurel Schnabel, Walaiporn Nakapan, Mi Jeong Kim, and Stanislav Roudavski. Living systems and micro-utopias: towards continuous designing. In Proceedings of the 21st International Conference of the Association for Computer-Aided Architectural Design Research in Asia CAADRIA 2016, pages 713–722, 2016.
6. Valdemar Danry, Pat Pataranutaporn, Adam Haar Horowitz, Paul Strohmeier, Josh Andres, Rakesh Patibanda, Zhuying Li, Takuto Nakamura, Jun Nishida, Pedro Lopes, et al. Do cyborgs dream of electric limbs? experiential factors in human-computer integration design and evaluation. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–6, 2021.
7. Valdemar Danry, Pat Pataranutaporn, Florian Mueller, Pattie Maes, and Sang-won Leigh. On eliciting a sense of self when integrating with computers. In Augmented Humans 2022, pages 68–81. 2022.
8. Rod Dickinson, Nathan Semertzidis, and Florian Mueller. Machine in the middle: Exploring dark patterns of emotional human-computer integration through media art. In CHI Conference on Human Factors in Computing Systems Extended Abstracts, pages 1–7, 2022.
9. Ellen Yi-Luen Do. Design for assistive augmentation – mind, might and magic. Assistive augmentation, pages 99–116, 2018.
10. Barrett Ens, Maxime Cordeil, Chris North, Tim Dwyer, Lonni Besançon, Arnaud Prouzeau, Jiazhou Liu, Andrew Cunningham, Adam Drogemuller, Kadek Ananta Satriadi, et al. Immersive analytics 2.0: Spatial and embodied sensemaking. In CHI Conference on Human Factors in Computing Systems Extended Abstracts, pages 1–7, 2022.
11. Xiao Fang, Nathan Semertzidis, Michaela Scary, Xinyi Wang, Josh Andres, Fabio Zambetta, and Florian Mueller. Telepathic play: Towards playful experiences based on brain-to-brain interfacing. In Extended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play, pages 268–273, 2021.
12. Umer Farooq, Jonathan Grudin, Ben Shneiderman, Pattie Maes, and Xiangshi Ren. Human computer integration versus powerful tools. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pages 1277–1282, 2017.
13. Umer Farooq and Jonathan T Grudin. Paradigm shift from human computer interaction to integration. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pages 1360–1363, 2017.
14. Lukas Gehrke, Pedro Lopes, and Klaus Gramann. Toward human augmentation using neural fingerprints of affordances. In Affordances in Everyday Life: A Multidisciplinary Collection of Essays, pages 173–180. Springer, 2022.
15. Masahiko Inami, Daisuke Uriu, Zendai Kashino, Shigeo Yoshida, Hiroto Saito, Azumi Maekawa, and Michiteru Kitazaki. Cyborgs, human augmentation, cybernetics, and jizai body. In Augmented Humans 2022, pages 230–242. 2022.
16. Hsin-Liu Cindy Kao. Hybrid body craft: toward culturally and socially inclusive design for on-skin interfaces. IEEE Pervasive Computing, 20(3):41–50, 2021.
17. Sang-won Leigh, Abhinandan Jain, and Pattie Maes. Exploring human-machine synergy and interaction on a robotic instrument. In NIME, pages 437–442, 2019.
18. Zhuying Li, Tianze Huang, Rakesh Patibanda, and Florian Mueller. Ai in the shell: Towards an understanding of integrated embodiment. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–7, 2023.
19. Joseph CR Licklider. Man-computer symbiosis. IRE transactions on human factors in electronics, (1):4–11, 1960.
20. Dominika Lisy. In-corpo-real robot-dreams: Empathy, skin, and boundaries. In 2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), pages 01–05. IEEE, 2021.
21. Azumi Maekawa, Hiroto Saito, Daisuke Uriu, Shunichi Kasahara, and Masahiko Inami. Machine-mediated teaming: Mixture of human and machine in physical gaming experience. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22, New York, NY, USA, 2022. Association for Computing Machinery.
22. Azumi Maekawa, Hiroto Saito, Daisuke Uriu, Shunichi Kasahara, and Masahiko Inami. Machine-mediated teaming: Mixture of human and machine in physical gaming experience. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1–11, 2022.
23. Florian Mueller, Tuomas Kari, Zhuying Li, Yan Wang, Yash Dhanpal Mehta, Josh Andres, Jonathan Marquez, and Rakesh Patibanda. Towards designing bodily integrated play. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction, pages 207–218, 2020.
24. Florian Mueller, Pedro Lopes, Josh Andres, Richard Byrne, Nathan Semertzidis, Zhuying Li, Jarrod Knibbe, Stefan Greuter, et al. Towards understanding the design of bodily integration. International Journal of Human-Computer Studies, 152:102643, 2021
25. Florian Mueller, Pedro Lopes, Paul Strohmeier, Wendy Ju, Caitlyn Seim, Martin Weigel, Suranga Nanayakkara, Marianna Obrist, Zhuying Li, Joseph Delfa, et al. Next steps for human-computer integration. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–15, 2020.
26. Florian Mueller, Pattie Maes, and Jonathan Grudin. Human-computer integration (dagstuhl seminar 18322). In Dagstuhl Reports, volume 8. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2019.
27. Florian Mueller, Nathan Semertzidis, Josh Andres, Joe Marshall, Steve Benford, Xiang Li, Louise Matjeka, and Yash Mehta. Towards understanding the design of intertwined human-computer integrations. ACM Transactions on Asian and Low-Resource Language Information Processing, 2023.
28. Florian Mueller, Nathan Semertzidis, Josh Andres, Martin Weigel, Suranga Nanayakkara, Rakesh Patibanda, Zhuying Li, Paul Strohmeier, Jarrod Knibbe, Stefan Greuter, Marianna Obrist, et al. Human–computer integration: Towards integrating the human body with the computational machine. Foundations and Trends® in Human-Computer Interaction, 16(1):1–64, 2022.
29. Pat Pataranutaporn, Valdemar Danry, Joanne Leong, Parinya Punpongsanon, Dan Novy, Pattie Maes, and Misha Sra. Ai-generated characters for supporting personalized learning and well-being. Nature Machine Intelligence, 3(12):1013–1022, 2021.
30. Nathan Semertzidis, Fabio Zambetta, and Florian Mueller. Brain-computer integration: A framework for the design of brain-computer interfaces from an integrations perspective. ACM Transactions on Computer-Human Interaction, 2023.
31. Nathan Arthur Semertzidis, Zoe Xiao Fang, Pedro Lopes, Kai Kunze, Paul Pangaro, Florian Mueller, and Pattie Maes. What we talk about when we talk about human-computer integration. In CHI Conference on Human Factors in Computing Systems Extended Abstracts, pages 1–4, 2022.
32. Nathan Arthur Semertzidis, Michaela Scary, Xiao Fang, Xinyi Wang, Rakesh Patibanda, Josh Andres, Paul Strohmeier, Kai Kunze, Pedro Lopes, Fabio Zambetta, et al. Sighint: Special interest group for human-computer integration. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–3, 2021.
33. Jürgen Steimle, Marie Muehlhaus, Madalina Luciana Nicolae, Aditya Shekhar Nittala, Narjes Pourjafarian, Adwait Sharma, Marc Teyssier, Marion Koelle, Bruno Fruchard, and Paul Strohmeier. Design and fabrication of body-based interfaces (demo of saarland hci lab). In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–4, 2023.
34. Anusha Withana, Daniel Groeger, and Jürgen Steimle. Tacttoo: A thin and feel-through tattoo for on-skin tactile output. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, pages 365–378, 2018. 23212

Creative Commons BY 4.0

Florian 'Floyd' Mueller