In the past decade, virtual reality (VR), augmented reality (AR), and extended reality (XR), in general, have seen a second wave due to consumer-level availability of VR devices. In the medical realm, XR has been investigated extensively and successfully for training, teaching, rehabilitation, and some forms of therapy (e.g., disorders).

However, XR also bears great potential for application in the operating room (OR) directly during operations, as an additional and powerful technology, which has been underexplored and under-researched so far. We believe that by making use of this technology during and for operations, patients’ well-being and many kinds of success metrics of operations can be improved.

The first goal of this Dagstuhl Seminar is to survey existing uses of any kind of XR technologies in the OR. More importantly, we will seek to foster and create ideas for novel research on using XR in the OR. This should lead to novel use cases of the technology, novel research questions, and an agenda for research on XR in the OR for the next 5 years. As one of the tangible outcomes, we plan to develop a position paper outlining this agenda during the seminar, and submit it to a major journal.

These goals will be achieved by bringing together researchers from all sub-fields of XR (VR, AR, sound and haptic rendering, new materials, and other kinds of mixed reality), along with practitioners (surgeons) from different medical areas, in order to bring the practical perspective of surgery to the seminar.

In the seminar, we will discuss the challenges in the OR (e.g., stressful work environment, digitalization, hand-eye coordination, occupancy of perception); we will conduct brainstorming (using creativity techniques such as brainwriting and brainwalking) to think about any potential use of XR technologies in the operating room; we will structure the ideas so as to prepare a potential position paper.

Creative Commons BY 4.0

Peter Haddawy, Anja Hennemuth, Ron Kikinis, and Gabriel Zachmann