For much of the time since the invention of the automobile, human-machine interaction (HMI) in vehicles was reasonably clear: drivers controlled the vehicle by manipulating the steering wheel, pedals, and a few levers, buttons, or similar mechanical input devices [2]. They received information about the state of the vehicle through dials, warning lights, and sounds. And, they interacted with a relatively simple in-vehicle entertainment device: the radio, or perhaps the cassette- or CD player.

It is true that the number of input and output devices increased dramatically over the years - for example in the late 1950s, the Ford Edsel was described as a “devilish assemblage of gadgets” [5]. The Edsel was soon out of production, but the number of gadgets kept climbing. It is also true that drivers sometimes operated the vehicle when they were tired, and fell asleep at the wheel. Other times they consumed too much alcohol, and were not able to safely control their vehicle. Yet, the basic concepts of human-machine interactions in the vehicle were well-defined for research and development purposes. The driver’s primary task was to drive: keep the vehicle on the road, avoid crashes, maneuver through traffic, and ultimately reach a destination. The driver also engaged in secondary tasks, such as manipulating the radio, as well as other non-driving-related tasks, such as talking to passengers, and eating. Creating good human-machine interfaces meant supporting the driver in these primary and secondary tasks, while assuring safety for everyone on the road.

Then, with the introduction of mobile computing devices, engagement in secondary tasks, such as talking to remote conversants, as well as sending text messages, and manipulating the interfaces of various mobile applications, became a significant issue in cars. In a sense these distractions were the same as those that drivers faced with the myriad buttons in the Ford Edsel. But, there were differences too: the Edsel did not allow the driver to communicate to remote conversants, nor did it have a touch-screen with ever-changing content.

Today, we again find ourselves at a crossroads. Our cars have myriad buttons, as well as different mobile technologies, both for drivers and for passengers. But, additionally, the primary task of driving is often shared between the driver and the vehicle [9]. Most studies in distracted driving tend to focus on how non-driving activities serve as a distraction from the primary task of vehicle control. In the context of highly automated vehicles (HAV), driving will be the distraction from non-driving activities [6]. Sometimes, the vehicle can effectively take over the driving task, and we can expect that this will become the norm rather than the exception in the foreseeable future: the driving task will be automated, at least on some road segments, and the driver will become a passenger. Yet, in this same foreseeable future we can also expect that the vehicle will sometimes hand the driving task back to the driver, who will have to transition back from the role of passenger to the role of the driver [14], [18]. This is the new landscape of in-vehicle human-machine interaction, and it presents a number of research questions that we addressed in this Dagstuhl seminar. In the rest of this report, we introduce pre-workshop tasks and summarize the activities and outcome of the seminar. Automated traffic is a challenge not limited to the interaction between a human driver and an automated vehicle. Automated vehicles will be part of a mixed traffic with other traffic participants of less or no automation. Also further traffic participants such as pedestrian and bicycles are part of this and requires a certain level of communication and recognition of the vehicles intention and actions among vehicles and the surrounding traffic participants.

Research questions tackled in Dagstuhl seminar 19132

1. Handover: One of the key questions in designing in-vehicle human-machine interaction for partially automated vehicles is, how can the vehicle safely hand back the task of controlling the vehicle to the driver. In the short term this is one of the most important questions for those designing vehicles with automation, because in the short term such vehicles will have to hand control back to the driver relatively often [14], [15]. We need to understand how the modality, conveyed information, and reliability of take-over requests (TORs), engagement in non-driving tasks, and motion perception can influence drivers performance in task switching in highly automated driving context [6].
2. Trust: Drivers must trust the automation features in order to take advantage of them [19]. We need to individually understand the trust in the individual actions of the vehicle starting out from assistance systems [21] to more automated functions [13], [20].
3. Creating a place for work and play: One important benefit of automation would be that drivers can become passengers, and thus use the time in the vehicle to either be productive (work), or relax (play). How can human-machine interaction for automated vehicles be designed, such that drivers can take advantage of their newfound freedom from driving [9], [12]? How can we do this, taking into account the physical and computational characteristics of the vehicle, as well as the potential for motion sickness?
4. Communication between all traffic participants: With the advent of automation, the transportation environment will include partially and fully automated vehicles. Yet, manually driven vehicles will remain for the foreseeable future, as will pedestrians, bicyclists, and other transportation users. For safe driving, all of these transportation users will have to communicate, but it is not yet clear how this can best be accomplished [16].
5. Advanced technologies for in-vehicle HMI: The technologies that are available for human-machine interaction are continuously improving. Two exciting technologies that will be worth examining in the context of automated vehicles are speech interaction (e. g.[8]), and augmented reality e. g. [11] and [10].
6. Legal aspects of in-vehicle interfaces: Automation, as well as the user interfaces built for partially and fully automated vehicles, will have to fit into the legal structures of the countries where the vehicles are used [7]. What are these structures, what do designers need to know about them, and how can they help develop the future legal structures?

References

1. Ingrid Pettersson, Florian Lachner, Anna-Katharina Frison, Andreas Riener, and Andreas Butz. A bermuda triangle?: A review of method application and triangulation in user experience evaluation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, pages 461:1–461:16, New York, NY, USA, 2018. ACM.
2. Akamatsu, M., Green, P., & Bengler, K. Automotive technology and human factors research: Past, present, and future. In International journal of vehicular technology, 2013.
3. Borojeni, S.S., Weber, L., Heuten, W., & Boll, S. From reading to driving: priming mobile users for take-over situations in highly automated driving. Paper presented at the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services, 2018.
4. Borojeni, S. S., Chuang, L., Heuten, W., & Boll, S. Assisting Drivers with Ambient Take-Over Requests in Highly Automated Driving. Paper presented at the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 2016.
5. Brooks, J. Business adventures: Twelve classic tales from the world of wall street: Open Road Media. 2014.
6. Hancock, P. A. Driven to distraction and back again. In In J. Lee (Ed.), Driver distraction and inattention: Advances in research and countermeasures, (pp. 9-25): CRC Press., 2013.
7. Inners, M., & Kun, A. L. Beyond Liability: Legal Issues of Human-Machine Interaction for Automated Vehicles. In 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, (pp. 245-253), Oldenburg, Germany, 2017. ACM.
8. Kun, A., Paek, T., & Medenica, Z. The effect of speech interface accuracy on driving performance. Paper presented at the Eighth Annual Conference of the International, 2007.
9. Kun, A. L., Boll, S., & Schmidt, A. Shifting Gears: User Interfaces in the Age of Autonomous Driving. In IEEE Pervasive Computing, 15(1), 32-38., 2016.
10. Kun, A. L., Tscheligi, M., Riener, A., & Meulen, H. v. d. ARV 2017: Workshop on Augmented Reality for Intelligent Vehicles. Paper presented at the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications Adjunct, Oldenburg, Germany, 2017. ACM.
11. Kun, A. L., van der Meulen, H., & Janssen, C.P. Calling while driving: An initial experiment with HoloLens. Paper presented at the Driving Assessment, 2017.
12. Kun, A. L., Wachtel, J., Miller, W. T., Son, P., & Lavallière, M. ARV 2017: Workshop on Augmented Reality for Intelligent Vehicles. Paper presented at the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 2015.
13. Löcken, A., Heuten, W., & Boll, S. AutoAmbiCar: Using Ambient Light to Inform Drivers About Intentions of Their Automated Cars. Paper presented at the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications Adjunct 2016.
14. Merat, N., Jamson, A. H., Lai, F. C., Daly, M., & Carsten, O. M. User interfaces for first responder vehicles: views from practitioners, industry, and academia. In Transportation research part F: traffic psychology and behaviour, 27, 274-282., 2014.
15. Mok, B., Johns, M., Miller, D., & Ju, W. Tunneled In: Drivers with Active Secondary Tasks Need More Time to Transition from Automation. Paper presented at the 2017 CHI Conference on Human Factors in Computing Systems, CHI , 2017.
16. Rasouli, A., Kotseruba, I., & Tsotsos, J. K. Agreeing to Cross: How Drivers and Pedestrians Communicate. In arXiv preprint, arXiv:1702.03555., 2017.
17. Riener, A., Boll, S., & Kun, A. L. Automotive User Interfaces in the Age of Automation (Dagstuhl Seminar 16262). Paper presented at the Dagstuhl Reports., 2016. 1
18. van der Meulen, H., Kun, A. L., & Janssen, C.P. Switching Back to Manual Driving: How Does it Compare to Simply Driving Away After Parking? Paper presented at the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Ann Arbor, MI., 2016.
19. Wintersberger, P., Riener, A. Trust in Technology as a Safety Aspect in Highly Automated Driving. In i-com 15(3), 2016.
20. Wintersberger, P., Frison, A.-K., Riener, A., & Boyle, L. N. Towards a Personalized Trust Model for Highly Automated Driving. Paper presented at the Mensch und Computer 2016–Workshopband, 2016.
21. Yan, F., Eilers, M., Lüdtke, A., & Baumann, M. Developing a model of driver’s uncertainty in lane change situations for trustworthy lane change decision aid systems. Paper presented at the Intelligent Vehicles Symposium (IV), IEEE, 2016.

Creative Commons BY 3.0 Unported license

Susanne Boll, Andrew Kun, Andreas Riener, and C. Y. David Yang