Visualizing Driving Maneuvers Through Peripheral Displays: A Comparative Study of HMI Modalities in Autonomous Vehicles

1. Introduction

Autonomous vehicles are expected to bring several advantages, including improved safety and efficiency. However, the adoption of AVs necessitates their users’ acceptance, which has been shown to depend on a variety of factors [1,2,3,4,5,6,7,8,9]. Among them, trust in autonomy is a central prerequisite.

Higher automation levels are accompanied by novel seating layouts, with the "living room seating arrangement" being one of the most prevalent concepts [10,11,12]. This concept encompasses two rearward seats facing two forward seats, enabling eye contact between the passengers and thus fostering social interaction during autonomous rides. However, there is a reluctance towards facing against the direction of travel, with motion sickness being one of the main reasons [13,14]. Motion sickness rises from a sensory mismatch between acceleration registered by the vestibular organs and acceleration that is expected based on a conglomerate of information, e.g. previous experiences, and visual and somatosensory input [15]. This mismatch is more likely to occur in AV passengers when they cannot see the road ahead. Additionally, rearward facing passengers also exhibit less trust in the autonomy, further decreasing their willingness of using this technology and thus benefiting from its advantages [16,17].

Several studies have investigated the use of HMIs to foster trust calibration and motion sickness mitigation in AVs [18,19,20,21]. To this end, visual representation of current and upcoming maneuvers is a common concept to increase the vehicle maneuver transparency. However, most of these HMIs are presented on displays that rely on the passengers’ visual attention to convey the information displayed. Thus, passengers need to disrupt their NDRTs to benefit from these HMIs, which lowers the overall usefulness of the automation [22]. An emerging alternative way of visual information transmission is via interior lighting, e.g. through LED strips [23,24]. As this method is physically limited in the level of detail that information can be displayed in, existing concepts focus on visualizing either lateral [24] or longitudinal acceleration [23,25] and yield mixed results regarding their effectiveness. However, both types of acceleration can occur simultaneously in cars. This raises the question how a simultaneous visualization of longitudinal and lateral acceleration can be achieved. Importantly, increasing the visual load through more complex cues bares the risk of distraction from the NDRT, potentially diminishing the automation’s use, especially in visual NDRTs [26].

Hainich et al. [24] evaluated an indicator animation using LED strips. While this enables anticipation of turning maneuvers, more nuanced maneuver details like acceleration in x and y direction, both relevant for the development of motion sickness, are not communicated. However, simultaneous coding of lateral and longitudinal accelerations is conceivable using LED strips. Adjusting the blinking frequency and / or LED color according to the steering angle might yield additional motion sickness mitigation compared to the results found by Hainich et al. [24]. Nevertheless, this poses a more abstract visualization of lateral vehicle dynamics than extending the optic flow of the vehicles surroundings [23,25], and might thus lead to increased mental load. Using large surfaces in the periphery of the passengers like De Winkel et al. [25] enables this extension of optic flow for both longitudinal and lateral acceleration. This can potentially increase visually perceived self-motion and thus decrease the sensory conflict between visual and vestibular motion cues during car journeys.

The present study compares HMIs in the form of large LED panels, LED strips and conventional displays with regard to their capability to visualize vehicle maneuvers, and the degree to which they distract from a visual NDRT. As rearward facing passengers are at greater risk of motion sickness and undertrust in AVs, and the optic flow reverses in a rearward facing seating orientation, each HMI is evaluated in both seating orientations [13,27].

2. Materials and Methods

2.1. Design

2.2. Design

The LED panels consist of LED matrix arrays and are referred to as "2D". The LED strips consist of single LED vectors and are thus referred to as "1D". The conventional displays show video captured from a vehicles windshield, therefore this HMI condition is referred to as "video". Distraction from a visual NDRT was measured through changes in performance on math tasks, while capability in maneuver visualization was measured with and without distraction. Therefore, participants either conducted math tasks or focused on a fixation cross. This resulted in a 3x2x2 mixed design with HMI condition (video vs. "1D" vs. "2D") and task (fixation cross vs. math) manipulated as within-subjects variables and seating orientation (forward facing vs. rearward facing) as a between-subjects variable.

2.3. Dependent Variables

2.3.1. Motion Perception

Visually perceived vehicle motion was measured via joysticks. The maximum vehicle velocity of 100 km/h corresponded to a joystick placement at the foremost position for forward facing subjects and and the rearmost position for rearward facing subjects. A velocity of 0km/h corresponded to the center joystick position for all subjects, with a linear progression in between. Negative velocities (driving backwards) were not part of the simulated journeys. The maximum steering angle of 540° to the right corresponded to a joystick placement at the rightmost position for forward facing subjects, again inverted for rearward facing subjects. A steering angle of 0° corresponded to the centre joystick position, analogously for steering angles to the left, with a linear progression in between. This was also inverted for rearward facing passengers. The subjects’ joystick input was shown on a coordinate grit at the bottom right of the displays or tablets, depending on the HMI condition, to ensure a constant visual reference frame for the current joystick position [28]. Accuracy of motion perception was operationalised as the subjects’ mean absolute deviation from ideal joystick input during each trial, with lower deviation indicating higher accuracy. The quadrants of the coordinate system used to visualize the joystick input had an edge length of 1, resulting in an average deviation on a scale between 0 and $\sqrt{8}$.

2.3.2. Distraction

Distraction was operationalised as the number of math tasks solved correctly during each trial, with lower scores indicating stronger distraction. Additionally, mental demand as an indicator for subjective distraction was measured on a scale from zero (very low) to 20 (very high) via the NASA-TLX after each trial [29].

2.4. Control Variables

As participants were subjected to visually simulated motion without corresponding vestibular stimulation, they were at risk of experiencing simulator sickness, potentially decreasing their performance during math tasks and joystick operation [30,31]. Simulator sickness was assessed on a scale from zero to three via the Simulator Sickness Questionnaire (SSQ, Kennedy et al. [32]).

In order to assess the representativeness of the sample in terms of susceptibility to motion sickness and technology affinity, these constructs were recorded using the Motion Sickness Susceptibility Questionnaire short-form (MSSQ-Short; Golding [33]) and the Affinity for Technology Interaction Scale (ATI; Franke et al. [34]).

To quantify whether the 1D and 2D HMIs were perceived in the peripheral field of view, gaze behavior was measured using eye tracking glasses (Tobii Pro Glasses 3, Tobii Technology GmbH). The proportion of time in which the HMIs were perceived peripherally was operationalised via the proportion of total fixation time on the tablet that displayed the tasks (math tasks or fixation cross).

2.5. Apparatus

The experiment was conducted in a dark laboratory with the interior lighting switched on to ensure a constant ambient luminance of x nits. The setup consisted of a cage made of aluminum profiles modeled on the interior dimensions of a VW Multivan T6. This ensured representative distances between the subjects and the HMIs. The subjects sat on one of four office chairs with the same seat height as the T6 seats. The two front seats were rotated 180° to create a "living room" seating arrangement. Subjects facing forward sat in seat A, while subjects facing backward sat in seat C (Figure 2). Tablets with Android OS (Samsung Galaxy Tab A8, 10.5") were mounted in front of seats A and C using a tablet holder. The tablets’ centres were fixed at 65 cm in front of the seats’ backrests and 84 cm above the cage’s floor. The fixation cross and the math tasks were presented on the tablets. Questionnaires were build using LimeSurvey (Limesurvey GmbH) and answered on a handheld tablet. Conventional 13" LC-displays were placed between the seats in both rows (1920x1080 pixels, Faytech n.d.). The displays’ lower edges were located at 85 cm above the cage’s floor and 7 cm behind the seats’ backrests. The video feed for the video condition was shown on these displays.

Speed and steering angle data were recorded during a previous study from a test vehicle equipped with a steering- and pedal robot. This data served as the basis for the 1D and 2D HMIs, as well as joystick input reference values. Each drive lasted 330 seconds. The course was driven clockwise and counterclockwise. Three of the simulated drives were clockwise and three were counterclockwise, with the order counterbalanced between the test subjects. This was done to reduce learning effects. In the video condition, the displays showed video recordings originating from the aforementioned study, recorded by a webcam trough the windshield of the experimental vehicle. Current maneuvers were additionally visualized through arrows as an overlay on top of the camera recordings (Figure 1).

The cage was lined with RGB LED panels (Model ‘Uslim 2’, Unilumin Germany GmbH, 3.9 mm pixel pitch, 1000 nits peak brightness). Four 500 mm x 1000 mm panels were mounted to the cage’s ceiling, four 250 mm x 500 mm panels on each side and one 500 mm x 1000 mm and two 250 mm x 500 mm panels on the floor. During the 2D condition, all panels were turned on, while only the side panels were turned on during the 1D condition (Figure 2).

Figure 2. (Side view into the cage. The seats’ centres were located at 39 cm above the floor and 34 cm to the side of the nearest side panels. The canopy panels were fixed at 133 cm above the cage’s floor. Both images show maneuver visualizations while driving straight.a) 1D HMI. (b) 2D HMI.

Figure 2. (Side view into the cage. The seats’ centres were located at 39 cm above the floor and 34 cm to the side of the nearest side panels. The canopy panels were fixed at 133 cm above the cage’s floor. Both images show maneuver visualizations while driving straight.a) 1D HMI. (b) 2D HMI.

In the 1D condition, vehicle speed and steering angle were visualized using the top row of LEDs on the side panels, just beneath the side windows. The cues consisted of stripes with a length of 70 px and a height of 1 px. As this "one-dimensional" HMI does not allow for the manipulation of light cue flow direction, the pulse frequency of the cues was used to visualize lateral acceleration. A steering angle of 0° corresponded to static light strips (0 Hz) and the maximum steering angle of 300° corresponded to a pulsing frequency of 3 Hz, with a linear interpolation in between. Only the light cues on the inside of each curve pulsed.

In the 2D condition, vehicle velocity and steering angle were coded using light dots (Figure 2). The dots were displayed on three visual layers, with the dots on the front layer being twice as large (20 px diameter) and fast as the dots on the middle layer (10 px diameter). These were, in turn, twice as large and fast as the dots on the rearmost layer (5 px diameter) This creates visual depth and, consequently, motion parallax, which is theorized to reinforce vection [11].

The velocity of the 1D and 2D light cues was determined based by the vehicle velocity. In a previous expert workshop, an average scaling of 17% for the dots in the foremost layer was favored, with the majority of participants preferring a logarithmic scaling [35]. Based on this, the logarithmic scaling used by Bohrmann et al. [23] was adapted such that the average scaling across the velocity range from zero to 100 km/h was 17%:

$$V_{\mathrm{long}}\left(V_{\mathrm{veh}}\right)=\left(\left(0.07-(0.07-1)·e^{-0.013·V_{\mathrm{veh}}}\right)·{\displaystyle \frac{1}{6}}\right)·{\displaystyle \frac{V_{\mathrm{veh}}}{5.39}}·2.23$$

where:

• $V_{\mathrm{long}}$ is the longitudinal cue velocity in m/s,
• $V_{\mathrm{veh}}$ is the vehicle velocity in km/h.

The resulting velocity scaling is visualized in Figure 3.

Lateral acceleration was visualized through the angle at which the dots flowed to the left or right (ceiling and ground) or up or down (sides). It was calculated as the vertical cue velocity in m/s in y-axis in relation to steering wheel angle and vehicle velocity using the following formula:

$$V_{\mathrm{vert}}({\theta }_s,V_{\mathrm{veh}})=sin\left({\displaystyle \frac{\pi {\theta }_s}{30}}\right)·\left({\displaystyle \frac{V_{\mathrm{veh}}}{3.6}}\right)·0.00195$$

where:

• $V_{\mathrm{vert}}$ is the vertical cue velocity in m/s,
• ${\theta }_s$ is the steering wheel angle in degrees,
• $V_{\mathrm{veh}}$ is the vehicle velocity in km/h,
• $0.00195$ is the product of the pixel pitch (3.9 mm) and a damping factor (0.5).

The resulting angle scaling is visualized in Figure 4.

Preliminary tests showed that these visualizations of lateral vehicle dynamics did not suffice to guarantee peripheral perceptibility for the 1D and 2D condition. Thus, change of the light cues’ color was used additionally. A steering angle of 0° corresponded to cyan (R0G255B255), which transitioned across green, yellow, orange and finally red (R255G0B0) at 300° (Figure 5). Initially, the color spectrum was implemented from cyan to orange because red is often associated with warnings and calls to action [36]. However, the preliminary tests showed that expanding the spectrum to include red greatly increased the peripheral perceptibility of color changes. The color change was applied only to the light cues on the inside of each curve.

Logitech G Extreme 3D Pro joysticks were used (Logitech International S.A.). The joysticks were fixed to 3D-printed mounts to ensure stability. The joysticks’ bases were fixed at 71 cm above the cage’s floor, 44 cm in front of the seats’ backrests and 44 cm right to the backrests’ centres. Joystick input was captured using a self-developed program build with Python and based on DARMA, a software used in previous studies to capture two-dimensional input from test subjects [28].

2.6. NDRT

During the fixation cross trials, subjects were instructed to look at a fixation cross displayed at the centre of the fixed tablets (1D and 2D HMI), or at the video (Figure 1). During the math trials, subjects were instructed to verbally and correctly answer the math tasks presented to them on the fixed tablets. Once an answer was given, the instructor triggered the next math task. The math tasks included all possible additions, subtractions and multiplications between two of the integers from 0 to 10. Four sets of tasks were generated using Python and their order was counterbalanced across the baseline, video, 1D and 2D trials to ensure equal mean difficulties in all conditions.

2.7. Procedure

Prior to the laboratory study, subjects completed an online questionnaire on demographic and control variables. On site, subjects received information about the content and procedure of the study and their participants’ rights. Subjects then gave their informed consent to participate. Subsequently, they answered the SSQ to record baseline simulator sickness. Afterwards, subjects were introduced to the eye tracking glasses and the glasses were equipped. Baseline performance on the math tasks was then recorded for 330 seconds without HMI.

Next, the functionality of the HMis was explained. This was followed by a 90-second test run with both the video HMI and the 1D HMI turned on. The subjects were able to practice operating the joystick in accordance with the maneuver visualizations. During this time, in addition to the visualization of the subjects’ joystick input by a red dot, the ideal joystick input was visualized by a yellow dot on the coordinate cross (Figure 1). This allowed the subjects to establish an internal frame of reference between the visualization of the vehicle maneuvers and the corresponding joystick movements. The 90-second track section was chosen to include both the maximum speed of 100 km/h and the maximum steering angle of 540°. This allowed the subjects to experience the full range of possible visualizations and joystick inputs. Any comprehension questions from the test subjects were answered. The test run was then repeated with the combination of video and 2D HMI.

This was followed by three simulated journeys in which the test subjects were instructed to look at the center of the fixation cross on the fixed tablets (1D and 2D HMI condition) or the video. The order of the HMI conditions within these three runs was counterbalanced across subjects. After each run, the SSQ and the NASA-TLX were queried. These three runs were followed by three runs in which the subjects were instructed to solve the math problems on the fixed tablets. Within these three trials, the order of the HMI condition was also counterbalanced across subjects. Again, the SSQ and NASA-TLX were queried after each trial. Thus, each subject experienced a total of two experimental trials per HMI condition (once with fixation cross, once with math problems). To reduce learning effects, one of these two trials was clockwise and one was counterclockwise, and this order was also counterbalanced between subjects.

A final questionnaire was completed after the last round. Here the subjects indicated which of the panel positions on the 2D HMI contributed to the perceptibility of the driving maneuvers. They were also asked to rank the three HMI conditions in order of preference. Finally, the subjects were asked for suggestions to improve the visualization of the maneuvers, received their compensation of 15 Euros, and were dismissed. Participants could cancel their participation at any time and without justification or adverse consequences.

2.8. Sample

The sample consisted of employees from non-development departments at HELLA GmbH & Co. KGaA, as well as University staff students. 34 subjects participated in the study (15 female, 19 male). Mean age was 36.71 years (SD = 12.86). Motion sickness susceptibility as assessed by the MSSQ-Short had an average score of 10.82 (SD = 11.92, min = 0.00, max = 42.75). This corresponds to the 48.19th percentile of the population tested by Golding [33] (min = 0th percentile, max = 100th percentile). Mean affinity for technology interaction as measured by the Affinity for Technology Interaction Scale (ATI, Franke et al. [34]) was 4.27 (SD = 0.57).

3. Results

Statistical analyses were carried out using R (R Core Team, 2024). The influences of the independent variables (seating orientation: forward vs. rearward, HMI: video vs. 1D vs. 2D and task: fixation cross vs. math) on the dependent variables (motion perception and distraction), as well as on simulator sickness were assessed using mixed ANOVAs. The assumption of sphericity was checked using Mauchly’s test. When this assumption was violated, sphericity corrected p-values are reported. The assumption of normally distributed residuals was checked using Shapiro-Wilk tests, the assumption of homogeneity of between-group variances for seating orientation was checked using Levene tests. If one of these two assumptions was violated, non-parametric aligned ranks transformation (ART) ANOVAs were carried out, instead [37]. Post-hoc t-tests are reported with Bonferroni-Holm corrected p-values. As seating orientation had no significant effect on either dependent variable, graphs include only the effects of HMI and task conditions.

3.1. Motion Perception

The influences of the three independent variables (seating orientation: forward vs. rearward, HMI: display vs. 1D vs. 2D and task: fixation cross vs. math) on accuracy of motion perception were analyzed using a 2*3*2 ART ANOVA. HMI had a significant effect (F(2,160) = 10.91, p < .001, $\eta p^2=0.12$), with medium effect size [38]. There were no further main effects or interactions. Post-hoc paired t-tests between each HMI condition revealed that joystick deviations were significantly higher in the video condition (M = 0.355, SD = 0.101) than in the 1D condition (M = 0.299, SD = 0.073, t(160) = 4.59, p < 0.001, d = 0.636) and than in the 2D condition (M = 0.315, SD = 0.071, t(160) = 3.02, p = 0.006, d = 0.459). There was no significant difference in joystick deviation between 1D and 2D. Figure 6 shows the mean joystick deviations for each condition.

3.2. Distraction

A 2x4 (seating orientation: forward vs. rearward and HMI: baseline vs. video vs. 1D vs. 2D) mixed ANOVA was carried out to analyze the influence of seating orientation and HMI on mean math scores. HMI showed a significant, large effect (F(3,96) = 44.49, p < .001, $\eta g^2=0.16$). There was no significant main effect of seating orientation and no significant interaction. Post-hoc paired t-tests between each possible pair from all four HMI conditions revealed that math scores in the baseline condition (M = 160, SD = 34.20) were significantly higher than in the 2D condition (M = 144, SD = 32.9, t(32) = 3.780, p = 0.001, d = 0.652), the 1D condition (M = 141, SD = 30.80, t(32) = 4.713, p < 0.001, d = 0.819) and the video condition (M = 120, SD = 33.70, t(32) = 9.764, p < 0.001, d = 1.69). Math scores in the 2D condition were significantly higher than in the video condition (t(32) = 8.406, p < 0.001, d = 1.46). Math scores in the 1D condition were also significantly higher than in the video condition (t(32) = 7.779, p < 0.001, d = 1.32). There was no significant difference between math scores in the 1D and 2D conditions. Mean math scores for each condition are shown in Figure 7.

A 2*3*2 ART ANOVA revealed a significant, large effect of task on mental demand as assessed by the NASA-TLX mental demand subscale (F(1,160) = 115.93, p < .001, $\eta g^2=0.42$). This main effect was qualified by a significant interaction with HMI (F(2,160) = 7.19, p = .001, $\eta g^2=0.08$). Post-hoc paired t-tests revealed that the tasks differed significantly only in the video HMI condition, with higher mental demand in the math task condition (M = 13.9, SD = 5.36) than in the fixation cross condition (M = 7.25, SD = 5.09, t(160) = 3.667, p = 0.005, d = 0.889). Mean mental demand scores are shown in Figure 8.

3.3. Simulator Sickness

Influence of seating orientation, HMI and task on simulator sickness was analyzed using a 2*3*2 ART ANOVA. This revealed a significant but small effect for HMI (F(2,160) = 3.34, p = .038, $\eta p^2=0.04$) and a significant effect of medium size for task (F(1,160) = 23.06, p < .001, $\eta p^2=0.13$). These effects were qualified by a significant interaction between HMI and task with small effect size (F(2,160) = 3.59, p = .029, $\eta p^2=0.04$). Post-hoc paired t-tests revealed no significant differences between any pairs of conditions within this interaction. Figure 9 gives an overview of mean SSQ scores for each condition.

3.4. Peripheral Perception

Two participants had to be excluded from the analysis of gaze behavior due to corrupted eye tracking data. A 2x2x2 ART ANOVA was carried out to investigate the effects of seating orientation, task and HMI condition on the proportion of time that the HMIs were perceived peripherally. As participants were allowed to look at the conventional display during the video HMI trials, this condition was omitted from the analysis. A significant effect of medium size for HMI was revealed, with a higher proportion of fixation time on the task during the 2D trials (M = 86.02%, SD = 19.28%) than the 1D trials (M = 80.47%, SD = 23.87%, F(1,87) = 5.72, p = .019, $\eta p^2=0.06$). Mean proportions are shown in Figure 10.

Ten participants had a proportion of total fixation time on task lower than 70% during at least one of the 1D and 2D trials. As the exclusion of these participants leads to a sample size to low for the respective ANOVAs, the data for this subsample, as well as the full sample is shown descriptively in Appendix A. Correlations between all dependent and control variables are listed in Appendix B.

4. Discussion

One- and two-dimensional light cues for AV maneuver visualization were compared to conventional displays showing a video image with regard to perceptibility and distraction. The peripheral HMIs showed better perceptibility and caused less distraction than the video. There were no differences between the 1D and 2D variant in terms of perceptibility or distraction.

Notably, the peripherally presented light cues enabled more precise maneuver perception than the video, even when the video could be observed foveally without a distracting NDRT. This suggests that such light cues may be capable of mitigating a sensory conflict between visually and vestibularly perceived movements, thereby preventing motion sickness. This is supported by the finding that motion perception is not effected as negatively by eccentricity as acuity [39]. Moreover, there is evidence suggesting that the peripheral field of view is more relevant for the perception of optical flow [40] and is specifically critical for motor response to this optical flow, namely postural control [41]. This is particularly relevant for motion sickness mitigation as a lack of postural control is thought to be a mayor cause for motion sickness [42].

Accuracy of maneuver perception was not negatively effected by the math tasks. For the peripheral cues, this is in line with findings that even subconsciously perceived visual cues can cause correct motor responses [43]. There is mixed evidence on whether time-critical tasks involving foveally perceived stimuli, such as the math task in this study, lead to tunnel vision [44]. If so, such an effect could have been masked by learning effects in this study, as the three trials with math tasks followed the three trials with a fixation cross. Evidence points towards an adaptation to distracting stimuli over prolonged exposure [45]. This could also explain why maneuver perception did not significantly differ between NDRT conditions in the video condition. Another possible explanation is that participants focused more on controlling the joysticks rather than on the math tasks during the video condition. This would also account for the significantly worse math scores compared to the other HMI conditions. However, the peripheral light cues also led to lower math scores compared to the baseline, suggesting that they are not entirely distraction-free either.

The objectively higher distraction caused by the video is also confirmed by the subjective ratings. While mental demand was descriptively higher during the math tasks across all HMIs, this effect was only significant for the video HMI. This suggests that perceiving the maneuvers via video required more cognitive resources, making multitasking more difficult. This could explain why the subjects tended to focus on one task at a time during the video and math condition.

The 2D light cues had a higher proportion of fixation duration on the tablets than the 1D light cues. This is likely due to the larger part of the peripheral field of view that is covered by the 2D cues. However, they did not show significantly better perceptibility than the 1D light cues, as measured by the accuracy of joystick input. Despite the familiarization drives and explanations of the HMIs, test subjects sitting backwards, in particular, expressed confusion regarding the cues displayed peripherally. They mentioned a lack of contextual information, such as vestibular feedback and a view of the vehicle’s surroundings from the windows, as the reason for the interpretation problems. Since the 2D cues are a visual extension of the vehicle environment, the absence of a real vehicle environment could inhibit their comprehensibility [23,46].

Neither accuracy of maneuver perception nor NDRT distraction were significantly influenced by seating orientation. As rearward facing passengers are at particular risk of motion sickness, this suggests that the tested HMIs should be effective in mitigation motion sickness not only on forward facing passengers, but also in rearward facing passengers, who are at particular risk of motion sickness.

4.1. Limitations

The study was conducted in a static laboratory setting to ensure maximum control of environmental influences. However, despite the detailed introduction to the HMIs and the test trials, some subjects had problems understanding the 2D light cues due to a lack of contextual information that would be present during real car journeys. These were particularly pronounced in subjects sitting backwards. As a result, fluctuations in the joystick deviations cannot solely be attributed to the differences in perception of the HMIs, but also reflect different levels of accuracy in their interpretation. However, the correct interpretation of the cues presupposes their perceptibility. The fact that the joystick deviations were higher for the peripheral light cues than for the video, and no subject reported difficulties perceiving the video, therefore indicates that the peripheral cues were certainly perceptible. Nevertheless, to enable a comparison between the HMIs and seating orientations in a more realistic environment, future studies should include contextual information, e.g. by utilizing a dynamic simulator with a 360° visual environment or by using a real vehicle on a test track.

Measuring maneuver perception using joystick input is a new method that, to our knowledge, has not been used in this form before. However, since joysticks are typically used to execute maneuvers (e.g., in flight simulators or for remote operating cars), we consider this method to be suitable. However, translating the visually perceived maneuvers into motor control of the joystick requires conscious processing. It is still unclear to what extend visually perceived movements need to be consciously processed to resolve the sensory conflict responsible for motion sickness. Thus, joystick input accuracy may underestimate motion-sickness-relevant maneuver perception.

Visual information about self-motion without matching vestibular information could have led to vection and thus to motion sickness [47]. This is supported by the fact that the peripheral light cues led to descriptively higher SSQ scores than the video. Simulator sickness and mental demand showed a medium correlation. Thus, higher NASA-TLX scores might have been caused by higher simulator sickness or vice versa.

4.2. Future Research

While we found substantial advantages of visualizing vehicle maneuvers in the peripheral instead of the foveal field of view, this method also entails disadvantages. Displaying future maneuvers instead of current ones may yield additional benefits in motion sickness mitigation, as anticipation plays a mayor role in the development of motion sickness [22]. If the approach of visually expanding the vehicle environment in the interior is pursued, this leads to incongruent visual information and can exacerbate motion sickness. Future research should investigate ways of avoiding this incongruence and address how the two methods can be meaningfully combined.

5. Conclusions

1D and 2D peripheral light cues enabled more accurate vehicle manuever perception and less distraction from an NDRT than conventional displays. This shows that peripheral light cues have great potential in increasing maneuver transparency, thereby increasing trust and reducing the sensory conflict responsible for motion sickness. Research on the effectiveness of such cues has thus far yielded mixed results [23,24,25]. In these studies, only either longitudinal or lateral acceleration was visualized, never both simultaneously. Additionally, none of these studies measured gaze behavior of the subjects, so it cannot be ruled out that the light signals were primarily perceived foveally. To this end, this is the first study that shows the potential of light cues for motion sickness mitigation through peripheral perception of lateral and longitudinal maneuvers.