Moses Mariajoseph will defend his doctoral thesis entitled “Assessing the Effectiveness of Immersive Visual Technologies in an Industrial Machine Framework” on April 20, 2023.

Abstract

Under various contexts, such as heavy work machine operators, control room operators, emergency handlers, etc., the operator’s decision-making is expected to be crucial for reaching positive outcomes on productivity, the environment, and even human lives. To deal with such challenges, the operator must hold complete situational awareness for readiness. In this context, situational awareness refers to the perception and comprehension of information that allows an operator to project future courses of action needed to respond to a dynamic environment. Considering an on-road vehicle driver, when a hazardous situation approaches, the semi-autonomous vehicle (i.e., SAE Level-3) relies on the driver as a fallback solution, unaware of the driver’s situation awareness for readiness. During such a situation, autonomous driving can be misused when the driver becomes overly reliant on it. We suggest that semi-autonomous vehicles should take into account the driver’s focus on the vehicle and the road.

Meanwhile, in the off-road environment where heavy machine vehicles (i.e., earth-moving vehicles) are deployed (e.g., mining environments, quarry sites), the environment changes constantly, so situational awareness for the driver is crucial. In both on-road and off-road vehicles, controllability is paramount. According to automotive standards such as ISO 26262 (i.e., for on-road vehicles) and ISO 19041-1 (i.e., for off-road vehicles), controllability refers to the ability to avoid harm to the person or group at risk through the timely reactions of the operator, possibly with the support of alternative controls. A high level of controllability necessitates that the operator has a high level of attention to the surrounding scenario. To examine the operator’s focus on the vehicle and the road, we propose that the vehicle should initiate an exploration of the driver’s situational awareness for readiness, which could potentially adjust the ISO 26262 and ISO 19041-1 controllability. Based on our literature study, we found that situational awareness for readiness is largely depends on mental stress. Therefore, monitoring the driver’s acute stress patterns can help to emphasize identifying the driver’s situational awareness for readiness level. The driver’s acute stress pattern can be obtained by sensing the driver’s physiology and presenting it to the driver in an immersive, intuitively understandable way.

However, our research aims to develop a model for situational awareness for readiness based on human physiological parameters in an augmented automotive environment. As a result, we created this model using innovative real-time non-invasive acquisition techniques, with ultra-short-term analysis of heart activity and galvanic skin response signals, in conjunction with speech data. we conducted preliminary experiments using Stroop and arithmetic tasks. Later, we generalized our situational awareness model to cover off-road vehicles and performed experimental work to validate it in an off-road virtual reality offers a realistic experience. Even though virtual reality provides a realistic experience emphasizing achieving a higher level of situational awareness for readiness, since such technology is human-centric, it requires humans to act effectively, and make timely decisions. Therefore, we analyzed the prolonged effect of virtual reality on the driver’s physiology and the variation in vehicle handling behavior of the driver while using a virtual reality display compared to a two-dimensional display. However, our experimental results indicate that prolonged exposure to the virtual environment causes physiological palpitations, which can lead to uncertainty in handling the virtual environment. Although virtual reality technology is becoming increasingly popular, its success depends on measuring the user’s situational awareness for readiness while using it..