High-Reliability Teleoperation Assistance based on Deep Learning and Human-Computer Interaction
Deep learning and HCI research on driver assistance, simulation, and delay compensation for off-road teleoperation.
About this project
I conducted this research at the Agency for Defense Development from 2021 to 2024 under the project “High-Reliability Teleoperation Assistance based on Deep Learning and Human-Computer Interaction.” The four studies connect driver control assistance, behavior simulation, interface design, and delay compensation for teleoperated driving in unstructured environments.
Implementation and research infrastructure
I built a teleoperated driving dataset containing LiDAR point clouds, vehicle dynamics, and control inputs from simulated wheeled, tracked, and aerial vehicles.
The implementation included LiDAR preprocessing and PyTorch models for traversability cost maps (U-Net and BEVDetNet), obstacle detection, driver intention recognition, and real-time driver assistance using an LSTM denoising autoencoder. I integrated the models into a ROS2-based teleoperation system on Linux and managed a four-GPU NVIDIA A100 server with Docker.
The research produced four papers across TAROS, AutomotiveUI, IEEE Robotics and Automation Letters, and IET Electronics Letters. The studies below retain their individual project years.
GoonDAE: Denoising-Based Driver Assistance for Off-Road Teleoperation
2022 · Denoising-based assistance that improves unskilled drivers’ control inputs for stable off-road teleoperation.
Generative Model-based Simulation of Driver Behavior when Using Control Input Interface for Teleoperated Driving in Unstructured Canyon Terrains
2023 · Generative driver behavior models that simulate teleoperated driving in unstructured canyon terrains.
Driving with Black Box Assistance: Teleoperated Driving Interface Design Guidelines for Computational Driver Assistance Systems in Unstructured Environments
2023 · Interface design guidelines to reduce workload and support trust when driving with computational assistance.
Forward to the Future: A Deep Learning-based Approach to Mitigate Control Transmission Delay of Teleoperated Driving in Unstructured Environments
2024 · Deep learning-based prediction of future control inputs to compensate for transmission delay in teleoperated driving.