Enhanced Q-learning algorithm with eligibility traces for safe path planning of unmanned surface vehicles in dynamic unknown environment

Jianxun Wu; Peng Liu; Jing Ma; Ning Li; Hao Wang; Xinghua Qian

doi:10.1117/12.3029165

6 May 2024 Enhanced Q-learning algorithm with eligibility traces for safe path planning of unmanned surface vehicles in dynamic unknown environment

Jianxun Wu, Peng Liu, Jing Ma, Ning Li, Hao Wang, Xinghua Qian

Author Affiliations +

Proceedings Volume 13107, Fourth International Conference on Sensors and Information Technology (ICSI 2024); 131072A (2024) https://doi.org/10.1117/12.3029165
Event: Fourth International Conference on Sensors and Information Technology (ICSI 2024), 2024, Xiamen, China

Abstract

This study explores the application of a path planning algorithm based on Q-learning and eligibility traces in autonomous task execution for Unmanned Surface Vehicles (USVs). The algorithm aims to provide secure path planning for USVs in dynamic unknown environments, taking into account obstacles, potential threats, and multiple constraints. Initially, a detailed Markov Decision Process (MDP) model was designed. Subsequently, the introduced Q-learning and eligibility trace algorithm demonstrated significant advantages in path planning, utilizing the Upper Confidence Bound (UCB) strategy for action selection. Finally, simulation experiment results indicate that, compared to traditional Q-learning methods, the algorithm can more effectively plan paths for USVs, avoid threat areas, and achieve faster convergence.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Jianxun Wu, Peng Liu, Jing Ma, Ning Li, Hao Wang, and Xinghua Qian "Enhanced Q-learning algorithm with eligibility traces for safe path planning of unmanned surface vehicles in dynamic unknown environment", Proc. SPIE 13107, Fourth International Conference on Sensors and Information Technology (ICSI 2024), 131072A (6 May 2024); https://doi.org/10.1117/12.3029165

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available