Special Issue on Heterogeneous Domain Adaptation Learning and its Applications in Intelligent System
Guest Editors: Fa Zhu, Xiaochun Cheng and Sotiris Kotsiantis

Abstract

The path planning of mobile robots helps robots to perceive environment using the information obtained from sensors and plan a route to reach the target. With the increasing difficulty of task, the environment the mobile robots face becomes more and more complex. Traditional path planning methods can no longer meet the requirements of mobile robot navigation in complex environment. Deep reinforcement learning (DRL) is introduced into robot navigation However, it may be time-consuming to train DRL model when the environment is very complex and the existing environment may differ from the unknown environment. In order to handle the robot navigation in heterogeneous environment, this paper utilizes deep transfer reinforcement learning (DTRL) for mobile robot path planning. Compared with DRL, DTRL does not require the distribution of the existing environment is the same as that of the unknown environment. Additionally, DTRL can transfer the knowledge of existing model to new scenario to reduce the training time. The simulations show that DTRL can reach higher success rate than DRL for heterogeneous environment robot navigation. By using local policy, it costs less time to train DTRL than DRL for a complex environment and DTRL can consume less navigation time.

Keywords:

Remember to check out the Most Cited Articles!
Check out Notable Titles in Artificial Intelligence.

References

1. A. Dzedzickis, J. Subačiūtė-Žemaitienė, E. Šutinys, U. Samukaitė-Bubnienė and V. Bučinskas, Advanced applications of industrial robotics: New trends and possibilities, Appl. Sci. 12(1) (2021) 135. Crossref, Google Scholar
2. A. M. Okamura, M. J. Matarić and H. I. Christensen, Medical and health-care robotics, IEEE Robot. Autom. Mag. 17(3) (2010) 26–37. Crossref, Web of Science, Google Scholar
3. T. Liu, P. Zheng and J. Bao, Deep learning-based welding image recognition: A comprehensive review, J. Manuf. Syst. 68 (2023) 601–625. Crossref, Web of Science, Google Scholar
4. T. Liu, P. Zheng, J. Bao and H. Chen, A state-of-the-art survey of welding radiographic image analysis: Challenges, technologies and applications, Measurement 2014 (2023) 112821. Crossref, Google Scholar
5. Z. Tang, W. Wu, J. Gao and P. Yang, Saw delay line based IoT smart sensing in water distribution system, in 2018 IEEE 20th Int. Conf. High Performance Computing and Communications; IEEE 16th Int. Conf. Smart City; IEEE 4th Int. Conf. Data Science and Systems (HPCC/SmartCity/DSS) (IEEE, 2018), pp. 1474–1478. Crossref, Google Scholar
6. B. Balusamy, R. K. Dhanaraj, T. Seetharaman, V. Sharma, A. Shankar and W. Viriyasitavat, Design control and management of intelligent and autonomous nanorobots with artificial intelligence for prevention and monitoring of blood related diseases, Eng. Appl. Artif. Intell. 131 (2024) 107798. Crossref, Web of Science, Google Scholar
7. B. K. Patle et al., A review: On path planning strategies for navigation of mobile robot, Def. Technol. 15(4) (2019) 582–606. Crossref, Web of Science, Google Scholar
8. P. Raja and S. Pugazhenthi, Optimal path planning of mobile robots: A review, Int. J. Phys. Sci. 7(9) (2012) 1314–1320. Crossref, Google Scholar
9. L. Liu, X. Wang, X. Yang, H. Liu, J. Li and P. Wang, Path planning techniques for mobile robots: Review and prospect, Expert Syst. Appl. 227 (2023) 120254. Crossref, Web of Science, Google Scholar
10. N. Buniyamin et al., A simple local path planning algorithm for autonomous mobile robots, Int. J. Syst. Appl. Eng. Dev. 5(2) (2011) 151–159. Google Scholar
11. Z. Ding, Y. Huang, H. Yuan and H. Dong, Introduction to reinforcement learning, in Deep Reinforcement Learning: Fundamentals, Research and Applications (Springer, Singapore, 2020), pp. 47–123. Crossref, Google Scholar
12. C. L. Chen, D. Y. Dong, H.-X. Li and T.-J. Tarn, Hybrid MDP based integrated hierarchical Q-learning, Sci. China Inf. Sci. 54 (2011) 2279–2294. Crossref, Web of Science, Google Scholar
13. P. C. Hammer, Adaptive Control Processes: A Guided Tour (R. Bellman) (1962). Google Scholar
14. K. Zhu and T. Zhang, Deep reinforcement learning based mobile robot navigation: A review, Tsinghua Sci. Technol. 26(5) (2021) 674–691. Crossref, Web of Science, Google Scholar
15. S. Kumar and A. Sikander, A modified probabilistic roadmap algorithm for efficient mobile robot path planning, Eng. Optim. 55(9) (2023) 1616–1634. Crossref, Web of Science, Google Scholar
16. P. E. Hart, N. J. Nilsson and B. Raphael, A formal basis for the heuristic determination of minimum cost paths, IEEE Trans. Syst. Sci. Cybern. 4(2) (1968) 100–107. Crossref, Web of Science, Google Scholar
17. C. Han and B. Li, Mobile robot path planning based on improved A* algorithm, in 2023 IEEE 11th Joint Int. Information Technology and Artificial Intelligence Conf. (ITAIC), Vol. 11 (IEEE, 2023), pp. 672–676. Crossref, Google Scholar
18. E. W. Dijkstra, A note on two problems in connexion with graphs, in Edsger Wybe Dijkstra: His Life, Work, and Legacy (Association for Computing Machinery, 2022), pp. 287–290. Crossref, Google Scholar
19. S. Sundarraj, R. V. K. Reddy, B. M. Babu, G. H. Lokesh, F. Flammini and R. Natarajan, Route planning for an autonomous robotic vehicle employing a weight-controlled particle swarm-optimized Dijkstra algorithm, IEEE Access 11 (2023) 92433–92442. Crossref, Web of Science, Google Scholar
20. V. J. Lumelsky and A. A. Stepanov, Path-planning strategies for a point mobile automaton moving amidst unknown obstacles of arbitrary shape, Algorithmica 2(1–4) (1987) 403–430. Crossref, Google Scholar
21. T. Chen, G. Zhang, X. Hu and J. Xiao, Unmanned aerial vehicle route planning method based on a star algorithm, in 2018 13th IEEE Conf. Industrial Electronics and Applications (ICIEA) (IEEE, 2018), pp. 1510–1514. Crossref, Google Scholar
22. O. Khatib, Real-time obstacle avoidance for manipulators and mobile robots, Int. J. Robot. Res. 5(1) (1986) 90–98. Crossref, Web of Science, Google Scholar
23. X. Tang, H. Zhou and T. Xu, Obstacle avoidance path planning of 6-DoF robotic arm based on improved A* algorithm and artificial potential field method, Robotica 42(2) (2024) 457–481. Crossref, Web of Science, Google Scholar
24. J.-S. Shaw and S.-Y. Lee, Using genetic algorithm for drawing path planning in a robotic arm pencil sketching system, Proc. Inst. Mech. Eng. C: J. Mech. Eng. Sci. 238(14) (2024) 7134–7142. Crossref, Web of Science, Google Scholar
25. J. Chakraborty and S. Mukhopadhyay, A robust cooperative multi-robot path-planning in noisy environment, in 2010 5th Int. Conf. Industrial and Information Systems (IEEE, 2010), pp. 626–631. Crossref, Google Scholar
26. Z. Nie, X. Yang, S. Gao, Y. Zheng, J. Wang and Z. Wang, Research on autonomous moving robot path planning based on improved particle swarm optimization, in 2016 IEEE Cong. Evolutionary Computation (CEC) (IEEE, 2016), pp. 2532–2536. Crossref, Google Scholar
27. Z. Bai, H. Pang, Z. He, B. Zhao and T. Wang, Path planning of autonomous mobile robot in comprehensive unknown environment using deep reinforcement learning, IEEE Internet Things J. 11 (2024) 22153–22166. Crossref, Web of Science, Google Scholar
28. J. Pajarinen and V. Kyrki, Robotic manipulation of multiple objects as a POMDP, Artif. Intell. 247 (2017) 213–228. Crossref, Web of Science, Google Scholar
29. S. Pellegrinelli, A. Orlandini, N. Pedrocchi, A. Umbrico and T. Tolio, Motion planning and scheduling for human and industrial-robot collaboration, CIRP Ann. 66(1) (2017) 1–4. Crossref, Web of Science, Google Scholar
30. D. L. Cruz and W. Yu, Path planning of multi-agent systems in unknown environment with neural kernel smoothing and reinforcement learning, Neurocomputing 233 (2017) 34–42. Crossref, Web of Science, Google Scholar
31. H. Wang, J. Duan, M. Wang, J. Zhao and Z. Dong, Research on robot path planning based on fuzzy neural network algorithm, in 2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conf. (IAEAC) (IEEE, 2018), pp. 1800–1803. Crossref, Google Scholar
32. H. Kretzschmar, M. Spies, C. Sprunk and W. Burgard, Socially compliant mobile robot navigation via inverse reinforcement learning, Int. J. Robot. Res. 35(11) (2016) 1289–1307. Crossref, Web of Science, Google Scholar
33. L. Tai and M. Liu, Towards cognitive exploration through deep reinforcement learning for mobile robots, arXiv:1610.01733. Google Scholar
34. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra and M. Riedmiller, Playing Atari with deep reinforcement learning, arXiv:1312.5602. Google Scholar
35. V. Mnih et al., Human-level control through deep reinforcement learning, Nature 518(7540) (2015) 529–533. Crossref, Web of Science, Google Scholar
36. J. Xin, H. Zhao, D. Liu and M. Li, Application of deep reinforcement learning in mobile robot path planning, in 2017 Chinese Automation Cong. (CAC) (IEEE, 2017), pp. 7112–7116. Crossref, Google Scholar
37. U. Muller, J. Ben, E. Cosatto, B. Flepp and Y. Cun, Off-road obstacle avoidance through end-to-end learning, in Advances in Neural Information Processing Systems, Vol. 18 (2005). Google Scholar
38. M. Pfeiffer, M. Schaeuble, J. Nieto, R. Siegwart and C. Cadena, From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots, in 2017 IEEE Int. Conf. on Robotics and Automation (ICRA) (IEEE, 2017), pp. 1527–1533. Crossref, Google Scholar
39. L. Tai, G. Paolo and M. Liu, Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation, in 2017 IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS) (IEEE, 2017), pp. 31–36. Crossref, Google Scholar
40. T. Yan, Y. Zhang and B. Wang, Path planning for mobile robot’s continuous action space based on deep reinforcement learning, in 2018 Int. Conf. Big Data and Artificial Intelligence (BDAI) (IEEE, 2018), pp. 42–46. Google Scholar
41. Z. Liu, F. Zhu, K. Zhang, Z. Lai and H. Huo, Manifold transfer subspace learning based on double relaxed discriminative regression, Artif. Intell. Rev. 56(Suppl. 1) (2023) 959–981. Crossref, Web of Science, Google Scholar
42. Z. Liu, T. Wang, F. Zhu, X. Chen, D. Pelusi and A. V. Vasilakos, Domain adaptive learning based on equilibrium distribution and dynamic subspace approximation, Expert Syst. Appl. 249 (2024) 123673. Crossref, Web of Science, Google Scholar
43. C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang and C. Liu, A survey on deep transfer learning, in Artificial Neural Networks and Machine Learning–ICANN 2018: 27th Int. Conf. Artificial Neural Networks, Rhodes, Greece, October 4–7, 2018, Proc., Part III 27 (Springer, 2018), pp. 270–279. Crossref, Google Scholar
44. C. Lu, C. Lv, J. Gong, W. Wang, D. Cao and F.-Y. Wang, Instance-level knowledge transfer for data-driven driver model adaptation with homogeneous domains, IEEE Trans. Intell. Transp. Syst. 23(10) (2022) 17015–17026. Crossref, Web of Science, Google Scholar
45. K. Gopalakrishnan, S. K. Khaitan, A. Choudhary and A. Agrawal, Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection, Constr. Build. Mater. 157 (2017) 322–330. Crossref, Web of Science, Google Scholar
46. P. Zhu, S. Dong, X. Pan, X. Hu and S. Zhu, A simulation-data-driven subdomain adaptation adversarial transfer learning network for rolling element bearing fault diagnosis, Meas. Sci. Technol. 33(7) (2022) 075101. Crossref, Web of Science, Google Scholar
47. Y. Dai, J. Yu, C. Zhang, B. Zhan and X. Zheng, A novel whale optimization algorithm of path planning strategy for mobile robots, Appl. Intell. 53(9) (2023) 10843–10857. Crossref, Web of Science, Google Scholar
48. Y. Li, X. C. Song and W. Guan, Mobile robot path planning based on ABC-PSO algorithm, in 2022 IEEE 6th Information Technology and Mechatronics Engineering Conf. (ITOEC), Vol. 6 (IEEE, 2022), pp. 530–534. Crossref, Google Scholar