Deep Reinforcement Learning with Transfer Learning for Efficient Solution of Fractional Optimal Control Problems
Abstract
This paper introduces a novel approach for solving Fractional Optimal Control Problems (FOCPs) using Deep Reinforcement Learning (DRL). Fractional-order systems, involving derivatives of arbitrary order, have shown great potential for accurately modeling complex, real-world systems that cannot be effectively captured by traditional integer-order models. The inherent nonlinearity and complexity of FOCPs often make them difficult to solve using conventional methods. In this work, we leverage DRL to learn optimal control strategies for fractional systems by interacting with the environment. Additionally, Transfer Learning (TL) is incorporated to accelerate the learning process and enhance model efficiency by utilizing pre-trained models. The proposed method provides significant improvements in computational efficiency, accuracy, and the ability to handle highly nonlinear dynamics compared to traditional approaches. Numerical simulations validate the effectiveness of our approach, demonstrating its potential for broader applications in fractional control systems.
Keywords:
Fractional optimal control problems, Deep reinforcement learning, Transfer learning, Nonlinear dynamics, OptimizationReferences
- [1] Jin, B. (2021). Fractional differential equations. Springer. https://doi.org/10.1007/978-3-030-76043-4%0A%0A
- [2] Kilbas, A. (2006). Theory and applications of fractional differential equations. Elsevier. https://sutlib2.sut.ac.th/sut_contents/H103746.pdf
- [3] Magin, R. (2004). Fractional calculus in bioengineering, part 1. Critical reviewsTM in biomedical engineering, 32(1), 104. 10.1615/CritRevBiomedEng.v32.i1.10
- [4] Agrawal, O. P. (2004). A general formulation and solution scheme for fractional optimal control problems. Nonlinear dynamics, 38(1), 323–337. https://doi.org/10.1007/s11071-004-3764-6
- [5] Agrawal, O. P. (2008). A formulation and numerical scheme for fractional optimal control problems. Journal of vibration and control, 14(9–10), 1291–1299. https://doi.org/10.1177/1077546307087451
- [6] Pooseh, S., Almeida, R., & Torres, D. F. M. (2013). Fractional order optimal control problems with free terminal time. https://doi.org/10.3934/jimo.2014.10.363
- [7] Sweilam, N. H., Al-Ajami, T. M., & Hoppe, R. H. W. (2013). Numerical solution of some types of fractional optimal control problems. The scientific world journal, 2013(1), 306237. https://doi.org/10.1155/2013/306237
- [8] Sabouri, J., Effati, S., & Pakdaman, M. (2017). A neural network approach for solving a class of fractional optimal control problems. Neural processing letters, 45(1), 59–74. https://doi.org/10.1007/s11063-016-9510-5
- [9] Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533. https://B2n.ir/sk3938
- [10] Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., … & Wierstra, D. (2015). Continuous control with deep reinforcement learning. https://doi.org/10.48550/arXiv.1509.02971
- [11] Pan, S. J., & Yang, Q. (2009). A survey on transfer learning. IEEE transactions on knowledge and data engineering, 22(10), 1345–1359. https://doi.org/10.1109/TKDE.2009.191
- [12] Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., … & He, Q. (2020). A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1), 43–76. https://doi.org/10.1109/JPROC.2020.3004555