Lecturer: Professor Xia Li from Sun Yat-sen University

Title: Variance Criterion in Markov Decision Processes and Reinforcement Learning

Host: Professor Mo Lipo

Time: December 10, 2024 (Tuesday) 19:30

Tencent Meeting: 754-5890-0694

Abstract: With the successful application of AlphaGo, reinforcement learning has gained increasing attention from both academia and industry. The theoretical foundation of reinforcement learning is the Markov decision process, and the vast majority of current research optimizes the cumulative discounted performance metric, which cannot handle variance metrics. Variance can be used to characterize risk, safety, and stability indicators. This report will mainly introduce the author's recent research results on the variance minimization optimization problem in Markov decision processes, implementing it as a variance criterion reinforcement learning algorithm. Furthermore, the application of the above methods to energy systems will be discussed, optimizing the scheduling of new energy sources such as wind power and photovoltaics with energy storage systems to reduce the variance of the system's combined output, smooth the fluctuation of new energy output, and improve the quality and utilization rate of new energy systems.

Lecturer's Profile:

Professor at the School of Management, Sun Yat-sen University. He obtained his bachelor's and doctoral degrees from the Department of Automation at Tsinghua University in 2002 and 2007, respectively. From 2011 to 2019, he taught at the Department of Automation at Tsinghua University, serving as a lecturer and associate professor (Ph.D. supervisor). In 2019, he transferred to Sun Yat-sen University. His main research directions include the theoretical studies of Markov decision processes, reinforcement learning, queuing theory, stochastic games, and their application research in energy and finance. He has published over 100 papers, obtained more than 10 Chinese and American invention patents, and presided over 5 National Natural Science Foundation projects, among others. He serves as an associate editor (AE) for international authoritative SCI journals such as IEEE Transactions on Automation Science and Engineering and Discrete Event Dynamic Systems.