| 引用本文: | 蔺伟山,王小君,孙庆凯,等.不确定性环境下基于深度强化学习的综合能源系统动态调度[J].电力系统保护与控制,2022,50(18):50-60.[点击复制] |
| LIN Weishan,WANG Xiaojun,SUN Qingkai,et al.Dynamic dispatch of an integrated energy system based on deep reinforcement learning in an uncertain environment[J].Power System Protection and Control,2022,50(18):50-60[点击复制] |
|
| 摘要: |
| 随着综合能源系统中间歇性能源和负荷不确定性的逐步增强，传统的调度方法局限于固定物理模型及参数设定，难以较好地动态响应源荷的随机波动。针对这一问题，提出了一种基于深度强化学习的综合能源系统动态调度方法。首先，以数据驱动方式构建面向综合能源系统的深度强化学习模型，通过智能体与综合能源系统的持续交互，自适应学习调度策略，降低对物理模型的依赖程度。其次，通过添加随机扰动的方式表征源荷不确定性变化特征，针对不确定性变化特征改进深度强化学习模型的状态空间、动作空间、奖励机制以及训练流程等关键环节，并经由近端策略优化算法优化求解，实现了综合能源系统的动态调度决策。最后，通过算例仿真验证了所提方法在不同时间尺度以及不确定性环境下的可行性和有效性。 |
| 关键词: 综合能源系统 动态调度 不确定性 深度强化学习 近端策略优化 |
| DOI：DOI: 10.19783/j.cnki.pspc.211685 |
| 投稿时间：2021-12-10修订日期：2022-01-24 |
| 基金项目:国家自然科学基金项目资助(51977005) |
|
| Dynamic dispatch of an integrated energy system based on deep reinforcement learning in an uncertain environment |
| LIN Weishan,WANG Xiaojun,SUN Qingkai,LIU Zhao,HE Jinghan,PU Tianjiao |
| (1. School of Electrical Engineering, Beijing Jiaotong University, Beijing 100044, China;
2. China Electric Power Research Institute Co., Ltd., Beijing 100192, China) |
| Abstract: |
| As the uncertainties of intermittent energy and load in the integrated energy system gradually increase, traditional dispatch methods are limited to fixed physical models and parameter settings that can hardly respond to the random fluctuations in the dynamic system with source-load. In this paper, a deep reinforcement learning-based dynamic dispatch method for the integrated energy system is proposed to address this problem. First, a data-driven deep reinforcement learning model is constructed for the integrated energy system. Through the continuous interaction between agent and integrated energy system, the dispatch strategies are learned adaptively to reduce dependence on the physical models. Secondly, the variations of source-load uncertainties are characterized by adding random disturbances. Pivotal aspects such as state spaces, action spaces, reward mechanisms and the training process of the deep reinforcement learning model are improved according to the characteristics of uncertainties. Then a proximal policy optimization algorithm is used to solve the problem, and the dynamic dispatch decisions of the integrated energy system are realized. Finally, simulation results verify the feasibility and effectiveness of the proposed method over different time scales and in uncertain environments.
This work is supported by the National Natural Science Foundation of China (No. 51977005). |
| Key words: integrated energy system dynamic dispatch uncertainties deep reinforcement learning proximal policy optimization |
