Coordinated Scheduling of Flexible Resources in Active Distribution Networks Using Multi-Agent Reinforcement Learning and Safety-Constrained Control

Zhishuang Wang, Chao Pang, Jian Chen, Shiqian Ma

Abstract


The rapid popularization of controllable loads such as electric vehicles (EVs), battery energy storage systems (BESS), and heating, ventilating and air conditioning (HVAC) has greatly improved the flexibility of active distribution networks, but it also brings challenges to safe and efficient real-time scheduling. However, most existing control strategies rely on either static optimization or single-agent reinforcement learning, which have shortcomings in scalability, adaptability to dynamic operating conditions, and ensuring network constraint satisfaction. To remedy these deficiencies, we propose a multi-agent reinforcement learning (MARL) -based scheduling framework combined with quadratic programming (QP) safety projection, which unifies flexible resource modeling, learns adaptive cost-aware and comfort-aware control policies, and guarantees real-time physical feasibility. The case study on the IEEE 123-node test system shows the superior performance of the proposed method compared to the existing methods. Specifically, in a typical scenario with 40% PV penetration, the proposed framework achieves 19.3% peak demand reduction and 13.4% operational cost savings. Moreover, it maintains a high EV user satisfaction of 96.5% and strictly guarantees network security with a zero voltage violation rate, which indicates its potential for reliable deployment in modern active distribution networks.

Keywords


Active distribution network; Flexible resource scheduling; Multi-agent reinforcement learning; Multi-objective coordinated control; Quadratic programming safety layer

Full Text:

PDF

References


Shi J., Ma L., Li C. et al., 2022. A comprehensive review of standards for distributed energy resource grid-integration and microgrid. Renewable and Sustainable Energy Reviews 170: 112957.

Wang X. and Dong B., 2024. Long-term experimental evaluation and comparison of advanced controls for HVAC systems. Applied Energy 371: 123706.

Nguyen A.T., Pham D.H., Oo B.L., Santamouris M., Ahn Y., and Lim B.T.H., 2024. Modelling building HVAC control strategies using a deep reinforcement learning approach. Energy and Buildings 310: 114065.

Hojjatinia Z., Mezher A.M., Castillo-Guerra E., Cardenas-Barrera J.L., and Saleh S.A., 2024. Peak shaving impact on load forecasting: A strategy for mitigation. IEEE Access 12: 160846-160863.

Barman P., Dutta L., Bordoloi S., Kalita A., Buragohain P., Bharali S. et al., 2023. Renewable energy integration with electric vehicle technology: A review of the existing smart charging approaches. Renewable and Sustainable Energy Reviews 183: 113518.

Lu Y., Fu M., Gao J., Shi Q., and Liu W., 2023. Voltage limit violation risk evaluation method considering communication failures in distributed voltage regulation system. International Journal of Electrical Power & Energy Systems 147: 108886.

Lai C.M. and Teh J., 2022. Network topology optimisation based on dynamic thermal rating and battery storage systems for improved wind penetration and reliability. Applied Energy 305: 117837.

Xiong J., Sun Y., Wang J., Li Z., Xu Z., and Zhai S., 2022. Multi-stage equipment optimal configuration of park-level integrated energy system considering flexible loads. International Journal of Electrical Power & Energy Systems 140: 108050.

Sun L., Xia J., She D., Ding W., Jiang J., Liu B. et al., 2024. A predictive fuzzy logic and rule-based control approach for practical real-time operation of urban stormwater storage system. Water Research 266: 122437.

Wang H., Shi M., Xie P., Lai C.S., Li K., and Jia Y., 2023. Electric vehicle charging scheduling strategy for supporting load flattening under uncertain electric vehicle departures. Journal of Modern Power Systems and Clean Energy 11 (5): 1634-1645.

Al-Othman A., Tawalbeh M., Martis R., Dhou S., Orhan M., Qasim M. et al., 2022. Artificial intelligence and numerical models in hybrid renewable energy systems with fuel cells: Advances and prospects. Energy Conversion and Management 253: 115154.

Zhao C., Sun D., Zhang L., Nian H., and Hu B., 2023. Two-stage optimal re/active power control of large-scale wind farm to mitigate voltage dip induced frequency excursion. IEEE Transactions on Sustainable Energy 14 (1): 730-733.

Al-Issa H.A., Qawaqzeh M., Kurashkin S., Halko S., Kvitka S., Vovk O. et al., 2022. Monitoring of power transformers using thermal model and permission time of overload. International Journal of Electrical and Computer Engineering 12 (3): 2323-2334.

Alvarez-Alvarado M.S., Donaldson D.L., Recalde A.A., Noriega H.H., Khan Z.A., Velasquez W. et al., 2022. Power system reliability and maintenance evolution: A critical review and future perspectives. IEEE Access 10: 51922-51950.

Li Q., Liu J., Cui B., Song W., and Ye J., 2024. Distribution system flexibility characterization: A network-informed data-driven approach. IEEE Transactions on Smart Grid 15 (1): 1188-1191.

Pourghaderi N., Fotuhi-Firuzabad M., Moeini-Aghtaie M., Kabirifar M., and Lehtonen M., 2023. Exploiting DERs' flexibility provision in distribution and transmission systems interface. IEEE Transactions on Power Systems 38 (2): 1963-1977.

Li Y., Wang J., Zhang Y., and Han Y., 2022. Day-ahead scheduling strategy for integrated heating and power system with high wind power penetration and integrated demand response: A hybrid stochastic/interval approach. Energy 253: 124189.

Sasaki K., Aki H. and Ikegami T., 2022. Application of model predictive control to grid flexibility provision by distributed energy resources in residential dwellings under uncertainty. Energy 239: 122183.

Escobar F., Viquez J.M., Garcia J., Aristidou P., and Valverde G., 2022. Coordination of DERs and flexible loads to support transmission voltages in emergency conditions. IEEE Transactions on Sustainable Energy 13 (3): 1344-1355.

Zhao Z., Guo J., Luo X., Lai C.S., Yang P., Lai L.L. et al., 2022. Distributed robust model predictive control-based energy management strategy for islanded multi-microgrids considering uncertainty. IEEE Transactions on Smart Grid 13 (3): 2107-2120.

Quintana E. and Inga E., 2022. Optimal reconfiguration of electrical distribution system using heuristic methods with geopositioning constraints. Energies 15 (15): 5317.

Lu Y., Xiang Y., Huang Y., Yu B., Weng L., and Liu J., 2023. Deep reinforcement learning based optimal scheduling of active distribution system considering distributed generation, energy storage and flexible load. Energy 271: 127087.

Ye Y., Papadaskalopoulos D., Yuan Q., Tang Y., and Strbac G., 2023. Multi-agent deep reinforcement learning for coordinated energy trading and flexibility services provision in local electricity markets. IEEE Transactions on Smart Grid 14 (2): 1541-1554.

Amato C., 2024. An introduction to centralized training for decentralized execution in cooperative multi-agent reinforcement learning. arXiv preprint arXiv:2409.03052.

Lu Y., Fu M., Gao J., Shi Q., and Liu W., 2023. Voltage limit violation risk evaluation method considering communication failures in distributed voltage regulation system. International Journal of Electrical Power & Energy Systems 147: 108886.

Zhang H., Xiao F., Zhang C., and Li R., 2023. A multi-agent system based coordinated multi-objective optimal load scheduling strategy using marginal emission factors for building cluster demand response. Energy and Buildings 281: 112765.

Wu S., Liu T., Egerstedt M., and Jiang Z.-P., 2023. Quadratic programming for continuous control of safety-critical multiagent systems under uncertainty. IEEE Transactions on Automatic Control 68 (11): 6664-6679.

Evangelopoulos V.A., Kontopoulos T.P. and Georgilakis P.S., 2022. Heterogeneous aggregators competing in a local flexibility market for active distribution system management: A bi-level programming approach. International Journal of Electrical Power & Energy Systems 136: 107639.




DOI: https://doi.org/10.64289/iej.26.0206.5530729