Reinforcement Learning-Driven Adaptive Control of Complex Systems: A Policy Learning Framework for Long-term Sustainability

摘要

Smart microgrids are important complex systems in the shift towards sustainable energy due to the double challenge of global climate change and resource depletion but they struggle to work sustainably over time due to the uncertainties of renewable energy variability, load uncertainty and degradation of storage. Conventional control methodologies including Model Predictive Control (MPC), conventional heuristic rule-based control methods, usually demand accurate system models or manual rules that might restrict their flexibility during changing operating conditions. The paper presents a low-cost and repeatable adaptive control policy learning scheme based on Deep Reinforcement Learning (DRL) to manage smart microgrids in the long-term sustainability-based approach. As a formulation of the energy management problem as a Markov Decision Process (MDP) with safety constraints and a lightweight proxy of storage degradation, the framework will use the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm to optimize continuous control actions. This research does not depend on physical deployment of microgrids or expensive laboratory experiments; instead, it makes use of publicly available hourly time-series data of photovoltaic (PV) generation, wind power, storage states, dynamic load and time-varying cost signals in a purely software simulation setting. Comparative simulation outcomes reveal that the proposed Sustainable-TD3 framework is able to minimize the simulated full operational cost without compromising the supply-demand balance and reducing the proxy cost of storage degradation of energy storage devices. The findings indicate that a storage-health-conscious reward design may enhance the trade-off between economic efficiency and long-term sustainability in normal computing operations. This paper offers a repeatable and minimal-resource-reference model of adaptive control research in smart energy systems and encourages the continued development of sustainable scheduling plans in zero carbon communities and sustainable infrastructure.