Rule-based deep reinforcement learning for optimal control of electrical batteries in an energy community

Roberto, Rocchetta and Lorenzo, Nespoli and Vasco, Medici and Saverio, Basso and Marco, Derboni and Matteo, Salani (2023) Rule-based deep reinforcement learning for optimal control of electrical batteries in an energy community. In: Proceedings of the 33rd European Safety and Reliability Conference (ESREL 2023) European Safety and Reliability Conference, 3-September-2023, Southampton.


Download (214kB) | Preview


This work investigates rule-based controllers (RBCs) and reinforcement learning (RL) agents for managing distributed electrical batteries in a net-zero energy community (NZEC) and reducing costs and emissions for the community. The RBCs are based on deterministic rules, hence, may fail to adapt to new scenarios and uncertainties. On the other hand, RL agents learn from direct interaction with uncertain environments and can better adapt to new conditions. A novel RL approach is proposed, combining MaskPPO and a deep neural network, to avoid the exploration of unsafe/unprofitable actions and enhance control efficacy through accurate predictions of future demand. These new approaches are demonstrated on the NeurIPS 2022 CityLearn challenge where real-world data from a district in California are embedded within a simulator for distributed battery control. Points of strength and limitations of the different tools discussed. For comparison sake, an oracle-driven controller is also considered as it gives a reference best-achievable optimum for the challenge problem, ie, lower bounds on costs and emissions reduction scores. Based on the results, RL agents generally offered robust control over the distributed batteries and often outperformed the rule-based controllers. Additionally, the combination of action masks and neural forecasters significantly improved the performance of the RL agents, bringing them very close to the scores achieved by the global optimum. A study of the model’s robustness to seasonality changes concludes this work and further illustrates the generalization ability of controllers.

Actions (login required)

View Item View Item