气候变化领域集成服务门户(CCPortal): Deep reinforcement learning for the real time control of stormwater systems

CCPortal

DOI	10.1016/j.advwatres.2020.103600
	Deep reinforcement learning for the real time control of stormwater systems
	Mullapudi A.; Lewis M.J.; Gruden C.L.; Kerkez B.
发表日期	2020
ISSN	0309-1708
卷号	140
英文摘要	A new generation of smart stormwater systems promises to reduce the need for new construction by enhancing the performance of the existing infrastructure through real-time control. Smart stormwater systems dynamically adapt their response to individual storms by controlling distributed assets, such as valves, gates, and pumps. This paper introduces a real-time control approach based on Reinforcement Learning (RL), which has emerged as a state-of-the-art methodology for autonomous control in the artificial intelligence community. Using a Deep Neural Network, a RL-based controller learns a control strategy by interacting with the system it controls - effectively trying various control strategies until converging on those that achieve a desired objective. This paper formulates and implements a RL algorithm for the real-time control of urban stormwater systems. This algorithm trains a RL agent to control valves in a distributed stormwater system across thousands of simulated storm scenarios, seeking to achieve water level and flow set-points in the system. The algorithm is first evaluated for the control of an individual stormwater basin, after which it is adapted to the control of multiple basins in a larger watershed (4 km2). The results indicate that RL can very effectively control individual sites. Performance is highly sensitive to the reward formulation of the RL agent. Generally, more explicit guidance led to better control performance, and more rapid and stable convergence of the learning process. While the control of multiple distributed sites also shows promise in reducing flooding and peak flows, the complexity of controlling larger systems comes with a number of caveats. The RL controller's performance is very sensitive to the formulation of the Deep Neural Network and requires a significant amount of computational resource to achieve a reasonable performance enhancement. Overall, the controlled system significantly outperforms the uncontrolled system, especially across storms of high intensity and duration. A frank discussion is provided, which should allow the benefits and drawbacks of RL to be considered when implementing it for the real-time control of stormwater systems. An open source implementation of the full simulation environment and control algorithms is also provided. © 2020 Elsevier Ltd
关键词	Controllers Deep neural networks Learning systems Real time control Reinforcement learning Storms Water levels Autonomous control Computational resources Control performance Open source implementation Performance enhancements Stormwater systems Uncontrolled systems Urban stormwater systems Deep learning algorithm machine learning real time stormwater
语种	英语
来源机构	Advances in Water Resources
文献类型	期刊论文
条目标识符	http://gcip.llas.ac.cn/handle/2XKMVOVA/131802
推荐引用方式 GB/T 7714	Mullapudi A.,Lewis M.J.,Gruden C.L.,et al. Deep reinforcement learning for the real time control of stormwater systems[J]. Advances in Water Resources,2020,140.
APA	Mullapudi A.,Lewis M.J.,Gruden C.L.,&Kerkez B..(2020).Deep reinforcement learning for the real time control of stormwater systems.,140.
MLA	Mullapudi A.,et al."Deep reinforcement learning for the real time control of stormwater systems".140(2020).