Reinforcement Learning for Adaptive Water Distribution Network Planning: Exploring its Feasibility and Potential

“Water distribution network (WDN) planning for the long term is a complex task, due to the deep uncertainty that characterizes important design parameters, such as urban development, demographic shifts, resource availability and climate. In an effort to address those challenges, researchers have moved towards dynamic design approaches, such as staged optimization, that allows taking into consideration the entire lifecycle of the network, while prioritizing the short-term needs [1]. However, recently it was demonstrated that the existing state-of-the-art approaches that rely on future scenarios and heuristic methods lack adaptivity as they need to be re-evaluated each time new information emerges.
Meanwhile, there have been notable breakthroughs in sequential decision-making problems under uncertainty in recent years, particularly in the field of reinforcement learning (RL). RL agents have demonstrated human (or even superhuman) performance in complex strategy games within stochastic environments [2]–[5]. In RL an agent learns by interacting with the environment and its goal is to select actions that maximize a cumulative reward. One distinctive RL characteristic is that agents learn how to adapt to unforeseen circumstances, making it a promising approach to decision-making in dynamic and uncertain environments.
Although RL algorithms have been successfully applied to real-time control in water engineering problems [6]–[9], their potential in the (short-term and long-term) design of water networks remains largely unexplored. In this work, we use an RL approach to address the deterministic design of WDNs, starting with the Hanoi network problem [10] which provides a simple benchmark to test our hypothesis. Our results demonstrate the feasibility of using RL for the design of WDNs and represent a first step towards the development of more adaptive planning approaches in the field.”

(Citation: Tsiami, L., Makropoulos, C., Savic, D.A. – Reinforcement Learning for Adaptive Water Distribution Network Planning: Exploring its Feasibility and Potential – 19th International Computing & Control for the Water Industry Conference, 7-9- September 2023)