Reinforcement Learning of Multi‐Timescale Forecast Information for Designing Operating Policies of Multi‐ Purpose Reservoirs

“Hydrological forecasts have significantly improved in skill over recent years, encouraging their systematic exploitation in multipurpose reservoir operations to improve reliability and resilience to extreme events. Despite the growing availability of multi‐timescale forecasts, there is still a lack of transparent and integrated methods for selecting the most suitable forecast products, variables, and lead times for specific operational challenges. In this work, we propose a holistic approach based on Reinforcement Learning (RL) to design multipurpose dam operating policies informed by available multi‐timescale forecast products. Our approach extends the traditional Evolutionary Multi‐Objective Direct Policy Search method by parametrizing both the operating policy and the forecast information extraction process. We compare our RL approach with a state‐of‐the‐art two‐step procedure in which the forecast selection and processing are performed before the policy optimization. We demonstrate the value of the method for the multipurpose operation of Lake Como (Italy) by considering multi‐timescale forecasts from short to seasonal lead times to manage flood‐ and drought‐ related operational objectives. Our approach identifies solutions achieving an 18% improvement in hypervolume indicator compared to policies not informed by forecasts and a 6% improvement over those designed using the two‐step reference methodology. These improvements are accompanied by increased flexibility in policy design and trade‐off analysis by directly extracting forecast information within the multi‐ objective optimization. This study demonstrates the feasibility and benefits of integrating policy design with forecast information extraction, particularly when multiple operational forecasts are available.”

(Citation: Zanutto, D., Ficchì, A., Giuliani, M., Castelletti, A. – Reinforcement Learning of Multi‐Timescale Forecast Information for Designing Operating Policies of Multi‐ Purpose Reservoirs – Water Resources Research 61(2025)2 art. no. e2023WR036724 – DOI: 10.1029/2023WR036724 – (Open Access))