Markov Decision Process Approximation Methods for Water Distribution Network Inspection and Maintenance: A Case Study of the U.S. Virgin Islands

Minsuk Seo, Daniel A. Eisenberg, Jefferson Huang

We develop a repair-oriented inspection and maintenance decision framework for water distribution networks. This work is motivated by utilities operating in data-sparse environments, such as in remote locations like the U.S. Virgin Islands, where data collection about network state and underground pipeline outages is limited to above-ground and easy to access information (e.g., water tank levels and pump operations). We formulate the problem as a discounted Markov decision process and integrate it with high-fidelity hydraulic simulation. The model captures latent system dynamics without requiring pipe-level sensing. The results reveal state-dependent optimal policies and heterogeneous failure characteristics across pipes, including rare but high-impact behaviors. We further show that certain observable system states uniquely correspond to specific pipe failures, enabling a form of virtual sensing. These findings demonstrate that system-level dynamics can support inspection planning and maintenance decisions under uncertainty in resource-constrained settings.