A Data Mining Tool for Planning Sanitary Sewer Condition Inspection

Abstract

The deterioration of aging sanitary sewer pipes poses a potentially significant environmental and economic threat. While accurate information on sewer condition required for proactive management can be obtained through closed-circuit television (CCTV) inspections, these inspections are expensive and hence frequently limited to relatively small portions of the sewer system. Hence, there is real value in alternatives to assist in determining sewer integrity. Data mining is demonstrated as a means of extracting information from limited inspection records, allowing sewer pipe condition to be predicted for pipes that have not yet been inspected. The paper describes a classification tree algorithm capable of providing insight into a pipe condition dataset obtained after inspecting a portion of the sanitary sewers in Guelph, Ontario, Canada. The model is developed with minimal data pre-processing effort and illustrates the influence of pipe-specific attributes (e.g. year of construction, diameter and length) on pipe condition in a format that can be easily shared with those unfamiliar with the data mining process. The predictive capability of the classification tree is validated using a stratified test set representative of the distribution of pipe condition existing in the sewer system. CCTV inspection datasets are often imbalanced—with significantly more pipes in one condition class than another and this is problematic as data mining algorithms tend to be most effective when observations available for model development are balanced across classes. An optimally tuned classification tree predicts binary pipe condition (good vs. poor condition) with an overall accuracy of 76 % (282 out of 364 instances of pipe condition correctly predicted in the stratified test set). The model achieved an acceptable test set area under the receiver operating characteristic (ROC) curve of 0.77 and can effectively identify individual pipes for future rounds of inspection. The data mining approach presented herein is found capable of unlocking information contained within inspection records and enhances existing management practices used in the wastewater industry.