An automatic, miniature and multi-parameter on-line water quality monitoring system based on a micro-spectrometer is designed and implemented. The system is integrated with the flow-batch analysis and spectrophotometric detection method. The effectiveness of the system is tested by measuring chemical oxygen demand (COD) and ammonia-nitrogen in water. The results show that the modified system provides a cost-effective, sensitive, reproducible and reliable way to measure COD and ammonia-nitrogen in water samples with automatic operation and low toxic chemical consumption. In addition, the experiment results show that the relative error of the system is less than 10%, the limit of detection is 2 mg/L COD and 0.032 mg/L ammonia-nitrogen, respectively, and the relative standard deviation was 6.6% at 15.0 mg/L COD (n = 7) and 5.0% at 0.300 mg/L ammonia-nitrogen (n = 7). Results from the newly designed system are consistent with the data collected through the Chinese national standard analysis methods.