A new method to detect and reduce the impulse noise in color images is presented in this paper. The method consists of two stages: detection and filtering. Since each of the individual channels (components) of the color image can be considered as a monochrome image, both stages are applied to each channel separately, and then the individual results are combined into one output image. The corrupted pixels are detected in the first stage based on a proposed innovative switching technique. The noise-free pixels are copied to their corresponding locations in the output image. In the second stage, average filtering is applied only to those pixels which are determined to be noisy in the first stage, and only noise-free pixel values are involved in calculating this average. The size of the sliding window depends on the estimated noise density and is very small even for high noise densities. The proposed method is effective in noise reduction while preserving edge details and color chromaticity. Simulation results show that the proposed method outperforms all the tested existing state-of-the-art methods used in digital color image restoration in both standard objective measurements and perceived image quality.