Skin color detection is a popular and useful technique because of its wide range of utilizations both in human computer interaction and content based analysis. Applications such as detecting and tracking of human body parts, face detection and recognition, naked people detection and people retrieval in multimedia databases all benefit from skin detection. Hence finding a proper method for segment the skin-like pixels can solve the presented problems. The Imperialist Competitive Algorithm (ICA) is a new evolutionary algorithm which was recently introduced and has a good performance in some optimization problems. The ICA was inspired by socio-political processes of imperialistic competition of mankind in the real world. In this paper a hybrid ICA-ANN is proposed for solving skin classification. A new algorithm that combines ICA and ANN to solve skin classification has been proposed, and then a hybrid Neural Network (NN)-Imperialist Competitive Algorithm (ICA) is applied to solve the classification problem. In the proposed algorithm, a multi layer perceptron network (MLP) manages the problem’s constraints and an ICA algorithm searches for high quality solutions and minimum cost. The proposed color segmentation algorithm operates directly on RGB color space without the need of color space conversion. Experimental results show that this method can improve the performance of the MLP algorithm significantly.