The task of pedestrian detection in video surveillance applications will face challenges like dynamic background changes, false human detection (shadow), and illumination variations. In literature, many approaches have been proposed to resolve these challenges. But their performance is not up to the mark. Thus this paper proposes efficient pedestrian detection including shadow removal and automatic dynamic background update. For this firstly, a background frame is initialized where no moving object is present. Then a background subtraction algorithm is applied to each of the key frames from the live video to detect the foreground objects (using fuzzy C means clustering followed by mean absolute difference). Later on this segmented foreground a contour is estimated and passed through the HOG classifier for pedestrian detection. The performance of the proposed approach has been compared using various datasets & state-of-the-art approaches and found to the best with an average precision of 98 %, unlike the others.