This paper presents a new gesture track recognition method based on the depth image information received from the Kinect sensor. First, a Kinect sensor is used to obtain the coordinates of a moving arm. Then, the gesture tracks corresponding to these coordinates are analyzed. Matching and recognition of gesture tracks are implemented by performing golden section search. The results show that this track-based method is highly effective in gesture recognition.