Previous studies have reported that directionally selective ganglion cells respond strongly in their preferred direction, but are only weakly excited by stimuli moving in the opposite null direction. Various studies have attempted to elucidate the mechanisms underlying direction selectivity with cellular basis. However, these studies have not elucidated the mechanism underlying motion direction detection. In this study, we propose the mechanism based on Barlow’s inhibitory scheme for motion direction detection. We described the local motion-sensing direction-selective neurons. Next, this model was used to construct the two-dimensional multi-directional detection neurons which detect the local motion directions. The information of local motion directions was finally used to infer the global motion direction. To verify the validity of the proposed mechanism, we conducted a series of experiments involving a dataset with a number of images. The proposed mechanism exhibited good performance in all experiments with high detection accuracy. Furthermore, we compare the performance of our proposed system and traditional Convolution Neural Network (CNN) on motion direction prediction. It is found that the performance of our system is much better than that of CNN in terms of accuracy, calculation speed and cost.