In the previous decades, a great deal of quick intra Coding Unit (CU) decision algorithms has developed for High Efficiency Video Coding (HEVC/H.265). Regardless of the way that such video processing algorithms accomplish minimum intracoding time with less coding productivity misfortune, these are not appropriate for normally utilized between inter-prediction setups. Here, an algorithm is proposed for HEVC named as speedy (quick) inter-coding unit decision algorithm. In this technique, a movement decent variety of collocated CU was figured to decide for collocated CU partition. Likewise, the previous mode-based detection and termination decision have performed by utilizing a discriminant function limiting expected hazard. However, it requires promoting a decrease in computational complexity. Therefore, an energy minimization function is introduced with motion diversity computation to split the CU as the min cut in the graph for obtaining more efficient gesture-based split-to-split decision of collocated CU in a neighboring frame and also, the estimated motion vectors are clustered to find non-homogeneous regions depend on the deep learning approach such as fuzzy c-means clustering algorithm. Moreover, the thresholds and parameters used in motion diversity and energy function were learned by using deep learning RF classifier to enhance the HEVC video coding performance of an initial CU splitting and termination decision for HEVC inter-prediction.