Gastrointestinal stromal tumors (GISTs) and leiomyomas are the most common submucosal tumors of the upper digestive tract, and the diagnosis of the tumors is essential for their treatment and prognosis. However, the ability of endoscopic ultrasonography (EUS) which could correctly identify the tumor types is limited and closely related to the knowledge, operational level, and experience of the endoscopists. Therefore, the convolutional neural network (CNN) is used to assist endoscopists in determining GISTs or leiomyomas with EUS. A model based on CNN was constructed according to GoogLeNet architecture to distinguish GISTs or leiomyomas. All EUS images collected from this study were randomly sampled and divided into training set (n=411) and testing set (n=103) in a ratio of 4:1. The CNN model was trained by EUS images from the training set, and the testing set was utilized to evaluate the performance of the CNN model. In addition, there were some comparisons between endoscopists and CNN models. It was shown that the sensitivity and specificity in identifying leiomyoma were 95.92%, 94.44%, sensitivity and specificity in identifying GIST were 94.44%, 95.92%, and accuracy in total was 95.15% of the CNN model. It indicates that the diagnostic accuracy of the CNN model is equivalent to skilled endoscopists, or even higher than them. While identifying GIST or leiomyoma, the performance of CNN model was robust, which is highlighting its promising role in supporting less-experienced endoscopists and reducing interobserver agreement.