AimsEvaluating the endoscopic images of patients with ulcerative colitis can effectively determine a reasonable treatment plan. However, the endoscopic evaluation is usually affected by the doctor's subjective judgment. This research aims to develop a computer-assisted diagnosis system that can objectively diagnose the degree of endoscopic image activity in patients with ulcerative colitis. MethodsWe proposed a neural network named “Efficient Attention Mechanism Network” (“EAM-Net”) which combines the efficient channel attention network and spatial attention module. The features extracted by convolutional neural network are divided into two branches and input into the recurrent neural network and EAM-Net modules to generate and splice the attention map. The proposed EAM-Net is formed into UC-DenseNet for the classification of ulcerative colitis. The proposed method was evaluated on two colonoscopy image datasets. ResultsBy using UC-DenseNet for the diagnosis of endoscopic remission in patients with ulcerative colitis, the accuracy, precision, recall rate, F1-score, and area under curve are increased by 0.5% to 2%, 0.5% to 2%, 1% to 3%, 0.6% to 2%, and 0.5% to 1.8%, respectively. In addition, for the diagnosis of the severity of endoscopic inflammation in patients with ulcerative colitis, the accuracy, precision, recall rate, and F1-score are increased by 1.5% to 4%, 1% to 3.5%, 2% to 8%, 2% to 7%, respectively. ConclusionsExperimental results show that the proposed UC-DenseNet can effectively diagnose ulcerative colitis. It can assist the endoscopist in formulating the best treatment strategy.