Gastrointestinal disorders are a class of prevalent disorders in the world. Capsule endoscopy is considered an effective diagnostic modality for diagnosing such gastrointestinal disorders, especially in small intestinal regions. The aim of this work is to leverage the potential of deep convolutional neural networks for automated classification of gastrointestinal abnormalities from capsule endoscopy images. This method developed a deep learning architecture, GastroNetV1, an automated classifier, to detect abnormalities in capsule endoscopy images. The gastrointestinal abnormalities considered are ulcerative colitis, polyps, and esophagitis. The curated dataset consists of 6000 images with “ground truth” labeling. The input image is automatically classified as ulcerative colitis, a polyp, esophagitis, or a normal condition by a web-based application designed with the trained algorithm. The classifier produced 99.2% validation accuracy, 99.3% specificity, 99.3% sensitivity, and 0.991 AUC. These results exceed that of the state-of-the-art systems. Hence, the GastroNetV1 could be used to identify the different gastrointestinal abnormalities in the capsule endoscopy images, which will, in turn, improve healthcare quality.