The development of Pakcoy cultivation holds good prospects, as seen from the demand for vegetable commodities in Indonesia. Its cultivation is consistently rising in terms of volume and value of vegetable imports. However, the cultivation process encounters multiple issues caused by pests and diseases. In addition, the volatile climate in Indonesia has resulted in uninterrupted pest development and the potential decline of Pakcoy’s productivity. Therefore, the detection system for pests and diseases in the Pakcoy plant is called upon to accurately and quickly assist farmers in determining the right treatment, thereby reducing economic losses and producing abundant quality crops. A web-based application with several well-known Convolutional Neural Network (CNN) were incorporated, such as MobileNetV2, GoogLeNet, and ResNet101. A total of 1,226 images were used for training, validating, and testing the dataset to address the problem in this study. The dataset consisted of several plant conditions with leaf miners, cabbage butterflies, powdery mildew disease, healthy plants, and multiple data labels for pests and diseases presented in the individual image. The results show that the MobileNetV2 provides a minimum loss compared to GoogLeNet and ResNet-101 with scores of 0.076, 0.239, and 0.209, respectively. Since the MobileNetV2 architecture provides a good model, the model was carried out to be integrated and tested with the web-based application. The testing accuracy rate reached 98% from the total dataset of 70 testing images. In this direction, MobileNetV2 can be a viable method to be integrated with web-based applications for classifying an image as the basis for decision-making.