Intestinal parasitic infections are still a serious public health problem in developing countries, and the diagnosis of parasitic infections requires the first step of parasite/egg detection of samples. Automated detection can eliminate the dependence on professionals, but the current detection algorithms require large computational resources, which increases the lower limit of automated detection. Therefore, we have designed a lightweight deep-learning model, YAC-Net, to achieve rapid and accurate detection of parasitic eggs and reduce the cost of automation. This paper uses the ICIP 2022 Challenge dataset for experiments, and the experiments are conducted using fivefold cross-validation. The YOLOv5n model is used as the baseline model, and then two improvements are made to the baseline model based on the specificity of the egg data. First, the neck of the YOLOv5n is modified to from a feature pyramid network (FPN) to an asymptotic feature pyramid network (AFPN) structure. Different from the FPN structure, which mainly integrates semantic feature information at adjacent levels, the hierarchical and asymptotic aggregation structure of AFPN can fully fuse the spatial contextual information of egg images, and its adaptive spatial feature fusion mode can help the model select beneficial feature and ignore redundant information, thereby reducing computational complexity and improving detection performance. Second, the C3 module of the backbone of the YOLOv5n is modified to a C2f module, which can enrich gradient information, improving the feature extraction capability of the backbone. Moreover, ablation studies are designed by us to verify the effectiveness of the AFPN and C2f modules in the process of model lightweighting. The experimental results show that compared with YOLOv5n, YAC-Net improves precision by 1.1%, recall by 2.8%, the F1 score by 0.0195, and mAP_0.5 by 0.0271 and reduces the parameters by one-fifth. Compared with some state-of-the-art detection methods, YAC-Net achieves the best performance in precision, F1 score, mAP_0.5, and parameters. The precision, recall, F1 score, mAP_0.5, and parameters of our method on the test set are 97.8%, 97.7%, 0.9773, 0.9913, and 1,924,302, respectively. Compared with the baseline model, YAC-Net optimizes the model structure and simplifies the parameters while ensuring the detection performance. It helps to reduce the equipment requirements for performing automated detection and can be used to realize the automatic detection of parasite eggs under microscope images.