Real-time Segmentation Research Articles

Leveraging Big Data and AI for Optimizing Digital Marketing Strategies: A Data-driven Approach

In this paper, the author discusses how big data and AI can improve the rate of conversion, engagement, and ROI in digital marketing through ’big data’ techniques in improving the effectiveness of the marketing mix. In this study, by counting and comparing campaigns from different industries, the researcher establishes that three strategies facilitated by AI have a positive impact on marketing- predictive analytics, real time personalization and segmentation. Findings further show that some industries such as e-commerce and the financial industry benefit most from these technologies, yet in other areas like the healthcare industry, regulatory issues prevent substantial results. The insights reveal how embedding of AI in the digital marketing strategy give companies a competitive advantage while the aspects of ethics such as data protection cannot be overlooked. Finally, the research study calls attention to the role of AI and big data in developing better, engaging, and relevant digital marketing solutions.

Real-time segmentation and classification of whole-slide images for tumor biomarker scoring

Histopathology image segmentation and classification are essential for diagnosing and treating breast cancer. This study introduced a highly accurate segmentation and classification for histopathology images using a single architecture. We utilized the famous segmentation architectures, SegNet and U-Net, and modified the decoder to attach ResNet, VGG and DenseNet to perform classification tasks. These hybrid models are integrated with Stardist as the backbone, and implemented in a real-time pathologist workflow with a graphical user interface. These models were trained and tested offline using the ER-IHC-stained private and H&E-stained public datasets (MoNuSeg). For real-time evaluation, the proposed model was evaluated using PR-IHC-stained glass slides. It achieved the highest segmentation pixel-based F1-score of 0.902 and 0.903 for private and public datasets respectively, and a classification-based F1-score of 0.833 for private dataset. The experiment shows the robustness of our method where a model trained on ER-IHC dataset able to perform well on real-time microscopy of PR-IHC slides on both 20x and 40x magnification. This will help the pathologists with a quick decision-making process.

Real-time segmentation of biliary structure in pure laparoscopic donor hepatectomy

Pure laparoscopic donor hepatectomy (PLDH) has become a standard practice for living donor liver transplantation in expert centers. Accurate understanding of biliary structures is crucial during PLDH to minimize the risk of complications. This study aims to develop a deep learning-based segmentation model for real-time identification of biliary structures, assisting surgeons in determining the optimal transection site during PLDH. A single-institution retrospective feasibility analysis was conducted on 30 intraoperative videos of PLDH. All videos were selected for their use of the indocyanine green near-infrared fluorescence technique to identify biliary structure. From the analysis, 10 representative frames were extracted from each video specifically during the bile duct division phase, resulting in 300 frames. These frames underwent pixel-wise annotation to identify biliary structures and the transection site. A segmentation task was then performed using a DeepLabV3+ algorithm, equipped with a ResNet50 encoder, focusing on the bile duct (BD) and anterior wall (AW) for transection. The model’s performance was evaluated using the dice similarity coefficient (DSC). The model predicted biliary structures with a mean DSC of 0.728 ± 0.01 for BD and 0.429 ± 0.06 for AW. Inference was performed at a speed of 15.3 frames per second, demonstrating the feasibility of real-time recognition of anatomical structures during surgery. The deep learning-based semantic segmentation model exhibited promising performance in identifying biliary structures during PLDH. Future studies should focus on validating the clinical utility and generalizability of the model and comparing its efficacy with current gold standard practices to better evaluate its potential clinical applications.

Development of advanced AI-based segmentation and prediction method for air entrainment in plunging water jets

Understanding air entrainment phenomena induced by plunging water jets is critical in the fields of nuclear and hydraulic engineering. Air entrainment is one of the key safety design parameters for nuclear systems. However, most existing studies rely on empirical correlations or curve-fitting models to estimate bubble penetration depth, and no agreed-upon calculation principle exists for varying jet conditions. To address these limitations, this research developed two advanced AI approaches: an improved YOLOv5 for segmenting air entrainment and the NSGA-III-BPNN method for predicting penetration depth. The improved YOLOv5 enables real-time segmentation and extraction of air entrainment motion and dynamics under diverse conditions, demonstrating high scalability and robustness. The penetration depth estimated using the improved YOLOv5 shows greater accuracy compared to conventional empirical correlationsand is more efficient than traditional image post-processing techniques for classifying shape regimes based on dynamic air entrainment patterns. To overcome the limitations of object segmentation, which typically relies on video or image data, the NSGA-III-BPNN method predicts maximum penetration depths with greater accuracy than YOLOv5, offering a more effective prediction model for air entrainment penetration depth. By leveraging advanced AI techniques, the research not only provides valuable segmentation data for refining computational fluid dynamics (CFD) modeling but also paves the way for significant advancements in both nuclear and hydraulic engineering.

An Improved Instance Segmentation Method for Complex Elements of Farm UAV Aerial Survey Images.

Farm aerial survey layers can assist in unmanned farm operations, such as planning paths and early warnings. To address the inefficiencies and high costs associated with traditional layer construction, this study proposes a high-precision instance segmentation algorithm based on SparseInst. Considering the structural characteristics of farm elements, this study introduces a multi-scale attention module (MSA) that leverages the properties of atrous convolution to expand the sensory field. It enhances spatial and channel feature weights, effectively improving segmentation accuracy for large-scale and complex targets in the farm through three parallel dense connections. A bottom-up aggregation path is added to the feature pyramid fusion network, enhancing the model's ability to perceive complex targets such as mechanized trails in farms. Coordinate attention blocks (CAs) are incorporated into the neck to capture richer contextual semantic information, enhancing farm aerial imagery scene recognition accuracy. To assess the proposed method, we compare it against existing mainstream object segmentation models, including the Mask R-CNN, Cascade-Mask, SOLOv2, and Condinst algorithms. The experimental results show that the improved model proposed in this study can be adapted to segment various complex targets in farms. The accuracy of the improved SparseInst model greatly exceeds that of Mask R-CNN and Cascade-Mask and is 10.8 and 12.8 percentage points better than the average accuracy of SOLOv2 and Condinst, respectively, with the smallest number of model parameters. The results show that the model can be used for real-time segmentation of targets under complex farm conditions.

A Novel Mamba Architecture with a Semantic Transformer for Efficient Real-Time Remote Sensing Semantic Segmentation

Real-time remote sensing segmentation technology is crucial for unmanned aerial vehicles (UAVs) in battlefield surveillance, land characterization observation, earthquake disaster assessment, etc., and can significantly enhance the application value of UAVs in military and civilian fields. To realize this potential, it is essential to develop real-time semantic segmentation methods that can be applied to resource-limited platforms, such as edge devices. The majority of mainstream real-time semantic segmentation methods rely on convolutional neural networks (CNNs) and transformers. However, CNNs cannot effectively capture long-range dependencies, while transformers have high computational complexity. This paper proposes a novel remote sensing Mamba architecture for real-time segmentation tasks in remote sensing, named RTMamba. Specifically, the backbone utilizes a Visual State-Space (VSS) block to extract deep features and maintains linear computational complexity, thereby capturing long-range contextual information. Additionally, a novel Inverted Triangle Pyramid Pooling (ITP) module is incorporated into the decoder. The ITP module can effectively filter redundant feature information and enhance the perception of objects and their boundaries in remote sensing images. Extensive experiments were conducted on three challenging aerial remote sensing segmentation benchmarks, including Vaihingen, Potsdam, and LoveDA. The results show that RTMamba achieves competitive performance advantages in terms of segmentation accuracy and inference speed compared to state-of-the-art CNN and transformer methods. To further validate the deployment potential of the model on embedded devices with limited resources, such as UAVs, we conducted tests on the Jetson AGX Orin edge device. The experimental results demonstrate that RTMamba achieves impressive real-time segmentation performance.

O-260 Identifying seminiferous tubules for sperm retrieval during microdissection testicular sperm extraction (M-TESE) using deep learning

Abstract Study question Is it possible to utilize artificial intelligence (AI) in the selection of seminiferous tubules for sperm retrieval during microdissection testicular sperm extraction (M-TESE)? Summary answer AI algorithm was able to segment seminiferous tubules deemed to be good candidates for sperm retrieval with IoU of 0.746 and F1 score of 72.5%. What is known already M-TESE is an advanced surgical procedure reserved for sperm retrieval in men with non-obstructive azoospermia (NOA). Meta-analysis have shown M-TESE might be superior in relation to sperm retrieval rates and postsurgical complications when compared to conventional techniques. Limitations of the technique are the relatively long learning curve for the surgeons and the longer intraoperative time required to identify the tubules with active spermatogenesis. Convolutional neural networks (CNNs) have been shown to successfully identify areas of interest in medical images such as CT and MRI scans. The possibility of utilizing trained CNNs in M-TESE procedures has so far not been studied. Study design, size, duration This is a cross sectional study analyzing image data gathered during M-TESE procedures including the first five men enrolled in a prospective study designed specifically to test the utilization of AI in the treatment of NOA. All participants signed written informed consent. Images were collected intraoperatively with low and high resolution cameras according to predefined protocol. Participants/materials, setting, methods CNN model with down and upsampling architecture was trained using patch-based technique with corresponding labelled truth image pairs based on combination of semi-automated hue saturation value thresholding and manual segmentation of seminiferous tubules. Tubules were allocated into two different classes by the operating surgeon according to the visual perception of their morphology as type 1 (i.e. larger, opaque, good candidates for micro dissection) and type 2 (i.e. sclerotic or atrophic; unlikely to harbor active spermatogenesis). Main results and the role of chance CNN models were trained on 1809 patches with corresponding ground truth segmented images. Two algorithms were tested: the first was trained solely to detect and semantically distinguish type 1 seminiferous tubules. The second was trained to semantically segment the major anatomical structures: tunica albuginea, testicular parenchyma, blood vessels; lobules and the two types of seminiferous tubules describe above. Validation was performed on 988 image patches not previously shown to the algorithm. After grid search for optimal learning rate and augmentation the first algorithm was able to segment type 1 tubules with F1 score of 72.5% and Jaccard Index (IoU) of 0.746. Training took 856 minutes on 48 core high performance computer (HPC) equipped with single graphics processing unit (GPU). Same computational environment was used in the training of the second algorithm. This model was able to segment anatomical structures such as parenchyma (IoU=0.732) and testicular lobules (IoU=0.683) but performed sub optimally on the clinically relevant type 1 tubules (IoU=0.432). The first trained algorithm was able to segment a single high resolution surgical image in 3.4 seconds on an HPC. However, the same procedure took 5.5 minutes on standard desktop computer with 8 core Intel processor without GPU. Limitations, reasons for caution The number of patients is relatively small and therefore more training data would be needed to achieve robust performance. Training data were labelled by single surgeon limiting the generalizability and possibly biasing the selection process. The need of HPC environment for real time segmentation can limit the clinical use. Wider implications of the findings This pilot study shows the potential of AI to help in choosing suitable tubules for micro dissection, shortening the learning curve for young andrological surgeons. The access to cluster based computation is getting easier by the day and can soon allow instant intraoperative AI segmentation, thus shortening the surgical procedure. Trial registration number NA

Real-time segmentation of short videos under VR technology in dynamic scenes

Abstract This work addresses the challenges of scene segmentation and low segmentation accuracy in short videos by employing virtual reality (VR) technology alongside a 3D DenseNet model for real-time segmentation in dynamic scenes. First, this work extracted short videos by frame and removed redundant background information. Then, the volume rendering algorithm in VR technology was used to reconstruct short videos in dynamic scenes in 3D. It enriched the detailed information of short videos, and finally used the 3D DenseNet model for real-time segmentation of short videos in dynamic scenes, improving the accuracy of segmentation. The experiment compared the performance of High resolution network, Mask region based convolutional neural network, 3D U-Net, Efficient neural network models on the Densely annotation video segmentation dataset. The experimental results showed that the segmentation accuracy of the 3D DenseNet model has reached 99.03%, which was 15.11% higher than that of the ENet model. The precision rate reached 98.33%, and the average segmentation time reached 0.64 s, improving the segmentation accuracy and precision rate. It can adapt to various scene situations and has strong robustness. The significance of this research lies in its innovative approach in tackling these issues. By integrating VR technology with advanced deep learning models, we can achieve more precise segmentation of dynamic scenes in short videos, enabling real-time processing. This has significant practical implications for fields such as video editing, VR applications, and intelligent surveillance. Furthermore, the outcomes of this research contribute to advancing computer vision in video processing, providing valuable insights for the development of future intelligent video processing systems.

A temporal enhanced semi-supervised training framework for needle segmentation in 3D ultrasound images

Objective. Automated biopsy needle segmentation in 3D ultrasound images can be used for biopsy navigation, but it is quite challenging due to the low ultrasound image resolution and interference similar to the needle appearance. For 3D medical image segmentation, such deep learning networks as convolutional neural network and transformer have been investigated. However, these segmentation methods require numerous labeled data for training, have difficulty in meeting the real-time segmentation requirement and involve high memory consumption. Approach. In this paper, we have proposed the temporal information-based semi-supervised training framework for fast and accurate needle segmentation. Firstly, a novel circle transformer module based on the static and dynamic features has been designed after the encoders for extracting and fusing the temporal information. Then, the consistency constraints of the outputs before and after combining temporal information are proposed to provide the semi-supervision for the unlabeled volume. Finally, the model is trained using the loss function which combines the cross-entropy and Dice similarity coefficient (DSC) based segmentation loss with mean square error based consistency loss. The trained model with the single ultrasound volume input is applied to realize the needle segmentation in ultrasound volume. Main results. Experimental results on three needle ultrasound datasets acquired during the beagle biopsy show that our approach is superior to the most competitive mainstream temporal segmentation model and semi-supervised method by providing higher DSC (77.1% versus 76.5%), smaller needle tip position (1.28 mm versus 1.87 mm) and length (1.78 mm versus 2.19 mm) errors on the kidney dataset as well as DSC (78.5% versus 76.9%), needle tip position (0.86 mm versus 1.12 mm) and length (1.01 mm versus 1.26 mm) errors on the prostate dataset. Significance. The proposed method can significantly enhance needle segmentation accuracy by training with sequential images at no additional cost. This enhancement may further improve the effectiveness of biopsy navigation systems.

Real-time segmentation algorithm of unstructured road scenes based on improved BiSeNet

Real-time segmentation algorithm of unstructured road scenes based on improved BiSeNet

RTSeg-net: A lightweight network for real-time segmentation of fetal head and pubic symphysis from intrapartum ultrasound images

The segmentation of the fetal head (FH) and pubic symphysis (PS) from intrapartum ultrasound images plays a pivotal role in monitoring labor progression and informing crucial clinical decisions. Achieving real-time segmentation with high accuracy on systems with limited hardware capabilities presents significant challenges. To address these challenges, we propose the real-time segmentation network (RTSeg-Net), a groundbreaking lightweight deep learning model that incorporates innovative distribution shifting convolutional blocks, tokenized multilayer perceptron blocks, and efficient feature fusion blocks. Designed for optimal computational efficiency, RTSeg-Net minimizes resource demand while significantly enhancing segmentation performance. Our comprehensive evaluation on two distinct intrapartum ultrasound image datasets reveals that RTSeg-Net achieves segmentation accuracy on par with more complex state-of-the-art networks, utilizing merely 1.86 M parameters—just 6 % of their hyperparameters—and operating seven times faster, achieving a remarkable rate of 31.13 frames per second on a Jetson Nano, a device known for its limited computing capacity. These achievements underscore RTSeg-Net's potential to provide accurate, real-time segmentation on low-power devices, broadening the scope for its application across various stages of labor. By facilitating real-time, accurate ultrasound image analysis on portable, low-cost devices, RTSeg-Net promises to revolutionize intrapartum monitoring, making sophisticated diagnostic tools accessible to a wider range of healthcare settings.

AM-UNet: Field Ridge Segmentation of Paddy Field Images Based on an Improved MultiResUNet Network

In order to solve the problem of image boundary segmentation caused by the irregularity of paddy fields in southern China, a high-precision segmentation method based on the improved MultiResUNet model for paddy field mapping is proposed, combining the characteristics of paddy field scenes. We introduce the attention gate (AG) mechanism at the end of the encoder–decoder skip connections in the MultiResUNet model to generate the weights and highlight the response of the field ridge area, add an atrous spatial pyramid pooling (ASPP) module after the end of the encoder down-sampling, use an appropriate combination of expansion rates to improve the identification of small-scale edge details, use 1 × 1 convolution to improve the range of the sensory field after bilinear interpolation to increase the segmentation accuracy, and, thus, construct the AM-UNet paddy field ridge segmentation model. The experimental results show that the IoU, precision, and F1 value of the AM-UNet model are 88.74%, 93.45%, and 93.95%, respectively, and that inference time for a single image is 168ms, enabling accurate and real-time segmentation of field ridges in a complex paddy field environment. Thus, the AM-UNet model can provide technical support for the development of vision-based automatic navigation systems for agricultural machines.

A serial semantic segmentation model based on encoder-decoder architecture

The thriving progress of Convolutional Neural Networks (CNNs) and the outstanding efficacy of Visual Transformers (ViTs) have delivered impressive outcomes in the domain of semantic segmentation. However, each model in isolation entails a trade-off between high computational complexity and compromised computational efficiency. To address this challenge, we effectively combine the CNN and encoder-decoder structures in a Transformer-inspired fashion, presenting the Serial Semantic Segmentation Trans via CNN Former (SSS-Former) model. To augment the feature extraction capability, we utilize the meticulously crafted SSS-CSPNet, resulting in a well-designed architecture for the holistic model. We propose a novel SSS-PN attention network that enhances the spatial topological connections of features, leading to improved overall performance. Additionally, the integration of SASPP bridges the semantic gap between multi-scale features and enhances segmentation ability for overlapping objects. To fulfill the requirement of real-time segmentation, we leverage a novel restructuring technique to devise a more lightweight and faster ResSSS-Former model. Abundant experimental results demonstrate that both SSS-Former and ResSSS-Former outperform existing state-of-the-art methods in terms of computational efficiency, result precision, and speed. Remarkably, SSS-Former achieves a mIoU of 58.63 % at 89.1FPS on the ADE20K dataset. On the validation and testing datasets of CityScapes, it obtains mIoU scores of 85.1 % and 85.2 % respectively, with a speed of 94.1FPS. Our optimized ResSSS-Former achieves impressive real-time segmentation results, with an astonishing 100+FPS while maintaining high segmentation accuracy. The compelling results from the ISPRS datasets further validate the effectiveness of our proposed models in segmenting multi-scale and overlapping objects.

Design and Experiment of Variable-Spray System Based on Deep Learning

In response to issues of the low pesticide-utilization rate caused by the traditional constant spraying method, a variable-spraying system for wheat-field weeds was proposed in this study based on real-time segmentation by deep learning. In this study, the weed density within the operational area was obtained by using the improved DeepLab V3+ semantic segmentation mode, and a variable spray-level model based on real-time weed density and speed was constructed by using PWM variable-spraying technology to adjust the spray volume. The lightweight MobileNet V2 network was selected as its backbone network, and the CA attention mechanism was integrated into the feature extraction module. The mean intersection over the union (MIoU) and mean pixel accuracy (MPA) of the improved DeepLab V3+ were 73.34% and 80.76%, respectively, and the segmentation time for a single image was 0.09 s. The results of field verification tests showed that (1) compared with constant spraying, variable spraying can increase droplet density and save the amount of pesticides, with the droplet density increased by 38.87 droplets/cm2 and the pesticide consumption saved by 46.3%; and (2) at the same speed, the average droplet-coverage rate in the areas with sparse weed density decreased by 13.98% compared with the areas with dense weeds. Under the same plant density, the average coverage rate of 0.5 m/s increased by 2.91% and 6.59% compared with 1 m/s and 1.5 m/s, respectively. These results further demonstrated that the system can automatically adjust the spray volume based on different travel speeds and weed densities. This research can provide theoretical and reference support for the development of new precision-spray plant-protection machinery for wheat fields.

SCTNet: Single-Branch CNN with Transformer Semantic Information for Real-Time Segmentation

Recent real-time semantic segmentation methods usually adopt an additional semantic branch to pursue rich long-range context. However, the additional branch incurs undesirable computational overhead and slows inference speed. To eliminate this dilemma, we propose SCTNet, a single branch CNN with transformer semantic information for real-time segmentation. SCTNet enjoys the rich semantic representations of an inference-free semantic branch while retaining the high efficiency of lightweight single branch CNN. SCTNet utilizes a transformer as the training-only semantic branch considering its superb ability to extract long-range context. With the help of the proposed transformer-like CNN block CFBlock and the semantic information alignment module, SCTNet could capture the rich semantic information from the transformer branch in training. During the inference, only the single branch CNN needs to be deployed. We conduct extensive experiments on Cityscapes, ADE20K, and COCO-Stuff-10K, and the results show that our method achieves the new state-of-the-art performance. The code and model is available at https://github.com/xzz777/SCTNet.

KDP-Net: An Efficient Semantic Segmentation Network for Emergency Landing of Unmanned Aerial Vehicles

As the application of UAVs becomes more and more widespread, accidents such as accidental injuries to personnel, property damage, and loss and destruction of UAVs due to accidental UAV crashes also occur in daily use scenarios. To reduce the occurrence of such accidents, UAVs need to have the ability to autonomously choose a safe area to land in an accidental situation, and the key lies in realizing on-board real-time semantic segmentation processing. In this paper, we propose an efficient semantic segmentation method called KDP-Net for characteristics such as large feature scale changes and high real-time processing requirements during the emergency landing process. The proposed KDP module can effectively improve the accuracy and performance of the semantic segmentation backbone network; the proposed Bilateral Segmentation Network improves the extraction accuracy and processing speed of important feature categories in the training phase; and the proposed edge extraction module improves the classification accuracy of fine features. The experimental results on the UDD6 and SDD show that the processing speed of this method reaches 85.25 fps and 108.11 fps while the mIoU reaches 76.9% and 67.14%, respectively. The processing speed reaches 53.72 fps and 38.79 fps when measured on Jetson Orin, which can meet the requirements of airborne real-time segmentation for emergency landing.

Real-time reliable semantic segmentation of thyroid nodules in ultrasound images

Objective. Low efficiency in medical image segmentation is a common issue that limits computer-aided diagnosis development. Due to the varying positions and sizes of nodules, it is not easy to accurately segment ultrasound images. This study aims to propose a segmentation model that maintains high efficiency while improving accuracy. Approach. We propose a novel layer that integrates the advantages of dense connectivity, dilated convolution, and factorized filters to maintain excellent efficiency while improving accuracy. Dense connectivity optimizes feature reuse, dilated convolution redesigns layers, and factorized convolution improves efficiency. Moreover, we propose a loss function optimization method from a pixel perspective to increase the network's accuracy further. Main results. Experiments on the Thyroid dataset show that our method achieves 81.70% intersection-over-union (IoU), 90.50% true positive rate (TPR), and 0.25% false positive rate (FPR). In terms of accuracy, our method outperforms the state-of-the-art methods, with twice faster inference and nearly 400 times fewer parameters. Meanwhile, in a test on an External Thyroid dataset, our method achieves 77.03% IoU, 82.10% TPR, and 0.16% FPR, demonstrating our proposed model's robustness. Significance. We propose a real-time semantic segmentation architecture for thyroid nodule segmentation in ultrasound images called fully convolution dense dilated network (FCDDN). Our method runs fast with a few parameters and is suitable for medical devices requiring real-time segmentation.

LPT-Net: A Line-Pad Transformer Network for efficiency coal gangue segmentation with linear multi-head self-attention mechanism

Accurately separating coal and gangue is a crucial step in coal production. However, existing methods often face efficiency challenges. The high computational complexity hinders real-time gangue sorting. Real-time segmentation is essential for improved sorting results, making efficiency a crucial evaluation metric. Additionally, ensuring real-time segmentation accuracy is crucial. To address these issues, we propose Line-Pad Transformer Network (LPT-Net), a efficient segmentation network for gangue sorting. LPT-Net incorporates L-MSA, a module designed for linear attention calculation. It utilizes linear feature sampling for efficient computation, enabling the rapid extraction of semantic features from coal gangue. Additionally, LPT-Net introduces MSA1 and MSA2, serving as semantic information providers to enable more effective extraction of gangue semantic features. Experimental results on GSTD demonstrate LPT’s effectiveness, achieving 74.71 IoU and 83.85 Acc at 95.7 FPS, outperforming other methods by 1.5–12 times in terms of inference speed. These results highlight LPT’s suitability for efficient gangue sorting tasks.

An efficient segmentation model for abnormal chicken droppings recognition based on improved deep dual-resolution network.

The characteristics of chicken droppings are closely linked to their health status. In prior studies, chicken droppings recognition is treated as an object detection task, leading to challenges in labeling and missed detection due to the diverse shapes, overlapping boundaries, and dense distribution of chicken droppings. Additionally, the use of intelligent monitoring equipment equipped with edge devices in farms can significantly reduce manual labor. However, the limited computational power of edge devices presents challenges in deploying real-time segmentation algorithms for field applications. Therefore, this study redefines the task as a segmentation task, with the main objective being the development of a lightweight segmentation model for the automated monitoring of abnormal chicken droppings. A total of 60 Arbor Acres broilers were housed in 5 specific pathogen-free cages for over 3 weeks, and 1650 RGB images of chicken droppings were randomly divided into training and testing sets in an 8:2 ratio to develop and test the model. Firstly, by incorporating the attention mechanism, multi-loss function and auxiliary segmentation head, the segmentation accuracy of the DDRNet was enhanced. Then, by employing the group convolution and an advanced knowledge distillation algorithm, a lightweight segmentation model named DDRNet-s-KD was obtained, which achieved a mean Dice coefficient (mDice) of 79.43% and an inference speed of 86.10 frames per second (FPS), showing a 2.91% and 61.2% increase in mDice and FPS compared to the benchmark model. Furthermore, the DDRNet-s-KD model was quantized from 32-bit floating-point values to 8-bit integers and then converted to TensorRT format. Impressively, the weight size of the quantized model was only 13.7 MB, representing an 82.96% reduction compared to the benchmark model. This makes it well-suited for deployment on the edge device, achieving an inference speed of 137.51 FPS on Jetson Xavier NX. In conclusion, the methods proposed in this study show significant potential in monitoring abnormal chicken droppings and can provide an effective reference for the implementation of other agricultural embedded systems.

A Protocol to Depict the Proteolytic Processes Using a Combination of Metal-Organic Materials (MOMs), Electron Paramagnetic Resonance (EPR), and Mass Spectrometry (MS).

Proteolysis is a critical biochemical process yet a challenging field to study experimentally due to the self-degradation of a protease and the complex, dynamic degradation steps of a substrate. Mass spectrometry (MS) is the traditional way for proteolytic studies, yet it is challenging when time-resolved, step-by-step details of the degradation process are needed. We recently found a way to resolve the cleavage site, preference/selectivity of cleavage regions, and proteolytic kinetics by combining site-directed spin labeling (SDSL) of protein substrate, time-resolved two-dimensional (2D) electron paramagnetic resonance (EPR) spectroscopy, protease immobilization via metal-organic materials (MOMs), and MS. The method has been demonstrated on a model substrate and protease, yet there is a lack of details on the practical operations to carry out our strategy. Thus, this protocol summarizes the key steps and considerations when carrying out the EPR/MS study on proteolytic processes, which can be generalized to study other protein/polypeptide substrates in proteolysis. Details for the experimental operation and cautions of each step are reported with figures illustrating the concepts. This protocol provides an effective approach to understanding the proteolytic process with the advantages of offering time-resolved, residue-level resolution of structural basis underlying the process. Such information is important for revealing the cleavage site and proteolytic mechanisms of unknown proteases. The advantage of EPR, probing the target substrate regardless of the complexities caused by the proteases and their self-degradation, offers a practically effective, rapid, and easy-to-operate approach to studying proteolysis. Key features • Combining protease immobilization, EPR, spin labeling, and MS experimental methods allows for the analysis of proteolysis process in real time. • Reveals cleavage site, kinetics of product generation, and preference of cleavage regions via time-resolved SDSL-EPR. • MS confirms EPR findings and helps depict the sequences and populations of the cleaved segments in real time. • The demonstrated method can be generalized to other proteins or polypeptide substrates upon proteolysis by other proteases.