Mean F1 Score Research Articles

COMA-Net: Towards generalized medical image segmentation using complementary attention guided bipolar refinement modules

Precise medical image segmentation is a crucial step for proper isolation of target regions, such as an organ or lesion for accurate medical diagnosis, prognosis and certain medical procedures. Taking advantage of the available annotated medical image datasets, many CNN-based approaches have been proposed for segmentation over the years. These conventional approaches lack appropriate supervised means to enhance the foreground/target regions relative to background at the feature level for improving the generalizability of these methods to obtain better performance across diverse imaging modalities. In this work, we introduce COMA-Net (COMplementary Attention guided bipolar refinement-based Network), which employs a complementary attention scheme between a pair of positive and negative refinement modules placed on top of two encoder structures for generating refined feature references for the decoder stage in a supervised manner. A four-way feature shifting operation is introduced in conjunction with a set of dilated convolutional layers so that it considers the spatial relationships across a wider footprint leading to better contextual feature extraction. We also formulate a novel Foreground-to-Background Ratio (FBR) index to highlight the differences in signal power between target region and background due to the refinement. Experimental results on five different publicly available medical image segmentation datasets, including BUSI, GLAS, ISIC-2018, MoNuSeg and CVC-ClinicDB reveal that on average, the proposed method can achieve an additional mean F1, IoU, precision, and recall score of +0.97%, +1.25%, +1.11%, and +0.22%, respectively over the state-of-the-art segmentation methods, suggesting its great potential for application on real-world patient image data.

Bi-Objective Crop Mapping from Sentinel-2 Images Based on Multiple Deep Learning Networks

Accurate assessment of the extent of crop distribution and mapping different crop types are essential for monitoring and managing modern agriculture. Medium and high spatial resolution remote sensing (RS) for Earth observation and deep learning (DL) constitute one of the most major and effective tools for crop mapping. In this study, we used high-resolution Sentinel-2 imagery from Google Earth Engine (GEE) to map paddy rice and winter wheat in the Bengbu city of Anhui Province, China. We compared the performance of different popular DL backbone networks with the traditional machine learning (ML) methods, including HRNet, MobileNet, Xception, and Swin Transformer, within the improved DeepLabv3+ architecture, Segformer and random forest (RF). The results showed that the Segformer based on the combination of the Transformer architecture encoder and the lightweight multilayer perceptron (MLP) decoder achieved an overall accuracy (OA) value of 91.06%, a mean F1 Score (mF1) value of 89.26% and a mean Intersection over Union (mIoU) value of 80.70%. The Segformer outperformed other DL methods by combining the results of multiple evaluation metrics. Except for Swin Transformer, which was slightly lower than RF in OA, all DL methods significantly outperformed RF methods in accuracy for the main mapping objects, with mIoU improving by about 13.5~26%. The predicted images of paddy rice and winter wheat from the Segformer were characterized by high mapping accuracy, clear field edges, distinct detail features and a low false classification rate. Consequently, DL is an efficient option for fast and accurate mapping of paddy rice and winter wheat based on RS imagery.

Merging and imputation of flow cytometry data: a critical assessment.

Although most modern techniques and analysis methods in multiparameter flow cytometry (MFC) allow for increased dimensionality for the characterization and quantification of cell populations, most MFC applications depend on flow cytometers measuring relatively small (<16) numbers of parameters. When more markers than the available parameters need to be acquired, these are commonly distributed over multiple independent measurements that include a backbone of common markers. Several methods have been proposed to impute values for combinations of markers that were not measured simultaneously. These imputation methods are frequently used without proper validation and knowledge of their effects on data analysis. We evaluated the performance of existing imputation software (Infinicyt, CyTOFmerge, CytoBackBone, and cyCombine) in approximating known measured expression data in terms of similarity in visual appearance, cell expression, and gating in different datasets by splitting MFC samples into separate measurements with partially overlapping markers and re-calculating missing marker expression. Out of the assessed packages, CyTOFmerge showed the most accurate approximation of the known expression in terms of similar expression values and concordance with manual gating, with a mean F-score between 0.53-0.87 when retrieving cell populations in different datasets. Performance remained inadequate for all methods, with only limited similarity at the cell level. In conclusion, the use of imputed MFC data should take such limitations into account and include independent validation of results to justify conclusions. This article is protected by copyright. All rights reserved.

Proceeding the categorization of microplastics through deep learning-based image segmentation

Microplastics (MPs) have been recognized as prominent anthropogenic pollutants that inflict significant harm to marine ecosystems. Various approaches have been proposed to mitigate the risks posed by MPs. Gaining an understanding of the morphology of plastic particles can provide valuable insights into the source and their interaction with marine organisms, which can assist the development of response measures. In this study, we present an automated technique for identifying MPs through segmentation of MPs in microscopic images using a deep convolutional neural network (DCNN) based on a shape classification nomenclature framework. We used MP images from diverse samples to train a Mask Region Convolutional Neural Network (Mask R-CNN) based model for classification. Erosion and dilation operations were added to the model to improve segmentation results. On the testing dataset, the mean F1-score (F1) of segmentation and shape classification was 0.7601 and 0.617, respectively. These results demonstrate the potential of proposed method for the automatic segmentation and shape classification of MPs. Furthermore, by adopting a specific nomenclature, our approach represents a practical step towards the global standardization of MPs categorization criteria. This work also identifies future research directions to improve accuracy and further explore the possibilities of using DCNN for MPs identification.

ATL-BP: A Student Engagement Dataset and Model for Affect Transfer Learning for Behavior Prediction

We propose a video-based transfer learning approach for predicting problem outcomes of students working with an intelligent tutoring system (ITS) by analyzing their faces and gestures. The ability to predict such outcomes enables tutoring systems to adjust interventions and ultimately yield improved student learning. We collected and released a labeled dataset of 2,749 problem-solving interaction samples of 54 students working with an intelligent online math tutor. Our transfer-learning challenge was then to design a representation in the source domain of images obtained from the Internet for facial expression analysis, and transfer this learned representation for human behavior prediction in the domain of webcam videos of students in a classroom environment. We developed a novel facial affect representation and a user-personalized training scheme that unlocks the potential of this representation. We designed several variants of a recurrent neural network that models the temporal structure of video sequences. Our final model, named ATL-BP for Affect Transfer Learning for Behavior Prediction, achieves a relative increase in the mean F-score of 50% over the state-of-the-art method on this new dataset. We also propose an additional set of annotations to predict students’ engagement while solving a specific problem, and present models that can predict such engagement.

Batch-balanced focal loss: a hybrid solution to class imbalance in deep learning.

To validate the effectiveness of an approach called batch-balanced focal loss (BBFL) in enhancing convolutional neural network (CNN) classification performance on imbalanced datasets. BBFL combines two strategies to tackle class imbalance: (1)batch-balancing to equalize model learning of class samples and (2) focal loss to add hard-sample importance to the learning gradient. BBFL was validated on two imbalanced fundus image datasets: a binary retinal nerve fiber layer defect (RNFLD) dataset () and a multiclass glaucoma dataset (). BBFL was compared to several imbalanced learning techniques, including random oversampling (ROS), cost-sensitive learning, and thresholding, based on three state-of-the-art CNNs. Accuracy, F1-score, and the area under the receiver operator characteristic curve (AUC) were used as the performance metrics for binary classification. Mean accuracy and mean F1-score were used for multiclass classification. Confusion matrices, t-distributed neighbor embedding plots, and GradCAM were used for the visual assessment of performance. In binary classification of RNFLD, BBFL with InceptionV3 (93.0% accuracy, 84.7% F1, 0.971 AUC) outperformed ROS (92.6% accuracy, 83.7% F1, 0.964 AUC), cost-sensitive learning (92.5% accuracy, 83.8% F1, 0.962 AUC), and thresholding (91.9% accuracy, 83.0% F1, 0.962 AUC) and others. In multiclass classification of glaucoma, BBFL with MobileNetV2 (79.7% accuracy, 69.6% average F1 score) outperformed ROS (76.8% accuracy, 64.7% F1), cost-sensitive learning (78.3% accuracy, 67.8.8% F1), and random undersampling (76.5% accuracy, 66.5% F1). The BBFL-based learning method can improve the performance of a CNN model in both binary and multiclass disease classification when the data are imbalanced.

Color-patterned fabric defect detection based on the improved YOLOv5s model

The complex background pattern of color-patterned fabrics, the small target of some defects, the difficulty separating them from the background, and the extreme aspect ratio present challenges for their automated, real-time detection. To solve the above problems, the YOLOv5s-based color-patterned fabric defect detection algorithm was proposed by combining lightweight modules. For the small target defects of color-patterned fabrics, coordinate attention was introduced in the feature extraction part to guide the model to focus entirely on the target defect area and suppress the background noise of color-patterned fabrics. Meanwhile, the bidirectional feature pyramid network was introduced in the feature fusion part to give different fusion weights to the extracted feature maps, improve the efficiency of feature fusion, and guide the model further to distinguish the fabric defects from the color-patterned background. Finally, the k-means algorithm was used to generate anchor boxes for the extreme aspect ratio of fabric defects to improve the training efficiency and accuracy of the model. Self-built datasets were experimented with to verify the improved model’s detection effect. The results show that the improved YOLOv5s model can reach 87.7% and 0.881 in mean average precision and F1 score, which are 2.3% and 0.02 better than the original model, respectively. The detection speed of the improved YOLOv5s model reached 60.24 frames per second (GPU 1660). After deployment on the fabric defect detection platform, the speed of detecting color-patterned fabrics can reach 15 m/min.

Negative Selection by Clustering for Contrastive Learning in Human Activity Recognition

Contrastive learning is an emerging and important self-supervised learning paradigm that has been successfully applied to sensor-based human activity recognition (HAR) because it can achieve competitive performance relative to supervised learning. Contrastive learning methods generally involve instance discrimination, which means that the instances are regarded as negatives of each other, and thus their representations are pulled away from each other during the training process. However, instance discrimination could cause overclustering, meaning that the representations of instances from the same class could be overly separated. To alleviate this overclustering phenomenon, we propose a new contrastive learning framework to select negatives by clustering in HAR, which is named ClusterCLHAR. First, ClusterCLHAR clusters the instance representations, and for each instance, only those from different clusters are regarded as negatives. Second, a new contrastive loss function is proposed to mask the same-cluster instances from the negative pairs. We evaluate ClusterCLHAR on three popular benchmark datasets, USC-HAD, MotionSense, and UCI-HAR, using the mean F1-score as an evaluation metric for downstream tasks. The experimental results show that ClusterCLHAR outperforms all the state-of-the-art methods applied to HAR in self-supervised learning and semi-supervised learning.

Attention-aided semantic segmentation network for weed identification in pineapple field

Based on Unmanned aerial vehicle (UAV) platforms, targeted weed management has become the mainstream method to address the harm caused by weeds in farmland, and accurate weed area identification is an important prerequisite for this approach. In this paper, we conducted research on the existing issues in weed recognition based on PSPNet. To enhance the feature extraction ability of the model and improve its performance in complex field environments, we explored the effects of three types of attention modules when inserted into different positions in the network. The experimental results showed that the best performance was achieved when the ECA module was inserted into the SPP layer of the network. Additionally, to improve the utilization of limited data and enhance model performance, we designed a semi-supervised semantic segmentation method and conducted ablation experiments to determine the optimal network parameters and verify their effectiveness. Finally, we combined these two comprehensive optimization strategies and tested them on a weed dataset. The results show that the mean pixel accuracy (MPA) and F-score of the improved weed identification model are 88.66 % and 87.79 %, 3.82 % and 2.74 % higher than that of the original network while effectively increasing prediction accuracy. This study can provide a practical reference for precise weed distribution mapping and subsequent implementation of adaptive spraying on UAV platforms.

Scope of Pre-trained Language Models for Detecting Conflicting Health Information

An increasing number of people now rely on online platforms to meet their health information needs. Thus identifying inconsistent or conflicting textual health information has become a safety-critical task. Health advice data poses a unique challenge where information that is accurate in the context of one diagnosis can be conflicting in the context of another. For example, people suffering from diabetes and hypertension often receive conflicting health advice on diet. This motivates the need for technologies which can provide contextualized, user-specific health advice. A crucial step towards contextualized advice is the ability to compare health advice statements and detect if and how they are conflicting. This is the task of health conflict detection (HCD). Given two pieces of health advice, the goal of HCD is to detect and categorize the type of conflict. It is a challenging task, as (i) automatically identifying and categorizing conflicts requires a deeper understanding of the semantics of the text, and (ii) the amount of available data is quite limited. In this study, we are the first to explore HCD in the context of pre-trained language models. We find that DeBERTa-v3 performs best with a mean F1 score of 0.68 across all experiments. We additionally investigate the challenges posed by different conflict types and how synthetic data improves a model's understanding of conflict-specific semantics. Finally, we highlight the difficulty in collecting real health conflicts and propose a human-in-the-loop synthetic data augmentation approach to expand existing HCD datasets. Our HCD training dataset is over 2x bigger than the existing HCD dataset and is made publicly available on Github.

Prediction of Safety Risk Levels of Benzopyrene Residues in Edible Oils in China Based on the Variable-Weight Combined LSTM-XGBoost Prediction Model.

To assess and predict the food safety risk of benzopyrene (BaP) in edible oils in China, this study collected national sampling data of edible oils from 20 Chinese provinces and their prefectures in 2019, and constructed a risk assessment model of BaP in edible oils with consumption data. Initially, the k-means algorithm was used for risk classification; then the data were pre-processed and trained to predict the data using the Long Short-Term Memory (LSTM) and the eXtreme Gradient Boosting (XGBoost) models, respectively, and finally, the two models were combined using the inverse error method. To test the effectiveness of the prediction model, this study experimentally validated the model according to five evaluation metrics: root mean square error (RMSE), mean absolute error (MAE), precision, recall, and F1 score. The variable-weight combined LSTM-XGBoost prediction model proposed in this paper achieved a precision of 94.62%, and the F1 score value reached 95.16%, which is significantly better than other neural network models; the results demonstrate that the prediction model has certain stability and feasibility. Overall, the combined model used in this study not only improves the accuracy but also enhances the practicality, real-time capabilities, and expandability of the model.

Automatic knowledge extraction from Chinese electronic medical records and rheumatoid arthritis knowledge graph construction.

Knowledge graphs are a powerful tool for organizing knowledge, processing information and integrating scattered information, effectively visualizing the relationships among entities and supporting further intelligent applications. One of the critical tasks in building knowledge graphs is knowledge extraction. The existing knowledge extraction models in the Chinese medical domain usually require high-quality and large-scale manually labeled corpora for model training. In this study, we investigate rheumatoid arthritis (RA)-related Chinese electronic medical records (CEMRs) and address the automatic knowledge extraction task with a small number of annotated samples from CEMRs, from which an authoritative RA knowledge graph is constructed. After constructing the domain ontology of RA and completing manual labeling, we propose the MC-bidirectional encoder representation from transformers-bidirectional long short-term memory-conditional random field (BERT-BiLSTM-CRF) model for the named entity recognition (NER) task and the MC-BERT + feedforward neural network (FFNN) model for the entity extraction task. The pretrained language model (MC-BERT) is trained with many unlabeled medical data and fine-tuned using other medical domain datasets. We apply the established model to automatically label the remaining CEMRs, and then an RA knowledge graph is constructed based on the entities and entity relations, a preliminary assessment is conducted, and an intelligent application is presented. The proposed model achieved better performance than that of other widely used models in knowledge extraction tasks, with mean F1 scores of 92.96% in entity recognition and 95.29% in relation extraction. This study preliminarily confirmed that using a pretrained medical language model could solve the problem that knowledge extraction from CEMRs requires a large number of manual annotations. An RA knowledge graph based on the above identified entities and extracted relations from 1,986 CEMRs was constructed. Experts verified the effectiveness of the constructed RA knowledge graph. In this paper, an RA knowledge graph based on CEMRs was established, the processes of data annotation, automatic knowledge extraction, and knowledge graph construction were described, and a preliminary assessment and an application were presented. The study demonstrated the viability of a pretrained language model combined with a deep neural network for knowledge extraction tasks from CEMRs based on a small number of manually annotated samples.

Deep Learning for Medium-Scale Agricultural Crop Detection through Aerial View Images

This research project focuses on utilizing two state-of-the-art YOLOv4-based deep learning models, for large-scale agricultural crop detection using Unmanned Aerial Vehicles (UAVs). The objective is to develop an accurate and efficient crop detection system capable of identifying chili crops, eggplant crops, and empty polybags in agricultural fields. Crops detection is important for the development of a robotic vision in maximize the productivity and efficiency in agriculture associate with the development of concept Industry 4.0. This study seek to explore the comparison between YOLOv4 and YOLOv4 tiny model in term of mean average precision (mAP), precision, recall, F1-score, detection time and memory consumption. A custom dataset with 300 images was collected and annotated into total bounding boxes of 23335 with 6969 chili tree, 15402 eggplant tree and 964 empty polybag. The dataset was separated into train, validation and test set with the ratio of 70:20:10. The dataset was trained into YOLOv4 and YOLOv4 tiny with 2000 iterations. The result has shown that the YOLOv4 has the higher mean AP of 91.49% with 244.2mb memory storage consumption while YOLOv4 tiny achieve lower mean AP of 71.83% with 22.4mb. In summary, this research has significated the implementation of deep learning models to perform large-scale agricultural crop detection and can be further develop into automation industrial 4.0 of local agricultural sector.

Developing a Supplementary Diagnostic Tool for Breast Cancer Risk Estimation Using Ensemble Transfer Learning.

Breast cancer is the most prevalent cancer worldwide. Thus, it is necessary to improve the efficiency of the medical workflow of the disease. Therefore, this study aims to develop a supplementary diagnostic tool for radiologists using ensemble transfer learning and digital mammograms. The digital mammograms and their associated information were collected from the department of radiology and pathology at Hospital Universiti Sains Malaysia. Thirteen pre-trained networks were selected and tested in this study. ResNet101V2 and ResNet152 had the highest mean PR-AUC, MobileNetV3Small and ResNet152 had the highest mean precision, ResNet101 had the highest mean F1 score, and ResNet152 and ResNet152V2 had the highest mean Youden J index. Subsequently, three ensemble models were developed using the top three pre-trained networks whose ranking was based on PR-AUC values, precision, and F1 scores. The final ensemble model, which consisted of Resnet101, Resnet152, and ResNet50V2, had a mean precision value, F1 score, and Youden J index of 0.82, 0.68, and 0.12, respectively. Additionally, the final model demonstrated balanced performance across mammographic density. In conclusion, this study demonstrates the good performance of ensemble transfer learning and digital mammograms in breast cancer risk estimation. This model can be utilised as a supplementary diagnostic tool for radiologists, thus reducing their workloads and further improving the medical workflow in the screening and diagnosis of breast cancer.

MortCam: An Artificial Intelligence-aided fish mortality detection and alert system for recirculating aquaculture

Mortality is an important production and fish welfare indicator in aquaculture. Unusual mortality patterns can be associated with abiotic or/and biotic stresses on fish in recirculating aquaculture systems (RAS). Real or near real-time mortality tracking can provide valuable inputs to farm managers, to make informed RAS management decisions and address root causes in an effort to prevent mass mortality events. While traditional systems use infrequent human operator observation and tracking - often in conjunction with an underwater camera - the proposed tool (i.e., ‘MortCam’) augments this approach with Artificial Intelligence (AI) and Internet of Things (IoT) deployed at the Edge to provide round-the-clock mortality monitoring and trigger alerts when mortality thresholds are exceeded. MortCam consists of an imaging sensor integrated with an edge computing device, customized for underwater applications. MortCam was deployed in a 150 m3 circular dual-drain RAS tank at 0.6 m above the bottom drain plate to acquire the imagery data in both ambient and supplemental light conditions. The images were collected every fifteen minutes for 90 days. Acquired images were annotated either as ‘alive’ or ‘dead’ fish and split into training (70 %), validation (20 %), and test (10 %) datasets to train a custom YOLOv7 mortality detection model. The optimized mixed model achieved a mean average precision (mAP) and F1 score of 93.4 % and 0.89, respectively. Additionally, the model performed well in terms of mortality count and was found robust despite changes in the imaging conditions. The model was deployed on the MortCam to achieve round-the-clock autonomous mortality monitoring. The system reliably generated email and text alerts to notify fish production staff of unusual mortality events.

Zero-shot transfer learning for structural health monitoring using generative adversarial networks and spectral mapping

Gathering properly labeled, adequately rich, and case-specific data for successfully training a purely data-driven or hybrid model for structural health monitoring (SHM) applications is a challenging task. We posit that a Transfer Learning (TL) method that utilizes available data in any relevant source domain and directly applies to the target domain through domain adaptation can provide substantial remedies to address this issue. Accordingly, we present a novel TL method that differentiates between the source’s no-damage and damage cases and utilizes a domain adaptation (DA) technique. The DA module transfers the accumulated knowledge in contrasting no-damage and damage cases in the source domain to the target domain, given only the target’s no-damage case. High-dimensional features allow employing signal processing domain knowledge to devise a generalizable DA approach. The Generative Adversarial Network (GAN) architecture is adopted for learning since its optimization process accommodates high-dimensional inputs in a zero-shot setting. At the same time, its training objective conforms seamlessly with the case of no-damage and damage data in SHM since its discriminator network differentiates between real (no-damage) and fake (possibly unseen damage) data. An extensive set of experimental results demonstrates the method’s success in transferring knowledge on differences between no-damage and damage cases across three strongly heterogeneous independent target structures. The area under the Receiver Operating Characteristics curves (Area Under the Curve — AUC) is used to evaluate the differentiation between no-damage and damage cases in the target domain, reaching values as high as 0.95. With no-damage and damage cases discerned from each other, zero-shot structural damage detection is carried out. The mean F1 scores for all damages in the three independent datasets are 0.971, 0.986, and 0.975. The success of the proposed TL approach is expected to pave the way for further improvements in the accuracy and generalizability of data-driven SHM applications, thereby offering new possibilities for large-scale SHM applications in urban settings.

Development and Validation of a Radiomics Model Based on 3-Dimensional Endoanal Rectal Ultrasound of Rectal Cancer for Predicting Lymph Node Metastasis.

Development of a radiomics model for predicting lymph node metastasis status in rectal cancer patients based on 3-dimensional endoanal rectal ultrasound images. This study retrospectively included 79 patients (41 with lymph node metastasis positive and 38 with lymph node metastasis negative) diagnosed with rectal cancer in our hospital from January 2018 to February 2022. The tumor's region of interest is first delineated by radiologists, from which radiomics features are extracted. Radiomics features were then selected by independent samples t-test, correlation coefficient analysis between features, and least absolute shrinkage and regression with selection operator. Finally, a multilayer neural network model is developed using the selected radiomics features, and nested cross-validation is performed on it. These models were validated by assessing their diagnostic performance and comparing the areas under the curve and recall rate curve in the test set. The areas under the curve of radiologist was 0.662 and the F1 score was 0.632. Thirty-four radiomics features were significantly associated with lymph node metastasis (P < .05), and 10 features were finally selected for developing multilayer neural network models. The areas under the curve of the multilayer neural network models were 0.787, 0.761, 0.853, and the mean areas under the curve was 0.800. The F1 scores of the multilayer neural network models were 0.738, 0.740, and 0.818, and the mean F1 score was 0.771. Radiomics models based on 3-dimensional endoanal rectal ultrasound can be used to identify lymph node metastasis status in rectal cancer patient with good diagnostic performance.

Automatic Branch-Leaf Segmentation and Leaf Phenotypic Parameter Estimation of Pear Trees Based on Three-Dimensional Point Clouds.

The leaf phenotypic traits of plants have a significant impact on the efficiency of canopy photosynthesis. However, traditional methods such as destructive sampling will hinder the continuous monitoring of plant growth, while manual measurements in the field are both time-consuming and laborious. Nondestructive and accurate measurements of leaf phenotypic parameters can be achieved through the use of 3D canopy models and object segmentation techniques. This paper proposed an automatic branch-leaf segmentation pipeline based on lidar point cloud and conducted the automatic measurement of leaf inclination angle, length, width, and area, using pear canopy as an example. Firstly, a three-dimensional model using a lidar point cloud was established using SCENE software. Next, 305 pear tree branches were manually divided into branch points and leaf points, and 45 branch samples were selected as test data. Leaf points were further marked as 572 leaf instances on these test data. The PointNet++ model was used, with 260 point clouds as training input to carry out semantic segmentation of branches and leaves. Using the leaf point clouds in the test dataset as input, a single leaf instance was extracted by means of a mean shift clustering algorithm. Finally, based on the single leaf point cloud, the leaf inclination angle was calculated by plane fitting, while the leaf length, width, and area were calculated by midrib fitting and triangulation. The semantic segmentation model was tested on 45 branches, with a mean Precisionsem, mean Recallsem, mean F1-score, and mean Intersection over Union (IoU) of branches and leaves of 0.93, 0.94, 0.93, and 0.88, respectively. For single leaf extraction, the Precisionins, Recallins, and mean coverage (mCoV) were 0.89, 0.92, and 0.87, respectively. Using the proposed method, the estimated leaf inclination, length, width, and area of pear leaves showed a high correlation with manual measurements, with correlation coefficients of 0.94 (root mean squared error: 4.44°), 0.94 (root mean squared error: 0.43 cm), 0.91 (root mean squared error: 0.39 cm), and 0.93 (root mean squared error: 5.21 cm2), respectively. These results demonstrate that the method can automatically and accurately measure the phenotypic parameters of pear leaves. This has great significance for monitoring pear tree growth, simulating canopy photosynthesis, and optimizing orchard management.

Ensuring privacy protection in the era of big laparoscopic video data: development and validation of an inside outside discrimination algorithm (IODA)

BackgroundLaparoscopic videos are increasingly being used for surgical artificial intelligence (AI) and big data analysis. The purpose of this study was to ensure data privacy in video recordings of laparoscopic surgery by censoring extraabdominal parts. An inside-outside-discrimination algorithm (IODA) was developed to ensure privacy protection while maximizing the remaining video data.MethodsIODAs neural network architecture was based on a pretrained AlexNet augmented with a long-short-term-memory. The data set for algorithm training and testing contained a total of 100 laparoscopic surgery videos of 23 different operations with a total video length of 207 h (124 min ± 100 min per video) resulting in 18,507,217 frames (185,965 ± 149,718 frames per video). Each video frame was tagged either as abdominal cavity, trocar, operation site, outside for cleaning, or translucent trocar. For algorithm testing, a stratified fivefold cross-validation was used.ResultsThe distribution of annotated classes were abdominal cavity 81.39%, trocar 1.39%, outside operation site 16.07%, outside for cleaning 1.08%, and translucent trocar 0.07%. Algorithm training on binary or all five classes showed similar excellent results for classifying outside frames with a mean F1-score of 0.96 ± 0.01 and 0.97 ± 0.01, sensitivity of 0.97 ± 0.02 and 0.0.97 ± 0.01, and a false positive rate of 0.99 ± 0.01 and 0.99 ± 0.01, respectively.ConclusionIODA is able to discriminate between inside and outside with a high certainty. In particular, only a few outside frames are misclassified as inside and therefore at risk for privacy breach. The anonymized videos can be used for multi-centric development of surgical AI, quality management or educational purposes. In contrast to expensive commercial solutions, IODA is made open source and can be improved by the scientific community.

A novel attention-based enhancement framework for face mask detection in complicated scenarios

In the context of COVID-19 pandemic prevention and control, it is of vital significance to realize accurate face mask detection via computer vision technique. In this paper, a novel attention improved Yolo (AI-Yolo) model is proposed, which can handle existing challenges in the complicated real-world scenarios with dense distribution, small-size object detection and interference of similar occlusions. In particular, a selective kernel (SK) module is set to achieve convolution domain soft attention mechanism with split, fusion and selection operations; a spatial pyramid pooling (SPP) module is applied to enhance the expression of local and global features, which enriches the receptive field information; and a feature fusion (FF) module is utilized to promote sufficient fusions of multi-scale features from each resolution branch, which adopts basic convolution operators without excessive computational complexity. In addition, the complete intersection over union (CIoU) loss function is adopted in the training stage for accurate positioning. Experiments are carried out on two challenging public face mask detection datasets, and the results demonstrate the superiority of the proposed AI-Yolo against other seven state-of-the-art object detection algorithms, which achieves the best results in terms of mean average precision and F1 score on both datasets. Furthermore, effectiveness of the meticulously designed modules in AI-Yolo is validated through extensive ablation studies. In a word, the proposed AI-Yolo is competent to accomplish face mask detection tasks under extremely complex situations with precise localization and accurate classification.