Segmentation Task Research Articles

Visual nutrition analysis: leveraging segmentation and regression for food nutrient estimation

IntroductionNutrition is closely related to body health. A reasonable diet structure not only meets the body’s needs for various nutrients but also effectively prevents many chronic diseases. However, due to the general lack of systematic nutritional knowledge, people often find it difficult to accurately assess the nutritional content of food. In this context, image-based nutritional evaluation technology can provide significant assistance. Therefore, we are dedicated to directly predicting the nutritional content of dishes through images. Currently, most related research focuses on estimating the volume or area of food through image segmentation tasks and then calculating its nutritional content based on the food category. However, this method often lacks real nutritional content labels as a reference, making it difficult to ensure the accuracy of the predictions.MethodsTo address this issue, we combined segmentation and regression tasks and used the Nutrition5k dataset, which contains detailed nutritional content labels but no segmentation labels, for manual segmentation annotation. Based on these annotated data, we developed a nutritional content prediction model that performs segmentation first and regression afterward. Specifically, we first applied the UNet model to segment the food, then used a backbone network to extract features, and enhanced the feature expression capability through the Squeeze-and-Excitation structure. Finally, the extracted features were processed through several fully connected layers to obtain predictions for the weight, calories, fat, carbohydrates, and protein content.Results and discussionOur model achieved an outstanding average percentage mean absolute error (PMAE) of 17.06% for these components. All manually annotated segmentation labels can be found at https://doi.org/10.6084/m9.figshare.26252048.v1.

Low-Resource Chinese Named Entity Recognition via CNN-based Multitask Learning

Named entity recognition (NER) is a fundamental subtask for information extraction that aims to locate and classify named entities in unstructured text into predefined categories. Recently, large-scale language models (LLMs) have achieved SOTA performance on a variety of natural language processing tasks. However, because NER is a sequence labeling task in nature while LLMs is a text-generation model, the performance of LLMs on NER is still significantly below supervised baselines, and NER remains a difficult task. Meanwhile, the word boundary and semantic information of Chinese words are usually quite vague, as words contained in Chinese texts are not separated by spaces. Thus, the NER task still requires supervised learning paradigm and heavily relies on large amounts of labeled data, such as entity type and boundary information. However, the cost of labeling data can be prohibitively large, and the purely supervised approaches usually suffer from poor generalization capability. In this article, we propose a multitask learning-based bidirectional iterated dilated convolution model, BCNN-CWS, for low-resource NER via leveraging word boundary information of Chinese word segmentation (CWS) task. Specifically, to efficiently recognize named entities, an iterated dilated convolutional model with a limited number of layers is implemented. In addition, a bidirectional causal convolution mechanism is presented for contextual information extraction. Results of extensive experiments on public Chinese datasets demonstrate that BCNN-CWS achieves superior performance over state-of-the-art models, and it yields up to about 50% speed improvement over existing methods. It is worth noting that BCNN-CWS can be further improved by combining with a pretrained model. Received: 25 Spetember 2024 | Revised: 4 November 2024 | Accepted: 28 November 2024 Conflicts of Interest The authors declare that they have no conflicts of interest to this work. Data Availability Statement The data that support the findings of this study are openly available in GitLab at https://github.com/jiangfeng13/BCNN-CWS Author Contribution Statement Tao Wu: Conceptualization, Methodology, Writing – original draft, Writing – review & editing, Visualization, Supervision. Xinwen Cao: Resources, Data curation. Feng Jiang: Software, Validation, Formal analysis, Investigation, Writing – original draft. Canyixing Cui: Data curation, Writing -review & editing. Xuehao Li: Resources. Xingping Xian: Supervision, Project administration, Funding acquisition.

Point-Cloud Instance Segmentation for Spinning Laser Sensors

In this paper, we face the point-cloud segmentation problem for spinning laser sensors from a deep-learning (DL) perspective. Since the sensors natively provide their measurements in a 2D grid, we directly use state-of-the-art models designed for visual information for the segmentation task and then exploit the range information to ensure 3D accuracy. This allows us to effectively address the main challenges of applying DL techniques to point clouds, i.e., lack of structure and increased dimensionality. To the best of our knowledge, this is the first work that faces the 3D segmentation problem from a 2D perspective without explicitly re-projecting 3D point clouds. Moreover, our approach exploits multiple channels available in modern sensors, i.e., range, reflectivity, and ambient illumination. We also introduce a novel data-mining pipeline that enables the annotation of 3D scans without human intervention. Together with this paper, we present a new public dataset with all the data collected for training and evaluating our approach, where point clouds preserve their native sensor structure and where every single measurement contains range, reflectivity, and ambient information, together with its associated 3D point. As experimental results show, our approach achieves state-of-the-art results both in terms of performance and inference time. Additionally, we provide a novel ablation test that analyses the individual and combined contributions of the different channels provided by modern laser sensors.

Aerial Images Segmentation with Graph Neural Network

Abstract. The development of remote sensing platforms and sensors, as well as the improvement of remote data processing tools and methods, create new opportunities for automatic updating of maps. Currently, aerial photographs serve as the main source for automatic map updates due to their accessibility and significant informational value. One of the core elements for image to maps transition is accurate image segmentation. Nowadays, machine learning methods demonstrate the best results in task of image segmentation. At its core, maps represent information about a certain area in a vector form, that not only contains visual information about area, but also reflects some relations between objects in the map. This quality makes a map more convenient for human perception than an aerial photograph (raster image). This study addresses the problem of accurate aerial image segmentation with taking the advantages of using graph neural network as the more adequate model of map structure. We use graph neural network for retrieving semantic and vector information about a captured area from its aerial image. The developed framework at first phase utilizes visual transformer for retrieving deep features from the input aerial image. The graph neural network then performs clustering of the extracted deep features to obtain semantic segmentation of the image. To train and evaluate the developed framework, a special dataset is collected and annotated. It contains more than 10k aerial photographs representing various types of objects taken in different years and seasons. The evaluation results on the created dataset proved the state-of-the-art performance of the developed framework.

Evaluating YOLO Models for Efficient Crack Detection in Concrete Structures Using Transfer Learning

The You Only Look Once (YOLO) network is considered highly suitable for real-time object detection tasks due to its characteristics, such as high speed, single-shot detection, global context awareness, scalability, and adaptability to real-world conditions. This work introduces a comprehensive analysis of various YOLO models for detecting cracks in concrete structures, aiming to assist in the selection of an optimal model for future detection and segmentation tasks. The YOLO models are initially trained on a dataset containing both images with and without cracks, producing a generalized model capable of extracting abstract features beneficial for crack detection. Subsequently, transfer learning is employed using a dataset that reflects real-world conditions, such as occlusions, varying crack sizes, and rotations, to further refine the model. Crack detection in concrete remains challenging due to the wide variation in crack sizes, aspect ratios, and complex backgrounds. To achieve optimal performance, we test different versions of YOLO, a state-of-the-art single-shot detector, and aim to balance inference speed and mean average precision (mAP). Our results indicate that YOLOv10 demonstrates superior performance, achieving a mean average precision (mAP) of 74.52% with an inference time of 19.5 milliseconds per image, making it the most effective among the models tested.

A dual-decoder banded convolutional attention network for bone segmentation in ultrasound images.

Ultrasound (US) has great potential for application in computer-assisted orthopedic surgery (CAOS) due to its non-radiative, cost-effective, and portable traits. However, bone segmentation from low-quality US images has been challenging. Traditional segmentation methods cannot achieve satisfactory results due to their high customization and dependence on bone morphology. Existing deep learning-based methods make it difficult to ensure efficient and accurate segmentation due to the ignorance of prior knowledge of bone features during feature learning. This paper aims to systematically investigate feature extraction and segmentation methodologies of bone US images and then proposes an innovative convolutional neural network to address the need for precise and efficient bone structure extraction in CAOS. This paper has proposed a dual-decoder banded convolutional attention network (BCA-Net), which takes the raw US image as input and simplified U-Net as the baseline network. Multiscale banded convolution kernels are employed internally in the BCA-Net model, leveraging the prior knowledge that bone surfaces in US images are exhibited as bright bands of a few millimeters in width. Additionally, a shared encoder to extract input features and two independent decoders to generate outputs for the bone surface and bone shadow mask are integrated into the BCA-Net model, leveraging the prior knowledge that US bone surfaces manifest low-intensity hollow shadows below. Then, a new task consistency loss is introduced that can utilize inter-task dependency fully and enhance the performance of our model. In the network construction process, a dataset containing 1623 sets of US images was adopted, and a five-fold cross-validation strategy was divided into the training and validation sets for the model's training and validation. Many vital metrics were introduced to comprehensively evaluate the model performance, including overlap, edge distance, area under curve, and model efficiency. Finally, the model performance was subjected to a confidence interval, Tukey's honest significant difference, and Cohen's d statistics at a significance level (5%) to ensure the accuracy and reliability of the obtained findings. The experimental results show that the BCA-Net model performs well in the bone surface segmentation task. Its average Dice coefficient reaches 87.51%, 4.04% higher than U-Net's, proving its superior bone surface segmentation accuracy. Meanwhile, the average distance error is 0.2mm, 0.33mm lower than U-Net's, highlighting its accuracy in detail capture and boundary recognition. Using a confidence distance threshold of 1.02mm, the Dice coefficient of the BCA-Net model exceeds 98%, an improvement of 1.87% over U-Net's, which is highly consistent with manual labeling. The BCA-Net model achieves a statistical significance of p-values<0.05 in the above accuracy comparisons. In addition, the BCA-Net model has a small parameter count (5.58M) and high computational efficiency (35.85 frames per second), further validating its excellent potential in bone surface segmentation tasks. The proposed method achieves excellent performance with high accuracy and efficiency, aligning well with clinical requirements and holding excellent potential for advancing the utilization of US images in CAOS.

Fundus Vascular Segmentation Based on Data Enhancement and Invariant Feature Extraction

Deep neural networks have emerged as the predominant method for medical image segmentation, owing to their robust feature learning capabilities, enabling accurate automatic segmentation of target structures within intricate medical images. Fundus images, being crucial in diagnosing ophthalmic diseases, underscore the importance of effective segmentation techniques. However, fundus vascular images pose challenges due to their high complexity and subtle individual differences, necessitating improvement in existing segmentation methodologies for enhanced disease classification accuracy. This paper introduces a fundus blood vessel segmentation model, employing data augmentation and invariant feature extraction, to systematically tackle the core challenges in medical image processing, particularly in fundus blood vessel segmentation. These challenges include limited source domain samples and inadequate model domain generalization. The model adopts a dual-dimensional strategy. Firstly, it delves into data augmentation technology to enhance the diversity and representativeness of samples within the finite source domain. This is achieved through an image enhancement module based on Fourier transform, mitigating the impact of data scarcity on model training effectiveness. Secondly, the research focuses on interdomain invariant feature extraction, aiming to extract feature representations that consistently characterize fundus blood vessel structure and pathology across different data distributions, thereby enhancing model generalization performance in unfamiliar domains. Specifically, the paper designs a fundus image enhancement module based on Fourier transform in the data augmentation dimension. In the feature extraction dimension, it proposes a normalization module based on uncertainty theory, departing from traditional normalization methods. Experimental results demonstrate the efficacy of the proposed method, showcasing superior generalization performance compared to existing techniques in retinal blood vessel and OD/OC segmentation tasks. Experience validates the model-agnostic nature of the learned strategy, indicating its potential for seamless transferability to other models, thereby offering robust support for advancing medical image segmentation research and applications.

LULC-SegNet: Enhancing Land Use and Land Cover Semantic Segmentation with Denoising Diffusion Feature Fusion

Deep convolutional networks often encounter information bottlenecks when extracting land object features, resulting in critical geometric information loss, which impedes semantic segmentation capabilities in complex geospatial backgrounds. We developed LULC-SegNet, a semantic segmentation network for land use and land cover (LULC), which integrates features from the denoising diffusion probabilistic model (DDPM). This network enhances the clarity of the edge segmentation, detail resolution, and the visualization and accuracy of the contours by delving into the spatial details of the remote sensing images. The LULC-SegNet incorporates DDPM decoder features into the LULC segmentation task, utilizing machine learning clustering algorithms and spatial attention to extract continuous DDPM semantic features. The network addresses the potential loss of spatial details during feature extraction in convolutional neural network (CNN), and the integration of the DDPM features with the CNN feature extraction network improves the accuracy of the segmentation boundaries of the geographical features. Ablation and comparison experiments conducted on the Circum-Tarim Basin Region LULC Dataset demonstrate that the LULC-SegNet improved the LULC semantic segmentation. The LULC-SegNet excels in multiple key performance indicators compared to existing advanced semantic segmentation methods. Specifically, the network achieved remarkable scores of 80.25% in the mean intersection over union (MIOU) and 93.92% in the F1 score, surpassing current technologies. The LULC-SegNet demonstrated an IOU score of 73.67%, particularly in segmenting the small-sample river class. Our method adapts to the complex geophysical characteristics of remote sensing datasets, enhancing the performance of automatic semantic segmentation tasks for land use and land cover changes and making critical advancements.

A foundation model for enhancing magnetic resonance images and downstream segmentation, registration and diagnostic tasks.

In structural magnetic resonance (MR) imaging, motion artefacts, low resolution, imaging noise and variability in acquisition protocols frequently degrade image quality and confound downstream analyses. Here we report a foundation model for the motion correction, resolution enhancement, denoising and harmonization of MR images. Specifically, we trained a tissue-classification neural network to predict tissue labels, which are then leveraged by a 'tissue-aware' enhancement network to generate high-quality MR images. We validated the model's effectiveness on a large and diverse dataset comprising 2,448 deliberately corrupted images and 10,963 images spanning a wide age range (from foetuses to elderly individuals) acquired using a variety of clinical scanners across 19 public datasets. The model consistently outperformed state-of-the-art algorithms in improving the quality of MR images, handling pathological brains with multiple sclerosis or gliomas, generating 7-T-like images from 3 T scans and harmonizing images acquired from different scanners. The high-quality, high-resolution and harmonized images generated by the model can be used to enhance the performance of models for tissue segmentation, registration, diagnosis and other downstream tasks.

SEMPNet: enhancing few-shot remote sensing image semantic segmentation through the integration of the segment anything model

ABSTRACT Few-shot semantic segmentation has attracted increasing attention due to its potential for low dependence on annotated samples. While extensively explored in the computer vision community, these techniques are primarily designed for natural images, resulting in limited generalization to remote sensing images. In contrast to the mostly individual and distinct objects presented in natural images, remote sensing images often feature clustered and regular patterns of objects. To bridge this gap, we propose a novel approach for few-shot remote sensing image semantic segmentation, which takes into account the specific characteristics of remote sensing imagery. Our approach introduces a mask classification pipeline, which initially extracts all independent objects within an image and subsequently assigns specific categories to each object guided by semantic information derived from few support images. To accomplish this, a robust mask extractor is imperative. Fortunately, the impressive segment anything model (SAM) possesses the potential to fulfill this role. Leveraging its remarkable zero-shot segmentation capabilities, we present the SAM-enhanced mask parsing network (SEMPNet), a novel few-shot remote sensing image semantic segmentation model. The method generates a set of masks for each image using SAM, transforming the segmentation task into a mask classification problem. To precisely classify each mask, we calculate pixel-wise correlations between each mask and the support features through cross-image position attention. Finally, a mask parsing module is utilized to decode the correlation maps and generate the segmentation results. The experiments on two remote sensing datasets testify the superiority of our method. Our code will be available at https://github.com/TinyAway/SEMPNet.

Automatic segmentation of pericardial adipose tissue from cardiac MR images via semi-supervised method with difference-guided consistency.

Accurate and automatic segmentation of pericardial adipose tissue (PEAT) in cardiac magnetic resonance (MR) images is essential for the diagnosis and treatment of cardiovascular diseases. Precise segmentation is challenging due to high costs and the need for specialized knowledge, as a large amount of accurately annotated data is required, demanding significant time and medicalresources. In order to reduce the burden of data annotation while maintaining the high accuracy of segmentation tasks, this paper introduces a semi-supervised learning method to solve the limitations of current PEAT segmentationmethods. In this paper, we propose a difference-guided collaborative mean teacher (DCMT) semi-supervised method, designed for the segmentation of PEAT from DCMT consists of two main components: a semi-supervised framework with a difference fusion strategy and a backbone network MCM-UNet using Mamba-CNN mixture (MCM) blocks. The differential fusion strategy effectively utilizes the uncertain areas in unlabeled data, encouraging the model to reach a consensus in predictions across these difficult-to-segment yet information-rich areas. In addition, considering the sparse and scattered distribution of PEAT in cardiac MR images, which makes it challenging to segment, we propose MCM-UNet as the backbone network in our semi-supervised framework. This not only enhances the processing ability of global information, but also accurately captures the detailed local features of the image, which greatly improves the accuracy of PEATsegmentation. Our experiments conducted on the MRPEAT dataset show that our DCMT method outperforms existing state-of-the-art semi-supervised methods in terms of segmentation accuracy. These findings underscore the effectiveness of our approach in handling the specific challenges associated with PEATsegmentation. The DCMT method significantly improves the accuracy of PEAT segmentation in cardiac MR images. By effectively utilizing uncertain areas in the data and enhancing feature capture with the MCM-UNet, our approach demonstrates superior performance and offers a promising solution for semi-supervised learning in medical image segmentation. This method can alleviate the extensive annotation requirements typically necessary for training accurate segmentation models in medicalimaging.

Geodl: An R package for geospatial deep learning semantic segmentation using torch and terra.

Convolutional neural network (CNN)-based deep learning (DL) methods have transformed the analysis of geospatial, Earth observation, and geophysical data due to their ability to model spatial context information at multiple scales. Such methods are especially applicable to pixel-level classification or semantic segmentation tasks. A variety of R packages have been developed for processing and analyzing geospatial data. However, there are currently no packages available for implementing geospatial DL in the R language and data science environment. This paper introduces the geodl R package, which supports pixel-level classification applied to a wide range of geospatial or Earth science data that can be represented as multidimensional arrays where each channel or band holds a predictor variable. geodl is built on the torch package, which supports the implementation of DL using the R and C++ languages without the need for installing a Python/PyTorch environment. This greatly simplifies the software environment needed to implement DL in R. Using geodl, geospatial raster-based data with varying numbers of bands, spatial resolutions, and coordinate reference systems are read and processed using the terra package, which makes use of C++ and allows for processing raster grids that are too large to fit into memory. Training loops are implemented with the luz package. The geodl package provides utility functions for creating raster masks or labels from vector-based geospatial data and image chips and associated masks from larger files and extents. It also defines a torch dataset subclass for geospatial data for use with torch dataloaders. UNet-based models are provided with a variety of optional ancillary modules or modifications. Common assessment metrics (i.e., overall accuracy, class-level recalls or producer's accuracies, class-level precisions or user's accuracies, and class-level F1-scores) are implemented along with a modified version of the unified focal loss framework, which allows for defining a variety of loss metrics using one consistent implementation and set of hyperparameters. Users can assess models using standard geospatial and remote sensing metrics and methods and use trained models to predict to large spatial extents. This paper introduces the geodl workflow, design philosophy, and goals for future development.

Semi-supervised medical image segmentation network based on mutual learning.

Semi-supervised learning provides an effective means to address the challenge of insufficient labeled data in medical image segmentation tasks. However, when a semi-supervised segmentation model is overfitted and exhibits cognitive bias, its performance will deteriorate. Errors stemming from cognitive bias can quickly amplify and become difficult to correct during the training process of neural networks, resulting in the continuous accumulation of erroneous knowledge. To address the issue of error accumulation, a novel learning strategy is required to enhance the accuracy of medical image segmentation. This paper proposes a semi-supervised medical image segmentation network based on mutual learning (MLNet) to alleviate the issue of continuous accumulation of erroneous knowledge. The MLNet adopts a teacher-student network as the backbone framework, training student and teacher networks on labeled data and mutually updating network parameter weights, enabling the two models to learn from each other. Additionally, an image partial exchange algorithm (IPE) as an appropriate perturbation addition method is proposed to reduce the introduction of erroneous information and the disruption to the contextual information of the image. In the 10% labeled experiment on the ACDC dataset, our Dice coefficient reached 89.48%, a 9.28% improvement over the baseline model. In the 10% labeled experiment on the BraTS2019 dataset, the proposed method still performs exceptionally well, achieving 84.56%, surpassing other comparative methods. Compared with other methods, experimental results demonstrate that our approach achieves optimal performance across all metrics, proving its effectiveness and reliability.

A Robust Malaria Cell Detection Framework Using Adaptive and Atrous Convolution-Based Recurrent Mobilenetv2 with Trans-MobileUNet + + -Based Abnormality Segmentation.

The highly contagious malaria disease is spread by the female Anopheles mosquito. This disease results in a patient's death or incapacity to move their muscles, if it is not appropriately identified in the early stages. A Rapid Diagnostic Test (RDT) is a frequently used approach to find malaria cells in red blood cells. However, it might not be able to identify infections with small amounts of samples. In the microscopic detection model, blood stains are placed under a microscope for diagnosing malaria. But accurate diagnosis is hard in this method, particularly in developing nations where the disease is most common. The microscopic detection processes are expensive and time-consuming due to the usage of microscopes. The quality of the blood smears and the availability of a qualified specialist, who is skilled in recognizing the disease, impact the accuracy of malaria detection results. The traditional deep learning-based malaria identification models need more processing power. Therefore, a deep learning-based adaptive method is designed to detect malaria cells through the medical image. Hence, the images are gathered from the standard sites and then fed to the segmentation process. Here, the abnormality segmentation is carried out with the help of a developed Trans-MobileUNet + + (T-MUnet + +) network. Trans-MobileUNet + + captures global context, so it is well-suited for segmentation tasks. The segmented image is applied to the adaptive detection phase where the Adaptive and Atrous Convolution-based Recurrent MobilenetV2 (AA-CRMV2) model is designed for the effective recognition of malaria cells. The efficiency of the designed approach is elevated by optimizing the parameters from the AA-CRMV2 network with the help of the Updated Random Parameter-based Fennec Fox Optimization (URP-FFO) algorithm. Several experimental analyses are evaluated in the implemented model over classical techniques to display their effectualness rate.

MT-SCnet: multi-scale token divided and spatial-channel fusion transformer network for microscopic hyperspectral image segmentation.

Hybrid architectures based on convolutional neural networks and Transformers, effectively captures both the local details and the overall structural context of lesion tissues and cells, achieving highly competitive segmentation results in microscopic hyperspectral image (MHSI) segmentation tasks. However, the fixed tokenization schemes and single-dimensional feature extraction and fusion in existing methods lead to insufficient global feature extraction in hyperspectral pathology images. Base on this, we propose a multi-scale token divided and spatial-channel fusion transformer network (MT-SCnet) for MHSIs segmentation. Specifically, we first designed a Multi-Scale Token Divided module. It divides token at different scale based on mirror padding and promotes information interaction and fusion between different tokens to obtain more representative features for subsequent global feature extraction. Secondly, a novel spatial channel fusion transformer was designed to capture richer features from spatial and channel dimensions, and eliminates the semantic gap between features from different dimensions based on cross-attention fusion block. Additionally, to better restore spatial information, deformable convolutions were introduced in decoder. The Experiments on two MHSI datasets demonstrate that MT-SCnet outperforms the comparison methods. This advance has significant implications for the field of MHSIs segmentation. Our code is freely available at https://github.com/sharycao/MT-SCnet.

Res2U++: Deep learning model for segmentation of ischemic stroke lesions

Stroke is a critical global health issue characterized by the interruption of blood flow to the brain, resulting in a depletion of oxygen and nutrients and causing severe damage or death to brain cells. Early detection and intervention are pivotal in preventing adverse outcomes. Diagnostic imaging modalities like CT and MRI are essential for accurate diagnosis and treatment planning. However, precise segmentation of stroke lesions remains challenging due to variations in appearance, size, location, and intensity within the brain, exacerbated by noise and variability in manual segmentation. To address these challenges, we propose a novel deep neural network model for automatic stroke segmentation. Our model features an encoder–decoder architecture, combining the robust deep and multi-scale feature extraction capabilities of the ResUnet++ encoder with a modified UNet decoder that enhances segmentation accuracy and preserves spatial information effectively. Incorporating the Atrous Spatial Pyramidal Pooling (ASPP) block in the bottleneck strengthens the model, enabling efficient capture of multi-scale contextual information and improving its capability to handle diverse object sizes and shapes in semantic segmentation tasks. We evaluate the model on the Ischemic Stroke Lesion Segmentation Challenge 2015 (SISS 2015) and ISLES 2022 databases. During training, our model achieves a Dice coefficient of 0.85 ± 0.18 on SISS 2015 and 0.85 ± 0.11 on ISLES 2022. For testing on SISS 2015, it achieves 0.80 ± 0.33, surpassing current state-of-the-art results for these datasets.

Liver tumor segmentation method combining multi-axis attention and conditional generative adversarial networks.

In modern medical imaging-assisted therapies, manual annotation is commonly employed for liver and tumor segmentation in abdominal CT images. However, this approach suffers from low efficiency and poor accuracy. With the development of deep learning, automatic liver tumor segmentation algorithms based on neural networks have emerged, for the improvement of the work efficiency. However, existing liver tumor segmentation algorithms still have several limitations: (1) they often encounter the common issue of class imbalance in liver tumor segmentation tasks, where the tumor region is significantly smaller than the normal tissue region, causing models to predict more negative samples and neglect the tumor region; (2) they fail to adequately consider feature fusion between global contexts, leading to the loss of crucial information; (3) they exhibit weak perception of local details such as fuzzy boundaries, irregular shapes, and small lesions, thereby failing to capture important features. To address these issues, we propose a Multi-Axis Attention Conditional Generative Adversarial Network, referred to as MA-cGAN. Firstly, we propose the Multi-Axis attention mechanism (MA) that projects three-dimensional CT images along different axes to extract two-dimensional features. The features from different axes are then fused by using learnable factors to capture key information from different directions. Secondly, the MA is incorporated into a U-shaped segmentation network as the generator to enhance its ability to extract detailed features. Thirdly, a conditional generative adversarial network is built by combining a discriminator and a generator to enhance the stability and accuracy of the generator's segmentation results. The MA-cGAN was trained and tested on the LiTS public dataset for the liver and tumor segmentation challenge. Experimental results show that MA-cGAN improves the Dice coefficient, Hausdorff distance, average surface distance, and other metrics compared to the state-of-the-art segmentation models. The segmented liver and tumor models have clear edges, fewer false positive regions, and are closer to the true labels, which plays an active role in medical adjuvant therapy. The source code with our proposed model are available at https://github.com/jhliao0525/MA-cGAN.git.

UNet-like transformer for 1D soil stratification using cone penetration test and borehole data

Subsurface stratification is crucial for the construction safety of underground projects. The one-dimensional (1D) soil stratification aims at identifying segmentation points that separate soil strata. Current engineering practice mainly requires human judgement, which is time-consuming, labour-intensive, and heavily relies on domain expertise. Other probabilistic methods, such as Bayesian approaches, usually involve complex expressions. With the advent of artificial intelligence, deep learning has emerged as a powerful tool in various domains. The UNet, as a typical convolutional neural network, has been extensively utilized for its superior performance in segmentation tasks, but struggles to capture global and long-range semantic information due to the locality of convolution operations. To realize intelligent and automatic 1D soil stratification, this paper introduces a UNet-like Transformer (ULTra) that integrates multiple data sources, including cone penetration test and borehole data, to incorporate prior knowledge. The architecture features a multi-level Transformer with shifted windows in both the encoder and decoder to extract context features and restore spatial resolution, respectively. Experimental results demonstrate that the ULTra outperforms other UNet variants, particularly in detecting minor textures and local details, underscoring the benefits of integrating Transformers into a standard UNet. Case studies indicate that compared with probabilistic methods, the ULTra enables automatic 1D soil stratification using original exploration data with less human intervention, which is fast, effective, and could be continuously improved through interaction with human knowledge, thus streamlining the intelligent data analysis.

A novel AI model for detecting periapical lesion on CBCT: CBCT-SAM.

Periapical lesions are not always evident on radiographic scans. Sometimes, asymptomatic or initial periapical lesions on cone-beam computed tomography (CBCT) could be missed by inexperienced dentists, especially when the scan has a large field of view and is not for endodontic treatment purposes. Previously, numerous algorithms have been introduced to assist radiographic assessment and diagnosis in the field of endodontics. This study aims to investigate the efficacy of CBCT-SAM, a new artificial intelligence (AI) model, in identifying periapical lesions on CBCT. Model training and validation in this study was performed using 185 CBCT scans with confirmed periapical lesions. Manual segmentation labels were prepared by a trained operator and validated by a maxillofacial radiologist. The diagnostic and segmentation performances of four AI models were evaluated and compared: CBCT-SAM, CBCT-SAM without progressive Prediction Refinement Module (PPR), and two previously developed models: Modified U-Net and PAL-Net. Accuracy was used to evaluated the diagnostic performance of the models, and accuracy, sensitivity, specificity, precision and Dice Similarity Coefficient (DSC) were used to evaluate the models' segmentation performance. CBCT-SAM achieved an average diagnostic accuracy of 98.92% ± 010.37% and an average segmentation accuracy of 99.65% ± 0.66%. The average sensitivity, specificity, precision and DSC were 72.36 ± 21.61%, 99.87% ± 0.11%, 0.73 ± 0.21 and 0.70 ± 0.19. CBCT-SAM and PAL-Net performed significantly better than Modified U-Net in segmentation accuracy (p = 0.023, p = 0.041), sensitivity (p = 0.000, p = 0.002), and DSC (p= 0.001, p= 0.004). There is no significant difference between CBCT-SAM, CBCT-SAM without PPR and PAL-Net. However, with PPR incorporated into the model, CBCT-SAM slightly surpassed PAL-Net in the diagnostic and segmentation tasks. CBCT-SAM is capable of providing expert-level assistance in the identification of periapical lesions on CBCT. The application of artificial intelligence could increase dentists' chairside diagnostic accuracy and efficiency. By assisting radiographic assessment, such as periapical lesions on CBCT, it helps reduce the chance of missed diagnosis by human errors and facilitates early detection and treatment of dental pathologies at the early stage.

Super-resolution reconstruction of wind fields with a swin-transformer-based deep learning framework

Deep learning approaches that allow for the rapid simulation of high-resolution atmospheric turbulence are expected by using the super-resolution (SR) technique. Recently, the shifted window attention mechanism in Swin-Transformer offers a significant improvement compared with the vanilla attention mechanism. This method restricts the attention computation to a local neighborhood, reducing the computational load to a linear relationship with sequence length. However, its original architecture is unsuitable for the SR in turbulence due to the mismatch with classification, detection, and segmentation tasks. In this study, the hierarchical structure is redesigned, and a new relative position representing approach is introduced to facilitate the SR procedures of turbulent wind. The channel-shuffled perceptual loss is integrated into the loss function to guide the update of weight parameters. The experimental cases of idealized two-dimensional turbulent flow and realistic boundary layer wind are employed to validate the performance. The results suggest that the proposed approach remarkably outperformed the previous Super-Resolution Convolutional Neural Network, Deep Statistical Downscaling, and Regional Climate Model Emulator in wind vectors. It exhibits lower values than the other three networks whether in terms of point-wise metrics like mean square error, mean absolute error, mean absolute percentage error, or perceptual metrics, including structural similarity index measure and probability density functions. The reconstructed wind vectors closely match the target high-resolution results. This study will help advance the application of shifted window attention mechanisms in wind field SR.