Input Pixel Research Articles

Bridging Convolutional Neural Networks and Transformers for Efficient Crack Detection in Concrete Building Structures.

Detecting cracks in building structures is an essential practice that ensures safety, promotes longevity, and maintains the economic value of the built environment. In the past, machine learning (ML) and deep learning (DL) techniques have been used to enhance classification accuracy. However, the conventional CNN (convolutional neural network) methods incur high computational costs owing to their extensive number of trainable parameters and tend to extract only high-dimensional shallow features that may not comprehensively represent crack characteristics. We proposed a novel convolution and composite attention transformer network (CCTNet) model to address these issues. CCTNet enhances crack identification by processing more input pixels and combining convolution channel attention with window-based self-attention mechanisms. This dual approach aims to leverage the localized feature extraction capabilities of CNNs with the global contextual understanding afforded by self-attention mechanisms. Additionally, we applied an improved cross-attention module within CCTNet to increase the interaction and integration of features across adjacent windows. The performance of CCTNet on the Historical Building Crack2019, SDTNET2018, and proposed DS3 has a precision of 98.60%, 98.93%, and 99.33%, respectively. Furthermore, the training validation loss of the proposed model is close to zero. In addition, the AUC (area under the curve) is 0.99 and 0.98 for the Historical Building Crack2019 and SDTNET2018, respectively. CCTNet not only outperforms existing methodologies but also sets a new standard for the accurate, efficient, and reliable detection of cracks in building structures.

An efficient blur kernel estimation method for blind image Super-Resolution

Many existing space-variant blind image Super-Resolution (SR) methods require the generation of blur kernels for all input pixels. However, the existing methods overlook the possibility of similar blur kernel patterns among adjacent pixels, leading to spatial incoherence and wasteful computational resources. In this study, we introduce an efficient two-stage method for estimating blur kernels. Instead of generating pixel-wise kernels, a limited set of kernels at pixels with distinct information are generated first and the remaining kernels are reconstructed based on this initial set. This novel method, referred to as Anchor Detection and Kernel Reconstruction, comprises two main components: an Anchor Detection Module (ADM) and a Kernel Reconstruction Module (KRM). The objective of ADM is to identify pixels in a given Low-Resolution image that contain rich information, which are called anchors. The corresponding blur kernels, denoted as anchor kernels, are then generated for these identified pixels using a complete backbone network. The remaining blur kernels are reconstructed using KRM with a lightweight interpolation method based on the anchor kernels to enhance spatial consistency among the reconstructed pixels. Extensive experiments demonstrate that the proposed ADKR method maintains comparable performance while estimating only 50% of blur kernels with a full backbone network, reaching approximately 20% reduction in FLOPs. The code has been made available at https://github.com/xuyimin0926/ADKR.

Approximate bilateral filters for real-time and low-energy imaging applications on FPGAs

Bilateral filtering is an image processing technique commonly adopted as intermediate step of several computer vision tasks. Opposite to the conventional image filtering, which is based on convolving the input pixels with a static kernel, the bilateral filtering computes its weights on the fly according to the current pixel values and some tuning parameters. Such additional elaborations involve nonlinear weighted averaging operations, which make difficult the deployment of bilateral filtering within existing vision technologies based on real-time and low-energy hardware architectures. This paper presents a new approximation strategy that aims to improve the energy efficiency of circuits implementing the bilateral filtering function, while preserving their real-time performances and elaboration accuracy. In contrast to the state-of-the-art, the proposed technique allows the filtering action to be on the fly adapted to both the current pixel values and to the tuning parameters, thus avoiding any architectural modification or tables update. When hardware implemented within the Xilinx Zynq XC7Z020 FPGA device, a 5 × 5 filter based on the proposed method processes 237.6 Mega pixels per second and consumes just 0.92 nJ per pixel, thus improving the energy efficiency by up to 2.8 times over the competitors. The impact of the proposed approximation on three different imaging applications has been also evaluated. Experiments demonstrate reasonable accuracy penalties over the accurate counterparts.

Plant Leaf Disease Detection using CNN

Abstract— Plants and crops that are infected by pests have an impact on the country's agricultural production. Usually, farmers or professionals keep a close eye on the plants in order to discover and identify diseases. However, this procedure is frequently time- consuming costly, andimprecise. Plant disease detection can be done by looking for a spot on the diseased plant's leaves. The goal of this paper is to create a Disease Recognition Model that is supported by leaf image classification. To detect plant diseases, we are utilizing image processing with a Convolution neural network (CNN). A convolutional neural network(CNN) is a form of artificial neural network that is specifically intended to process pixel input and is used in image recognition.Farmers do not expertise in leaf disease so they produce less production. Plant leaf diseases detection is the important because profit and loss are depends on production. Index Terms: Image processing,Crops,Support vector Machine,Plant disease,Classification. Keywords— Heart Disease, Diabetes, Machine Learning.

Detecting and counting sorghum aphid alates using smart computer vision models

Sorghum aphid [Melanaphis sorghi (Theobald)] is considered an economic pest causing significant yield losses in susceptible sorghum in the southern U.S. Infestations start with the migration of alates (i.e., winged adults) to sorghum and establishing aphid colonies. In favorable conditions, sorghum aphid can exponentially reproduce via asexual reproduction. A suggested strategy is to monitor alates to determine initial infestations and take preventive strategies, which can result in more efficient pest monitoring and management. To reduce the time of monitoring and better understand of alate establishment under field conditions, we propose using computer vision models, specifically deep learning, to detect and count alates using field-collected images. During pest monitoring, we captured 2527 images and assessed the performance of five models within the YOLOv5 architecture family using two different image sizes, including input resolutions of 640 × 640, and 1280 × 1280 pixels. We trained models to detect and count individual alates, which ranged between 1 and 100 alates/leaf. Among models, the YOLOv5l Pytorch detection model had the best overall performance at 1280 × 1280 input pixel resolution. The YOLOv5l model is a candidate model for quantifying alates on sorghum leaves using deep learning with a precision of 83.80%, 85.60% recall, and 89% mAP@0.5 with a lower mean percent error of misdetection. To enable the use of our best deep learning model by the research community, we developed a web-based application that is freely available to the public. Using this application, users can upload images to detect and count alates with a low error of misdetection.

Explainable artificial intelligence‐driven mask design for self‐supervised seismic denoising

AbstractThe presence of coherent noise in seismic data leads to errors and uncertainties, and as such it is paramount to suppress noise as early and efficiently as possible. Self‐supervised denoising circumvents the common requirement of deep learning procedures of having noisy‐clean training pairs. However, self‐supervised coherent noise suppression methods require extensive knowledge of the noise statistics. We propose the use of explainable artificial intelligence approaches to ‘see inside the black box’ that is the denoising network and use the gained knowledge to replace the need for any prior knowledge of the noise itself. This is achieved in practice by leveraging bias‐free networks and the direct linear link between input and output provided by the associated Jacobian matrix; we show that a simple averaging of the Jacobian contributions over a number of randomly selected input pixels provides an indication of the most effective mask to suppress noise present in the data. The proposed method, therefore, becomes a fully automated denoising procedure requiring no clean training labels or prior knowledge. Realistic synthetic examples with noise signals of varying complexities, ranging from simple time‐correlated noise to complex pseudo‐rig noise propagating at the velocity of the ocean, are used to validate the proposed approach. Its automated nature is highlighted further by an application to two field data sets. Without any substantial pre‐processing or any knowledge of the acquisition environment, the automatically identified blind masks are shown to perform well in suppressing both trace‐wise noise in common shot gathers from the Volve marine data set and coloured noise in post‐stack seismic images from a land seismic survey.

GMConv: Modulating Effective Receptive Fields for Convolutional Kernels.

In convolutional neural networks (CNNs), the convolutions are conventionally performed using a square kernel with a fixed N × N receptive field (RF). However, what matters most to the network is the effective receptive field (ERF), which indicates the extent to which input pixels contribute to an output pixel. Inspired by the property that ERFs typically exhibit a Gaussian distribution, we propose a Gaussian Mask convolutional kernel (GMConv). Specifically, GMConv utilizes the Gaussian function to generate a concentric symmetry mask that is placed over the kernel to refine the RF. We analyze the RFs of CNN kernels in different CNN layers and evaluate our approach through extensive experiments on image classification and object detection tasks. Over several tasks and standard base models, our approach compares favorably against the standard convolution. For instance, using GMConv for AlexNet and ResNet-50, the top-1 accuracy on ImageNet classification is boosted by 0.98% and 0.85% , respectively.

Cloth Interactive Transformer for Virtual Try-On

The 2D image-based virtual try-on has aroused increased interest from the multimedia and computer vision fields due to its enormous commercial value. Nevertheless, most existing image-based virtual try-on approaches directly combine the person-identity representation and the in-shop clothing items without taking their mutual correlations into consideration. Moreover, these methods are commonly established on pure convolutional neural networks (CNNs) architectures which are not simple to capture the long-range correlations among the input pixels. As a result, it generally results in inconsistent results. To alleviate these issues, in this article, we propose a novel two-stage cloth interactive transformer (CIT) method for the virtual try-on task. During the first stage, we design a CIT matching block, aiming at precisely capturing the long-range correlations between the cloth-agnostic person information and the in-shop cloth information. Consequently, it makes the warped in-shop clothing items look more natural in appearance. In the second stage, we put forth a CIT reasoning block for establishing global mutual interactive dependencies among person representation, the warped clothing item, and the corresponding warped cloth mask. The empirical results, based on mutual dependencies, demonstrate that the final try-on results are more realistic. Substantial empirical results on a public fashion dataset illustrate that the suggested CIT attains competitive virtual try-on performance.

Image Inpainting based on Gated Convolution and spectral Normalization

Traditional image inpainting methods based on deep learning have the problem of insufficient discrimination between the missing area and the global area information in the feature extraction of image inpainting tasks because of the characteristics of the model constructed by the convolutional layer. At the same time, the traditional generative adversarial network often has problems such as training difficulties and model collapse in the training process. To solve the above problems and improve the repair effect of the model, this paper proposes a dual discriminator image inpainting model based on generative adversarial network combining gated convolution and spectral normalization. The model is mainly composed of an image inpainting module and an image recognition module. The traditional image inpainting model considers all input pixels as valid pixels when extracting the features of the image to be inpainted, which is unreasonable for the image inpainting task. In order to solve this problem, the gated convolution is designed to replace the role of traditional convolution in the image inpainting module. Gated convolutions address the irrationality of ordinary convolutions in image inpainting tasks by providing learnable dynamic feature selection mechanisms for each channel at each spatial location in all layers. At the same time, generative image inpainting models usually have problems such as training hard mode collapse during the training process. In this paper, we intend to introduce the spectral normalization mechanism in the convolutional layer design of the discriminator module. By introducing Lipschitz continuity constraints from the spectral norm of the parameter matrix of each layer of the neural network, the neural network is better insensitive to input perturbations, making the training process more stable and easier to converge. It solves the problems of mode collapse and model training difficulty in the training of image inpainting model based on generative adversarial network. Finally, qualitative and quantitative experiments show that the image inpainting model based on gated convolution and spectral normalization solves the above problems, and the inpainted image has reasonable texture structure and contextual semantic information.

Invariant Representations Learning with Future Dynamics

High-dimension inputs limit the sample efficiency of deep reinforcement learning, increasing the difficulty of applying it to real-world continuous control tasks, especially in uncertain environments. A good state embedding is crucial for the performance of agents on downstream tasks. The bisimulation metric is an excellent representation learning method that can abstract task-relevant invariant latent embeddings of states based on behavior similarity. However, since only one time-step transition is considered, the features captured by this method we call short-term dynamics. We think long-term dynamics are also important for state representation learning. In this paper, we present Invariant Representations Learning with Future Dynamics (RLF), which uses graph neural networks to learn long-term dynamics and trains the representation network based on a new state metric inspired by bisimulation relation. We experimented with our method on continuous control tasks from DeepMind Control Suite and showed that the RLF can mine more stable embeddings than the state-of-the-art representation learning methods for both state and pixel inputs. The learned policy on top of the embeddings has higher sample efficiency and performance and generalizes well in different tasks.

Mapping landscape ecological patterns using numeric and categorical maps.

The reciprocal relationships between ecological process and landscape pattern are fundamental to landscape ecology. Landscape ecologists traditionally use raster maps portraying classified features such as land use or land cover categories, and metrics suggested by the patch-corridor-matrix conceptual model of pattern. Less attention has been given to the landscape gradient conceptual model and raster maps portraying numeric features such as greenness or percent vegetation cover. We introduce the open-source tool GraySpatCon to calculate and map a variety of landscape pattern metrics from both conceptual models using either categorical or numeric maps. The 51 metrics, drawn mostly from the landscape ecology and image processing literatures, are calculated from the frequencies of input pixel values and/or the pixel value adjacencies in an analysis region. GraySpatCon conducts either a moving window analysis which produces a continuous map of a pattern metric, or a global analysis which produces a single metric value. We describe an implementation in the GuidosToolbox desktop application which allows novice users to interactively explore GraySpatCon functionality. In the R desktop environment, we demonstrate several metrics using an example map of percent tree cover and illustrate a multi-scale moving window analysis to identify scale domains. Comparisons of computational efficiency indicate a substantial GraySpatCon advantage over related software in the R environment.

Abstract 18776: ECG-GPT: Automated Complete Diagnosis Generation From ECG Images Using Novel Vision-Text Transformer Model

Background: Artificial intelligence can automate the diagnosis of a multitude of rhythm and conduction disorders from ECGs, but is limited by a dependency on discretely defined labels. Moreover, most models also frequently rely on signal data, limiting their application in low-resource settings, where such models are needed most and ECGs are available in image form. Methods: 1.4 million 12-lead ECGs and accompanying cardiologist diagnosis statements from Yale were used to develop a model capable of generating ECG report statements. This encompassed a wide array of conditions. To develop a model capable of report generation across ECG layouts, 12-lead signal recordings were plotted randomly in multiple formats. The gold-standard diagnosis statements were processed using to remove identifiable information and standardize diagnosis reporting. An ECG-specific vision encoder-decoder model with a pretrained BEiT transformer encoder with a 384x384 pixel input size and a generative pretrained transformer, GPT-2, as a decoder was developed (B). Results: In a held-out test set of 100,000 ECGs, representing ECG interpretation with up to 255 tokens and over 41 distinct cardiovascular conditions, the model performed well across conventional metrics, with ROUGE-L and BLEU-1 scores of 0.695 and 0.502, respectively (C). The model’s clinical validity was assessed by computing agreement between cardiologist and model-generated diagnosis statements. The model generated text reports with accurate identification of diverse set of rhythm and conduction disorders, with AUROCs for predicting atrial fibrillation, right bundle branch block, and sinus tachycardia of 0.95, 0.94, and 0.96, respectively (D). Conclusions: Our vision encoder-decoder model generates an ECG report from ECG images in any layout. This approach represents an automated and accessible strategy for generating cardiologist-level reporting on ECGs, useful for low-resource settings.

Spatial Downscaling of Near-Surface Air Temperature Based on Deep Learning Cross-Attention Mechanism

Deep learning methods can achieve a finer refinement required for downscaling meteorological elements, but their performance in terms of bias still lags behind physical methods. This paper proposes a statistical downscaling network based on Light-CLDASSD that utilizes a Shuffle–nonlinear-activation-free block (SNBlock) and Swin cross-attention mechanism (SCAM), and is named SNCA-CLDASSD, for the China Meteorological Administration Land Data Assimilation System (CLDAS). This method aims to achieve a more accurate spatial downscaling of a temperature product from 0.05° to 0.01° for the CLDAS. To better utilize the digital elevation model (DEM) for reconstructing the spatial texture of the temperature field, a module named SCAM is introduced, which can activate more input pixels and enable the network to correct and merge the extracted feature maps with DEM information. We chose 90% of the CLDAS temperature data with DEM and station observation data from 2016 to 2020 (excluding 2018) as the training set, 10% as the verification set, and chose the data in 2018 as the test set. We validated the effectiveness of each module through comparative experiments and obtained the best-performing model. Then, we compared it with traditional interpolation methods and state-of-the-art deep learning super-resolution algorithms. We evaluated the experimental results with HRCLDAS, national stations, and regional stations, and the results show that our improved model performs optimally compared to other methods (RMSE of 0.71 °C/0.12 °C/0.72 °C, BIAS of −0.02 °C/0.02 °C/0.002 °C), with the most noticeable improvement in mountainous regions, followed by plains. SNCA-CLDASSDexhibits the most stable performance in intraday hourly bias at temperature under the conditions of improved feature extraction capability in the SNBlock and a better utilization of the DEM by the SCAM. Due to the replacement of the upsampling method from sub pixels to CARAFE, it effectively suppresses the checkerboard effect and shows better robustness than other models. Our approach extends the downscaling model for CLDAS data products and significantly improves performance in this task by enhancing the model’s feature extraction and fusion capabilities and improving upsampling methods. It offers a more profound exploration of historical high-resolution temperature estimation and can be migrated to the downscaling of other meteorological elements.

Development of a Mammography Calcification Detection Algorithm Using Deep Learning with Resolution-Preserved Image Patch Division

Convolutional neural networks (CNNs) in deep learning have input pixel limitations, which leads to lost information regarding microcalcification when mammography images are compressed. Segmenting images into patches retains the original resolution when inputting them into the CNN and allows for identifying the location of calcification. This study aimed to develop a mammographic calcification detection method using deep learning by classifying the presence of calcification in the breast. Using publicly available data, 212 mammograms from 81 women were segmented into 224 × 224-pixel patches, producing 15,049 patches. These were visually classified for calcification and divided into five subsets for training and evaluation using fivefold cross-validation, ensuring image consistency. ResNet18, ResNet50, and ResNet101 were used for training, each of which created a two-class calcification classifier. The ResNet18 classifier achieved an overall accuracy of 96.0%, mammogram accuracy of 95.8%, an area under the curve (AUC) of 0.96, and a processing time of 0.07 s. The results of ResNet50 indicated 96.4% overall accuracy, 96.3% mammogram accuracy, an AUC of 0.96, and a processing time of 0.14 s. The results of ResNet101 indicated 96.3% overall accuracy, 96.1% mammogram accuracy, an AUC of 0.96, and a processing time of 0.20 s. This developed method offers quick, accurate calcification classification and efficient visualization of calcification locations.

CNN-Res: deep learning framework for segmentation of acute ischemic stroke lesions on multimodal MRI images

Background Accurate segmentation of stroke lesions on MRI images is very important for neurologists in the planning of post-stroke care. Segmentation helps clinicians to better diagnose and evaluation of any treatment risks. However, manual segmentation of brain lesions relies on the experience of neurologists and is also a very tedious and time-consuming process. So, in this study, we proposed a novel deep convolutional neural network (CNN-Res) that automatically performs the segmentation of ischemic stroke lesions from multimodal MRIs.MethodsCNN-Res used a U-shaped structure, so the network has encryption and decryption paths. The residual units are embedded in the encoder path. In this model, to reduce gradient descent, the residual units were used, and to extract more complex information in images, multimodal MRI data were applied. In the link between the encryption and decryption subnets, the bottleneck strategy was used, which reduced the number of parameters and training time compared to similar research.ResultsCNN-Res was evaluated on two distinct datasets. First, it was examined on a dataset collected from the Neuroscience Center of Tabriz University of Medical Sciences, where the average Dice coefficient was equal to 85.43%. Then, to compare the efficiency and performance of the model with other similar works, CNN-Res was evaluated on the popular SPES 2015 competition dataset where the average Dice coefficient was 79.23%.ConclusionThis study presented a new and accurate method for the segmentation of MRI medical images using a deep convolutional neural network called CNN-Res, which directly predicts segment maps from raw input pixels.

Improved distinct bone segmentation in upper-body CT through multi-resolution networks

PurposeAutomated distinct bone segmentation from CT scans is widely used in planning and navigation workflows. U-Net variants are known to provide excellent results in supervised semantic segmentation. However, in distinct bone segmentation from upper-body CTs a large field of view and a computationally taxing 3D architecture are required. This leads to low-resolution results lacking detail or localisation errors due to missing spatial context when using high-resolution inputs.MethodsWe propose to solve this problem by using end-to-end trainable segmentation networks that combine several 3D U-Nets working at different resolutions. Our approach, which extends and generalizes HookNet and MRN, captures spatial information at a lower resolution and skips the encoded information to the target network, which operates on smaller high-resolution inputs. We evaluated our proposed architecture against single-resolution networks and performed an ablation study on information concatenation and the number of context networks.ResultsOur proposed best network achieves a median DSC of 0.86 taken over all 125 segmented bone classes and reduces the confusion among similar-looking bones in different locations. These results outperform our previously published 3D U-Net baseline results on the task and distinct bone segmentation results reported by other groups.ConclusionThe presented multi-resolution 3D U-Nets address current shortcomings in bone segmentation from upper-body CT scans by allowing for capturing a larger field of view while avoiding the cubic growth of the input pixels and intermediate computations that quickly outgrow the computational capacities in 3D. The approach thus improves the accuracy and efficiency of distinct bone segmentation from upper-body CT.

Hybrid self-attention NEAT: a novel evolutionary self-attention approach to improve the NEAT algorithm in high dimensional inputs

This article presents a "Hybrid Self-Attention NEAT" method to improve the original NeuroEvolution of Augmenting Topologies (NEAT) algorithm in high-dimensional inputs. Although the NEAT algorithm has shown a significant result in different challenging tasks, as input representations are high dimensional, it cannot create a well-tuned network. Our study addresses this limitation by using self-attention as an indirect encoding method to select the most important parts of the input. In addition, we improve its overall performance with the help of a hybrid method to evolve the final network weights. The main conclusion is that Hybrid Self- Attention NEAT can eliminate the restriction of the original NEAT. The results indicate that in comparison with evolutionary algorithms, our model can get comparable scores in Atari games with raw pixels input with a much lower number of parameters.

Geometric control of thermoformable knitted textiles using raster images

Computer numerical control (CNC) knitting technology offers great potential in the creation of thermoformable textiles that can be shaped and stiffened in response to heat. Our research explores how CNC knitting can be used to design and fabricate textiles with precisely allocated material and microstructure layouts. These layouts pre-program specific deformation mechanism(s) into the textile that bias it to form an intended geometry, forgoing the need for a mould during the thermoforming process. We fabricate these smart textiles by knitting two thermal-reactive yarns with different extents of shrinkage, in a double layered structure akin to a bilayer strip. We develop a computational design-to-fabrication pipeline that translates raster images into machine-knittable instructions. Referencing multi-material additive manufacturing principles and self-actuating textiles, this paper proposes several design strategies of dithered gradients, tessellated patches and origami creases, to convert input pixel data into a material distribution layout. When paired with our assisted thermoforming process, this layout induces specific deformations of the textile, such as uni-/multi-axial curling, periodic buckling and sharp folding. Our prototypes implement these strategies on the micro-, meso- and macroscale, leading to the design and fabrication of an architectural cladding panel (700 × 535 × 110 mm) and a patterned clutch bag (200 × 420 × 65 mm).

A divide and conquer approach to maximise deep learning mammography classification accuracies.

Breast cancer claims 11,400 lives on average every year in the UK, making it one of the deadliest diseases. Mammography is the gold standard for detecting early signs of breast cancer, which can help cure the disease during its early stages. However, incorrect mammography diagnoses are common and may harm patients through unnecessary treatments and operations (or a lack of treatment). Therefore, systems that can learn to detect breast cancer on their own could help reduce the number of incorrect interpretations and missed cases. Various deep learning techniques, which can be used to implement a system that learns how to detect instances of breast cancer in mammograms, are explored throughout this paper. Convolution Neural Networks (CNNs) are used as part of a pipeline based on deep learning techniques. A divide and conquer approach is followed to analyse the effects on performance and efficiency when utilising diverse deep learning techniques such as varying network architectures (VGG19, ResNet50, InceptionV3, DenseNet121, MobileNetV2), class weights, input sizes, image ratios, pre-processing techniques, transfer learning, dropout rates, and types of mammogram projections. This approach serves as a starting point for model development of mammography classification tasks. Practitioners can benefit from this work by using the divide and conquer results to select the most suitable deep learning techniques for their case out-of-the-box, thus reducing the need for extensive exploratory experimentation. Multiple techniques are found to provide accuracy gains relative to a general baseline (VGG19 model using uncropped 512 × 512 pixels input images with a dropout rate of 0.2 and a learning rate of 1 × 10-3) on the Curated Breast Imaging Subset of DDSM (CBIS-DDSM) dataset. These techniques involve transfer learning pre-trained ImagetNet weights to a MobileNetV2 architecture, with pre-trained weights from a binarised version of the mini Mammography Image Analysis Society (mini-MIAS) dataset applied to the fully connected layers of the model, coupled with using weights to alleviate class imbalance, and splitting CBIS-DDSM samples between images of masses and calcifications. Using these techniques, a 5.6% gain in accuracy over the baseline model was accomplished. Other deep learning techniques from the divide and conquer approach, such as larger image sizes, do not yield increased accuracies without the use of image pre-processing techniques such as Gaussian filtering, histogram equalisation and input cropping.

Fully automated imaging protocol independent system for pituitary adenoma segmentation: a convolutional neural network—based model on sparsely annotated MRI

This study aims to develop a fully automated imaging protocol independent system for pituitary adenoma segmentation from magnetic resonance imaging (MRI) scans that can work without user interaction and evaluate its accuracy and utility for clinical applications. We trained two independent artificial neural networks on MRI scans of 394 patients. The scans were acquired according to various imaging protocols over the course of 11 years on 1.5T and 3T MRI systems. The segmentation model assigned a class label to each input pixel (pituitary adenoma, internal carotid artery, normal pituitary gland, background). The slice segmentation model classified slices as clinically relevant (structures of interest in slice) or irrelevant (anterior or posterior to sella turcica). We used MRI data of another 99 patients to evaluate the performance of the model during training. We validated the model on a prospective cohort of 28 patients, Dice coefficients of 0.910, 0.719, and 0.240 for tumour, internal carotid artery, and normal gland labels, respectively, were achieved. The slice selection model achieved 82.5% accuracy, 88.7% sensitivity, 76.7% specificity, and an AUC of 0.904. A human expert rated 71.4% of the segmentation results as accurate, 21.4% as slightly inaccurate, and 7.1% as coarsely inaccurate. Our model achieved good results comparable with recent works of other authors on the largest dataset to date and generalized well for various imaging protocols. We discussed future clinical applications, and their considerations. Models and frameworks for clinical use have yet to be developed and evaluated.