Large Size Images Research Articles

Application of convolutional neural networks towards nuclei segmentation in localization-based super-resolution fluorescence microscopy images

BackgroundAutomated segmentation of nuclei in microscopic images has been conducted to enhance throughput in pathological diagnostics and biological research. Segmentation accuracy and speed has been significantly enhanced with the advent of convolutional neural networks. A barrier in the broad application of neural networks to nuclei segmentation is the necessity to train the network using a set of application specific images and image labels. Previous works have attempted to create broadly trained networks for universal nuclei segmentation; however, such networks do not work on all imaging modalities, and best results are still commonly found when the network is retrained on user specific data. Stochastic optical reconstruction microscopy (STORM) based super-resolution fluorescence microscopy has opened a new avenue to image nuclear architecture at nanoscale resolutions. Due to the large size and discontinuous features typical of super-resolution images, automatic nuclei segmentation can be difficult. In this study, we apply commonly used networks (Mask R-CNN and UNet architectures) towards the task of segmenting super-resolution images of nuclei. First, we assess whether networks broadly trained on conventional fluorescence microscopy datasets can accurately segment super-resolution images. Then, we compare the resultant segmentations with results obtained using networks trained directly on our super-resolution data. We next attempt to optimize and compare segmentation accuracy using three different neural network architectures.ResultsResults indicate that super-resolution images are not broadly compatible with neural networks trained on conventional bright-field or fluorescence microscopy images. When the networks were trained on super-resolution data, however, we attained nuclei segmentation accuracies (F1-Score) in excess of 0.8, comparable to past results found when conducting nuclei segmentation on conventional fluorescence microscopy images. Overall, we achieved the best results utilizing the Mask R-CNN architecture.ConclusionsWe found that convolutional neural networks are powerful tools capable of accurately and quickly segmenting localization-based super-resolution microscopy images of nuclei. While broadly trained and widely applicable segmentation algorithms are desirable for quick use with minimal input, optimal results are still found when the network is both trained and tested on visually similar images. We provide a set of Colab notebooks to disseminate the software into the broad scientific community (https://github.com/YangLiuLab/Super-Resolution-Nuclei-Segmentation).

Multi-Resolution Supervision Network with an Adaptive Weighted Loss for Desert Segmentation

Desert segmentation of remote sensing images is the basis of analysis of desert area. Desert images are usually characterized by large image size, large-scale change, and irregular location distribution of surface objects. The multi-scale fusion method is widely used in the existing deep learning segmentation models to solve the above problems. Based on the idea of multi-scale feature extraction, this paper took the segmentation results of each scale as an independent optimization task and proposed a multi-resolution supervision network (MrsSeg) to further improve the desert segmentation result. Due to the different optimization difficulty of each branch task, we also proposed an auxiliary adaptive weighted loss function (AWL) to automatically optimize the training process. MrsSeg first used a lightweight backbone to extract different-resolution features, then adopted a multi-resolution fusion module to fuse the local information and global information, and finally, a multi-level fusion decoder was used to aggregate and merge the features at different levels to get the desert segmentation result. In this method, each branch loss was treated as an independent task, AWL was proposed to calculate and adjust the weight of each branch. By giving priority to the easy tasks, the improved loss function could effectively improve the convergence speed of the model and the desert segmentation result. The experimental results showed that MrsSeg-AWL effectively improved the learning ability of the model and has faster convergence speed, lower parameter complexity, and more accurate segmentation results.

Efficient Folded Attention for Medical Image Reconstruction and Segmentation

Recently, 3D medical image reconstruction (MIR) and segmentation (MIS) based on deep neural networks have been developed with promising results, and attention mechanism has been further designed for performance enhancement. However, the large size of 3D volume images poses a great computational challenge to traditional attention methods. In this paper, we propose a folded attention (FA) approach to improve the computational efficiency of traditional attention methods on 3D medical images. The main idea is that we apply tensor folding and unfolding operations to construct four small sub-affinity matrices to approximate the original affinity matrix. Through four consecutive sub-attention modules of FA, each element in the feature tensor can aggregate spatial-channel information from all other elements. Compared to traditional attention methods, with the moderate improvement of accuracy, FA can substantially reduce the computational complexity and GPU memory consumption. We demonstrate the superiority of our method on two challenging tasks for 3D MIR and MIS, which are quantitative susceptibility mapping and multiple sclerosis lesion segmentation.

TSDLPP: A Novel Two-Stage Deep Learning Framework For Prognosis Prediction Based on Whole Slide Histopathological Images.

Recently, digital pathology image-based prognosis prediction has become a hot topic in healthcare research to make early decisions on therapy and improve the treatment quality of patients. Therefore, there has been a recent surge of interest in designing deep learning method solving the problem of prognosis prediction with digital pathology images. However, whole slide histopathological images (WSIs) based prognosis prediction is still a challenge due to the large size of pathological images, the heterogeneity of tumors and the high cost of region of interests (ROIs) labeling. In this study, we design a novel two-stage deep learning framework for prognosis prediction (TSDLPP) based on WSIs. Our proposed framework consists of two-stage paradigms: 1) training tissue decomposition network (TDNet) to divide WSIs into cancerous and non-cancerous regions, 2) integrating general prognosis-related densely connected CNN (GPR-DCCNN) and morphology-specific prognosis-related densely connected CNNs (MSPR-DCCNNs) to extract different level features of pathological images. In the end, we apply TSDLPP to the prognosis prediction of breast cancer using The Cancer Genome Atlas (TCGA) datasets. Experiment results demonstrate that TSDLPP obtains superior performance of prognosis prediction compared with the existing state-of-arts methods.

Strongly convex optimization for joint fractal feature estimation and texture segmentation

The present work investigates the segmentation of textures by formulating it as a strongly convex optimization problem, aiming to favor piecewise constancy of fractal features (local variance and local regularity) widely used to model real-world textures in numerous applications very different in nature. Two objective functions combining these two features are compared, referred to as joint and coupled , promoting either independent or co-localized changes in local variance and regularity. To solve the resulting convex nonsmooth optimization problems, because the processing of large size images and databases are targeted, two categories of proximal algorithms (dual forward-backward and primal-dual), are devised and compared. An in-depth study of the objective functions, notably of their strong convexity, memory and computational costs, permits to propose significantly accelerated algorithms. A class of synthetic models of piecewise fractal texture is constructed and studied. They enable, by means of large-scale Monte-Carlo simulations, to quantify the benefits in texture segmentation of combining local regularity and local variance (as opposed to regularity only) while using strong-convexity accelerated primal-dual algorithms. Achieved results also permit to discuss the gains/costs in imposing co-localizations of changes in local regularity and local variance in the problem formulation. Finally, the potential of the proposed approaches is illustrated on real-world textures taken from a publicly available and documented database.

Dual pH- and Glutathione-Responsive CO2-Generating Nanodrug Delivery System for Contrast-Enhanced Ultrasonography and Therapy of Prostate Cancer.

Ultrasonography (US) contrast imaging using US contrast agents has been widely applied for the diagnosis and differential diagnosis of tumors. Commercial US contrast agents have limited applications because of their large size and shorter imaging time. At the same time, the desired therapeutic purpose cannot be achieved by applying only conventional US contrast agents. The development of nanoscale US agents with US imaging and therapeutic functions has attracted increasing attention. In this study, we successfully developed DOX-loaded poly-1,6-hexanedithiol-sodium bicarbonate nanoparticles (DOX@HADT-SS-NaHCO3 NPs) with pH-responsive NaHCO3 and GSH-responsive disulfide linkages. DOX@HADT-SS-NaHCO3 NPs underwent acid-triggered decomposition of NaHCO3 to generate CO2 bubbles and a reduction of disulfide linkages to further promote the release of CO2 and DOX. The potential of DOX@HADT-SS-NaHCO3 NPs for contrast-enhanced US imaging and therapy of prostate cancer was thoroughly evaluated using in vitro agarose gel phantoms and a C4-2 tumor-bearing nude mice model. These polymeric NPs displayed significantly enhanced US contrast at acidic pH and antitumor efficacy. Therefore, the NaHCO3 and DOX-encapsulated polymeric NPs hold tremendous potential for effective US imaging and therapy of prostate cancer.

Image stitching by feature positioning and seam elimination

Parallax, exposure differences, ghost and efficiency handling are the challenging tasks for image stitching, which is regarded as the promising approach to resolve the issues in the tasks. In this paper, we propose a novel stitching method that locates the overlapped regions of the input images, and records the feature points at the same time. The warping of each image is then guided by a mesh interpolation map in a local warp model. We also propose an arc function weight model to eliminate image chromatic aberration. It is proved via the validation cases that our approach shows constantly the better performance than the AutoStitch, APAP, SPHP, ANAP, ELA and many other state-of-the-art methods. Our method can effectively avoid mismatched points, improve the matching efficiency of feature points of large-size images by about 60%, eliminate the color difference seam and ghost of the image, and still have good accuracy and stability in complex scenes.

A multi-instance networks with multiple views for classification of mammograms

Breast cancer is the most common malignant disease in women, and early screening of breast cancer is crucial for improving the survival rate. Mammography is one of the most popular imaging methods for breast cancer screening with the characteristics of practicality, effectiveness, and low cost. However, the classification of mammograms suffers from large image sizes, indistinct image characteristics of lesions, small proportion of abnormalities, and class imbalance. To address these difficulties, the multi-view input and weighted multi-instance learning (MIL) methods are proposed. A novel model called the weighted MIL DenseNet with multi-view input (WMDNet) is presented that integrates the two methods above. The multi-view inputs method is used to enhance the abnormalities of mammograms and obtain more potential features from mammograms with different views, simultaneously. The weighted MIL is designed to extract the most suspicious lesions from mammograms to resolve the problems of small proportion of abnormalities and class imbalance. To verify the effectiveness of the proposed methods, three binary classification models are evaluated on two public datasets, the INbreast and MIAS datasets. The experimental results demonstrate that the proposed methods can achieve preferably results compared with several other state-of-the-art approaches, especially the proposed WMDNet model gains the best classification results on both datasets.

FAST INTERACTIVE REGIONAL PATTERN MERGING FOR GENERIC TISSUE SEGMENTATION IN HISTOPATHOLOGY IMAGES

The image segmentation of histopathological tissue images has always been a challenge due to the overlapping of tissue color distributions, the complexity of extracellular texture and the large image size. In this paper, we introduce a new region-merging algorithm, namely, the Regional Pattern Merging (RPM) for interactive color image segmentation and annotation, by efficiently retrieving and applying the user’s prior knowledge of stroke-based interaction. Low-level color/texture features of each region are used to compose a regional pattern adapted to differentiating a foreground object from the background scene. This iterative region-merging is based on a modified Region Adjacency Graph (RAG) model built from initial segmented results of the mean shift to speed up the merging process. The foreground region of interest (ROI) is segmented by the reduction of the background region and discrimination of uncertain regions. We then compare our method against state-of-the-art interactive image segmentation algorithms in both natural images and histological images. Taking into account the homogeneity of both color and texture, the resulting semi-supervised classification and interactive segmentation capture histological structures more completely than other intensity or color-based methods. Experimental results show that the merging of the RAG model runs in a linear time according to the number of graph edges, which is essentially faster than both traditional graph-based and region-based methods.

A Novel and Efficient Approach for Content- Based Image Retrieval

In content-based image retrieval (CBIR) means the quest of an image continues at the real content material of picture to a positive volume to its metadata. The CBIR System has utilized for the remove the features, indexing individual’s features by inappropriate formation and capably given a response to user’s uncertainty. To provide the significant response to client question, CBIR gave some run of work. A novel scheme is proposed in this research work for color picture indexing by introducing the ease of the SVM system. A novel method is introduced for color picture indexing by exploit for it easy of the SVM system. This feature Extraction was seen as the binary classification problem and SVM was used for the problem’s solution. It is concluded that to attain the extraction at the high speed as well as for making it so flexible that it can also adjust with the images of large size. The objective of this paper is to stand for the importance of maintaining vector machine in the well-organized retrieval of an image in SVM is utilizes as classifier that performing task of classification the image and this process of classification are providing to all picture that are mined later by feature extraction process

MDRSteg: large-capacity image steganography based on multi-scale dilated ResNet and combined chi-square distance loss

Image steganography has emerged as a method of hiding secret data within an image file to ensure the security of the transmitted data. In this study, we propose an architecture named MDRSteg to unobtrusively hide a large-size image in another image based on a residual neural network with dilated convolution and multi-scale fusion. The architecture consists of an embedding network to hide the secret image in the cover-image and a revealing network to reveal the secret image from the stego-image, both networks are made up of fully convolutional residual modules. The networks are jointly trained with a loss function which is the combination of chi-square distance (CSD) and mean-square error. The proposed MDRSteg are trained and tested on three datasets, Labeled Faces in the Wild, Pascal visual object classes, and ImageNet. Extensive experiments have been done and the experimental results suggest that the proposed model can not only hide a large size image in another image with good invisibility and large hiding capacity (24 bits-per-pixel), but also exhibits good generalization ability. The experimental results also show that dilated convolution, multi-scale fusion, and combined CSD loss function have positive effects on the delicate image steganography results and proves that the model is practically useful for many applications.

Performance analysis of a new fractal compression method for medical images based on fixed partition

Applications of image compression have been gaining importance over the years in medical fields. The medical imaging modalities produced a large amount of information on all levels of hospital care. This information, in the form of images, needs to be stored for future references. While compressing the medical images, it is necessary to maintain high diagnostic quality with a high compression rate. Several image compression approaches have been developed towards the direction of the storage space problem in such a way that the doctors accurately and reliably diagnose the patient’s diseases from the reconstructed image after decompression. A high compression ratio is required to reduce the storage space due to the large size of medical images. Fractal image compression is a lossy technique to compress the image in a coded form instead of pixels and is differentiated by its long encoding time with a high compression ratio, resolution-independent, fast decoding, and self-similarity. The main purpose of this paper is to present a comparative performance study of the three coding schemes of fractal compression for grayscale medical images based on fixed partition. The first two coding schemes are based on the pixel-pattern measure and the third scheme is a proposed method based on the fractal dimension for complexity measure of range and domain blocks. The comparative study included encoding time, peak signal to noise ratio, and compression ratio, as a result, has been accomplished.

A multi-channel uncertainty-aware multi-resolution network for MR to CT synthesis.

Synthesising computed tomography (CT) images from magnetic resonance images (MRI) plays an important role in the field of medical image analysis, both for quantification and diagnostic purposes. Convolutional neural networks (CNNs) have achieved state-of-the-art results in image-to-image translation for brain applications. However, synthesising whole-body images remains largely uncharted territory involving many challenges, including large image size and limited field of view, complex spatial context, and anatomical differences between images acquired at different times. We propose the use of an uncertainty-aware multi-channel multi-resolution 3D cascade network specifically aiming for whole-body MR to CT synthesis. The Mean Absolute Error on the synthetic CT generated with the MultiResunc network (73.90 HU) is compared to multiple baseline CNNs like 3D U-Net (92.89 HU), HighRes3DNet (89.05 HU) and deep boosted regression (77.58 HU) and shows superior synthesis performance. We ultimately exploit the extrapolation properties of the MultiRes networks on sub-regions of the body.

AN EFFICIENT MAXIMUM LIKELIHOOD ESTIMATOR FOR TWO-DIMENSIONAL FRACTIONAL BROWNIAN MOTION

For pattern recognition, natural scenes and medical images are often modeled as two-dimensional fractional Brownian motion (2D FBM), which can be easily described by the Hurst exponent ([Formula: see text], a real number between 0 and 1. The Hurst exponent is directly related to the fractal dimension ([Formula: see text]) by [Formula: see text]–[Formula: see text], and hence it very suitably serves as a characteristic index. Therefore, how to estimate the Hurst exponent effectively and efficiently is very important in pattern recognition. In this paper, a partially iterative algorithm, simply called an iterative maximum likelihood estimator (MLE) for 2D DFBM is first proposed and then a more sufficiently iterative algorithm, simply called an efficient MLE for 2D DFBM, is further proposed via a perfect structure of the log-likelihood function related to the Hurst exponent. Except for theoretical knowledge, two practical algorithms are also correspondingly provided for easy applications. Experimental results show that the MLE for 2D DFBM is effective and workable; its accuracy gets higher as the image size increases, and hence its recognition resolution is much finer to identify small difference among patterns. Furthermore, the efficient MLE is much quicker than the iterative MLE, especially at larger image sizes.

Tilt Correction Toward Building Detection of Remote Sensing Images

Building detection is a crucial task in the field of remote sensing, which can facilitate urban construction planning, disaster survey, and emergency landing. However, for large-size remote sensing images, the great majority of existing works have ignored the image tilt problem. This problem can result in partitioning buildings into separately oblique parts when the large-size images are partitioned. This is not beneficial to preserve semantic completeness of the building objects. Motivated by the above fact, we first propose a framework for detecting objects in a large-size image, particularly for building detection. The framework mainly consists of two phases. In the first phase, we particularly propose a tilt correction (TC) algorithm, which contains three steps: texture mapping, tilt angle assessment, and image rotation. In the second phase, building detection is performed with object detectors, especially deep-neural-network-based methods. Last but not least, the detection results will be inversely mapped to the original large-size image. Furthermore, a challenging dataset named Aerial Image Building Detection is contributed for the public research. To evaluate the TC method, we also define an evaluation metric to compute the cost of building partition. The experimental results demonstrate the effects of the proposed method for building detection.

GreedyCenters: Satellite imagery adaptive sampling method for artificial neural networks training

The one of many significant particularities of satellite imagery is large size of images within orders of magnitude exceeds capability of modern GPGPU to train neural networks on its full size. On the other hand satellite imagery tends to be limitedly available. Moreover, the objects of interest tends to constitute a small fraction of whole dataset. This leads to the demand of sample extraction and augmentation method specialized on satellite imagery. Yet this area is immensely underrated so almost all widely used method limited to grid-based sample extraction and augmentation via combinations of 90-degrees rotations and mirroring on vertical or horizontal axes. This paper proposes the domain-agnostic method of sample extraction and augmentation. Adoption of this method to specific subject area is based on domain-specific way to generate significance field of image. In contrast to trivial greedy solutions and more advanced stochastic optimization methods the design of proposed method is focused on maximizing per-step progress. This makes its performance reasonably good even without low-level optimizations without significant quality loss. It can be easily implemented using widely known and open source software libraries.

Skull Segmentation from CBCT Images via Voxel-Based Rendering.

Skull segmentation from three-dimensional (3D) cone-beam computed tomography (CBCT) images is critical for the diagnosis and treatment planning of the patients with craniomaxillofacial (CMF) deformities. Convolutional neural network (CNN)-based methods are currently dominating volumetric image segmentation, but these methods suffer from the limited GPU memory and the large image size (e.g., 512 × 512 × 448). Typical ad-hoc strategies, such as down-sampling or patch cropping, will degrade segmentation accuracy due to insufficient capturing of local fine details or global contextual information. Other methods such as Global-Local Networks (GLNet) are focusing on the improvement of neural networks, aiming to combine the local details and the global contextual information in a GPU memory-efficient manner. However, all these methods are operating on regular grids, which are computationally inefficient for volumetric image segmentation. In this work, we propose a novel VoxelRend-based network (VR-U-Net) by combining a memory-efficient variant of 3D U-Net with a voxel-based rendering (VoxelRend) module that refines local details via voxel-based predictions on non-regular grids. Establishing on relatively coarse feature maps, the VoxelRend module achieves significant improvement of segmentation accuracy with a fraction of GPU memory consumption. We evaluate our proposed VR-U-Net in the skull segmentation task on a high-resolution CBCT dataset collected from local hospitals. Experimental results show that the proposed VR-U-Net yields high-quality segmentation results in a memory-efficient manner, highlighting the practical value of our method.

A Novel Coarse-to-Fine Sea-Land Segmentation Technique Based on Superpixel Fuzzy C-Means Clustering and Modified Chan-Vese Model

The sea-land segmentation for optical remote sensing images (RSIs) has a valuable role in water resources and coastal zones management. However, it is challenging because optical RSIs mainly suffer from low contrast, intensity inhomogeneity with mixed pixels, and large image size with redundant information. This paper introduces a novel coarse-to-fine sea-land segmentation method that incorporates Superpixel Fuzzy C-Means (SPFCM) and a Modified Chan-Vese active contour model (MCV). First, the image is over-segmented into superpixels to reduce the information redundancy and utilize spectral and spatial information. The SPFCM employs local relationships among neighboring superpixels to cluster the superpixels based on their color and texture features, deal with mixed pixels, and produce coarse segmentation results. The CV depends on the initial contour. If the contour is incorrectly initialized, the CV may trap in local minima and needs many iterations. The purpose of the SPFCM result is to provide an automatic initial contour for an MCV model instead of manual initialization to improve the CV model's performance. Finally, color and texture features are combined in vector-valued images to solve the traditional CV's problems to deal with complicated nature images with intensity inhomogeneity or rich texture features to produce fine segmentation results. The proposed method is evaluated and achieved an average accuracy of 98.9%, an average Jaccard Similarity Coefficient (JSC) equals 97.1%, an average Disc Similarity Coefficient (DSC) equals 98.5%, and an average recall equals 99.3%. The proposed method results are promising, and it outperformed the results of other state-of-the-art sea-land segmentation methods.

Fabric Defect Detection Based on the Improved Cascade R-CNN

There is a large amount of fabric produced in the process of industrial production, thus fabric defects automatic detection can bring great benefits to enterprises. With the development of computer technology, deep learning has more advantages on fabric defect detection compared with traditional image processing. By comparing the advantages and disadvantages of different target detection models, we chose to use the Cascade R-CNN model for fabric defect detection finally. However, due to the large size of fabric image, small number of partial defects and complex image background, there are many difficulties in fabric defect detection. To solve the difficulties, we propose some methods to improve the accuracy of defect detection. First, in order to enhance the detection accuracy of small targets, we cut large fabric images into blocks for training and detection, and combine the detection results. Then, to solve the problem of the small number of partial defects in the fabric Dataset, a new method of multi-morphological data augmentation was proposed to increase the number of the Dataset. Finally, we improved the Feature Pyramid Networks module to enhance positional accuracy of defect detection. Experimental results show that the model can complete the task of defect detection efficiently and improve the accuracy of target detection effectively.

FastPathology: An Open-Source Platform for Deep Learning-Based Research and Decision Support in Digital Pathology

Deep convolutional neural networks (CNNs) are the current state-of-the-art for digital analysis of histopathological images. The large size of whole-slide microscopy images (WSIs) requires advanced memory handling to read, display and process these images. There are several open-source platforms for working with WSIs, but few support deployment of CNN models. These applications use third-party solutions for inference, making them less user-friendly and unsuitable for high-performance image analysis. To make deployment of CNNs user-friendly and feasible on low-end machines, we have developed a new platform, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FastPathology</i> , using the FAST framework and C++. It minimizes memory usage for reading and processing WSIs, deployment of CNN models, and real-time interactive visualization of results. Runtime experiments were conducted on four different use cases, using different architectures, inference engines, hardware configurations and operating systems. Memory usage for reading, visualizing, zooming and panning a WSI were measured, using FastPathology and three existing platforms. FastPathology performed similarly in terms of memory to the other C++-based application, while using considerably less than the two Java-based platforms. The choice of neural network model, inference engine, hardware and processors influenced runtime considerably. Thus, FastPathology includes all steps needed for efficient visualization and processing of WSIs in a single application, including inference of CNNs with real-time display of the results. Source code, binary releases, video demonstrations and test data can be found online on GitHub at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/SINTEFMedtek/FAST-Pathology/</uri> .