Combined Loss Function Research Articles

Construction of a Medical Micro-Object Cascade Network for Automated Segmentation of Cerebral Microbleeds in Susceptibility Weighted Imaging

Aim: The detection and segmentation of cerebral microbleeds (CMBs) images are the focus of clinical diagnosis and treatment. However, segmentation is difficult in clinical practice, and missed diagnosis may occur. Few related studies on the automated segmentation of CMB images have been performed, and we provide the most effective CMB segmentation to date using an automated segmentation system. Materials and Methods: From a research perspective, we focused on the automated segmentation of CMB targets in susceptibility weighted imaging (SWI) for the first time and then constructed a deep learning network focused on the segmentation of micro-objects. We collected and marked clinical datasets and proposed a new medical micro-object cascade network (MMOC-Net). In the first stage, U-Net was utilized to select the region of interest (ROI). In the second stage, we utilized a full-resolution network (FRN) to complete fine segmentation. We also incorporated residual atrous spatial pyramid pooling (R-ASPP) and a new joint loss function. Results: The most suitable segmentation result was achieved with a ROI size of 32 × 32. To verify the validity of each part of the method, ablation studies were performed, which showed that the best segmentation results were obtained when FRN, R-ASPP and the combined loss function were used simultaneously. Under these conditions, the obtained Dice similarity coefficient (DSC) value was 87.93% and the F2-score (F2) value was 90.69%. We also innovatively developed a visual clinical diagnosis system that can provide effective support for clinical diagnosis and treatment decisions. Conclusions: We created the MMOC-Net method to perform the automated segmentation task of CMBs in an SWI and obtained better segmentation performance; hence, this pioneering method has research significance.

Video frame interpolation via down–up scale generative adversarial networks

Frame interpolation finds many applications in video applications, including frame rate up-conversion and video compression. Deep learning-based methods have been proposed for frame interpolation, but a long runtime is typically required to achieve good visual quality. In this paper, we introduce an efficient frame interpolation method based on a modified generative adversarial network. The proposed framework consists of a generator with a pair of down–up scale modules, where the down-scaled-input module attempts to capture the overall structure of the scene while the original-scale-input module aims to restore finer textures. Skip connections and an input processing block are further incorporated into the minimal two-scale generator design to expedite processing without losing image features. The difference between the synthesized frame and the ground truth is measured by a combined loss function, including one adversarial loss and three reconstruction losses. Compared to the state-of-the-art motion compensation and deep-learning based frame interpolation approaches, the proposed framework achieves the most satisfactory trade-off between the synthesis quality and runtime.

SCL—Segmentation–Classification combined Loss for surface defect detection

Deep learning for surface defect detection has recently received increased attention in relation to quality control of industrial products. In this paper, we present a combined loss function called Segmentation–Classification combined Loss (SCL). It exploits the binary information (normal/anomalous sample) implicit in the segmentation masks to reduce false positives and false negatives in surface defect detection tasks. We evaluate the method on images in three different domains: defects in electrical commutators, defects in a weakly labeled synthetic dataset, and defects of glass bottles.Our combined loss function outperforms the state of the art in the Kolektor (electrical commutators) dataset and significantly improves the classical Dice segmentation loss function in terms of both average precision, false positives, and false negatives. Moreover, the combined loss function introduced in this paper can be applied in a straightforward manner to any loss-based segmentation model. Finally, in this work we introduce and publish a new dataset for the still poorly explored task of surface defect detection of glass bottles.

Voids Filling of DEM with Multiattention Generative Adversarial Network Model

The digital elevation model (DEM) acquired through photogrammetry or LiDAR usually exposes voids due to phenomena such as instrumentation artifact, ground occlusion, etc. For this reason, this paper proposes a multiattention generative adversarial network model to fill the voids. In this model, a multiscale feature fusion generation network is proposed to initially fill the voids, and then a multiattention filling network is proposed to recover the detailed features of the terrain surrounding the void area, and the channel-spatial cropping attention mechanism module is proposed as an enhancement of the network. Spectral normalization is added to each convolution layer in the discriminator network. Finally, the training of the model by a combined loss function, including reconstruction loss and adversarial loss, is optimized. Three groups of experiments with four different types of terrains, hillsides, valleys, ridges and hills, are conducted for validation of the proposed model. The experimental results show that (1) the structural similarity surrounding terrestrial voids in the three types of terrains (i.e., hillside, valley, and ridge) can reach 80–90%, which implies that the DEM accuracy can be improved by at least 10% relative to the traditional interpolation methods (i.e., Kriging, IDW, and Spline), and can reach 57.4%, while other deep learning models (i.e., CE, GL and CR) only reach 43.2%, 17.1% and 11.4% in the hilly areas, respectively. Therefore, it can be concluded that the structural similarity surrounding the terrestrial voids filled using the model proposed in this paper can reach 60–90% upon the types of terrain, such as hillside, valley, ridge, and hill.

Identification of maize leaf diseases by using the SKPSNet-50 convolutional neural network model

Maize leaf disease is the main factor that affects the yield and quality of maize. Early diagnosis and control of maize leaf diseases help ensure maize yield and quality. However, early maize leaf diseases show small spots and irregular shapes in natural scene images, resulting in current models having difficulty identifying maize leaf diseases accurately. This study proposes a SKPSNet-50 convolutional neural network model to realize the accurate identification of maize leaf diseases in natural scene images. The proposed model replaces the 3 × 3 convolution kernel in the backbone network ResNet-50 with Select Kernel–Point–Swish_B (SKPS), which is an improved building block of the selected kernel (SK) unit, and replaces the ReLU activation function with the Swish_B activation function to improve the feature extraction capability of diseased leaves with small spots and irregular shapes. In addition, this study adopts the combined focal loss function to guide the parameter adjustment of the model to solve the problem of data imbalance. Experimental results show that compared with classic machine learning models (such as BP neural network and support vector machine) and deep neural network models (such as AlexNet, VGG16, ResNet-50, SENet-50, SKNet-50 and GhostNet), the proposed model is more effective in the identification of maize leaf diseases in natural scene images, with an average identification accuracy of 92.9%, which is about 6% higher than that of the SKNet-50 model.

Volumetric segmentation of ground glass nodule based on 3D attentional cascaded residual U‐Net and conditional random field

Ground glass nodule (GGN) segmentation is one of the important and challenging tasks in diagnosing early-stage lung adenocarcinomas. Manually delineating of 3D GGN in a computed tomography (CT) image is a subjective, laborious, and tedious task, which presents poor repeatability. To reduce the annotation burden and improve the segmentation performance, this study proposes a 3D deep learning-based volumetric segmentation model to segment the GGN in CT images. A total of 379 GGNs were retrospectively collected from the public database, Shanghai Pulmonary Hospital (SHPH), and Fudan University Shanghai Cancer Center (FUSCC). First, a series of image preprocessing techniques involving image resampling, intensity normalization, 3D nodule patch cropping, and data augmentation, were adopted to generate the input images for the deep learning model by using CT scans. Then, a 3D attentional cascaded residual network (ACRU-Net) was proposed to develop the deep learning-based segmentation model by using the residual network and the atrous spatial pyramid pooling module. To improve the model performance, a voxel-based conditional random field (CRF) method was used to optimize the segmentation results. Finally, a balanced cross-entropy and Dice combined loss function was applied to train and build the segmentation model. Testing on SHPH and FUSCC datasets, the proposed method generates the Dice coefficients of 0.721±0.167 and 0.733±0.100, respectively, which are higher than those of 3D residual U-Net and ACRU-Net without CRF optimization. The results demonstrated that combining 3D ACRU-Net and CRF effectively improved the GGN segmentation performance. The proposed segmentation model may provide a potential tool to help the radiologist in the segmentation and diagnosis of 3D GGN.

Time-Domain Joint Training Strategies of Speech Enhancement and Intent Classification Neural Models.

Robustness against background noise and reverberation is essential for many real-world speech-based applications. One way to achieve this robustness is to employ a speech enhancement front-end that, independently of the back-end, removes the environmental perturbations from the target speech signal. However, although the enhancement front-end typically increases the speech quality from an intelligibility perspective, it tends to introduce distortions which deteriorate the performance of subsequent processing modules. In this paper, we investigate strategies for jointly training neural models for both speech enhancement and the back-end, which optimize a combined loss function. In this way, the enhancement front-end is guided by the back-end to provide more effective enhancement. Differently from typical state-of-the-art approaches employing on spectral features or neural embeddings, we operate in the time domain, processing raw waveforms in both components. As application scenario we consider intent classification in noisy environments. In particular, the front-end speech enhancement module is based on Wave-U-Net while the intent classifier is implemented as a temporal convolutional network. Exhaustive experiments are reported on versions of the Fluent Speech Commands corpus contaminated with noises from the Microsoft Scalable Noisy Speech Dataset, shedding light and providing insight about the most promising training approaches.

Simple Multiscale UNet for Change Detection With Heterogeneous Remote Sensing Images

Change detection with heterogeneous remote sensing images (HRSIs) is attractive for observing the Earth’s surface when homogeneous images are unavailable. However, HRSIs cannot be compared directly because the imaging mechanisms for bitemporal HRSIs are different, and detecting change with HRSIs is challenging. In this letter, a simple yet effective deep learning approach based on the classical UNet is proposed. First, a pair of image patches are concatenated together to learn a shared abstract feature in both image patch domains. Then, a multiscale convolution module is embedded in a UNet backbone to cover the various sizes and shapes of ground targets in an image scene. Finally, a combined loss function, which incorporates the focal and dice losses with an adjustable parameter, was incorporated to alleviate the effect of the imbalanced quantity of positive and negative samples in the training progress. By comparisons with five state-of-the-art methods in three pairs of real HRSIs, the experimental results achieved by our proposed approach have the best overall accuracy (OA), average accuracy (AA), recall (RC), and F-Score that are more than 95%, 79%, 60%, and 61%, respectively. The quantitative results and visual performance indicated the feasibility and superiority of the proposed approach for detecting land cover change with HRSIs.

Unsupervised SAR and Optical Image Matching Using Siamese Domain Adaptation

Due to its highly complementary information about remote sensing, synthetic aperture radar (SAR) and optical imagery matching have drawn much attention in recent years. Compared with traditional methods, deep learning-based SAR-optical image matching models largely rely on supervision with ground truths, where the matching accuracy suffers because of unseen image domains. To mitigate loads in burdensome labeling tasks, transferring deep learning models trained with annotated source domains to nonannotated target domains has attracted great concern. Due to the domain gap, the difference between the source and target domains is likely to deteriorate the matching accuracy on target data if the training process is directly conducted without proper domain adaptation (DA). In this research, a Siamese DA (SDA) approach with a combined loss function is developed in the context of multimodality image matching. Then, a novel rotation/scale-invariant transformation module with regression modules is designed to extract rotation/scale-equivariant features. Finally, the causal inference-based self-learning method and the multiresolution histogram matching approach are employed to enhance the unsupervised matching performance. Experimental results on the RadarSat/Planet dataset and the Sentinel-1/2 dataset demonstrate that the developed model can achieve competitive matching performance with a low overlap ratio between domains and little data labeling. By alleviating the domain discrepancy, the developed model drastically reduces the average L2 score of the unsupervised matching from 9.576 to 0.658, while the less-than-one-pixel matching error rate is enhanced from 66.3% to 90.6%.

Denoising Letter Images from Scanned Invoices Using Stacked Autoencoders

Invoice document digitization is crucial for efficient management in industries. The scanned invoice image is often noisy due to various reasons. This affects the OCR (optical character recognition) detection accuracy. In this paper, letter data obtained from images of invoices are denoised using a modified autoencoder based deep learning method. A stacked denoising autoencoder (SDAE) is implemented with two hidden layers each in encoder network and decoder network. In order to capture the most salient features of training samples, a undercomplete autoencoder is designed with non-linear encoder and decoder function. This autoencoder is regularized for denoising application using a combined loss function which considers both mean square error and binary cross entropy. A dataset consisting of 59,119 letter images, which contains both English alphabets (upper and lower case) and numbers (0 to 9) is prepared from many scanned invoices images and windows true type (.ttf) files, are used for training the neural network. Performance is analyzed in terms of Signal to Noise Ratio (SNR), Peak Signal to Noise Ratio (PSNR), Structural Similarity Index (SSIM) and Universal Image Quality Index (UQI) and compared with other filtering techniques like Nonlocal Means filter, Anisotropic diffusion filter, Gaussian filters and Mean filters. Denoising performance of proposed SDAE is compared with existing SDAE with single loss function in terms of SNR and PSNR values. Results show the superior performance of proposed SDAE method.

Enhanced Channel Attention Network With Cross-Layer Feature Fusion for Spectral Reconstruction in the Presence of Gaussian Noise

Spectral reconstruction from RGB images has made significant progress. Previous works usually utilized the noise-free RGB images as input to reconstruct the corresponding hyperspectral images (HSIs). However, due to instrumental limitation or atmospheric interference, it is inevitable to suffer from noise (e.g., Gaussian noise) in the actual image acquisition process, which further increases the difficulty of spectral reconstruction. In this paper, we propose an enhanced channel attention network (ECANet) to learn a nonlinear mapping from noisy RGB images to clean HSIs. The backbone of our proposed ECANet is stacked with multiple enhanced channel attention (ECA) blocks. The ECA block is the dual residual version of channel attention block, which makes the network focus on key auxiliary information and features that are more conducive to spectral reconstruction. For the case that the input RGB images are disturbed by Gaussian noise, cross-layer feature fusion (CLFF) unit is used to concatenate the multiple feature maps at different depths for more powerful feature representations. In addition, we design a novel combined loss function as the constraint of the ECANet to achieve more accurate reconstruction result. Experimental results on two HSI benchmarks, CAVE and NTIRE 2020, demonstrate that the effectiveness of our method in terms of both visual and quantitative over other state-of-the-art methods.

Multilevel Dual-Direction Modifying Variational Autoencoders for Hyperspectral Feature Extraction

Hyperspectral images (HSIs) provide abundant high-quality spectral information through an immense number of spectral channels, which can be used to classify on-ground objects for Earth observation accurately. However, these highly correlated channels and complex informative features always limit the application of HSIs. In this letter, we propose a multilevel dual-direction modifying variational autoencoder (MD²MVAE) for hyperspectral feature extraction. Its architecture is inspired by the spectral–spatial coherence in HSIs. Our motivation is to modify spectral sequential features by spatial sequential features in a multilevel dual-direction network. The dual-direction strategy has two implications: 1) the spatial continuity is captured by flattening neighboring samples in dual classic directions and 2) the spectral–spatial continuities are captured in the forward and backward directions. This multilevel dual-direction network provides a feasible way to avoid the loss of spatial information when flattening samples in the spatial domain, without using any convolutional layers. Inspired by our previous work, a variational autoencoder (VAE)-based network is used to enhance the noise immunity of latent features. To preserve the consistency of spectral–spatial sequential features, a combined loss function based on the solid angle on a unit sphere is proposed for parameter optimization. Two typical datasets are selected as benchmarks to show the effectiveness of MD²MVAE, compared with the state-of-the-art methods.

Weighted composite asymmetric Huber estimation for partial functional linear models

<abstract><p>In this paper, we first investigate a new asymmetric Huber regression (AHR) estimation procedure to analyze skewed data with partial functional linear models. To automatically reflect distributional features as well as bound the influence of outliers effectively, we further propose a weighted composite asymmetric Huber regression (WCAHR) estimation procedure by combining the strength across multiple asymmetric Huber loss functions. The slope function and constant coefficients are estimated through minimizing the combined loss function and approximating the slope function with principal component analysis. The asymptotic properties of the proposed estimators are derived. To realize the WCAHR estimation, we also develop a practical algorithm based on pseudo data. Numerical results show that the proposed WCAHR estimators can well adapt to the different error distributions, and thus are more useful in practice. Two real data examples are presented to illustrate the applications of the proposed methods.</p></abstract>

Deep Siamese Semantic Segmentation Network for PCB Welding Defect Detection

Deep learning has been widely used in recent years for printed circuit board (PCB) defect detection because of its excellent performance. However, deep-learning-based approaches often suffer from the over-fitting problem due to the lack of sufficient training data in real applications. Meanwhile, these approaches still have some challenges to detect these defects with small sizes and irregular shapes. To address these problems, this article has developed a novel deep Siamese semantic segmentation network which integrates the similarity measurement of the Siamese network with the encoder–decoder semantic segmentation network for PCB welding defect detection. This network includes two encoders sharing weighted values, a decoder, and some correlation modules, in which the decoder integrates deep features from two decoders with their feature difference computed by some correlation modules via skipping connections to recover spatial information on multiple output layers, and thus this proposed network can perform PCB welding small defect semantic segmentation. Moreover, via these correlation modules, this proposed network can pay more attention to semantic difference and further alleviate the over-fitting issue because of insufficient defect samples. Finally, we propose a combined loss function which combines the weighted cross-entropy loss, the Lovasz softmax loss, and the weighted precision–recall loss for network training to further improve small defect segmentation and recall improvement. Experimental results demonstrate that the proposed network can be trained on limited training images and achieve high efficiency and outstanding effects for PCB welding small defect segmentation.

Manifold Learning-Based Semisupervised Neural Network for Hyperspectral Image Classification

Feature extraction (FE), an important preprocessing step in hyperspectral image (HSI) classification, has received growing attention in the remote sensing community. In recent years, the FE ability of deep learning (DL) methods has been widely recognized. However, most DL models focus on training networks with strong nonlinear mapping ability. They fail to explore the intrinsic manifold structure in HSI, and their performance depends on large size of the labeled training set. To address the above problems, a novel FE approach, termed manifold learning-based semisupervised neural network (MSSNet), was proposed in this article. By introducing the graph embedding (GE) framework, MSSNet develops a semisupervised graph model to explore the manifold structure in HSI with both labeled and unlabeled data. On the basis of this graph model, MSSNet constructs a combined loss function to take into account the metric of difference values and the exploration of manifold margins; thus, it reduces the difference between the predictive value and the actual value to enhance the separability of the features extracted by the network. Experiments conducted on real-world HSI datasets demonstrate that the performance of the proposed MSSNet outperforms some related state-of-the-art FE approaches.

Automatic segmentation of concrete aggregate using convolutional neural network

The segmentation of concrete aggregates in sedimentation images plays a significant role in evaluating the particle distribution and stability properties. To address the limitations of traditional evaluation methodologies that are subjective and labor-intensive, a deep-learning-based concrete aggregate segmentation method is proposed. In the proposed approach, an encoder–decoder structure was adopted. Specifically, our model adds a squeeze-and-excitation block to ResNeXt50 to adaptively recalibrate the channel-wise feature response and increase the efficiency of feature extraction. A decoder module was adopted to refine the segmentation results. The combined loss function was applied to overcome the problem of sample imbalance while improving model performance. The trained model achieves accurate segmentation of the aggregate and suspension on the images in the test set, and its performance is superior to that of the other three popular segmentation methods, achieving 97.85% precision, 95.81% recall, 96.81% F1 score, and 93.85% intersection over union.

A cascaded fully convolutional network framework for dilated pancreatic duct segmentation.

Pancreatic duct dilation can be considered an early sign of pancreatic ductal adenocarcinoma (PDAC). However, there is little existing research focused on dilated pancreatic duct segmentation as a potential screening tool for people without PDAC. Dilated pancreatic duct segmentation is difficult due to the lack of readily available labeled data and strong voxel imbalance between the pancreatic duct region and other regions. To overcome these challenges, we propose a two-step approach for dilated pancreatic duct segmentation from abdominal computed tomography (CT) volumes using fully convolutional networks (FCNs). Our framework segments the pancreatic duct in a cascaded manner. The pancreatic duct occupies a tiny portion of abdominal CT volumes. Therefore, to concentrate on the pancreas regions, we use a public pancreas dataset to train an FCN to generate an ROI covering the pancreas and use a 3D U-Net-like FCN for coarse pancreas segmentation. To further improve the dilated pancreatic duct segmentation, we deploy a skip connection on each corresponding resolution level and an attention mechanism in the bottleneck layer. Moreover, we introduce a combined loss function based on Dice loss and Focal loss. Random data augmentation is adopted throughout the experiments to improve the generalizability of the model. We manually created a dilated pancreatic duct dataset with semi-automated annotation tools. Experimental results showed that our proposed framework is practical for dilated pancreatic duct segmentation. The average Dice score and sensitivity were 49.9% and 51.9%, respectively. These results show the potential of our approach as a clinical screening tool. We investigate an automated framework for dilated pancreatic duct segmentation. The cascade strategy effectively improved the segmentation performance of the pancreatic duct. Our modifications to the FCNs together with random data augmentation and the proposed combined loss function facilitate automated segmentation.

Row anchor selection classification method for early-stage crop row-following

The field navigation tasks of agricultural robots are the basis for implementing field management projects in the early stages of seedlings guided by precision agriculture policies. High accuracy and execution speed of navigation tasks are needed for the escalating field machine implementations in a significantly unstructured and irregular field environment, contributing to the difficulty and urgency of achieving a field navigation solution with high navigation accuracy while maintaining proper execution speed. In our study, the row anchor selection classification method (RASCM) is presented to track early-stage crop rows, and the early corn seedling dataset (E-dataset) and LRC-dataset are developed for the measurement of the experimental results. RASCM is divided into three parts, including a row anchor selection method (RasM), a lightweight deep convolutional neural network (DCNN) network GhostNet, and a combined loss function (C-loss). It is suggested that RASCM has achieved good results in terms of calculation speed and accuracy by comparing the end-to-end regression operation based on the steering wheel angle (E2Er) and crop row pixel segmentation (SeG) approaches with RASCM on the LRC-dataset, which can be effectively applied in actual projects. Finally, a class activation map (CAM) is used to visualize the internal working mechanism of GhostNet, and the demonstrated effectiveness of GhostNet is not limited to the RasM labeling area. In contrast, broad global features are learned, and the previous noise is highly mitigated. Strong generalization performance for different environments is displayed by using RACSM, contributing to the effectiveness of early-stage crop row-following. The main contribution of our paper is the proposal of RASCM, which is a novel method in the field of early-stage crop row-following and can be used as a basic reference for subsequent work in this research field.

System of license plate recognition considering large camera shooting angles

The system of automatic license plate recognition (ALPR) is a combination of software and hardware technologies implementing ALPR algorithms. It seems to be easy to achieve the goal but recognition of license plate requires many difficult solutions to some non-trivial tasks. If the license plate is oriented horizontally, uniformly lighted, has a clean surface, clearly distinguishable characters, then it’ll be not too difficult to recognize such a license plate. However, the reality is much worse. The lighting of each part of the plate isn’t equal; the picture from the camera is noisy. Besides, the license plate can have a big angle relative to the camera and be dirty. These obstacles make it difficult to recognize the license plate characters and determine their location on the image. For instance, the accuracy of recognition is much worse on large camera angles. To solve these problems, the developers of automatic license plate recognition systems use a different approach to processing and analysis of images. The work shows an automatic license plate recognition system, which increases the recognition accuracy at large camera angles. The system is based on the technology of recognition of images with the use of highly accurate convolutional neural networks. The proposed system improves stages of normalization and segmentation of an image of the license plate, taking on large camera angles. The goal of improvements is to increase of accuracy of recognition. On the stage of normalization, before histogram equalization, the affine transformation of the image is performed. For the process of segmentation and recognition, Mask R-CNN is used. As the main segment-search algorithm, selective search is chosen. The combined loss function is used to fasten the process of training and classification of the network. The additional module to the convolutional neural network is added for solving the interclass segmentation. The input for this module is generated feature tensor. The output is segmented data for semantic processing. The developed system was compared to well-known systems (SeeAuto.USA and Nomeroff.Net). The invented system got better results on large camera shooting angles.

Spatio-Temporal Features Based Surgical Phase Classification Using CNNs.

In this paper, we propose a novel encoder-decoder based surgical phase classification technique leveraging on the spatio-temporal features extracted from the videos of laparoscopic cholecystectomy surgery. We use combined margin loss function to train on the computationally efficient PeleeNet architecture to extract features that exhibit: (1) Intra-phase similarity, (2) Inter-phase dissimilarity. Using these features, we propose to encapsulate sequential feature embeddings, 64 at a time and classify the surgical phase based on customized efficient residual factorized CNN architecture (ST-ERFNet). We obtained surgical phase classification accuracy of 86.07% on the publicly available Cholec80 dataset which consists of 7 surgical phases. The number of parameters required for the computation is approximately reduced by 84% and yet achieves comparable performance as the state of the art.Clinical relevance- Autonomous surgical phase classification sets the platform for automatically analyzing the entire surgical work flow. Additionally, could streamline the process of assessment of a surgery in terms of efficiency, early detection of errors or deviation from usual practice. This would potentially result in increased patient care.