Weighted Loss Function Research Articles

Deep learning-based registration of two-dimensional dental images with edge specific loss.

Image registration is a very common procedure in dental applications for aligning images. Registration between pairs of images taken from different angles can improve diagnosis. Our study presents an edge-enhanced unsupervised deep learning (DL)-based deformable registration framework for aligning two-dimensional (2D) pairs of dental x-ray images. The proposed neural network is based on the combination of a U-Net like structure, which produces a displacement field, combined with spatial transformer networks, which produce the transformed image. The proposed structure is trained end-to-end by minimizing a weighted loss function consisting of three parts corresponding to image similarity, edge similarity, and registration restrictions. In this regard, the proposed edge specific loss enhances the unsupervised training of the registration framework without the need of supervision through anatomical structures. The proposed framework was applied to two datasets, a set of 104 x-ray images of mandibles, arranged in 2600 pairs for training and testing and a set of 17 pairs of pre- and post-operative reconstructed panoramic images. The proposed model outperformed both conventional registration methods and DL-based techniques for both qualitative and quantitative assessment, in most of the compared metrics concerning intensity similarity and edge distances. The proposed framework achieved accurate and fast deformable alignment of pairs of 2D dental radiographic images. The edge-based module of the loss function enhances the unsupervised learning by directing the network toward deformations that take into consideration the edges of the depicted objects (teeth, bone, and tissue), which are crucial in diagnosis.

0517 Changes in AutoCPAP Settings as a Function of Weight Loss Over Time in Bariatric Surgery Patients with Obstructive Sleep Apnea

Abstract Introduction Positive pressure remains the main treatment for obstructive sleep apnea (OSA). Compared to traditional continuous positive airway pressure (CPAP) machines with fixed pressure settings, AutoCPAP machines can titrate settings based on user requirements and transmit data to the pulmonologist. While bariatric surgery has been associated with improvement in OSA, there is little data examining associations between postoperative weight loss and positive pressure requirements longitudinally. The purpose of this study was to evaluate changes in AutoCPAP pressure settings as a function of weight loss over time. Methods A retrospective review of bariatric surgery patients at a single institution from 2018-2022 was performed. Participants included patients with OSA who used an AutoCPAP machine and had at least one appointment within the first 18 months postoperatively. Preoperative BMI, postoperative BMI and percent excess weight loss were analyzed. AutoCPAP parameters used as markers of OSA severity included 95th or 90th percentile pressure depending on the machine and apnea-hypopnea index (AHI). Paired t-tests were performed to evaluate changes in BMI and 90th/95th percentile pressures. Linear regressions with fixed effects were performed to investigate changes in pressures and BMI over time. Results 345 patients with OSA who underwent bariatric surgery and were prescribed AutoCPAP were evaluated. Of those, twenty-eight (8%) were compliant with their AutoCPAP and were included in this study. Overall, 90th/95th percentile pressures improved postoperatively by 1.32 cmH2O± 0.33 (P= 0.003). Patients with a preoperative BMI ≥40 demonstrated improved 90th/95th percentile pressures over time by 1.22 cmH2O± 0.34 (P=0.002) while those with a preoperative BMI < 40 did not (-1.7 cmH2O ± 0.86; P= 0.106). Reduction in BMI was associated with reduction in 90th/95th percentile pressures at 2-6 weeks and 2-6 months postoperatively. Conclusion Patients with a higher preoperative BMI demonstrated improved positive pressure requirements postoperatively compared to patients with a lower preoperative BMI. This suggests that there may be a critical BMI after which further weight loss is not associated with significant improvements in OSA. Further, positive pressure requirements may decrease most significantly within the first several months postoperatively. Support (if any)

Deep learning characterization of rock conditions based on tunnel boring machine data

Rock condition perception based on tunnel boring machine (TBM) data is of great importance for not only ensuring tunnel boring safety but also improving construction efficiency. The prediction of TBM boring responses (i.e., torque and total thrust of the cutterhead) largely determines the reliability of rock condition perception. In this paper, a new architecture of a two-dimensional convolutional neural network (2D-CNN) with a dual-input strategy is proposed to predict the TBM responses. The TBM Lot 3 of the Yinsong project in Jilin province, China, is taken as the case study in this paper. Two types of models that follow different learning strategies are compared: one is defined as the point model, which only learns data of the stable phase, and the other is defined as the line model, which learns data from both the loading and stable boring phases. The line model is further improved by the weighted loss function method. The results indicate that the strategy of learning data from both the loading phase and stable boring phase and increasing the weight of samples from the stable phase is shown to be optimal in predicting TBM boring responses. In terms of learning strategies, the line model can learn the influence of active control parameters on passive response parameters, but the point model cannot. In terms of machine learning algorithms, 2D-CNN has the best performance, with R2 values of 0.865 and 0.923 for torque and total thrust, respectively. The proposed line model can overcome the problem that the traditional model failed to learn the influence of control parameters. Such a model can provide a solid base for the timely optimization of the control parameters in TBM boring process.

JAED-Net: joint attention encoder–decoder network for road extraction from remote sensing images

In road extraction from remote sensing images, the road environment is complex and blocked by trees, buildings, and other objects, making it impossible to extract practical (continuous and complete) road information. We propose a joint attention encoder–decoder network (JAED-Net) for road extraction from remote sensing images to solve these problems. First, JAED-Net encodes a modified residual network as the backbone for road feature extraction. A joint attention module is added to the encoder to enhance the network’s ability to learn and express road features. Then, strip convolution is added to the decoder, so the network retains more spatial features, such as the width and connectivity of roads during upsampling. Finally, a hybrid weighted loss function is introduced to train the network and ensure stability because of the unbalanced ratio of road and background pixels in remote sensing images. Experimental validation of the proposed network is performed on three publicly available datasets.

Moving Point Target Detection Based on Temporal Transient Disturbance Learning in Low SNR

Moving target detection in optical remote sensing is important for satellite surveillance and space target monitoring. Here, a new moving point target detection framework under a low signal-to-noise ratio (SNR) that uses an end-to-end network (1D-ResNet) to learn the distribution features of transient disturbances in the temporal profile (TP) formed by a target passing through a pixel is proposed. First, we converted the detection of the point target in the image into the detection of transient disturbance in the TP and established mathematical models of different TP types. Then, according to the established mathematical models of TP, we generated the simulation TP dataset to train the 1D-ResNet. In 1D-ResNet, the structure of CBR-1D (Conv1D, BatchNormalization, ReLU) was designed to extract the features of transient disturbance. As the transient disturbance is very weak, we used several skip connections to prevent the loss of features in the deep layers. After the backbone, two LBR (Linear, BatchNormalization, ReLU) modules were used for further feature extraction to classify TP and identify the locations of transient disturbances. A multitask weighted loss function to ensure training convergence was proposed. Sufficient experiments showed that this method effectively detects moving point targets with a low SNR and has the highest detection rate and the lowest false alarm rate compared to other benchmark methods. Our method also has the best detection efficiency.

Asymmetric learning based deep denoiser for nonstationary desert seismic noise suppression

Random noise in desert seismic data has the weak similarity to the effective signals and spatiotemporally variable noise intensity. Mismatch noise intensity degrades the denoising performance of the convolutional neural network in incomplete noise suppression and damaged seismic signals. We propose an asymmetric learning-based nonstationary convolutional denoising neural network (AL-NCDNet) by introducing an asymmetric loss function and the noise level (NL) estimation block. Since the denoised result is robust to the overestimated NL and more sensitive to the underestimated NL, an asymmetric learning framework is adopted by leveraging a weighted loss function for constraining the noise level estimation block. Under the asymmetric learning framework, more penalties are imposed on the underestimated NL than the overestimated ones, alleviating the problem of noise residue caused by underestimated NL. Moreover, the estimated NL is taken as the weight between noise suppression and signal fidelity to guide the main denoising block adaptively suppressing nonstationary noise. The AL-NCDNet is tested on the synthetic, public and field seismic data. The denoised results verify that asymmetric learning is able to improve the accuracy of the estimated NL and the efficiency of the denoising model, thereby allows AL-NCDNet to effectively suppress spatiotemporally variable noise while preserving seismic events.

A power line segmentation model in aerial images based on an efficient multibranch concatenation network

Inspection of a power transmission line by an unmanned aerial vehicle (UAV) is an important measure to ensure the operational safety of the power grid. A semantic segmentation model based on deep learning can assist UAVs in extracting power lines, which is a crucial task to ensure the safe navigation of UAVs. However, the existing models are vulnerable to complex background interference in aerial images and the thin structure of power lines. To solve these problems, an improved power line segmentation model based on Deeplabv3+ (PL-Deeplab) is proposed in this paper. To create our model, the multibranch concatenation network (MCNet) with stronger feature extraction capability is introduced to the backbone network. Then, we design a one-shot aggregation feature pyramid (OSAFP) to effectively extract the contextual information and provide a larger receptive field. Furthermore, the feature fusion module (FFM) in the decoder is used to increase the utilization of high-low level feature information and improve the ability of the model to accurately segment the boundaries of power lines. In addition, the weighted loss function is used to solve the problem of class imbalance in the power lines dataset. A large number of experiments on the power lines public dataset show that the proposed model can complete the power line inspection task better and provide security for the UAV power line inspection.

Doubly Type II Censoring of Two Stress-Strength System Reliability Estimation for Generalized Exponential-Poisson Distribution

In this paper, a Bayesian analysis is made to estimate the Reliability of two stress-strength model systems. First: the reliability of a one component strengths X under stress Y. Second, reliability of one component strength under three stresses. Where X and Y are independent generalized exponential-Poison random variables with parameters (α,λ,θ) and (β,λ,θ) . The analysis is concerned with and based on doubly type II censored samples using gamma prior under four different loss functions, namely quadratic loss function, weighted loss functions, linear and non-linear exponential loss function. The estimators are compared by mean squared error criteria due to a simulation study. We also find that the mean square error is the best performance of the estimator from that found in quadratic, weighted, linear and non-linear exponential loss functions.

A Semi-Supervised Graph Convolutional Network for Early Prediction of Motor Abnormalities in Very Preterm Infants.

Approximately 32-42% of very preterm infants develop minor motor abnormalities. Earlier diagnosis soon after birth is urgently needed because the first two years of life represent a critical window of opportunity for early neuroplasticity in infants. In this study, we developed a semi-supervised graph convolutional network (GCN) model that is able to simultaneously learn the neuroimaging features of subjects and consider the pairwise similarity between them. The semi-supervised GCN model also allows us to combine labeled data with additional unlabeled data to facilitate model training. We conducted our experiments on a multisite regional cohort of 224 preterm infants (119 labeled subjects and 105 unlabeled subjects) who were born at 32 weeks or earlier from the Cincinnati Infant Neurodevelopment Early Prediction Study. A weighted loss function was applied to mitigate the impact of an imbalanced positive:negative (~1:2) subject ratio in our cohort. With only labeled data, our GCN model achieved an accuracy of 66.4% and an AUC of 0.67 in the early prediction of motor abnormalities, outperforming prior supervised learning models. By taking advantage of additional unlabeled data, the GCN model had significantly better accuracy (68.0%, p = 0.016) and a higher AUC (0.69, p = 0.029). This pilot work suggests that the semi-supervised GCN model can be utilized to aid early prediction of neurodevelopmental deficits in preterm infants.

Robust scalar-on-function partial quantile regression

Compared with the conditional mean regression-based scalar-on-function regression model, the scalar-on-function quantile regression is robust to outliers in the response variable. However, it is susceptible to outliers in the functional predictor (called leverage points). This is because the influence function of the regression quantiles is bounded in the response variable but unbounded in the predictor space. The leverage points may alter the eigenstructure of the predictor matrix, leading to poor estimation and prediction results. This study proposes a robust procedure to estimate the model parameters in the scalar-on-function quantile regression method and produce reliable predictions in the presence of both outliers and leverage points. The proposed method is based on a functional partial quantile regression procedure. We propose a weighted partial quantile covariance to obtain functional partial quantile components of the scalar-on-function quantile regression model. After the decomposition, the model parameters are estimated via a weighted loss function, where the robustness is obtained by iteratively reweighting the partial quantile components. The estimation and prediction performance of the proposed method is evaluated by a series of Monte-Carlo experiments and an empirical data example. The results are compared favorably with several existing methods. The method is implemented in an R package robfpqr.

A deep supervised cross-attention strategy for ischemic stroke segmentation in MRI studies

The key component of stroke diagnosis is the localization and delineation of brain lesions, especially from MRI studies. Nonetheless, this manual delineation is time-consuming and biased by expert opinion. The main purpose of this study is to introduce an autoencoder architecture that effectively integrates cross-attention mechanisms, together with hierarchical deep supervision to delineate lesions under scenarios of remarked unbalance tissue classes, challenging geometry of the shape, and a variable textural representation. This work introduces a cross-attention deep autoencoder that focuses on the lesion shape through a set of convolutional saliency maps, forcing skip connections to preserve the morphology of affected tissue. Moreover, a deep supervision training scheme was herein adapted to induce the learning of hierarchical lesion details. Besides, a special weighted loss function remarks lesion tissue, alleviating the negative impact of class imbalance. The proposed approach was validated on the public ISLES2017 dataset outperforming state-of-the-art results, achieving a dice score of 0.36 and a precision of 0.42. Deeply supervised cross-attention autoencoders, trained to pay more attention to lesion tissue, are better at estimating ischemic lesions in MRI studies. The best architectural configuration was achieved by integrating ADC, TTP and Tmax sequences. The contribution of deeply supervised cross-attention autoencoders allows better support the discrimination between healthy and lesion regions, which in consequence results in favorable prognosis and follow-up of patients.

Program Factors Affecting Weight Loss and Mobility in Older Adults: Evidence From the Mobility and Vitality Lifestyle Program (MOVE UP).

Background. The Mobility and Vitality Lifestyle Program (MOVE UP) is a behavioral weight-management intervention for improving mobility among community-dwelling older adults. We examined program factors that affect implementation outcomes and participant-level health outcomes. Methods. The MOVE UP program was implemented in the greater Pittsburgh area from January 2015 to June 2019 to improve lower extremity performance in community-dwelling older adults who were overweight or obese. Thirty-two sessions were delivered over 13 months. All sessions were designed to be 1-hour in length, on-site, group-based, and led by trained and supported community health workers (CHWs). Participants completed weekly Lifestyle Logs for self-monitoring of body weight, diet, and physical activity. We evaluated the MOVE UP program using the RE-AIM framework, and collected quantitative data at baseline, 5-, 9-, and 13-months. Multilevel linear regression models assessed the impacts of program factors (site, CHW, and participant characteristics) on implementation outcomes and participant-level health outcomes. Results. Twenty-two CHWs delivered MOVE UP program to 303 participants in 26 cohorts. Participants were similar to the target source population in weight but differed in some demographic characteristics. The program was effective for weight loss and lower extremity function in both intervention and maintenance periods (ps < .01), with an independent effect for Lifestyle Logs submission but not session attendance. Discussion. CHWs were able to deliver a multi-component weight loss intervention effectively in community settings. CHW and site characteristics had independent impacts on participants' adherence. Lifestyle Log submission may be a more potent measure of adherence in weight loss interventions than attendance.

A novel complex-valued classifier with a weighted loss function employed for W-band PS-16QAM transmission over 4.6 km

Probabilistic shaping(PS) technique can reduce Signal-Noise Ratio (SNR) requirement and save the transmission power, which is meaningful for millimeter wave (mm-wave) long-distance wireless transmission. Traditional Digital Signal Processing(DSP) algorithms are affected by the uneven probability distribution of constellation points when working on PS signals. We propose a novel complex-valued classifier with a weighted loss function to replace traditional algorithms in PS-16QAM W-band long-distance transmission. The proposed algorithm can provide a receiving sensitivity improvement of about 0.5 dBm. It can reduce the training sequence length by 4%–7%, the computational complexity by 10.53%–16.89%, and the number of training iterations by 23.08%, respectively.

A graph‐based edge attention gate medical image segmentation method

AbstractFor the purpose of solving the problems of missing edges and low segmentation accuracy in medical image segmentation, a medical image segmentation network (EAGC_UNet++) based on residual graph convolution UNet++ with edge attention gate (EAG) is proposed in the study. With UNet++ as the backbone network, the idea of graph theory is introduced into the model. First, the dropout residual graph convolution block (DropRes_GCN Block) and the traditional convolution structure in UNet++ are used as encoders. Second, EAGs are adopted so that the model pays more attention to image edge features during decoding. Finally, aiming at the imbalance problem of positive and negative samples in medical image segmentation, a new weighted loss function is introduced to enhance segmentation accuracy. In the experimental part, three datasets (LiTS2017, ISIC2018, COVID‐19 CT scans) were used to evaluate the performances of various models; multiple groups of ablation experiments were designed to verify the effectiveness of each part of the model. The experimental results showed that EAGC_UNet++ had better segmentation performance than the other models under three quantitative evaluation indicators and better solved the problem of missing edges in medical image segmentation.

Adaptive convolution kernel network for change detection in hyperspectral images.

Feature extraction is a key step in hyperspectral image change detection. However, many targets with great various sizes, such as narrow paths, wide rivers, and large tracts of cultivated land, can appear in a satellite remote sensing image at the same time, which will increase the difficulty of feature extraction. In addition, the phenomenon that the number of changed pixels is much less than unchanged pixels will lead to class imbalance and affect the accuracy of change detection. To address the above issues, based on the U-Net model, we propose an adaptive convolution kernel structure to replace the original convolution operations and design a weight loss function in the training stage. The adaptive convolution kernel contains two various kernel sizes and can automatically generate their corresponding weight feature map during training. Each output pixel obtains the corresponding convolution kernel combination according to the weight. This structure of automatically selecting the size of the convolution kernel can effectively adapt to different sizes of targets and extract multi-scale spatial features. The modified cross-entropy loss function solves the problem of class imbalance by increasing the weight of changed pixels. Study results on four datasets indicate that the proposed method performs better than most existing methods.

Soft sensor method of multimode BOF steelmaking endpoint carbon content and temperature based on vMF-WSAE dynamic deep learning

Abstract The difficulty of endpoint determination in basic oxygen furnace (BOF) steelmaking lies in achieving accurate real-time measurements of carbon content and temperature. For the characteristics of serious nonlinearity between process data, deep learning can perform excellent nonlinear feature representation for complex structural data. However, there is a process drift phenomenon in BOF steelmaking, and the existing deep learning-based soft sensor models cannot adapt to changes in the characteristics of samples, which may lead to their performance degradation. To deal with this problem, considering the characteristics of multimode distribution of process data, an adaptive updating deep learning model based on von-Mises Fisher (vMF) mixture model and weighted stacked autoencoder is proposed. First, the stacked autoencoder (SAE) and vMF mixture model are constructed for complex structural data, which can initially establish nonlinear mapping relationships and division of different distributions. Second, for each query sample, the basic SAE network will perform online adaptive fine-tuning according to its data with the same distribution to achieve dynamic updating. Moreover, each sample is assigned a weight according to its similarity with the query sample. Through the designed weighted loss function, the updated deep network will better match the working conditions of the query sample. Experimental studies with numerical examples and actual BOF steelmaking process data are provided to demonstrate the effectiveness of the proposed method.

MCAFNet: Multi-Channel Attention Fusion Network-Based CNN For Remote Sensing Scene Classification

Remote sensing scene images are characterized by intra-class diversity and inter-class similarity. When recognizing remote sensing images, traditional image classification algorithms based on deep learning only extract the global features of scene images, ignoring the important role of local key features in classification, which limits the ability of feature expression and restricts the improvement of classification accuracy. Therefore, this paper presents a multi-channel attention fusion network (MCAFNet). First, three channels are used to extract the features of the image. The channel "spatial attention module" is added after the maximum pooling layer of two channels to get the global and local key features of the image. The other channel uses the original model to extract the deep features of the image. Second, features extracted from different channels are effectively fused by the fusion module. Finally, an adaptive weight loss function is designed to automatically adjust the losses in different types of loss functions. Three challenging data sets, UC Merced Land-Use Dataset (UCM), Aerial Image Dataset (AID), and Northwestern Polytechnic University Dataset (NWPU), are selected for the experiment. Experimental results show that our algorithm can effectively recognize scenes and obtain competitive classification results.

Fine-Grained Facial Expression Recognition in Multiple Smiles

Smiling has often been incorrectly interpreted as “happy” in the popular facial expression datasets (AffectNet, RAF-DB, FERPlus). Smiling is the most complex human expression, with positive, neutral, and negative smiles. We focused on fine-grained facial expression recognition (FER) and built a new smiling face dataset, named Facial Expression Emotions. This dataset categorizes smiles into six classes of smiles, containing a total of 11,000 images labeled with corresponding fine-grained facial expression classes. We propose Smile Transformer, a network architecture for FER based on the Swin Transformer, to enhance the local perception capability of the model and improve the accuracy of fine-grained face recognition. Moreover, a convolutional block attention module (CBAM) was designed, to focus on important features of the face image and suppress unnecessary regional responses. For better classification results, an image quality evaluation module was used to assign different labels to images with different qualities. Additionally, a dynamic weight loss function was designed, to assign different learning strategies according to the labels during training, focusing on hard yet recognizable samples and discarding unidentifiable samples, to achieve better recognition. Overall, we focused on (a) creating a novel dataset of smiling facial images from online annotated images, and (b) developing a method for improved FER in smiling images. Facial Expression Emotions achieved an accuracy of 88.56% and could serve as a new benchmark dataset for future research on fine-grained FER.

Deep Clustering-Based Anomaly Detection and Health Monitoring for Satellite Telemetry

Satellite telemetry data plays an ever-important role in both the safety and the reliability of a satellite. These two factors are extremely significant in the field of space systems and space missions. Since it is challenging to repair space systems in orbit, health monitoring and early anomaly detection approaches are crucial for the success of space missions. A large number of efficient and accurate methods for health monitoring and anomaly detection have been proposed in aerospace systems but without showing enough concern for the patterns that can be mined from normal operational telemetry data. Concerning this, the present paper proposes DCLOP, an intelligent Deep Clustering-based Local Outlier Probabilities approach that aims at detecting anomalies alongside extracting realistic and reasonable patterns from the normal operational telemetry data. The proposed approach combines (i) a new deep clustering method that uses a dynamically weighted loss function with (ii) the adapted version of Local Outlier Probabilities based on the results of deep clustering. The DCLOP approach effectively monitors the health status of a spacecraft and detects the early warnings of its on-orbit failures. Therefore, this approach enhances the validity and accuracy of anomaly detection systems. The performance of the suggested approach is assessed using actual cube satellite telemetry data. The experimental findings prove that the suggested approach is competitive to the currently used techniques in terms of effectiveness, viability, and validity.

The Semantic Segmentation of Standing Tree Images Based on the Yolo V7 Deep Learning Algorithm

The existence of humans and the preservation of the natural ecological equilibrium depend greatly on trees. The semantic segmentation of trees is very important. It is crucial to learn how to properly and automatically extract a tree’s elements from photographic images. Problems with traditional tree image segmentation include low accuracy, a sluggish learning rate, and a large amount of manual intervention. This research suggests the use of a well-known network segmentation technique based on deep learning called Yolo v7 to successfully accomplish the accurate segmentation of tree images. Due to class imbalance in the dataset, we use the weighted loss function and apply various types of weights to each class to enhance the segmentation of the trees. Additionally, we use an attention method to efficiently gather feature data while reducing the production of irrelevant feature data. According to the experimental findings, the revised model algorithm’s evaluation index outperforms other widely used semantic segmentation techniques. In addition, the detection speed of the Yolo v7 model is much faster than other algorithms and performs well in tree segmentation in a variety of environments, demonstrating the effectiveness of this method in improving the segmentation performance of the model for trees in complex environments and providing a more effective solution to the tree segmentation issue.