Loss Function Research Articles

Style Migration Based on the Loss Function of Location Information

Style Migration Based on the Loss Function of Location Information

EB-Net: an efficient balanced network for accuracy and speed of remote sensing detection

ABSTRACT Remote sensing detection is a difficult task that requires not only identifying objects with complex background, quality, and angle issues, but also being lightweight enough to be carried by edge devices. In real remote sensing scenarios, achieving accurate, fast, and low-resource-consumption automated detection remains a significant challenge. Therefore, this paper proposes an efficient balanced network (EB-Net) for real remote sensing devices. First, a dynamic sparse attention (DSA) mechanism is proposed and has been proven with high performance via the complexity analysis. In addition, a new dynamic sparse transformer (DSFormer) is constructed using DSA, which enhances feature information and adapts to image resolution by self-attention and multi-headed attention, and achieves more flexible computation by random sampling. Then, three versions of discrete distribution IoU (DDIoU) are defined for adapting various scenarios and tasks in remote sensing, and this loss function makes the model achieve high accuracy and more lightweight. Finally, to make the model more lightweight, a cropped SPPF (CroSPPF) is presented, which significantly improves the computational efficiency by the lightweight of the sequence structure and activation function. Ablation experiments are conducted on the NWPU VHR-10 dataset, and demonstrate the effectiveness of the proposed methods. Numerous comparisons with state-of-the-art detectors are conducted on the NWPU VHR-10, RSOD and DOTA datasets. The experiments show that EB-Net outperforms state-of-the-art remote sensing detection models in comprehensive performance and achieves end-to-end accurate and lightweight detection.

First-principles study on the structural, elastic, electronic, optical and thermophysical properties of NaNO3 and K-doped NaNO3

The current work illustrates the optical, thermophysical, electronic, elastic, and structural characteristics of pure and K-doped NaNO3 obtained using DFT calculations. Electronic characteristics such as band structure, projected and total density of states of pure and doped NaNO3 are analyzed where the calculated band gaps for pure and K-doped NaNO3 were discovered to be 3.05 eV and 2.90 eV, respectively. The Berry–phase calculations carried out on both systems showed the existence of ferroelectric polarization, where an enhanced spontaneous polarization value of 36 μC/cm2 for K-doped NaNO3, as compared to 20 μC/cm2 for intrinsic NaNO3 is obtained. The thermophysical properties such as bulk modulus (B0), Debye temperature (θ D), and specific heat capacity (Cv) were calculated over a wide temperature range from 0 K to 800 K. The linear optical properties namely the dielectric constant, refractive index, electron energy loss function, absorption and extinction coefficient are calculated in wide energy range from 0 to 30 eV and discussed in detail.

Sequential estimation for the multiple linear regression models with balanced loss functions

Sequential estimation for the multiple linear regression models with balanced loss functions

Boundary Gaussian Distance Loss Function for Enhancing Character Extraction from High-Resolution Scans of Ancient Metal-Type Printed Books

Boundary Gaussian Distance Loss Function for Enhancing Character Extraction from High-Resolution Scans of Ancient Metal-Type Printed Books

Motion Correction for Brain MRI Using Deep Learning and a Novel Hybrid Loss Function

Purpose: Motion-induced magnetic resonance imaging (MRI) artifacts can deteriorate image quality and reduce diagnostic accuracy, but motion by human subjects is inevitable and can even be caused by involuntary physiological movements. Deep-learning-based motion correction methods might provide a solution. However, most studies have been based on directly applying existing models, and the trained models are rarely accessible. Therefore, we aim to develop and evaluate a deep-learning-based method (Motion Correction-Net, or MC-Net) for suppressing motion artifacts in brain MRI scans. Methods: A total of 57 subjects, providing 20,889 slices in four datasets, were used. Furthermore, 3T 3D sagittal magnetization-prepared rapid gradient-echo (MP-RAGE) and 2D axial fluid-attenuated inversion-recovery (FLAIR) sequences were acquired. The MC-Net was derived from a UNet combined with a two-stage multi-loss function. T1-weighted axial brain images contaminated with synthetic motions were used to train the network to remove motion artifacts. Evaluation used simulated T1- and T2-weighted axial, coronal, and sagittal images unseen during training, as well as T1-weighted images with motion artifacts from real scans. The performance indices included the peak-signal-to-noise ratio (PSNR), the structural similarity index measure (SSIM), and visual reading scores from three blinded clinical readers. A one-sided Wilcoxon signed-rank test was used to compare reader scores, with p < 0.05 considered significant. Intraclass correlation coefficients (ICCs) were calculated for inter-rater evaluations. Results: The MC-Net outperformed other methods in terms of PSNR and SSIM for the T1 axial test set. The MC-Net significantly improved the quality of all T1-weighted images for all directions (i.e., the mean SSIM of axial, sagittal, and coronal slices improved from 0.77, 0.64, and 0.71 to 0.92, 0.75, and 0.84; the mean PSNR improved from 26.35, 24.03, and 24.55 to 29.72, 24.40, and 25.37, respectively) and for simulated as well as real motion artifacts, both using quantitative measures and visual scores. However, MC-Net performed poorly for images with untrained T2-weighted contrast because the T2 contrast was unseen during training and is different from T1 contrast. Conclusion: The proposed two-stage multi-loss MC-Net can effectively suppress motion artifacts in brain MRI without compromising image quality. Given the efficiency of MC-Net (with a single-image processing time of ~40 ms), it can potentially be used in clinical settings.

Enhancing quality and speed in database-free neural network reconstructions of undersampled MRI with SCAMPI.

We present SCAMPI (Sparsity Constrained Application of deep Magnetic resonance Priors for Image reconstruction), an untrained deep Neural Network for MRI reconstruction without previous training on datasets. It expands the Deep Image Prior approach with a multidomain, sparsity-enforcing loss function to achieve higher image quality at a faster convergence speed than previously reported methods. Two-dimensional MRI data from the FastMRI dataset with Cartesian undersampling in phase-encoding direction were reconstructed for different acceleration rates for single coil and multicoil data. The performance of our architecture was compared to state-of-the-art Compressed Sensing methods and ConvDecoder, another untrained Neural Network for two-dimensional MRI reconstruction. SCAMPI outperforms these by better reducing undersampling artifacts and yielding lower error metrics in multicoil imaging. In comparison to ConvDecoder, the U-Net architecture combined with an elaborated loss-function allows for much faster convergence at higher image quality. SCAMPI can reconstruct multicoil data without explicit knowledge of coil sensitivity profiles. Moreover, it is a novel tool for reconstructing undersampled single coil k-space data. Our approach avoids overfitting to dataset features, that can occur in Neural Networks trained on databases, because the network parameters are tuned only on the reconstruction data. It allows better results and faster reconstruction than the baseline untrained Neural Network approach.

Noise source localization using deep learning

Summary Ambient noise source localization is of great significance for estimating seismic noise source distribution, understanding source mechanisms and imaging subsurface structures. The commonly used methods for source localization, such as the matched field processing and the full-waveform inversion, are time-consuming and not applicable for time-lapse monitoring of the noise source distribution. We propose an efficient alternative of using deep learning for noise source localization. In the neural network, the input data are noise cross-correlation functions and the output are matrices containing the information of noise source distribution. It is assumed that the subsurface structure is a horizontally layered earth model and the model parameters are known. A wavefield superposition method is employed to efficiently simulate ambient noise data with quantities of local noise sources labelled as training datasets. We use a weighted binary cross-entropy loss function to address the prediction inaccuracy caused by a sparse label matrix during training. The proposed deep learning framework is validated by synthetic tests and two field data examples. The successful applications to locate an anthropogenic noise source and a carbon dioxide (CO2) degassing area demonstrate the accuracy and efficiency of the proposed deep learning method for noise source localization, which has great potential for monitoring the changes of the noise source distribution in a survey area.

Semi-Supervised Medical Image Classification with Pseudo Labels Using Coalition Similarity Training

The development of medical image classification models necessitates a substantial number of labeled images for model training. In real-world scenarios, sample sizes are typically limited and labeled samples often constitute only a small portion of the dataset. This paper aims to investigate a collaborative similarity learning strategy that optimizes pseudo-labels to enhance model accuracy and expedite its convergence, known as the joint similarity learning framework. By integrating semantic similarity and instance similarity, the pseudo-labels are mutually refined to ensure their quality during initial training. Furthermore, the similarity score is utilized as a weight to guide samples away from misclassification predictions during the classification process. To enhance the model’s generalization ability, an adaptive consistency constraint is introduced into the loss function to improve performance on untrained datasets. The model achieved a satisfactory accuracy of 93.65% at 80% labeling ratio, comparable to supervised learning methods’ performance. Even with very low labeling ratio (e.g., 5%), the model still attained an accuracy of 74.28%. Comparison with other techniques such as Mean Teacher and FixMatch revealed that our approach significantly outperforms them in medical image classification tasks through improving accuracy by approximately 2%, demonstrating this framework’s leadership in medical image classification.

Non-Differentiable Loss Function Optimization and Interaction Effect Discovery in Insurance Pricing Using the Genetic Algorithm

Non-Differentiable Loss Function Optimization and Interaction Effect Discovery in Insurance Pricing Using the Genetic Algorithm

Application of improved DBN and GRU based on intelligent optimization algorithm in power load identification and prediction

Non intrusive load monitoring belongs to the key technologies of intelligent power management systems, playing a crucial role in smart grids. To achieve accurate identification and prediction of electricity load, intelligent optimization algorithms are introduced into deep learning optimization for improvement. A load recognition model combining sparrow search algorithm and deep confidence network is designed, as well as a gated recurrent network prediction model on the grounds of particle swarm optimization. The relevant results showed that the sparrow search algorithm used in the study performed well on the solution performance evaluation metrics with a minimum value of 0.209 for the inverse generation distance and a maximum value of 0.814 for the hyper-volume. The accuracy and recall values of the optimized load identification model designed in the study were relatively high. When the accuracy was 0.9, the recall rate could reach 0.94. The recognition accuracy of the model on the basis of the test set could reach up to 0.924. The lowest classification error was only 0.05. The maximum F1 value of the harmonic evaluation index of the bidirectional gated recurrent network optimized by particle swarm optimization converged to 90.06%. The loss function had been optimized by particle swarm optimization, and both the convergence value and convergence speed had been markedly enhanced. The average absolute error and root mean square error of the prediction model were both below 0.3. Compared to the bidirectional gated recurrent model before optimization, the particle swarm optimization strategy had a significant improvement effect on prediction details. In addition, the research method had superior recognition response speed and adaptability in real application environments. This study helps to understand the load demand of the power system, optimize the operation of the power grid, and strengthen the reliability, efficiency, and sustainability of the power system.

Managing Expectations and Imbalanced Training Data in Reactive Force Field Development: An Application to Water Adsorption on Alumina.

ReaxFF is a computationally efficient model for reactive molecular dynamics simulations that has been applied to a wide variety of chemical systems. When ReaxFF parameters are not yet available for a chemistry of interest, they must be (re)optimized, for which one defines a set of training data that the new ReaxFF parameters should reproduce. ReaxFF training sets typically contain diverse properties with different units, some of which are more abundant (by orders of magnitude) than others. To find the best parameters, one conventionally minimizes a weighted sum of squared errors over all of the data in the training set. One of the challenges in such numerical optimizations is to assign weights so that the optimized parameters represent a good compromise among all the requirements defined in the training set. This work introduces a new loss function, called Balanced Loss, and a workflow that replaces weight assignment with a more manageable procedure. The training data are divided into categories with corresponding "tolerances", i.e., acceptable root-mean-square errors for the categories, which define the expectations for the optimized ReaxFF parameters. Through the Log-Sum-Exp form of Balanced Loss, the parameter optimization is also a validation of one's expectations, providing meaningful feedback that can be used to reconfigure the tolerances if needed. The new methodology is demonstrated with a nontrivial parametrization of ReaxFF for water adsorption on alumina. This results in a new force field that reproduces both the rare and frequent properties of a validation set not used for training. We also demonstrate the robustness of the new force field with a molecular dynamics simulation of water desorption from a γ-Al2O3 slab model.

STAM-LSGRU: a spatiotemporal radar echo extrapolation algorithm with edge computing for short-term forecasting

With the advent of Mobile Edge Computing (MEC), shifting data processing from cloud centers to the network edge presents an advanced computational paradigm for addressing latency-sensitive applications. Specifically, in radar systems, the real-time processing and prediction of radar echo data pose significant challenges in dynamic and resource-constrained environments. MEC, by processing data near its source, not only significantly reduces communication latency and enhances bandwidth utilization but also diminishes the necessity of transmitting large volumes of data to the cloud, which is crucial for improving the timeliness and efficiency of radar data processing. To meet this demand, this paper proposes a model that integrates a spatiotemporal Attention Module (STAM) with a Long Short-Term Memory Gated Recurrent Unit (ST-ConvLSGRU) to enhance the accuracy of radar echo prediction while leveraging the advantages of MEC. STAM, by extending the spatiotemporal receptive field of the prediction units, effectively captures key inter-frame motion information, while optimizations to the convolutional structure and loss function further boost the model’s predictive performance. Experimental results demonstrate that our approach significantly improves the accuracy of short-term weather forecasting in a mobile edge computing environment, showcasing an efficient and practical solution for processing radar echo data under dynamic, resource-limited conditions.

Insulator defect detection in transmission line based on an improved lightweight YOLOv5s algorithm

As one of the most important components on the transmission line, the insulator is prone to failure, which affects the safe operation of electrical power system. Hence, it is crucial to accurately detect the insulator defects for further maintenance in time. Recently, with the development of artificial intelligence and target detection algorithms, the insulator defect detection has received more and more attention. However, there are still existing some difficulties: insufficient samples and low detection accuracy. To improve the accuracy of insulator defect detection, this paper proposes an auto-detection method based on an improved lightweight YOLOv5s model. First, this paper introduces the basic network frame of YOLOv5s and proposes an improved algorithm by utilizing the GIoU loss function, Mish activation function, and CBAM module. Then, performs data enhancement in the insulator dataset to enhance the robustness of the model. Finally, trains and tests the improved YOLOv5s model, and compares it with traditional target detection algorithms. Compared with traditional target detection algorithms, the AP value of the proposed algorithm in detecting insulator defects can be improved by 5 %. The results demonstrate the improved algorithm proposed in this paper can effectively identify and position the insulator defects.

Singular Value Decomposition-Based Penalized Multinomial Regression for Classifying Imbalanced Medulloblastoma Subgroups Using Methylation Data.

Medulloblastoma (MB) is a molecularly heterogeneous brain malignancy with large differences in clinical presentation. According to genomic studies, there are at least four distinct molecular subgroups of MB: sonic hedgehog (SHH), wingless/INT (WNT), Group 3, and Group 4. The treatment and outcomes depend on appropriate classification. It is difficult for the classification algorithms to identify these subgroups from an imbalanced MB genomic data set, where the distribution of samples among the MB subgroups may not be equal. To overcome this problem, we used singular value decomposition (SVD) and group lasso techniques to find DNA methylation probe features that maximize the separation between the different imbalanced MB subgroups. We used multinomial regression as a classification method to classify the four different molecular subgroups of MB using the reduced DNA methylation data. Coordinate descent is used to solve our loss function associated with the group lasso, which promotes sparsity. By using SVD, we were able to reduce the 321,174 probe features to just 200 features. Less than 40 features were successfully selected after applying the group lasso, which we then used as predictors for our classification models. Our proposed method achieved an average overall accuracy of 99% based on fivefold cross-validation technique. Our approach produces improved classification performance compared with the state-of-the-art methods for classifying MB molecular subgroups.

Reducing Training Data Needs with Minimal Multilevel Machine Learning (M3L)

Abstract For many machine learning applications in science, data acquisition, not training, is the bottleneck even when avoiding experiments and relying on computation and simulation. 
Correspondingly, and in order to reduce cost and carbon footprint, training data efficiency is key. 
We introduce minimal multilevel machine learning (M3L) which 
optimizes training data set sizes using a loss function at multiple levels of reference data in order to minimize a combination of prediction error with overall training data acquisition costs (as measured by computational wall-times).
Numerical evidence has been obtained for calculated atomization energies and electron affinities of thousands of organic molecules at various levels of theory including HF, MP2, DLPNO-CCSD(T), DFHFCABS, PNOMP2F12, and PNOCCSD(T)F12, and treating them with 
basis sets TZ, cc-pVTZ, and AVTZ-F12.
Our M3L benchmarks for reaching chemical accuracy in distinct chemical compound sub-spaces 
indicate substantial computational cost reductions by factors of 
$\sim$ 1.01, 1.1, 3.8, 13.8, and 25.8 when compared to heuristic sub-optimal multilevel machine learning (M2L) for the data sets QM7b, QM9$^\mathrm{LCCSD(T)}$, EGP, QM9$^\mathrm{CCSD(T)}_\mathrm{AE}$, and QM9$^\mathrm{CCSD(T)}_\mathrm{EA}$, respectively.
Furthermore, we use M2L to investigate the performance for 76 density functionals when used within multilevel learning and building on the following levels drawn from the hierarchy of Jacobs Ladder:~LDA, GGA, mGGA, and hybrid functionals.
Within M2L and the molecules considered, 
mGGAs do not provide any noticeable advantage over GGAs. 
Among the functionals considered and in combination with LDA, the three on average top performing GGA and Hybrid levels for atomization energies on QM9 using M3L correspond respectively to PW91, KT2, B97D, and $\tau$-HCTH, B3LYP$\ast$(VWN5), and TPSSH.

LSTM-Autoencoder Deep Learning Model for Anomaly Detection in Electric Motor

Anomaly detection is the process of detecting unusual or unforeseen patterns or events in data. Many factors, such as malfunctioning hardware, malevolent activities, or modifications to the data’s underlying distribution, might cause anomalies. One of the key factors in anomaly detection is balancing the trade-off between sensitivity and specificity. Balancing these trade-offs requires careful tuning of the anomaly detection algorithm and consideration of the specific domain and application. Deep learning techniques’ applications, such as LSTMs (long short-term memory algorithms), which are autoencoders for detecting an anomaly, have garnered increasing attention in recent years. The main goal of this work was to develop an anomaly detection solution for an electrical machine using an LSTM-autoencoder deep learning model. The work focused on detecting anomalies in an electrical motor’s variation vibrations in three axes: axial (X), radial (Y), and tangential (Z), which are indicative of potential faults or failures. The presented model is a combination of the two architectures; LSTM layers were added to the autoencoder in order to leverage the LSTM capacity for handling large amounts of temporal data. To prove the LSTM efficiency, we will create a regular autoencoder model using the Python programming language and the TensorFlow machine learning framework, and compare its performance with our main LSTM-based autoencoder model. The two models will be trained on the same database, and evaluated on three primary points: training time, loss function, and MSE anomalies. Based on the obtained results, it is clear that the LSTM-autoencoder shows significantly smaller loss values and MSE anomalies compared to the regular autoencoder. On the other hand, the regular autoencoder performs better than the LSTM, comparing the training time. It appears then, that the LSTM-autoencoder presents a superior performance although it was slower than the standard autoencoder due to the complexity of the added LSTM layers.

Self-supervised monocular visual odometry based on cross-correlation

Visual odometry constitutes a critical component in enabling autonomous navigation. However, the existing methods are limited by the feature extraction and matching accuracy, and cannot show good real-time performance while combining accuracy and robustness. In this paper, we propose a novel monocular visual odometry framework based on cross-correlation. The framework starts with a parameter-sharing Siamese network to build feature extractors that can simultaneously process multiple images as inputs. Moreover, we design cross-correlation modules and define a cross-correlation matrix to describe the strength of correlation between different parts of the input feature maps, reflecting the rotational and translational transformations of the input images. Furthermore, a novel loss function is introduced to impose constraints on the network. Additionally, a fully convolutional network is designed for pose estimation, computing poses alterations from the structure of the cross-correlation matrix. Channel attention and spatial attention mechanisms are introduced to improve the performance. More importantly, our method innovatively uses time intervals as labels, enables self-supervised training, and relies only on a monocular camera. Experimental results on the KITTI visual odometry dataset and the Oxford Robotcar Dataset show that our method produces competitive performance, demonstrating the superiority of the proposed method.

Semi-supervised method for visual detection of automotive paint defects

Automotive paint defect detection plays a crucial role in the automotive production process. Current research on visual defect detection methods is mainly based on supervised learning, which requires a large number of labeled image samples for model training. The labeling work is not only time consuming but also expensive, seriously hindering the testing and application of these models in practice. To address this issue, this study proposes a new method for automotive paint defect detection based on a semi-supervised training strategy. First, a semi-supervised automotive paint defect detection framework, which can use labeled and unlabeled samples to reduce the cost of data labeling effectively, is presented. Then, a spatial pyramid pooling fast external attention module that introduces an external attention mechanism is proposed to improve the traditional YOLOv7 network structure, called YOLOv7-EA, to obtain good detection performance. This network acts as a detector to generate high-quality pseudo labels for the unlabeled samples, providing additional data to train the model; meanwhile, it performs the final detection task. Lastly, a Wise-intersection over union loss function that considers the quality of the anchor box is introduced to reduce the interference of low-quality samples and improve the convergence speed and detection accuracy of the model. Using this method, we can accomplish the task of automotive paint defect detection with a small number of labeled image samples. Experimental results on the automotive paint defect dataset show that mean average precision (mAp)@.5, mAp@.75, and mAp@.5:.95 are superior to other methods under the condition of 10% and 15% labeled data, achieving good defect detection performance.

Improved YOLOv8 and SAHI Model for the Collaborative Detection of Small Targets at the Micro Scale: A Case Study of Pest Detection in Tea

Pest target identification in agricultural production environments is challenging due to the dense distribution, small size, and high density of pests. Additionally, changeable environmental lighting and complex backgrounds further complicate the detection process. This study focuses on enhancing the recognition performance of tea pests by introducing a lightweight pest image recognition model based on the improved YOLOv8 architecture. First, slicing-aided fine-tuning and slicing-aided hyper inference (SAHI) are proposed to partition input images for enhanced model performance on low-resolution images and small-target detection. Then, based on an ELAN, a generalized efficient layer aggregation network (GELAN) is designed to replace the C2f module in the backbone network, enhance its feature extraction ability, and construct a lightweight model. Additionally, the MS structure is integrated into the neck network of YOLOv8 for feature fusion, enhancing the extraction of fine-grained and coarse-grained semantic information. Furthermore, the BiFormer attention mechanism, based on the Transformer architecture, is introduced to amplify target characteristics of tea pests. Finally, the inner-MPDIoU, based on auxiliary borders, is utilized as a replacement for the original loss function to enhance its learning capacity for complex pest samples. Our experimental results demonstrate that the enhanced YOLOv8 model achieves a precision of 96.32% and a recall of 97.95%, surpassing those of the original YOLOv8 model. Moreover, it attains an mAP@50 score of 98.17%. Compared to Faster R-CNN, SSD, YOLOv5, YOLOv7, and YOLOv8, its average accuracy is 17.04, 11.23, 5.78, 3.75, and 2.71 percentage points higher, respectively. The overall performance of YOLOv8 outperforms that of current mainstream detection models, with a detection speed of 95 FPS. This model effectively balances lightweight design with high accuracy and speed in detecting small targets such as tea pests. It can serve as a valuable reference for the identification and classification of various insect pests in tea gardens within complex production environments, effectively addressing practical application needs and offering guidance for the future monitoring and scientific control of tea insect pests.