Loss Function Research Articles

Quantifying Morphologies of Developing Neuronal Cells using Deep Learning with Imperfect Annotations

The functionality of human intelligence relies on the interaction and health of neurons, hence, quantifying neuronal morphologies can be crucial for investigating the functionality of the human brain. This paper proposes a deep learning (DL) based method for segmenting and quantifying neuronal structures in fluorescence microscopy images of developing neuronal cells cultured in vitro. Compared to the majority of supervised DL-based segmentation methods that heavily rely on creating exact corresponding masks of neuronal structures for the preparation of training samples, the proposed approach allows for imperfect annotation of neurons, as it only requires tracing the centrelines of the neurites. This ability accelerates the preparation of training data by several folds. Our proposed framework is built on a modified version of PSPNet with an EfficientNet backbone pre-trained on the CityScapes dataset. To handle the imperfectness of training samples, we incorporated a weighted combination of two loss functions, namely the Dice loss and Lovász loss functions, into our network. We evaluated the proposed framework and several other state-of-the-art methods on a published dataset of approximately 900 manually quantified cultured mouse neurons. Our results indicate a close correlation between the proposed method and manual quantification in terms of neuron length and the number of branches while demonstrating improved analysis speed. Furthermore, the proposed method achieved high accuracy in neuron segmentation, as evidenced by the evaluation of the neurons’ length and number of branches.

A General Strategy for Improving the Performance of PINNs -- Analytical Gradients and Advanced Optimizers in the NeuralSchrödinger Framework

In this work, the previously introduced NeuralSchrödinger PINN is extended towards the use of analytical gradient expressions of the loss function. It is shown that the analytical gradients derived in this work increase the convergence properties for both the BFGS and ADAM optimizers compared to the previously employed numerical gradient implementation. In addition, the use of parallelised GPU computations via CUDA greatly increased the computational performance over the previous implementation using single-core CPU computations. As a consequence, an extension of the NeuralSchrödinger PINN towards two-dimensional quantum systems became feasible as also demonstrated in this work.

Face recognition with occluded face using improve intersection over union of region proposal network on Mask region convolutional neural network

Face recognition entails detecting and identifying facial attributes. Mask region convolutional neural network (R-CNN) method is a prominent approach, while prior research predominantly delved into refining loss functions and perfecting object and face detection, recognizing, and identifying faces using imperfect data remained relatively unexplored. This study focuses on an occluded dataset comprising Indonesian faces, wherein 'occluded' denotes facial data that lacks complete visibility-encompassing instances where objects obscure faces or are partially cropped. This investigation involves a deliberate experiment that tailors the intersection over union (IoU) of the region proposal network (RPN) to suit the nuances of occluded Indonesian faces, thereby augmenting accuracy in recognition and segmentation tasks. The innovation IoU in the strategic utilization of Anchors, which involves the exclusion of anchors falling beyond the image borders to optimize computational efficiency. The outcomes of this research are striking; it showcases a remarkable 14.75%, 10.9%, and 12.97% surge based on mean average precision (mAP), mean average recall (mAR), and F1-Scores compared to the conventional Mask R-CNN approach. Notably, our proposed model elevates the average accuracy by 10% to 15% and decreases running time by 21%, a noteworthy enhancement compared to the preceding model. This progress is substantiated by validation utilizing 300 instances dataset, reinforcing the robustness of our approach.

Optimizing loss functions for improved energy demand prediction in smart power grids

In this paper, our aim is to improve the accuracy and effectiveness of energy demand forecasting, particularly within modern electricity transmission systems and smart grid technology. To achieve this, we developed a hybrid approach that combines machine learning, representation learning, and other deep learning techniques. This approach is based on extracting essential features, including time-based attributes, identifiable trends, and optimal lags. The outcome of our investigation is the observation that triplet losses demonstrate remarkable accuracy, particularly when employed with a larger margin size and for longer prediction lengths. This finding signifies a substantial improvement in the precision and reliability of energy demand forecasting within modern electricity transmission systems. Our research not only improves predictive modeling in the power grid but also demonstrates the practical use of advanced analytics in addressing renewable energy integration challenges, refining energy demand forecasting for efficient management, system operation, and market analysis.

Incorporating Season and Solar Specificity Into Renderings Made by a NeRF Architecture Using Satellite Images.

As a result of Shadow NeRF and Sat-NeRF, it is possible to take the solar angle into account in a NeRF-based framework for rendering a scene from a novel viewpoint using satellite images for training. Our work extends those contributions and shows how one can make the renderings season-specific. Our main challenge was creating a Neural Radiance Field (NeRF) that could render seasonal features independently of viewing angle and solar angle while still being able to render shadows. We teach our network to render seasonal features by introducing one more input variable - time of the year. However, the small training datasets typical of satellite imagery can introduce ambiguities in cases where shadows are present in the same location for every image of a particular season. We add additional terms to the loss function to discourage the network from using seasonal features for accounting for shadows. We show the performance of our network on eight Areas of Interest containing images captured by the Maxar WorldView-3 satellite. This evaluation includes tests measuring the ability of our framework to accurately render novel views, generate height maps, predict shadows, and specify seasonal features independently from shadows. Our ablation studies justify the choices made for network design parameters.

Next-Gen Linguistic Intelligence Image Summarizer for Documents Using Attention Model

Abstract: The process of producing a succinct synopsis of a text involves two primary techniques: both abstractive and extractive While abstractive summarising rephrases text in a way that is more human-like, extractive summarization chooses important statements straight. handbook Because it takes time and money to summarise because it requires reading through big texts, an automated text model with lots of features It is suggested to use both extractive and abstractive methods for summarising. Important sentences that are enhanced by linguistic features are identified by a feature-rich extractor. These sentences are then further enhanced by an abstracter that adds components such named entity tags and part-of-speech tags. Moreover, a loss function is presented to match attentions at the word and sentence levels. At last the summarized text is converted into image by using parsing and other relatable techniques.

Object Detection Model: An Enhanced SSD

Abstract: Object Detection plays a significant role in today’s era. Object detection plays a crucial role in various applications such as surveillance, medical images, and autonomous vehicles. The primary goal of object detection is to accurately detect the boundaries of objects of interest and classify those objects into predefined categories. In this research paper, we enhanced an object detection technique i.e. Single shot multibox detector (SSD) which is one of the top object detection technique in both aspect accuracy and speed. SSD is an object detection model that predicts object bounding boxes and class probabilities in a single forward pass. It uses multiple feature maps at different scales to cater to various object sizes and aspect ratios.The main contribution of this research is to enhance the accuracy, recall time, precision time and mean average time (mAP). The performance of this object detection technique is improved as the number of feature maps is increases, improved backbone, and enhanced activation functions, additional layers for enhanced detection and loss function modifications. An accuracy of 0.75 indicates that the model correctly predicts the class label for approximately 75% of all objects in the dataset. The enhanced model showed these results: a precision score of 0.75 implies that 75% of the objects identified by the model as positive (detected objects) are indeed true positives. A recall of 0.73 indicates that the model successfully identifies and detects 73% of all positive instances in the dataset. A mAP of 75.25 signifies the average precision across all classes, computed at different IOU thresholds (e.g., 0.5, 0.75).

Correlation of Seismic Loss Functions Based on Stories and Core Locations in Vertical-Irregular Structures

Correlation of Seismic Loss Functions Based on Stories and Core Locations in Vertical-Irregular Structures

Forecasting fish mortality from water and air quality data using deep learning models.

High rate of aquatic mortality incidents recorded in Taiwan and worldwide is creating an urgent demand for more accurate fish mortality prediction. Present study innovatively integrated air and water quality data to measure water quality degradation, and utilized deep learning methods to predict accidental fish mortality from the data. Keras library was used to build multilayer perceptron and long short-term memory models for training purposes, and the models' accuracies in fish mortality prediction were compared with that of the naïve Bayesian classifier. Environmental data from the 5 days before a fish mortality event proved to be the most important data for effective model training. Multilayer perceptron model reached an accuracy of 93.4%, with a loss function of 0.01, when meteorological and water quality data were jointly considered. It was found that meteorological conditions were not the sole contributors to fish mortality. Predicted fish mortality rate of 4.7% closely corresponded to the true number of fish mortality events during the study period, that is, four. A significant surge in fish mortality, from 20% to 50%, was noted when the river pollution index increased from 5.36 to 6.5. Moreover, the probability of fish mortality increased when the concentration of dissolved oxygen dropped below 2mg/L. To mitigate fish mortality, ammonia nitrogen concentrations should be capped at 5mg/L. Dissolved oxygen concentration was found to be the paramount factor influencing fish mortality, followed by the river pollution index and meteorological data. Results of the present study are expected to aid progress toward achieving the Sustainable Development Goals and to increase the profitability of water resources.

Physics-guided neural network for grinding temperature prediction

Creep-feed grinding is a high-efficiency, high-precision grinding process widely used in the manufacturing of aviation engines. However, the workpiece burn and other quality issues caused by high processing temperature limit the yield of grinding. Therefore, the accurate model of grinding temperature has become the key to improving processing efficiency and quality. Different from the traditional physical or data-driven models, this paper attempts to combine both perspectives based on Physics-Guided Neural Networks (PGNN) to accurately predict grinding temperature with a small number of experiments. At the level of data acquisition, real grinding experiment data was obtained and a data augmentation method had been proposed. At the level of neural network structure, optimisation processes were implemented to enhance prediction performance, and a physics-guided loss function was inserted to guide network training. The experiment results shows that PGNN had better prediction accuracy than the physical model, while also mitigating the limitations of data-driven models on small sample sets. PGNN also performed better with noisy data and predictions out of the training data range, this reveals the benefits of PGNN for small sample problems in processing scenarios.

Detecting Road Intersections from Crowdsourced Trajectory Data Based on Improved YOLOv5 Model

In recent years, the rapid development of autonomous driving and intelligent driver assistance has brought about urgent demands on high-precision road maps. However, traditional road map production methods mainly rely on professional survey technologies, such as remote sensing and mobile mapping, which suffer from high costs, object occlusions, and long updating cycles. In the era of ubiquitous mapping, crowdsourced trajectory data offer a new and low-cost data resource for the production and updating of high-precision road maps. Meanwhile, as key nodes in the transportation network, maintaining the currency and integrity of road intersection data is the primary task in enhancing map updates. In this paper, we propose a novel approach for detecting road intersections based on crowdsourced trajectory data by introducing an attention mechanism and modifying the loss function in the YOLOv5 model. The proposed method encompasses two key steps of training data preparation and improved YOLOv5s model construction. Multi-scale training processing is first adopted to prepare a rich and diverse sample dataset, including various kinds and different sizes of road intersections. Particularly to enhance the model’s detection performance, we inserted convolutional attention mechanism modules into the original YOLOv5 and integrated other alternative confidence loss functions and localization loss functions. The experimental results demonstrate that the improved YOLOv5 model achieves detection accuracy, precision, and recall rates as high as 97.46%, 99.57%, and 97.87%, respectively, outperforming other object detection models.

Enhanced Network Security Protection through Data Analysis and Machine Learning: An Application of GraphSAGE for Anomaly Detection and Operational Intelligence

With the Internet's rapid expansion, network security challenges have become increasingly complex and prominent. Traditional protection methods, largely dependent on predefined rules and patterns, demonstrate limited effectiveness against sophisticated and unknown network attacks, failing to harness the full potential of extensive network data. This study addresses the challenges faced by modern cybersecurity, particularly the limitations of traditional defense methods in countering unknown and complex attacks, by proposing a solution that integrates data analysis and machine learning technologies. The focus of this research is placed on network security anomaly detection as well as on intelligent network operations and maintenance exception management based on graph network algorithms, aiming to enhance security defense capabilities and operational efficiency. Specifically, the main contributions and innovations of this paper include: 1. Innovations in sampling, aggregation, and loss functions within the Graph Sample and Aggregation (GraphSAGE) model to improve the accuracy and robustness of the model for network anomaly detection; 2. The introduction of a novel network anomaly root cause analysis and localization model, which, combined with an optimized root cause likelihood assessment method and search scheme, significantly enhances the speed and accuracy of anomaly localization; 3. The design of an integrated decision support system that can automatically adjust protection strategies as network conditions change, achieving a high level of automation and intelligence in cybersecurity management. This work not only provides effective technical support for network security protection but also opens new avenues for future cybersecurity research.

Fine Segmentation of Chinese Character Strokes Based on Coordinate Awareness and Enhanced BiFPN

Considering the complex structure of Chinese characters, particularly the connections and intersections between strokes, there are challenges in low accuracy of Chinese character stroke extraction and recognition, as well as unclear segmentation. This study builds upon the YOLOv8n-seg model to propose the YOLOv8n-seg-CAA-BiFPN Chinese character stroke fine segmentation model. The proposed Coordinate-Aware Attention mechanism (CAA) divides the backbone network input feature map into four parts, applying different weights for horizontal, vertical, and channel attention to compute and fuse key information, thus capturing the contextual regularity of closely arranged stroke positions. The network’s neck integrates an enhanced weighted bi-directional feature pyramid network (BiFPN), enhancing the fusion effect for features of strokes of various sizes. The Shape-IoU loss function is adopted in place of the traditional CIoU loss function, focusing on the shape and scale of stroke bounding boxes to optimize the bounding box regression process. Finally, the Grad-CAM++ technique is used to generate heatmaps of segmentation predictions, facilitating the visualization of effective features and a deeper understanding of the model’s focus areas. Trained and tested on the public Chinese character stroke datasets CCSE-Kai and CCSE-HW, the model achieves an average accuracy of 84.71%, an average recall rate of 83.65%, and a mean average precision of 80.11%. Compared to the original YOLOv8n-seg and existing mainstream segmentation models like SegFormer, BiSeNetV2, and Mask R-CNN, the average accuracy improved by 3.50%, 4.35%, 10.56%, and 22.05%, respectively; the average recall rates improved by 4.42%, 9.32%, 15.64%, and 24.92%, respectively; and the mean average precision improved by 3.11%, 4.15%, 8.02%, and 19.33%, respectively. The results demonstrate that the YOLOv8n-seg-CAA-BiFPN network can accurately achieve Chinese character stroke segmentation.

A de-texturing model for enhancing the accuracy of centroid positioning

In tasks guided by microvision, extracting centroids is a common method for positioning, which is negatively affected by texture. Here, an attention-related de-texturing model is proposed to eliminate the texture of microparts and preserve accurate edges. A network with an attention module called De-texturing Net is built, in which both the transformer and channel attention modules are included. Considering the importance of texture, the additional factor in loss function is constructed based on the Gram matrix difference between target images and generated images. Results show that De-texturing Net can generate de-texturized images with high Peak Signal to Noise Ratio/SSIM, indicating the similarity between de-texturized and target images. Moreover, for the centroid positioning, the error in de-texturized images is significantly lower than the error in original images. This study helps improve the accuracy of centroid positioning due to the de-texturized images with accurate edges.

A feature separation simulation-assisted transfer framework for rotating machinery fault intelligent diagnosis

High-quality labeled data are crucial prerequisites for ensuring the effectiveness of fault diagnosis methods based on deep learning technology. However, in practical scenarios, providing abundant training data with accurate labels for these approaches is unfeasible owing to the constraints imposed by the operating and working conditions. To tackle this realistic challenge, we propose an innovative feature separation simulation-assisted transfer framework (FSSATF) for the fault diagnosis of rotating machinery. The primary concept of FSSATF is to leverage dynamic simulation-assisted data as a surrogate for the labeled data of actual equipment and integrate the feature separation network to explicitly extract domain-independent and fault-discriminative features from the simulated and actual domains, facilitating knowledge transfer and enhancing fault diagnosis capabilities. Specifically, we design a feature separation network consisting of two feature extractors. The special feature extractor is trained with the proposed target domain classification loss to explicitly separate the noisy features from the actual data. Moreover, our proposed domain adaptive loss function effectively narrows the distribution discrepancy between the simulated and actual data, promoting the shared feature extractor to capture domain-invariant and fault-discriminative features. Additionally, clustering learning is embedded into FSSATF to minimize the distance between samples of the same category, strengthening the model’s capabilities for feature extraction, and improving its performance in real machinery fault diagnosis. Artificially damaged and run-to-failure datasets were employed to validate the effectiveness and superiority of FSSATF. The comparative analysis results demonstrate that the fault diagnosis performance surpasses those of other advanced transfer learning fault diagnosis methods.

A Two-branch Edge Guided Lightweight Network for infrared image saliency detection

In the dynamic landscape of saliency detection, convolutional neural networks have emerged as catalysts for innovation, but remain largely tailored for RGB imagery, falling short in the context of infrared images, particularly in memory-restricted environments. These existing approaches tend to overlook the wealth of contour information vital for a nuanced analysis of infrared images. Addressing this notable gap, we introduce the novel Two-branch Edge Guided Lightweight Network (TBENet), designed explicitly for the robust analysis of infrared image saliency detection. The main contributions of this paper are as follows. First, we formulate the saliency detection task as two subtasks, contour enhancement and foreground segmentation. Therefore, the TBENet is divided into two specialized branches: a contour prediction branch for extracting target contour and a saliency map generation branch for separating the foreground from the background. The first branch employs an encoder–decoder architecture to meticulously delineate object contours, serving as a guiding blueprint for the second branch. This latter segment adeptly integrates spatial and semantic data, creating a precise saliency map that is refined further by an innovative edge-weighted contour loss function. Second, to enhance feature integration capabilities, we propose depthwise multi-scale and multi-cue modules, facilitating sophisticated feature aggregation. Third, a high-level linear bottleneck module is devised to ensure the extraction of rich semantic information, and by replacing the standard convolution with the depthwise convolution, it is beneficial to reduce model complexity. Additional, we reduce the number of channels of the feature maps from each stage of the decoder to further enhance the lightweight of the model. Last, we construct a novel infrared ship dataset Small-IRShip to train and evaluate our proposed model. Experimental results on the homemade dataset Small-IRShip and two publicly available datasets, namely RGB-T and IRSTD-1k, demonstrate TBENet’s superior performance over state-of-the-art methods, affirming its effectiveness in harnessing edge information and incorporating advanced feature integration strategies.

Machine vision-based autonomous road hazard avoidance system for self-driving vehicles

The resolution of traffic congestion and personal safety issues holds paramount importance for human’s life. The ability of an autonomous driving system to navigate complex road conditions is crucial. Deep learning has greatly facilitated machine vision perception in autonomous driving. Aiming at the problem of small target detection in traditional YOLOv5s, this paper proposes an optimized target detection algorithm. The C3 module on the algorithm’s backbone is upgraded to the CBAMC3 module, introducing a novel GELU activation function and EfficiCIoU loss function, which accelerate convergence on position loss lbox, confidence loss lobj, and classification loss lcls, enhance image learning capabilities and address the issue of inaccurate detection of small targets by improving the algorithm. Testing with a vehicle-mounted camera on a predefined route effectively identifies road vehicles and analyzes depth position information. The avoidance model, combined with Pure Pursuit and MPC control algorithms, exhibits more stable variations in vehicle speed, front-wheel steering angle, lateral acceleration, etc., compared to the non-optimized version. The robustness of the driving system's visual avoidance functionality is enhanced, further ameliorating congestion issues and ensuring personal safety.

FedKG: A Knowledge Distillation-Based Federated Graph Method for Social Bot Detection

Malicious social bots pose a serious threat to social network security by spreading false information and guiding bad opinions in social networks. The singularity and scarcity of single organization data and the high cost of labeling social bots have given rise to the construction of federated models that combine federated learning with social bot detection. In this paper, we first combine the federated learning framework with the Relational Graph Convolutional Neural Network (RGCN) model to achieve federated social bot detection. A class-level cross entropy loss function is applied in the local model training to mitigate the effects of the class imbalance problem in local data. To address the data heterogeneity issue from multiple participants, we optimize the classical federated learning algorithm by applying knowledge distillation methods. Specifically, we adjust the client-side and server-side models separately: training a global generator to generate pseudo-samples based on the local data distribution knowledge to correct the optimization direction of client-side classification models, and integrating client-side classification models’ knowledge on the server side to guide the training of the global classification model. We conduct extensive experiments on widely used datasets, and the results demonstrate the effectiveness of our approach in social bot detection in heterogeneous data scenarios. Compared to baseline methods, our approach achieves a nearly 3–10% improvement in detection accuracy when the data heterogeneity is larger. Additionally, our method achieves the specified accuracy with minimal communication rounds.

Improved YOLOv8-Based Target Precision Detection Algorithm for Train Wheel Tread Defects

Train wheels are crucial components for ensuring the safety of trains. The accurate and fast identification of wheel tread defects is necessary for the timely maintenance of wheels, which is essential for achieving the premise of conditional repair. Image-based detection methods are commonly used for detecting tread defects, but they still have issues with the misdetection of water stains and the leaking of small defects. In this paper, we address the challenges posed by the detection of wheel tread defects by proposing improvements to the YOLOv8 model. Firstly, the impact of water stains on tread defect detection is avoided by optimising the structure of the detection layer. Secondly, an improved SPPCSPC module is introduced to enhance the detection of small targets. Finally, the SIoU loss function is used to accelerate the convergence speed of the network, which ensures defect recognition accuracy with high operational efficiency. Validation was performed on the constructed tread defect dataset. The results demonstrate that the enhanced YOLOv8 model in this paper outperforms the original network and significantly improves the tread defect detection indexes. The average precision, accuracy, and recall reached 96.95%, 96.30%, and 95.31%.

Boosting active domain adaptation with exploration of samples

Nowadays, the idea of active learning is gradually adopted to assist domain adaptation. However, due to the existence of domain shift, the traditional active learning methods originating from semi-supervised scenarios can not be directly applied to domain adaptation. To solve the problem, active domain adaptation is proposed as a new domain adaptation paradigm, which aims to improve the performance of the model by annotating a small amount of target domain samples. In this regard, we propose an active domain adaptation method named Boosting Active Domain Adaptation with Exploration of Samples (BADA), dividing Active DA into two related issues: sample selection and sample utilization. We design the instability selection criterion based on predictive consistency and the diversity selection criterion. For the remaining unlabeled samples, we design a self-training framework, which screens out reliable samples and unreliable samples through the sample screening mechanism similar to selection criteria. And we adopt respective loss functions for reliable samples and unreliable samples. Experiments show that BADA remarkably outperforms previous active learning methods and Active DA methods on several domain adaptation datasets.