Specific Loss Function Research Articles

Assessing physics-informed neural network performance with sparse noisy velocity data

The utilization of data in physics-informed neural network (PINN) may be considered as a necessity as it allows the simulation of more complex cases with a significantly lower computational cost. However, doing so would also make it prone to any issue with the data quality, including its noise. This study would primarily focus on developing a special loss function in the PINN to allow an effective utilization of noisy data. However, a study regarding the data location and amount was also conducted in order to allow a better data utilization in PINN. This study was conducted on a lid-driven cavity flow at Re = 200, 1000, and 5000 with a dataset of less than 100 velocity data and a maximum noise of 10% of the maximum velocity. The results show that by ensuring the data are distributed in a certain configuration, it has zero noise, and by using as much data as possible, the computational cost of PINN can be significantly reduced compared to without using any data at all. For Re = 200, it is 7.4 faster by using data, and this speedup is potentially higher for higher Re cases. For the noise in particular, it does not only make the PINN more inaccurate but also necessitate the usage of more data as this is the only way to make it more accurate. This issue though is capable to be solved with our new method, which only uses the data as an approximate solution, and the governing equation would figure out the details. This method was also shown to be capable to improve the PINN accuracy with the potential to almost completely eliminating the noise effect.

Simultaneous energy and mass calibration of large-radius jets with the ATLAS detector using a deep neural network

Abstract The energy and mass measurements of jets are crucial tasks for the Large Hadron Collider experiments. This paper presents a new calibration method to simultaneously calibrate these quantities for large-radius jets measured with the ATLAS detector using a deep neural network (DNN). To address the specificities of the calibration problem, special loss functions and training procedures are employed, and a complex network architecture, which includes feature annotation and residual connection layers, is used. The DNN-based calibration is compared to the standard numerical approach in an extensive series of tests. The DNN approach is found to perform significantly better in almost all of the tests and over most of the relevant kinematic phase space. In particular, it consistently improves the energy and mass resolutions, with a 30% better energy resolution obtained for transverse momenta p T > 500 GeV.

Copyright protection framework for federated learning models against collusion attacks

Federated learning (FL) models are constructed by multiple participants who provide their training datasets and collaborate in joint training. However, training and deployment processes have encountered various challenges in terms of intellectual property protection, such as illegal theft and data leakage. Existing FL protection frameworks focus on each client independently and verify the model ownership. When they are under collusion attacks (i.e., when multiple clients negotiate to steal together), they cannot accurately identify the clients that have stolen the model. To address this challenge, a novel watermarking protection scheme against collusion attacks for federated learning is proposed in this work. It employs anti-collusion coding to design unique watermark information for each client, which can effectively detect colluders. Furthermore, it utilizes a specific regularized loss function for watermark information embedding along with the incorporation of skip connections to embed the watermark information within each batch normalization layer. The experimental results demonstrated that embedding different watermark information into each client did not affect the accuracy of the original task. The accuracy was approximately 100% when identifying the colluders. The embedding and extraction times for the original task were only 1.53% and 0.29%, respectively. Further, it exhibited high robustness against various common attacks, including fine-tuning, shearing, and collusion attacks.

Distributional loss for convolutional neural network regression and application to parameter estimation in satellite navigation signals

Convolutional Neural Network (CNN) have been widely used in image classification. Over the years, they have also benefited from various enhancements and they are now considered state-of-the-art techniques for image-like data. However, when they are used for regression to estimate some function value from images, few recommendations are available to construct robust CNN regressor models. In this study, a robustness enforcing mechanism is proposed for CNN regression models. It combines convolutional neural layers to extract high level features representations from images with a soft labelling technique that helps generalization performance. More specifically, as the deep regression task is challenging, the idea is to account for some uncertainty in the targets that are seen as distributions around their mean. Building from earlier work (Imani and White, 2018), a specific histogram loss function based on the Kullback-Leibler (KL) divergence is applied during training. The prior distributions are selected according to the physical characteristics of the parameters to estimate. To assess and illustrate the technique, the model is applied to Global Navigation Satellite System (GNSS) multipath estimation where multipath signal parameters have to be estimated from correlator output images from the I and Q channels. The multipath signal delay, magnitude, Doppler shift frequency and phase parameters are estimated from synthetically generated datasets of satellite signals. Experiments are conducted under various receiving conditions and various input images resolutions to test the estimation performances quality and robustness. The results show that the proposed soft labelling CNN technique using distributional loss outperforms classical CNN regression under all conditions. Furthermore, the extra learning performance achieved by the model allows the reduction of input image resolution from 80x80 down to 40x40 or sometimes 20x20.

Physics-Informed Machine Learning for Structural Damage Diagnosis in Aluminium Plates

Damage diagnosis plays a crucial role in Structural Health Monitoring (SHM) by facilitating the identification, localization, and estimation of the extent of defects in structures. Lamb waves, known for their sensitivity to defects, are widely employed in SHM methods for thin-walled structures. Most of those traditional methods require extracting damage indices from Lamb wave signals. This operation involves substantial post-processing and implies that part of the diagnostic information is lost. To solve those limitations and improve the damage diagnosis accuracy, machine learning methods have recently been proposed in the literature. However, the reluctance of the industrial sector to adopt conventional black-box models due to their lack of explainability poses a challenge. This study proposes a physics-informed machine-learning approach to address the limitations of standard black-box methods. Particularly, a Physics-Informed Neural Network (PINN) is implemented to predict the density of an aluminium plate based on measurements of plate displacements caused by Lamb wave excitation. This is made possible by the implementation of a specific loss function, which leverages physical knowledge in the form of the partial differential equation governing Lamb waves. Predicting the plate density based on measured displacements eliminates the need for artificial damage indices, utilizing the density variation itself to detect and localize damage. Additionally, the outputs of the PINN, rooted in physics equations, offer enhanced explainability compared to standard black-box models. The versatility of this framework extends to predicting material properties distributions for components, and efforts will be directed towards adapting the method for composite materials, where the approach may pose additional challenges.

GraphDecoder: Recovering Diverse Network Graphs From Visualization Images via Attention-Aware Learning.

DNGs are diverse network graphs with texts and different styles of nodes and edges, including mind maps, modeling graphs, and flowcharts. They are high-level visualizations that are easy for humans to understand but difficult for machines. Inspired by the process of human perception of graphs, we propose a method called GraphDecoder to extract data from raster images. Given a raster image, we extract the content based on a neural network. We built a semantic segmentation network based on U-Net. We increase the attention mechanism module, simplify the network model, and design a specific loss function to improve the model's ability to extract graph data. After this semantic segmentation network, we can extract the data of all nodes and edges. We then combine these data to obtain the topological relationship of the entire DNG. We also provide an interactive interface for users to redesign the DNGs. We verify the effectiveness of our method by evaluations and user studies on datasets collected on the internet and generated datasets.

Conquering class imbalances in deep learning-based segmentation of dental radiographs with different loss functions

ObjectiveThe imbalanced nature of real-world datasets is an ongoing challenge in the field of machine and deep learning. In medicine and in dentistry, most data samples represent patients not affected by pathologies, and on imagery, pathologic image areas are often smaller than healthy ones. Selecting suitable loss functions during deep learning is essential and may help to overcome the resulting imbalance. We assessed six different loss functions for one exemplary task, tooth structure segmentation on bitewing radiographs, for their performance. MethodsSix different loss functions (Focal Loss, Dice Loss, Tversky Loss and hybrid losses of Cross-Entropy and Dice Loss, Focal and Dice Loss, Focal and Generalized Dice Loss) were compared on a tooth structure segmentation task of 1,625 bitewing radiographs. Training was performed using three different model architectures (U-Net, Linknet, DeepLavbV3+) over a 5-fold cross-validation. Tooth structures consisted of the classes (occurrence in% of samples/captures areas measured on pixel level) enamel (100 %/25 %), dentin (100 %/50 %), root canal (100 %/10 %), filling (81 %/8 %) and crown (28 %/5 %). ResultsHybrid loss functions significantly outperformed standalone ones and provided robust results over the different architectures for the classes enamel, dentin, root canal and filling. Specifically, the Dice Focal loss reached high performance to conquer both image level and pixel level class imbalance, respectively. Clinical SignificanceIn dental use cases it is often important to predict minority classes such as pathologies accurately. Using specific loss function may be an effective strategy to overcome data imbalance when training deep learning models.

A General Search-Based Framework for Generating Textual Counterfactual Explanations

One of the prominent methods for explaining the decision of a machine-learning classifier is by a counterfactual example. Most current algorithms for generating such examples in the textual domain are based on generative language models. Generative models, however, are trained to minimize a specific loss function in order to fulfill certain requirements for the generated texts. Any change in the requirements may necessitate costly retraining, thus potentially limiting their applicability. In this paper, we present a general search-based framework for generating counterfactual explanations in the textual domain. Our framework is model-agnostic, domain-agnostic, anytime, and does not require retraining in order to adapt to changes in the user requirements. We model the task as a search problem in a space where the initial state is the classified text, and the goal state is a text in a given target class. Our framework includes domain-independent modification operators, but can also exploit domain-specific knowledge through specialized operators. The search algorithm attempts to find a text from the target class with minimal user-specified distance from the original classified object.

Extracting unstructured roads for smart Open-Pit mines based on computer vision: Implications for intelligent mining

The mining industry is rapidly advancing towards automation and intelligence, with smart mines emerging as a future trend. Open-pit mining areas are semi-enclosed, and roads are essential for unmanned trucks to perceive the mining environment and execute various production tasks. The dynamic nature of open-pit mines, driven by production progress, leads to frequent alterations in roadways. As a consequence, roads become unstructured, with indistinct edges that easily blend into the surrounding mine environment. This poses a challenging operational environment for unmanned vehicles. To address this challenge in the realm of intelligent mining, this study establishes a dataset of mining roads based on different rock types and proposes an unstructured road segmentation method for mines by integrating residual networks, Contrast Limited Adaptive Histogram Equalization (CLAHE), and the Efficient Channel Attention (ECA) mechanism. This method is applied to four semantic segmentation networks: FCN, UNet, PSPNet, and DeepLab v3 +. The dataset and network model undergo validation using a specific hybrid loss function and relevant evaluation metrics. The results show that the established road dataset has good applicability, with an ablation experiment confirming the effectiveness of the added modules. This study introduces a new perspective for advancing unmanned driving in smart mines.

MOTION-STIMULATED EEG RECOGNITION BY YOLO DEEP NETWORK

In order to elucidate the relationship between sportsman motion awareness and brain waves from a physiological perspective, research was conducted on the recognition of motion-stimulated brain waves. A motion-stimulated electroencephalogram (EEG) recognition method based on the You Only Look Once (YOLO) deep network is proposed to address the high complexity, high interference, strong randomness, and strong time-varying characteristics of EEG signals. Under the YOLO deep network framework, replacing the Efficient Layer Aggregation Networks-A (ELAN-A) module with the Convolution-Next (CvNX) module achieves the goal of lightweight processing of the YOLO network. Furthermore, the Simple Attention Mechanism (SimAM) attention mechanism is introduced, and a special loss function is set. During the experiment, the short-time Fourier transform was used to convert the EEG signals of the test subjects into time-frequency EEG images with rich features, which can be used as inputs for the YOLO network. The experimental results show that our method achieves recognition accuracy of over 90% and the highest of over 95% in the recognition of five types of motion-induced EEG waves. Compared to CNN and traditional YOLO methods, our method has 3–5% improved with recognition accuracy.

Reliable fault diagnosis using evidential aggregated residual network under varying working conditions and noise interference

Deep neural networks (DNNs) have been widely used for intelligent fault diagnosis under the implicit assumption that the testing data draws from the same distribution of the training data. However, the assumption often falls short in open-world scenarios due to the varying working conditions and noise interference, resulting in unreliable fault diagnosis behavior. To address these issues, this paper develops an evidential aggregated residual network (EAResNet) incorporated by evidence theory, quantifying the predictive uncertainty that can be a sign of trustworthiness. By placing a Dirichlet distribution over class probabilities, accurate prediction and uncertainty estimation can be achieved. More evidence would be assigned to the correct class assisted by adopting the specific evidential loss function. The proposed aggregated residual network with the multi-branch feature learning architecture has been employed as the deep classifier in varying working conditions. In addition, a joint denoising method that merges fast iterative filtering (FIF) and independent component analysis (ICA), with fuzzy entropy threshold discriminant to filter noise components, has been developed to reduce data uncertainty caused by noise interference. Experimental results demonstrate that the proposed framework successfully achieves accurate and trustworthy prediction in varying working conditions and noise interference. Moreover, uncertainty estimation offers a confidence boundary for the predictions, mitigating the risk of erroneous decision-making.

A Gibbs Posterior Framework for Fair Clustering.

The rise of machine learning-driven decision-making has sparked a growing emphasis on algorithmic fairness. Within the realm of clustering, the notion of balance is utilized as a criterion for attaining fairness, which characterizes a clustering mechanism as fair when the resulting clusters maintain a consistent proportion of observations representing individuals from distinct groups delineated by protected attributes. Building on this idea, the literature has rapidly incorporated a myriad of extensions, devising fair versions of the existing frequentist clustering algorithms, e.g., k-means, k-medioids, etc., that aim at minimizing specific loss functions. These approaches lack uncertainty quantification associated with the optimal clustering configuration and only provide clustering boundaries without quantifying the probabilities associated with each observation belonging to the different clusters. In this article, we intend to offer a novel probabilistic formulation of the fair clustering problem that facilitates valid uncertainty quantification even under mild model misspecifications, without incurring substantial computational overhead. Mixture model-based fair clustering frameworks facilitate automatic uncertainty quantification, but tend to showcase brittleness under model misspecification and involve significant computational challenges. To circumnavigate such issues, we propose a generalized Bayesian fair clustering framework that inherently enjoys decision-theoretic interpretation. Moreover, we devise efficient computational algorithms that crucially leverage techniques from the existing literature on optimal transport and clustering based on loss functions. The gain from the proposed technology is showcased via numerical experiments and real data examples.

Unawareness detection: Discovering black-box malicious models and quantifying privacy leakage risks

Because machine learning models, especially black-box malicious models vulnerable to attribute inference attacks, are capable of generating a great deal of privacy leakage, recent work has focused on assessing these models in an attempt to prevent unexpected attribute privacy leakage. While there has been some success at model privacy risk evaluations, these traditional solutions are almost brittle in practice because they not only require white-box access to obtain model feature layer outputs but also their evaluation results are heavily influenced by the training dataset and the model structure, leading to difficulty in generalization. In this paper, we propose a novel unawareness detection mechanism for discovering black-box malicious models and quantifying potential unawareness privacy leakage risk along with machine learning models to overcome the two limitations. A new method for quantifying the privacy risk caused by a specific loss function has been proposed to mitigate the impact of the training dataset and the model structure. A new evaluation model has also been proposed that uses Matthew's correlation coefficient score as a new metric and the final output of the target model as a new input. In addition, the theoretical upper bound of the model privacy risk has also been given a mathematical formula that is positively correlated to the mutual information between the sensitive attributes and the target model outputs. Compared with traditional detection methods, our evaluation model reduces the requirement for model access and minimizes evaluation errors caused by data imbalance, and our privacy risk assessment method and theoretical upper bounds on privacy risk can be applied to a broader range of datasets and target model structures. The experimental results show that the adversary's prediction capability is affected by the distribution of datasets and the level of malicious intent of the model, which is consistent with the theoretical prediction, and the detecting method can find potential model privacy leaks in public datasets UTKFace and FairFace.

Guided Deep Learning Manifold Linearization of Porous Media Flow Equations

Summary Integrated reservoir studies for performance prediction and decision-making processes are computationally expensive. In this paper, we develop a novel linearization approach to reduce the computational burden of intensive reservoir simulation execution. We achieve this by introducing two novel components: (1) augmention of the state-space to yield a bilinear system and (2) an autoencoder based on a deep neural network to linearize physics reservoir equations in a reduced manifold using a Koopman operator. Recognizing that reservoir simulators execute expensive Newton-Raphson iterations after each timestep to solve the nonlinearities of the physical model, we propose “lifting” the physics to a more amenable manifold where the model behaves close to a linear system, similar to the Koopman theory, thus avoiding the iteration step. We use autoencoder deep neural networks with specific loss functions and structure to transform the nonlinear equation and frame it as a bilinear system with constant matrices over time. In such a way, it forces the states (pressures and saturations) to evolve in time by simple matrix multiplications in the lifted manifold. We also adopt a “guided” training approach, which is performed in three steps: (1) We initially train the autoencoder, (2) then we use a “conventional” model order reduction (MOR) as an initializer for the final (3) full training, when we use reservoir knowledge to improve and to lead the results to physically meaningful output. Many simulation studies exhibit extremely nonlinear and multiscale behavior, which can be difficult to model and control. Koopman operators can be shown to represent any dynamical system through linear dynamics. We applied this new framework to a 2D two-phase (oil and water) reservoir subject to a waterflooding plan with three wells (one injector and two producers) with speedups around 100 times faster and accuracy in the order of 1% to 3% on the pressure and saturation predictions. It is worthwhile noting that this method is a nonintrusive data-driven method because it does not need access to the reservoir simulation internal structure; thus, it is easily applied to commercial reservoir simulators and is also extendable to other studies. In addition, an extra benefit of this framework is to enable the plethora of well-developed tools for MOR of linear systems. To the authors’ knowledge, this is the first work that uses the Koopman operator for linearizing the system with controls. As with any MOR method, this can be directly applied to a well-control optimization problem and well-placement studies with low computational cost in the prediction step and good accuracy.

Ranking-Based Color Constancy With Limited Training Samples.

Computational color constancy is an important component of Image Signal Processors (ISP) for white balancing in many imaging devices. Recently, deep convolutional neural networks (CNN) have been introduced for color constancy. They achieve prominent performance improvements comparing with those statistics or shallow learning-based methods. However, the need for a large number of training samples, a high computational cost and a huge model size make CNN-based methods unsuitable for deployment on low-resource ISPs for real-time applications. In order to overcome these limitations and to achieve comparable performance to CNN-based methods, an efficient method is defined for selecting the optimal simple statistics-based method (SM) for each image. To this end, we propose a novel ranking-based color constancy method (RCC) that formulates the selection of the optimal SM method as a label ranking problem. RCC designs a specific ranking loss function, and uses a low rank constraint to control the model complexity and a grouped sparse constraint for feature selection. Finally, we apply the RCC model to predict the order of the candidate SM methods for a test image, and then estimate its illumination using the predicted optimal SM method (or fusing the results estimated by the top k SM methods). Comprehensive experiment results show that the proposed RCC outperforms nearly all the shallow learning-based methods and achieves comparable performance to (sometimes even better performance than) deep CNN-based methods with only 1/2000 of the model size and training time. RCC also shows good robustness to limited training samples and good generalization crossing cameras. Furthermore, to remove the dependence on the ground truth illumination, we extend RCC to obtain a novel ranking-based method without ground truth illumination (RCC_NO) that learns the ranking model using simple partial binary preference annotations provided by untrained annotators rather than experts. RCC_NO also achieves better performance than the SM methods and most shallow learning-based methods with low costs of sample collection and illumination measurement.

A Feasibility Study of Dose Band Prediction in Radiotherapy: Predicting a Dose Spectrum

Current deep learning-based dose prediction methods can only predict a specific dose distribution. If the predicted dose is inaccurate, no more options can be selected. We proposed a novel dose prediction method named dose band prediction, which outcomes a spectrum of predicted dose distribution for planning and quality assurance (QA). Upper-Band and Lower-Band losses were involved in 3D convolution neural networks to establish the Upper-Band Network (UBN) and Lower-Band Network (LBN). Each voxel's ideal dose spectrum (dose band) was defined by the maximum/minimum rational dose predicted by UBN/LBN. 130 NPC cases with Tomotherapy (dataset 1), 49 cervix cases with IMRT (dataset 2) and 43 cervix cases with VMAT (dataset 3) were enrolled to establish and evaluate our dose band prediction method. The dose band prediction method can successfully predict a spectrum of doses. Upper-Band/Lower-Band presents maximum/minimum rational dose; Middle-Line presents the average of Upper-Band and Lower-Band. The clinical implement dose was used as the reference dose. We evaluated the maximum interval between the reference and Upper-Band/Middle-Line/Lower-Band doses, and the percentage dose difference was used as the evaluation method. The differences in PTV for Upper-Band, Middle-Line and Lower-Band in dataset 1 were within 2.47%, 0.54%, and 2.8%; in dataset 2, they were within 0.37%, 1.15%, and 2.69%; in dataset 3, they were within 0.96%, 0.35%, and 1.66%. The mean difference of OARs for the Upper-Band, Middle-Line and Lower-Band in dataset 1 were within 8.13%, 4.97%, and 8.19%; in dataset 2, they were within 8.8%, 4.48%, and 5.52%; in dataset 3, they were within 4.01%, 3.13%, and 5.79% (shown in Table 1). Dose Band prediction achieved high-accuracy dose prediction by the Middle-Line. More importantly, the Upper-Band/Lower-Band provided a spectrum of possible rational doses. Our Dose Band prediction method is based on a specific loss function, so it can easily be applied in various network and patient cases. Dose Band prediction towards a more robust plan QA and planning assistance. Table 1. The maximum interval of doses (percentage dose difference, %).

A new type of bridge structure reinforcement effect evaluation algorithm based on time series

A supplement method based on reusing the original bridge structure is suggested to expedite construction and reduce costs. Furthermore, theoretical analysis of the single pile bearing capacity is conducted, primarily involving the calculation of the bridge pile foundation settlement, horizontal resistance and axial bearing capacity. At the same time, considering that the time series data generated in the actual process often have characteristics of imbalance between classes, a cost-sensitive hybrid network model is also constructed. Its significance is the analysis of bridge reinforcement performance data. The experimental results show that compared with the cross-entropy loss function, the introduction of a cost-sensitive loss function can significantly improve the monitoring accuracy of abnormal data sets. The ACC– score, G-means and F-measures are higher than those of the specific cross-entropy loss function. The permissible bearing capacity of muddy clay determined using a network model complies with engineering standards in the evaluation of bridge reinforcement performance. As a result, the reliability of this form of reinforcement is improved.

Unsupervised registration for liver CT-MR images based on the multiscale integrated spatial-weight module and dual similarity guidance.

Multimodal registration is a key task in medical image analysis. Due to the large differences of multimodal images in intensity scale and texture pattern, it is a great challenge to design distinctive similarity metrics to guide deep learning-based multimodal image registration. Besides, since the limitation of the small receptive field, existing deep learning-based methods are mainly suitable for small deformation, but helpless for large deformation. To address the above issues, we present an unsupervised multimodal image registration method based on the multiscale integrated spatial-weight module and dual similarity guidance. In this method, a U-shape network with our multiscale integrated spatial-weight module is embedded into a multi-resolution image registration architecture to achieve end-to-end large deformation registration, where the spatial-weight module can effectively highlight the regions with large deformation and aggregate discriminative features, and the multi-resolution architecture further helps to solve the optimization problem of the network in a coarse-to-fine pattern. Furthermore, we introduce a special loss function based on dual similarity, which represents both global gray-scale similarity and local feature similarity, to optimize the unsupervised multimodal registration network. We verified the effectiveness of the proposed method on liver CT-MR images. Experimental results indicate that the proposed method achieves the optimal DSC value and TRE value of 92.70±1.75(%) and 6.52±2.94(mm), compared with other state-of-the-art registration algorithms. The proposed method can accurately estimate the large deformation field by aggregating multiscale features, and achieve higher registration accuracy and fast registration speed. Comparative experiments also demonstrate the effectiveness and generalization ability of the algorithm.

Detection of post-COVID-19-related pulmonary diseases in X-ray images using Vision Transformer-based neural network

Objective:Computer methods related to the diagnosis of COVID-19 disease have progressed significantly in recent years. Chest X-ray analysis supported by artificial intelligence is one of the most important parts of the diagnosis. Unfortunately, there is no digital tool dedicated to post-acute pulmonary changes related to COVID-19 and modern diagnostic tools are needed. Methods:This paper introduces a novel neural network architecture for chest X-ray analysis, which consists of two parts. The first is an Inception architecture that captures global features, and the second is a combination of Inception modules and a Vision Transformer network to analyze the local features. Considering that several diseases can occur in X-ray images together, a specific loss function for multilabel classification was applied — asymmetric loss function (ASL), which we modified for our purpose. In contrast to other works, we focus only on the subgroup of 9 diseases from the chestX-ray14 dataset, which can appear as a consequence of COVID-19. Results:This work proves the effectiveness of the proposed neural network architecture combined with the asymmetric loss function on post-COVID-related diseases. The results were compared with several well-known classification architectures, such as VGG19, DenseNet121, EfficientNetB4, InceptionV3 and ResNet101. According to the results, the proposed method outperforms the mentioned models with AUC – 0.819, accuracy – 0.736, sensitivity – 0.7683, and specificity – 0.7221. Significance:Our work is the first one, which focuses on the diagnosis of post-COVID-19 related pulmonary diseases from X-ray images that uses deep learning. The proposed neural network reaches better accuracy than existing well-known architectures.

Solving multiscale elliptic problems by sparse radial basis function neural networks

Machine learning has been successfully applied to various fields of scientific computing in recent years. For problems with multiscale features such as flows in porous media and mechanical properties of composite materials, however, it remains difficult for machine learning methods to resolve such multiscale features, especially when the small scale information is present. In this work, we propose a sparse radial basis function neural network method to solve elliptic partial differential equations (PDEs) with multiscale coefficients. Inspired by the deep mixed residual method, we rewrite the second-order problem into a first-order system and employ multiple radial basis function neural networks (RBFNNs) to approximate unknown functions in the system. To avoid the overfitting and improve the performance of RBFNN, an additional regularization is introduced in the loss function. Thus the loss function contains two parts: the L2 loss for the residual of the first-order system and boundary conditions, and the ℓ1 regularization term for the weights of radial basis functions (RBFs). An algorithm for optimizing the specific loss function is introduced to accelerate the training process. The accuracy and effectiveness of the proposed method are demonstrated through a collection of multiscale problems with scale separation, discontinuity and multiple scales from one to three dimensions. Notably, the ℓ1 regularization can achieve the goal of representing the solution by fewer RBFs. As a consequence, the total number of RBFs scales like O(ε−nτ), where ε is the smallest scale, n is the dimensionality, and τ is typically smaller than 1. It is worth mentioning that the proposed method not only has the numerical convergence and thus provides a reliable numerical solution in three dimensions when a classical method is typically not affordable, but also outperforms most other available machine learning methods in terms of accuracy and robustness. Executable files are available at https://github.com/wangzhiwensuda/SRBFNN-for-multiscale-elliptic-problems. The program was implemented in PyTorch and was run on Linux.