Value Of Loss Function Research Articles

Deep-Learning-Based CT Imaging in the Quantitative Evaluation of Chronic Kidney Diseases.

This study focused on the application of deep learning algorithms in the segmentation of CT images, so as to diagnose chronic kidney diseases accurately and quantitatively. First, the residual dual-attention module (RDA module) was used for automatic segmentation of renal cysts in CT images. 79 patients with renal cysts were selected as research subjects, of whom 27 cases were defined as the test group and 52 cases were defined as the training group. The segmentation results of the test group were evaluated factoring into the Dice similarity coefficient (DSC), precision, and recall. The experimental results showed that the loss function value of the RDA-UNET model rapidly decayed and converged, and the segmentation results of the model in the study were roughly the same as those of manual labeling, indicating that the model had high accuracy in image segmentation, and the contour of the kidney can be segmented accurately. Next, the RDA-UNET model achieved 96.25% DSC, 96.34% precision, and 96.88% recall for the left kidney and 94.22% DSC, 95.34% precision, and 94.61% recall for the right kidney, which were better than other algorithms. The results showed that the algorithm model in this study was superior to other algorithms in each evaluation index. It explained the advantages of this model compared with other algorithm models. In conclusion, the RDA-UNET model can effectively improve the accuracy of CT image segmentation, and it is worth of promotion in the quantitative assessment of chronic kidney diseases through CT imaging.

Online distributed dual averaging algorithm for multi-agent bandit optimization over time-varying general directed networks

The paper deals with online distributed convex optimization over a multi-agent network, where a swarm of agents attempts to minimize coordinately a sequence of time-changing global loss functions. At per time slot, the global loss function is decomposed into a sum of several local loss functions, each of which is available sequentially and held privately by one agent over the network, and individual agent does not possess prior knowledge of the future global loss function. We are interested in a bandit setting, where only the values of local loss functions at queried points are disclosed to individual agent. Meanwhile, we consider a general multi-agent network where the agents’ communication is represented as a sequence of time-changing unbalanced digraphs and the corresponding weight matrices are only row stochastic. We investigate two bandit scenarios including one- and two-point bandit feedback, and then design two corresponding online distributed bandit algorithms for such a class of problems by exploiting the classical dual averaging method. We show that both algorithms can achieve the sub-linear expected dynamic regret provided that the accumulative variation of the benchmark sequence grows sub-linearly with time. In particular, the bounds of the expected static regret obtained in this paper can reduce the relevant results when restricted to the centralized bandit online convex optimization. In contrast to existing algorithms with gradient feedback, numerical tests verify the competitive performances of the proposed algorithms.

Distributed Online Bandit Learning in Dynamic Environments Over Unbalanced Digraphs

This work is concerned with distributed online <i>bandit learning</i> over a multi-agent network, where a group of agents aim to seek the minimizer of a time-changing global loss function cooperatively. At each epoch, the global loss function can be represented as the sum of local loss functions known privately by individual agent over the network. Furthermore, local functions are sequentially accessible to agents, and all the agents have no knowledge of future loss functions. Thus, agents of the network must interchange messages to pursue an online estimation of the global loss function. In this paper, we are interested in a bandit setup, where only values of local loss functions at sampling points are disclosed to agents. Meanwhile, we consider a more general network with unbalanced digraphs that the corresponding weight matrix is allowed to be only row stochastic. By extending the celebrated mirror descent algorithm, we first design a distributed bandit online leaning method for the online distributed convex problem. We then establish the sub-linear expected dynamic regret attained by the algorithm for convex and strongly convex loss functions, respectively, when the accumulative deviation of the minimizer sequence increases sub-linearly. Moreover, the expected dynamic regret bound is analysed for strongly convex loss functions. In addition, the expected static regret bound with the order of <inline-formula><tex-math notation="LaTeX">$O(\sqrt{T})$</tex-math></inline-formula> is obtained in the bandit setting while the corresponding static regret bound with the order of <inline-formula><tex-math notation="LaTeX">$O(\ln {T})$</tex-math></inline-formula> is also provided for the strongly convex case. Finally, numerical examples are provided to illustrate the efficiency of the method and to verify the theoretical findings.

Distributed Bandit Online Convex Optimization With Time-Varying Coupled Inequality Constraints

Distributed bandit online convex optimization with time-varying coupled inequality constraints is considered, motivated by a repeated game between a group of learners and an adversary. The learners attempt to minimize a sequence of global loss functions and at the same time satisfy a sequence of coupled constraint functions, where the constraints are coupled across the distributed learners at each round. The global loss and the coupled constraint functions are the sum of local convex loss and constraint functions, respectively, which are adaptively generated by the adversary. The local loss and constraint functions are revealed in a bandit manner, i.e., only the values of loss and constraint functions are revealed to the learners at the sampling instance, and the revealed function values are held privately by each learner. Both one- and two-point bandit feedback are studied with the two corresponding distributed bandit online algorithms used by the learners. We show that sublinear expected regret and constraint violation are achieved by these two algorithms, if the accumulated variation of the comparator sequence also grows sublinearly. In particular, we show that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(T^{\theta })$</tex-math></inline-formula> expected static regret and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(T^{7/4-\theta })$</tex-math></inline-formula> constraint violation are achieved in the one-point bandit feedback setting, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(T^{\max \lbrace \kappa,1-\kappa \rbrace })$</tex-math></inline-formula> expected static regret and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(T^{1-\kappa /2})$</tex-math></inline-formula> constraint violation in the two-point bandit feedback setting, where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\theta \in (3/4,5/6]$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\kappa \in (0,1)$</tex-math></inline-formula> are user-defined tradeoff parameters. Finally, the tightness of the theoretical results is illustrated by numerical simulations of a simple power grid example, which also compares the proposed algorithms to algorithms existing in the literature.

DFT based investigations of BAWO4: Electronic and optical properties

Barium tungstate (BaWO4) has fascinating applications in the area of laser technology, solar cell, super capacitor, electronic devices and in photoluminescence. Electronic and optical behavior of BaWO4 is the reasons behind all of these fascinating applications. This research work provides the evaluation of electronic and optical properties of BaWO4 at the optimized cut-off energy value of 1200 eV by the implementation of Density Functional Theory (DFT) within the framework of CASTEP in Material STUDIO. The direct band gap and diamagnetic behavior of semiconductor BaWO4 have been evaluate by band structure, PDOS and TDOS calculation. O 2p of V.B and W, Ba 5d of C.B are the occupied electronic states of BaWO4. The highest band gap value of BaWO4 corresponds to highest loss function and smallest value of dielectric constant. Absorption spectrum exhibits that BaWO4 is an excellent UV absorber so it suitable for the formation of sensor, filter and for transparent conducting film for window layers on solar cell, solar control, and warming coatings. High refractive index value of BaWO4 makes it useful for detector for dark matter and for scintillators.

Infrared and visible image fusion based on variational auto-encoder and infrared feature compensation

With high sensitivity to capture rich details, visible imaging equipment can take images containing more textures and contours which are important to visual perception. Unlike visible cameras, infrared imaging devices can detect targets invisible in visible images, because the imaging principle of infrared sensors derives from differences of thermal radiation. Thus, the purpose of image fusion is to merge as much meaningful feature information from the infrared and visible images into the fused image as possible, such as contours as well as textures of the visible image and thermal targets of the infrared image. In this paper, we propose an image fusion network based on variational auto-encoder (VAE), which performs the image fusion process in deep hidden layers. We divide the proposed network into image fusion network and infrared feature compensation network. Firstly, in the image fusion network, the encoder of the image fusion network is created to generate the latent vectors in hidden layers from the input visible image and infrared image. Secondly, two different latent vectors merge into one based on the product of Gaussian probability density; accordingly, the decoder begins to reconstruct the fused image with the descent of the loss function value. Meanwhile, Residual block and symmetric skip connection methods are added to the network to enhance the efficiency of network training. Finally, due to the defect of the loss function setting in the fusion network, an infrared feature compensation network is designed to compensate critical radiation features of the infrared image. Experimental results on public available datasets demonstrate that the proposed method is superior to other traditional and deep learning methods in both objective metrics and subjective visual perception.

Dataset segmentation considering the information about impact factors

Introduction: The application of machine learning methods involves the collection and processing of data which comes from the recording elements in the offline mode. Most models are trained on historical data and then used in forecasting, classification, search for influencing factors or impacts, and state analysis. In the long run, the data value ranges can change, affecting the quality of the classification algorithms and leading to the situation when the models should be constantly trained or readjusted taking into account the input data. Purpose: Development of a technique to improve the quality of machine learning algorithms in a dynamically changing and non-stationary environment where the data distribution can change over time. Methods: Splitting (segmentation) of multiple data based on the information about factors affecting the ranges of target variables. Results: A data segmentation technique has been proposed, based on taking into account the factors which affect the change in the data value ranges. Impact detection makes it possible to form samples based on the current and alleged situations. Using PowerSupply dataset as an example, the mass of data is split into subsets considering the effects of factors on the value ranges. The external factors and impacts are formalized based on production rules. The processing of the factors using the membership function (indicator function) is shown. The data sample is divided into a finite number of non-intersecting measurable subsets. Experimental values of the neural network loss function are shown for the proposed technique on the selected dataset. Qualitative indicators (Accuracy, AUC, F-measure) of the classification for various classifiers are presented. Practical relevance: The results can be used in the development of classification models of machine learning methods. The proposed technique can improve the classification quality in dynamically changing conditions of the functioning.

Best Selection of Generative Adversarial Networks Hyper-Parameters Using Genetic Algorithm

Generative adversarial networks (GANs) are most popular generative frameworks that have achieved compelling performance. They follow an adversarial approach where two deep models generator and discriminator compete with each other. In this paper, we propose a Generative Adversarial Network with best hyper-parameters selection to generate fake images for digit numbers 1–9 with generator and train discriminator to decide whereas the generated images are fake or true. Genetic algorithm (GA) technique was used to adapt GAN hyper-parameters, the resulted algorithm is named GANGA: generative adversarial network with genetic algorithm. The resulted algorithm has achieved high performance; it was able to get zero value of loss function for the generator and discriminator separately. Anaconda environment with tensorflow library facilitates was used; python as programming language was adapted with needed libraries. The implementation was done using MNIST dataset to validate the work. The proposed method is to let genetic algorithm choose best values of hyper-parameters depending on minimizing a cost function such as a loss function or maximizing accuracy function used to find best values of learning rate, batch normalization, number of neurons and a parameter of dropout layer.

Decentralized online convex optimization based on signs of relative states

In this paper, we study a class of decentralized online convex optimization problems with time-varying loss functions over multi-agent networks. We propose a decentralized online subgradient method by using only the signs of the relative states of neighbors, which considerably reduces the sensing and communication requirements for the agents. We show that, despite the loss of information, the proposed algorithm can still achieve O(T) regret bound (T is the time horizon), which matches that of the standard distributed online subgradient method with exact relative state information. We further investigate the scenario of using noisy signs, where the measurements of the directions of the relative states are perturbed by noise. We show that the regret bound is not affected as long as the noise is not too large, which manifests certain noise-tolerance property of the proposed algorithm. Additionally, we extend the algorithm to the case of bandit feedback, where only the values of the local loss functions at two points are revealed to agents at each time. We demonstrate that the regret bound is not influenced by bandit feedback in order sense. Finally, numerical experiments on decentralized online least squares and logistic regression are conducted to corroborate the efficacy of the proposed algorithms.

Interpretable deep learning-assisted laser-induced breakdown spectroscopy for brand classification of iron ores

Brand classification of iron ores using laser-induced breakdown spectroscopy (LIBS) combined with artificial neural networks can quickly realize the compliance verification and guarantee the interests of both trading partners. However, its practical application is impeded by complex pretreatments and unexplained feature learning problems. According to the LIBS data characteristics of iron ores, a convolutional neural network (CNN) is designed to predict 16 types of brand iron ores from Australia, Brazil, and South Africa. The accuracies of the calibration set and the prediction set with five-fold cross-validation (5-CV) were 99.86% and 99.88%, and the value of loss function was 0.0356. Meanwhile, the established CNN method was also compared with common machine learning methods using raw spectra as input variables, and it outperformed other methods. For the first time, this work interprets the CNN's effectiveness layer by layer in self-adaptively extracting LIBS features through t-distributed stochastic neighbor embedding (t-SNE) and the quantitative data of major chemical components in iron ores. Our approach shows that deep learning assisted LIBS is able to significantly reduce manual factors in preprocessing and feature selection and has broad application prospects in the brand classification of iron ores.

A novel IoT network intrusion detection approach based on Adaptive Particle Swarm Optimization Convolutional Neural Network

In the field of network security, it is of great significance to accurately detect various types of Internet of Things (IoT) network intrusion attacks which launched by the attacker-controlled zombie hosts. In this paper, we propose a novel IoT network intrusion detection approach based on Adaptive Particle Swarm Optimization Convolutional Neural Network (APSO-CNN). In particular, the PSO algorithm with change of inertia weight is used to adaptively optimize the structure parameters of one-dimensional CNN. The cross-entropy loss function value of the validation set, which is obtained from the first training of CNN, is taken as the fitness value of PSO. Especially, we define a new evaluation method that considers both the prediction probability assigned to each category and prediction label to compare the proposed APSO-CNN algorithm with CNN set parameters manually (R-CNN). Meanwhile, the comprehensive performance of proposed APSO-CNN and other three well known algorithms are compared in the five traditional evaluation indicators and the accuracy statistical characteristics of 10 times independent experiments. The simulation results show that the multi-type IoT network intrusion attack detection task based on APSO-CNN algorithm is effective and reliable.

Automatic, dynamic, and nearly optimal learning rate specification via local quadratic approximation

In deep learning tasks, the update step size determined by the learning rate at each iteration plays a critical role in gradient-based optimization. However, determining the appropriate learning rate in practice typically relies on subjective judgment. In this work, we propose a novel optimization method based on local quadratic approximation (LQA). In each update step, we locally approximate the loss function along the gradient direction by using a standard quadratic function of the learning rate. Subsequently, we propose an approximation step to obtain a nearly optimal learning rate in a computationally efficient manner. The proposed LQA method has three important features. First, the learning rate is automatically determined in each update step. Second, it is dynamically adjusted according to the current loss function value and parameter estimates. Third, with the gradient direction fixed, the proposed method attains a nearly maximum reduction in the loss function. Extensive experiments were conducted to prove the effectiveness of the proposed LQA method.

A Discriminative Locality-Sensitive Dictionary Learning With Kernel Weighted KNN Classification for Video Semantic Concepts Analysis

Video semantic concept analysis has received a lot of research attention in the area of human computer interactions in recent times. Reconstruction error classification methods based on sparse coefficients do not consider discrimination, essential for classification performance between video samples. To further improve the accuracy of video semantic classification, a video semantic concept classification approach based on sparse coefficient vector (SCV) and a kernel-based weighted KNN (KWKNN) is proposed in this paper. In the proposed approach, a loss function that integrates reconstruction error and discrimination is put forward. The authors calculate the loss function value between the test sample and training samples from each class according to the loss function criterion, and then vote on statistical results. Finally, this paper modifies the vote results combined with the kernel weight coefficient of each class and determine the video semantic concept. The experimental results show that this method effectively improves the classification accuracy for video semantic analysis and shorten the time used in the semantic classification compared with some baseline approaches.

NeighborLoss: A Loss Function Considering Spatial Correlation for Semantic Segmentation of Remote Sensing Image

House segmentation of remote sensing image based on deep learning has become the main segmentation method because it can automatically extract features. However, the accuracy of image segmentation is affected not only by the network model, but also by the loss function, but the existing loss functions, except Binary Cross Entropy, are designed to deal with imbalanced dataset, no new research on improving Binary Cross Entropy for balanced dataset, and all loss function treat each pixel in isolation, without considering the spatial correlation between pixel and its neighbor pixels. To solve this problem, a new loss function, named NeighborLoss function, is proposed. Firstly, the deep learning network is used to get the prediction results of each pixel. According to whether the prediction results of the eight neighboring pixels of each pixel are consistent with the each pixel prediction, different weights are given to each pixel. Finally, the weighted average value of cross entropy of all pixels in the batch is taken as the final loss function value. We use the main deep learning semantic segmentation networks SegNet, PSPNet, UNET ++, MUNet with both NeighborLoss and cross entropy loss respectively to extract houses on the open data set named WHU dataset for remote sensing. The results show that compared with cross entropy loss functions, the MIoU, Precision, Recall, and Accuracy of NeighborLoss function are improved. From the predicted graph, the NeighborLoss function is more accurate to extract the edge of the house, especially in the corner of the house. NeighborLoss function is a more effective loss function for remote sensing image segmentation.

Study on Fast-Changing Mixed-Modulation Recognition Based on Neural Network Algorithms

Modulation recognition (MR) plays a key role in cognitive radar, cognitive radio, and some other civilian and military fields. While existing methods can identify the signal modulation type by extracting the signal characteristics, the quality of feature extraction has a serious impact on the recognition results. In this paper, an end-to-end MR method based on long short-term memory (LSTM) and the gated recurrent unit (GRU) is put forward, which can directly predict the modulation type from a sampled signal. Additionally, the sliding window method is applied to fast-changing mixed-modulation signals for which the signal modulation type changes over time. The recognition accuracy on training datasets in different SNR ranges and the proportion of each modulation method in misclassified samples are analyzed, and it is found to be reasonable to select the evenly-distributed and full range of SNR data as the training data. With the improvement of the SNR, the recognition accuracy increases rapidly. When the length of the training dataset increases, the neural network recognition effect is better. The loss function value of the neural network decreases with the increase of the training dataset length, and then tends to be stable. Moreover, when the fast-changing period is less than 20ms, the error rate is as high as 50%. As the fast-changing period is increased to 30ms, the error rates of the GRU and LSTM neural networks are less than 5%.

Decentralized Online Convex Optimization With Event-Triggered Communications

Decentralized multi-agent optimization usually relies on information exchange between neighboring agents, which can incur unaffordable communication overhead in practice. To reduce the communication cost, we apply event-triggering technique to the decentralized multi-agent online convex optimization problem, where each agent is associated with a time-varying local loss function and the goal is to minimize the accumulated total loss (the sum of all local loss functions) by choosing appropriate actions sequentially. We first develop an event-triggered decentralized online subgradient descent algorithm for the full information case, where the local loss function is fully revealed to each agent at each time. We establish an upper bound for the regret of each agent in terms of the event-triggering thresholds. It is shown that the regret is sublinear provided that the event-triggering thresholds converge to zero as time goes to infinity. The algorithm and analysis are further extended to the scenario of bandit feedback, where only the values of the local loss function at two random points close to the current action are disclosed to each agent. We show that the two-point bandit feedback does not degrade the performance of the proposed algorithm in order sense and a regret bound similar to the full information case can be established. Finally, numerical results on the problem of decentralized online least squares are presented to validate the proposed algorithms.

Supervision of the Infection in an SI (SI-RC) Epidemic Model by Using a Test Loss Function to Update the Vaccination and Treatment Controls

This paper studies and proposes some supervisory techniques to update the vaccination and control gains through time in a modified SI (susceptible-infectious) epidemic model involving the susceptible and subpopulations. Since the presence of linear feedback controls are admitted, a compensatory recovered (or immune) extra subpopulation is added to the model under zero initial conditions to deal with the recovered subpopulations transferred from the vaccination and antiviral/antibiotic treatment on the susceptible and the infectious, respectively. Therefore, the modified model is referred to as an SI(RC) epidemic model since it integrates the susceptible, infectious and compensatory recovered subpopulations. The defined time-integral supervisory loss function can evaluate weighted losses involving, in general, both the susceptible and the infectious subpopulations. It is admitted, as a valid supervisory loss function, that which involves only either the infectious or the susceptible subpopulations. Its concrete definition involving only the infectious is related to the Shannon information entropy. The supervision problem is basically based on the implementation of a parallel control structure with different potential control gains to be judiciously selected and updated through time. A higher decision level structure of the supervisory scheme updates the appropriate active controller (i.e., that with the control gain values to be used along the next time window), as well as the switching time instants. In this way, the active controller is that which provides the best associated supervisory loss function along the next inter-switching time interval. Basically, a switching action from one active controller to another one is decided as a better value of the supervisory loss function is detected for distinct controller gain values to the current ones.

Multi-classification decision-making method for interval-valued intuitionistic fuzzy three-way decisions and its application in the group decision-making

With the introduction of the interval-valued intuitionistic fuzzy sets, the interval-valued intuitionistic fuzzy numbers are used instead of precise numbers to provide fuzzy characterization of feature attribute values and misclassification loss function values, which is more in line with the realistic fuzzy decision-making environment. Also, the constructive covering algorithm is introduced into the three-way decisions model, which effectively solves the shortcomings of the traditional decision-theoretic rough sets model in dealing with multi-classification problems, such as too many artificial parameters, complicated computation, redundant decisions, decisional conflicts and excessively large boundary domains. At the same time, in order to avoid the one-sidedness of individual decisions, the group decision-making method is introduced into the preliminarily constructed multi-classification model in this paper to build a multi-classification group decision-making model for interval-valued intuitionistic fuzzy three-way decisions based on the constructive covering algorithm. This model determines the initial weights of feature attributes by the precise weighting method, and determines the expert weights by the grey relational precise weighting method, which effectively achieves the consistency of group decision-making. The decision-making process and rules are also deduced, which expand the model of three-way decisions as well as its practical application value and scope.

DC-MMD-GAN: A New Maximum Mean Discrepancy Generative Adversarial Network Using Divide and Conquer

Generative adversarial networks (GANs) have a revolutionary influence on sample generation. Maximum mean discrepancy GANs (MMD-GANs) own competitive performance when compared with other GANs. However, the loss function of MMD-GANs is an empirical estimate of maximum mean discrepancy (MMD) and not precise in measuring the distance between sample distributions, which inhibits MMD-GANs training. We propose an efficient divide-and-conquer model, called DC-MMD-GANs, which constrains the loss function of MMD to tight bound on the deviation between empirical estimate and expected value of MMD and accelerates the training process. DC-MMD-GANs contain a division step and conquer step. In the division step, we learn the embedding of training images based on auto-encoder, and partition the training images into adaptive subsets through k-means clustering based on the embedding. In the conquer step, sub-models are fed with subsets separately and trained synchronously. The loss function values of all sub-models are integrated to compute a new weight-sum loss function. The new loss function with tight deviation bound provides more precise gradients for improving performance. Experimental results show that with a fixed number of iterations, DC-MMD-GANs can converge faster, and achieve better performance compared with the standard MMD-GANs on celebA and CIFAR-10 datasets.

IDETIFIKASI UKURAN PAKAIAN BERBASIS IMAGE PROCESSING

Data becomes a valuable asset today. With so much data, current technology can open up the possibility to process and make predictions. Machine Learning (ML) is part of Artificial Intelligence (AI) which has become an important tool in software systems in optimizing running systems. Image Classification is one of the topics of Machine Learning that is widely used in machine learning. The application of image classification on the server-side is often done in every complex classification problem because it requires large computing power. This application uses the ml5.js library as a javascript library to process data in machine learning. The application of machine learning for clothing measurement allows users to make it easier to determine the size of clothes at the Reevant Supply IND store. Meanwhile, to support the use of machine learning on the website, a pre-trained model is needed. This study examines neural network settings such as epoch and optimal learning rate. In addition, to prove that you can be superior in training time and loss function values. The results of this study prove that it is easier for users to determine the size of clothes purchased at the Reevant Supply IND store much faster than prior consultation with the store admin