Number Of Training Iterations Research Articles

Deep Learning for Dynamic Modeling and Coded Information Storage of Vector‐Soliton Pulsations in Mode‐Locked Fiber Lasers

AbstractSoliton pulsations are ubiquitous feature of non‐stationary soliton dynamics in mode‐locked lasers and many other physical systems. To overcome difficulties related to a huge amount of necessary computations and low efficiency of traditional numerical methods in modeling the evolution of non‐stationary solitons, a two‐parallel bidirectional long short‐term memory recurrent neural network (TP‐Bi_LSTM RNN) is proposed, with the main objective to predict dynamics of vector‐soliton pulsations (VSPs) in various complex states, whose real‐time dynamics is verified by experiments. For two examples, viz., single‐ and bi‐periodic VSPs, with period‐21 and a combination of period‐3 and period‐43, the prediction results are better than provided by direct simulations – namely, deviations produced by the TP‐Bi_LSTM RNN results are 36% and 18% less than those provided by the simulations, respectively. This means that predicted results provided by the neural network are better than numerical simulations. Moreover, the prediction results for unstable VSP state with period‐9 indicate that the optimization of training sets and the number of training iterations are particularly important for the predictability. Besides, the scheme of coded information storage based on the TP‐Bi_LSTM RNN, instead of actual pulse signals, is realized too. The findings offer new applications of deep learning to ultrafast optics and information storage.

A novel maternal thyroid disease prediction using multi-scale vision transformer architecture with improved linguistic hedges neural-fuzzy classifier.

Early pregnancy thyroid function assessment in mothers is covered. The benefits of using load-specific reference ranges are well-established. We pondered whether the categorization of maternal thyroid function would change if multiple blood samples obtained early in pregnancy were used. Even though binary classification is a common goal of current disease diagnosis techniques, the data sets are small, and the outcomes are not validated. Most current approaches concentrate on model optimization, focusing less on feature engineering. The suggested method can predict increased protein binding, non-thyroid syndrome (NTIS) (simultaneous non-thyroid disease), autoimmune thyroiditis (compensated hypothyroidism), and Hashimoto's thyroiditis (primary hypothyroidism). In this paper, we develop an automatic thyroid nodule classification system using a multi-scale vision transformer and image enhancement. Graph equalization is the chosen technique for image enhancement, and in our experiments, we used neural networks with four-layer network nodes. This work presents an enhanced linguistic coverage neuro-fuzzy classifier with chosen features for thyroid disease feature selection diagnosis. The training procedure is optimized, and a multi-scale vision transformer network is employed. Each hop connection in Dense Net now has trainable weight parameters, altering the architecture. Images of thyroid nodules from 508 patients make up the data set for this article. Sets of 80% training and 20% validation and 70% training and 30% validation are created from the data. Simultaneously, we take into account how the number of training iterations, network structure, activation function of network nodes, and other factors affect the classification outcomes. According to the experimental results, the best number of training iterations is 500, the logistic function is the best activation function, and the ideal network structure is 2500-40-2-1. K-fold validation and performance comparison with previous research validate the suggested methodology's enhanced effectiveness.

Enhanced frequency and time domain feature extraction for communication infrastructure type classification using optical fiber sensing

We present a feature extraction method for a feedforward-type neural network (FNN) designed to realize a distributed acoustic sensing (DAS) technique that suits conventional optical fiber. This is, to the best of our knowledge, the first trial in which an FNN is used to interpret the field communication infrastructure type surrounding optical cables. Three classes are taken to represent the field environment: the cable tunnel (class 1), circular duct (class 2), and overhead area (class 3). We investigate and compare frequency- and time-domain feature extraction. We also show that the frequency-domain features yielded by spectral envelope shape (SES) processing have better performance than simple fast Fourier transform features. Two types of time-domain features are verified: one is the short-time maximum magnitude (STMM), which shows the largest value in the time frame, and the other is the short-time average magnitude (STAM), which indicates the average value in a time frame. Note that all features are optimized for multi-class classification. In this paper, we present the suitable number of both features and the number of training iterations. An accuracy rate of 79.0% is achieved using FNN analysis with the features studied here. Furthermore, by considering the similarity of neighboring classes, classes are refined into higher probability classes. As a result, accuracy is improved to 87.2%.

A rehearsal framework for computational efficiency in online continual learning

In the realm of online continual learning, models are expected to adapt to an ever-changing environment. One of the most persistent hurdles in this adaptation is the mitigation of a phenomenon called "Catastrophic Forgetting" (CF). This critical condition occurs when models trained on non-identically distributed data lose performance in previously learned tasks. Rehearsal methods, leveraging the ability to replay older samples, aim to address this challenge by incorporating a buffer of past training samples. However, the absence of known task boundaries complicates the adaptation of current CF mitigation methods. This paper proposes a method attuned to data stream characteristics and online model performance in a resource-constrained environment. The number of training iterations and learning rate emerges as crucial hyperparameters, impacting the efficacy and efficiency of online continual learning. Up to this point, we propose a combination of Experience Replay methodologies, a Drift Detector, and various training convergence policies, specially tailored for scenarios with unknown task boundaries. Experimental results demonstrate the effectiveness of our approach, maintaining or enhancing performance compared to baseline methods, while significantly improving computational efficiency.

Generating intelligent agent behaviors in multi-agent game AI using deep reinforcement learning algorithm

<span>The utilization of games in training the reinforcement learning (RL) agent is to describe the complex and high-dimensional real-world data. By utilizing games, RL researchers will be able to evade high experimental costs in training an agent to do intelligence tasks. The objective of this research is to generate intelligent agent behaviors in multi-agent game artificial intelligence (AI) using deep reinforcement learning (DRL) algorithm. A basic RL algorithm called deep Q network is chosen to be implemented. The agent is trained by the environment's raw pixel images and the action list information. The experiments conducted by using this algorithm show the agent’s decision-making ability in choosing a favorable action. In the default setting for the algorithm, the training is set into 1 epoch and 0.0025 learning rate. The number of training iterations is set to one as the training function will be repeatedly called for every 4-timestep. However, the author also experimented with two different scenarios in training the agent and compared the results. The experimental findings demonstrate that our agents learn correctly and successfully while actively participating in the game in real time. Additionally, our agent can quickly adjust against a different enemy on a varied map because of the observed knowledge from prior training.</span>

A BiLSTM-attention-based point-of-interest recommendation algorithm

Abstract Aiming at the problem that users’ check-in interest preferences in social networks have complex time dependences, which leads to inaccurate point-of-interest (POI) recommendations, a location-based POI recommendation model using deep learning for social network big data is proposed. First, the original data are fed into an embedding layer of the model for dense vector representation and to obtain the user’s check-in sequence (UCS) and space-time interval information. Then, the UCS and spatiotemporal interval information are sent into a bidirectional long-term memory model for detailed analysis, where the UCS and location sequence representation are updated using a self-attention mechanism. Finally, candidate POIs are compared with the user’s preferences, and a POI sequence with three consecutive recommended locations is generated. The experimental analysis shows that the model performs best when the Huber loss function is used and the number of training iterations is set to 200. In the Foursquare dataset, Recall@20 and NDCG@20 reach 0.418 and 0.143, and in the Gowalla dataset, the corresponding values are 0.387 and 0.148.

Carbon Brush Temperature Prediction under Bayesian-BP Neural Network

Accurate prediction of carbon brush temperature is of vital importance for the safe, stable, and reliable operation of hydro turbines and hydropower plants. The carbon brush temperature prediction model based on the traditional BP neural network exhibits slow convergence and is prone to local minima. Therefore, this study proposes a Bayesian-based optimization model for the BP neural network, combining the principles of Bayesian inference and the BP neural network model. This model constructs the likelihood function of the weights and thresholds of the BP neural network to obtain the prior probability distribution and posterior probability distribution using Bayesian theorem. The optimal hyperparameters for the weights and thresholds are iteratively updated using the maximum likelihood method, and the learning efficiency is iteratively updated based on the number of training iterations. When the model’s convergence conditions and the maximum number of iterations are met, the optimal values for the weights, thresholds, and learning efficiency are obtained, effectively avoiding local optimization problems. Experimental results demonstrate that compared to the traditional BP neural network and LSTM neural network, the Bayesian-BP neural network achieves higher fidelity in fitting historical data and exhibits superior convergence accuracy and speed in temperature prediction. Therefore, it better fulfills the requirements for predicting carbon brush temperature in hydro turbines.

Aircraft parameter estimation using a stacked long short-term memory network and Levenberg-Marquardt method

To effectively estimate the unknown aerodynamic parameters from the aircraft’s flight data, this paper proposes a novel aerodynamic parameter estimation method incorporating a stacked Long Short-Term Memory (LSTM) network model and the Levenberg-Marquardt (LM) method. The stacked LSTM network model was designed to realize the aircraft dynamics modeling by utilizing a frame of nonlinear functional mapping based entirely on the measured input–output data of the aircraft system without requiring explicit postulation of the dynamics. The LM method combines the already-trained LSTM network model to optimize the unknown aerodynamic parameters. The proposed method is applied by using the real flight data, generated by ATTAS aircraft and a bio-inspired morphing Unmanned Aerial Vehicle (UAV). The investigation reveals that for the two different flight data, the designed stacked LSTM network structure can maintain the efficacy of the network prediction capability only by appropriately adjusting the dropout rates of its hidden layers without changing other network parameters (i.e., the initial weights, initial biases, number of hidden cells, time-steps, learning rate, and number of training iterations). Besides, the proposed method’s effectiveness and potential are demonstrated by comparing the estimated results of the ATTAS aircraft or the bio-inspired morphing UAV with the corresponding reference values or wind-tunnel results.

Interlayer Link Prediction in Multiplex Social Networks Based on Multiple Types of Consistency Between Embedding Vectors.

Online users are typically active on multiple social media networks (SMNs), which constitute a multiplex social network. With improvements in cybersecurity awareness, users increasingly choose different usernames and provide different profiles on different SMNs. Thus, it is becoming increasingly challenging to determine whether given accounts on different SMNs belong to the same user; this can be expressed as an interlayer link prediction problem in a multiplex network. To address the challenge of predicting interlayer links, feature or structure information is leveraged. Existing methods that use network embedding techniques to address this problem focus on learning a mapping function to unify all nodes into a common latent representation space for prediction; positional relationships between unmatched nodes and their common matched neighbors (CMNs) are not utilized. Furthermore, the layers are often modeled as unweighted graphs, ignoring the strengths of the relationships between nodes. To address these limitations, we propose a framework based on multiple types of consistency between embedding vectors (MulCEVs). In MulCEV, the traditional embedding-based method is applied to obtain the degree of consistency between the vectors representing the unmatched nodes, and a proposed distance consistency index based on the positions of nodes in each latent space provides additional clues for prediction. By associating these two types of consistency, the effective information in the latent spaces is fully utilized. In addition, MulCEV models the layers as weighted graphs to obtain representation. In this way, the higher the strength of the relationship between nodes, the more similar their embedding vectors in the latent representation space will be. The results of our experiments on several real-world and synthetic datasets demonstrate that the proposed MulCEV framework markedly outperforms current embedding-based methods, especially when the number of training iterations is small.

A novel complex-valued classifier with a weighted loss function employed for W-band PS-16QAM transmission over 4.6 km

Probabilistic shaping(PS) technique can reduce Signal-Noise Ratio (SNR) requirement and save the transmission power, which is meaningful for millimeter wave (mm-wave) long-distance wireless transmission. Traditional Digital Signal Processing(DSP) algorithms are affected by the uneven probability distribution of constellation points when working on PS signals. We propose a novel complex-valued classifier with a weighted loss function to replace traditional algorithms in PS-16QAM W-band long-distance transmission. The proposed algorithm can provide a receiving sensitivity improvement of about 0.5 dBm. It can reduce the training sequence length by 4%–7%, the computational complexity by 10.53%–16.89%, and the number of training iterations by 23.08%, respectively.

H3O-LGBM: hybrid Harris hawk optimization based light gradient boosting machine model for real-time trading.

The gross domestic product (GDP) of a country is mainly dependent on its trade and external sector which improves the country's income. According to FY2021-2022, India's nominal GDP is estimated to be 3.12 trillion US dollars. Overall exports and imports have a year-over-year increase of 49.6% and 68% respectively. Machine learning techniques have the potential to improve India's Current gross value by up to 15% by the year 2035. The integration of data, Technology, and talent helps to create intelligent models that enhance artificial intelligence growth. This paper presents an optimized light gradient boosting machine (Light GBM) model using the hybrid Harris hawk optimization (H3O) algorithm for trade forecasting. The overfitting problem in the conventional Harris Hawk Optimization is overcome using the exclusive feature bundling (EFB) and the gradient-based one-side sampling (GOSS) methodologies. The H3O optimization algorithm offers fast convergence by optimizing different lightGBM parameters such as a number of training iterations, maximum depth, minimal data in the leaf, etc. To improve the performance, one step further, the residual errors of the optimized lightGBM model are corrected using the Markov Chain model. The main aim of the optimized lightGBM model is to extract the crucial input values of certain variables such as imports and exports of goods and services, service trade, and merchandise trade and predict the price movement decision. The proposed model identifies the interrelationship with the external market and future market growth along with analyzing the variation in market conditions. The prediction decision is mainly to hold, but, or sell the stocks. When evaluated using the precious metal price forecast and stock market datasets, the proposed methodology shows that the hybrid approach can enhance the prediction performance. The results show that the input parameters were efficient in predicting the economic growth regarding the Intermarket trading system (ITS) and services with higher accuracy.

NAS-FPNLite object detection method fused with cross stage connection and inverted residual

NAS-FPNLite object detection method fused with cross stage connection and inverted residual

Differentially private stochastic gradient descent via compression and memorization

We propose a novel approach for achieving differential privacy for neural network training models through compression and memorization of gradients. The compression technique, which makes gradient vectors sparse, reduces the sensitivity so that differential privacy can be achieved with less noise; whereas the memorization technique, which remembers unused gradient parts, keeps track of the descent direction and thereby maintains the accuracy of the proposed algorithm. Our differentially private algorithm, called dp-memSGD for short, converges mathematically at the same rate of 1/T as standard stochastic gradient descent (SGD) algorithm, where T is the number of training iterations. Experimentally, we demonstrate that dp-memSGD converges with reasonable privacy losses on many benchmark datasets.

PEGANs: Phased Evolutionary Generative Adversarial Networks with Self-Attention Module

Generative adversarial networks have made remarkable achievements in generative tasks. However, instability and mode collapse are still frequent problems. We improve the framework of evolutionary generative adversarial networks (E-GANs), calling it phased evolutionary generative adversarial networks (PEGANs), and adopt a self-attention module to improve upon the disadvantages of convolutional operations. During the training process, the discriminator will play against multiple generators simultaneously, where each generator adopts a different objective function as a mutation operation. Every time after the specified number of training iterations, the generator individuals will be evaluated and the best performing generator offspring will be retained for the next round of evolution. Based on this, the generator can continuously adjust the training strategy during training, and the self-attention module also enables the model to obtain the modeling ability of long-range dependencies. Experiments on two datasets showed that PEGANs improve the training stability and are competitive in generating high-quality samples.

Rapid fault extraction from seismic images via deep learning

Using deep learning to automatically and quickly extract faults from seismic images is of practical significance. An improved U-Net algorithm is proposed by reducing convolutional layers, designing skip connections, enforcing deep supervision, and improving the loss function and learning rate to build a new model. In the operation, the feature map parameters in the network are further revised, the number of training iterations is increased, a callback function is added, and the parameter adjustment training consumes less time and space and has higher accuracy. Experiments on real public datasets show that the improved network can limit the time required to extract a 128 × 128 × 128 three-dimensional image within 200 ms, which not only requires less time and computing power than existing methods but also has an extraction accuracy as high as 97.6%.

Cross-Scenario Device-Free Gesture Recognition Based on Self-Adaptive Adversarial Learning

Device-free gesture recognition (DFGR) is an emerging technique which could leverage the influence of human gestures on surrounding wireless signals to recognize gestures. It has gained widespread attention due to its promising prospect of empowering pervasive wireless devices with the sensing ability. Due to the inconsistency of the feature distribution in different scenarios, a well-trained DFGR system often fails to get satisfactory performance in cross-scenario conditions. Researchers have done valuable exploration on alleviating the feature distribution shift from a global distribution point of view. However, global feature distribution alignment could not solve the feature distribution shift problem completely. In this paper, we develop a self-adaptive adversarial learning network which could further reduce the feature distribution shift through aligning the local feature distribution. Specifically, we design an adversarial network which is consisted of a feature extractor, a scenario discriminator, and two diverse classifiers. It could evaluate the degree of local feature distribution alignment by analyzing the prediction inconsistent of the classifiers. We design a self-adaptive adversarial loss which can be adjusted adaptively according to the degree of local alignment. If the features have been aligned locally, we reduce their impact on the loss to protect these aligned features. Otherwise, we increase their influence to accelerate the training process. Extensive experiments conducted on a designed mmWave testbed demonstrate that the proposed method could achieve an accuracy of at least 4% higher than those of existing cross-scenario DFGR methods, while the number of training iterations can be reduced by nearly half.

Diagnosis of Thyroid Nodules Based on Image Enhancement and Deep Neural Networks

The diagnosis of thyroid nodules at an early stage is a challenging task. Manual diagnosis of thyroid nodules is labor-intensive and time-consuming. Meanwhile, due to the difference of instruments and technical personnel, the original thyroid nodule ultrasound images collected are very different. In order to make better use of ultrasound image information of thyroid nodules, some image processing methods are indispensable. In this paper, we developed a method for automatic thyroid nodule classification based on image enhancement and deep neural networks. The selected image enhancement method is histogram equalization, and the neural networks have four-layer network nodes in our experiments. The dataset in this paper consists of thyroid nodule images of 508 patients. The data are divided into 80% training and 20% validation sets. A comparison result demonstrates that our method can achieve a better performance than other normal machine learning methods. The experimental results show that our method has achieved 0.901961 accuracy, 0.894737 precision, 1 recall, and 0.944444 F1-score. At the same time, we also considered the influence of network structure, activation function of network nodes, number of training iterations, and other factors on the classification results. The experimental results show that the optimal network structure is 2500-40-2-1, the optimal activation function is logistic function, and the best number of training iterations is 500.

LSTM with particle Swam optimization for sales forecasting

• Propose a sale forecasting approach based on LSTM with PSO for E-commerce companies. • The number of hidden neurons and iterations in LSTM are optimized by PSO. • We compare the proposed approach with 9 competing approaches. • Evaluated on the real datasets from an E-commerce company and 3 benchmark datasets. • Proposed models achieved good results in forecasting accuracy. Sales volume forecasting is of great significance to E-commerce companies. Accurate sales forecasting enables managers to make reasonable resource allocation in advance. In this paper, we propose a novel approach based on Long Short-Term Memory with Particle Swam Optimization (LSTM-PSO) for sale forecasting in E-commerce companies. In the proposed approach, the number of hidden neurons in different LSTM layers, and the number of iterations for training are optimized by Particle Swam Optimization metaheuristic. In the experiments, we compare the proposed approach with 9 competing approaches. The effectiveness of the proposed approach is evaluated on the real datasets from an E-commerce company as well as on the publicly available benchmark datasets. In the experiments, neural network design, activation functions, methods of regularization, and the training method of neural network are also analyzed. Experiment results show that the proposed PSO-LSTM models achieved good results in forecasting accuracy.

Deep Optimization of Parametric IIR Filters for Audio Equalization

This paper describes a novel Deep Learning method for the design of IIR parametric filters for automatic multipoint audio equalization, that is the task of improving the sound quality of a listening environment at multiple listening points employing multiple loudspeakers. The filters are designed to approximate the inverse of the RIR and achieve almost flat magnitude response. A simple and effective neural architecture, named BiasNet, is proposed to determine the IIR equalizer parameters. This novel architecture is conceived for optimization and, as such, is able to produce optimal IIR equalizer parameters at its output, after training, with no input required. In absence of input, the presence of learnable non-zero bias terms ensures that the network works properly. An output scaling method is used to obtain accurate tuning of the IIR filters center frequency, quality factor and gain. All layers involved in the proposed method are shown to be differentiable, allowing backpropagation to optimize the network weights and achieve, after a number of training iterations, the optimal output according to a given RIR. The parameters are optimized with respect to a loss function based on a spectral distance between the measured and desired magnitude response, and a regularization term is used to keep the same microphone-loudspeaker energy balance after equalization. Two experimental scenarios are employed, a room and a car cabin, with several loudspeakers. The performance of the proposed method improves over the baseline techniques and achieves an almost flat band at a lower computational cost.

Application of POD reduced-order algorithm on data-driven modeling of rod bundle

As a valid numerical method to obtain a high-resolution result of a flow field, computational fluid dynamics (CFD) have been widely used to study coolant flow and heat transfer characteristics in fuel rod bundles. However, the time-consuming, iterative calculation of Navier-Stokes equations makes CFD unsuitable for the scenarios that require efficient simulation such as sensitivity analysis and uncertainty quantification. To solve this problem, a reduced-order model (ROM) based on proper orthogonal decomposition (POD) and machine learning (ML) is proposed to simulate the flow field efficiently. Firstly, a validated CFD model to output the flow field data set of the rod bundle is established. Secondly, based on the POD method, the modes and corresponding coefficients of the flow field were extracted. Then, an deep feed-forward neural network, due to its efficiency in approximating arbitrary functions and its ability to handle high-dimensional and strong nonlinear problems, is selected to build a model that maps the non-linear relationship between the mode coefficients and the boundary conditions. A trained surrogate model for modes coefficients prediction is obtained after a certain number of training iterations. Finally, the flow field is reconstructed by combining the product of the POD basis and coefficients. Based on the test dataset, an evaluation of the ROM is carried out. The evaluation results show that the proposed POD-ROM accurately describe the flow status of the fluid field in rod bundles with high resolution in only a few milliseconds.