Forward Neural Network Research Articles

Machine Learning for Advanced Emission Monitoring and Reduction Strategies in Fossil Fuel Power Plants

Fossil fuel power plants are a significant contributor to global carbon dioxide (CO2) and nitrogen oxide (NOx) emissions. Accurate monitoring and effective reduction of these emissions are crucial for mitigating climate change. This systematic review examines the current state of research on the application of machine learning techniques in evaluating the emissions from fossil fuel power plants. This review first briefly introduces the continuous emission monitoring (CEM) systems and predictive emission monitoring (PEM) systems that are commonly used in power plants and highlights that machine learning models can significantly improve PEM systems through their capability to process and interpret large datasets intelligently to transform traditional emission monitoring systems by enhancing their precision, effectiveness, and cost-efficiency. Compared to previously published review articles, the key contribution and innovation in this present review is the discussion of machine learning models in CO2/NOx emissions according to the different algorithms used, including their advantages and disadvantages in a systematic way, which aims to help future researchers to develop more effective machine learning models. The most popular machine learning model includes reinforcement learning, a forward neural network, a long short-term memory neural network, and support vector regression. While each model method has its own advantages and disadvantages, we noted that training data quality, as well as the proper selection of model parameters, plays an important role. The challenges and research gaps, such as model transferability, a deep understanding of the physics of CO2/NOx emissions, and the availability of high-quality data for training machine learning models, are identified, and recommendations as well as potential future research directions to address these challenges are proposed and discussed.

Deep learning-based inverse design of multi-functional metasurface absorbers.

A novel approach-integrating a simulated annealing (SA) algorithm with deep learning (DL) acceleration-is presented for the rapid and accurate development of terahertz perfect absorbers through forward prediction and backward design. The forward neural network (FNN) effectively deduces the absorption spectrum based on metasurface geometry, resulting in an 80,000-fold increase in computational speed compared to a full-wave solver. Furthermore, the absorber's structure can be precisely and promptly derived from the desired response. The incorporation of the SA algorithm significantly enhances design efficiency. We successfully designed low-frequency, high-frequency, and broadband absorbers spanning the 4 to 16 THz range with an error margin below 0.02 and a remarkably short design time of only 10 min. Additionally, the proposed model in this Letter introduces a novel, to our knowledge, method for metasurface design at terahertz frequencies such as the design of metamaterials across optical, thermal, and mechanical domains.

Active mode detection for hybrid systems using feed- forward neural networks

Active mode detection for hybrid systems using feed- forward neural networks

Underwater Engineering Crack Identification based on Lightweight Convolutional Neural Network

Convolutional neural network is an effective model for image feature recognition, which is widely used in crack recognition and risk detection. The crack identification of underwater engineering involves a huge scale of data, and the traditional R-CNN model and Faster R-CNN model have the problems of large number of model parameters and low efficiency for crack identification. Therefore, a lightweight convolutional neural network based crack identification method for underwater engineering is proposed to optimize the efficiency and accuracy of crack identification. Deep separable convolution is used to replace the common convolution of YOLOX network (an open source high-performance object detection algorithm), reducing the redundant parameters of the model to achieve lightweight design. Following the backbone feature extraction network framework of YOLOX, Transformer (a structure based on attention mechanism and forward neural network) vision module is adopted to replace the CSP structure (a feature extraction structure) at the end of the backbone network, adding attention mechanism and enhancing key information extraction capability. Make up for the problem of decreasing feature extraction accuracy caused by decreasing parameter number; Finally, the adaptive spatial feature fusion strategy is used instead of the traditional feature pyramid cascade to implement feature fusion and improve the feature extraction ability of small targets. The test results show that the improved YOLOX lightweight convolutional neural network has good prediction ability for underwater engineering cracks, and can accurately select the crack location, with fewer false detection and missed detection cases. Ablation experiments show that this study builds a lightweight convolutional neural network model, which improves the accuracy of the model to identify underwater engineering cracks. The improved lightweight convolutional neural network is suitable for crack identification in underwater engineering and can be widely applied in the field of underwater engineering risk identification.

Deep learning-assisted inverse design of metasurfaces for active color image tuning.

Metasurfaces, artificial planar nanostructures, offer numerous advantages for color printing applications, including ultra-high resolution, resistance to fading, wide color gamut coverage, and multifunctional capabilities. Due to the sensitivity of their resonance spectra to the external environment, metasurfaces have garnered significant interest for color tuning applications. However, most existing approaches are limited to passive color tuning, wherein only the color changes passively while the composite color image remains unaltered. Active color image tuning, on the other hand, requires precise matching of both color and intensity to the designed targets before and after the tuning process. In this study, we propose a novel approach for active metasurface color image tuning by modulating the environmental refractive index. Building upon a forward neural network that establishes the relationship between the metasurface geometric parameters and color/intensity information, we employ a multi-objective inverse adjoint neural network. This network not only overcomes the inherent 'one-to-many' problem in inverse design using neural networks but also facilitates active color image tuning under three distinct environmental conditions. Our work provides a new approach for the inverse design of metasurfaces and opens up possibilities for applications in dynamic color printing, information encryption, and other related fields.

Computer-Aided Design of Hand-Drawn art Food Packaging Design Based on Deep Neural Network Model

Abstract High-speed assembly line food packaging quality defect detection methods have poor error detection rates, missing rates and accuracy. This paper advances a process of computer-aided online monitoring of food packaging based on a deep neural network model. Firstly, this paper uses the deep convolution method to analyze the defects in food packaging. Then the convolution method of food packaging defects is improved. The correct identification of defects in food packaging can be enhanced by adjusting VGG16. This paper uses a convolutional neural network, transfer learning and adaptive neural network to compare the recognition effect of food packaging defects based on a forward neural network. It is proved that the recognition accuracy of this method is 0.0005. Good identification results can be obtained after 10 times of repeated practices. This method has a good classification effect.

Physics-aware learning of nonlinear limit cycles and adjoint limit cycles

Thermoacoustic oscillations occur when the heat released by a flame is sufficiently in phase with the acoustic pressure. Under this condition, the linear instability can saturate to a nonlinear self-excited oscillation with a large amplitude. A typical nonlinear regime is a limit cycle, which is characterised by a periodic orbit in the thermoacoustic dynamics. In this paper, we develop a physics-aware data-driven method to predict periodic solutions using forward neural networks. The physics is constrained in two ways. First, the training is informed by a physical residual, which penalises solutions that violate the conservation of mass, momentum, and energy. Second, periodicity is imposed by introducing periodic activation functions in the neural network. We test the algorithm on a nonlinear time-delayed model of a Rijke tube. Adjoint methods offer a cheap and easy way to calculate the gradients with respect to design parameters, hence we extend our study to learning the adjoint variables of the Rijke system, which also settle onto periodic oscillations. We find that (i) periodic solutions of thermoacoustic systems can be accurately learned with this method, (ii) for periodic data, periodic activations outperform conventional activations in terms of prediction capability beyond the training range, and (iii) under the physical constraints, fewer data is sufficient to achieve a good performance. This work opens up possibilities for the prediction of nonlinear thermoacoustics by combining physical knowledge and data.

A Neural-Network Based MPAS—Shallow Water Model and Its 4D-Var Data Assimilation System

The technique of machine learning has been increasingly applied in numerical weather predictions. The aim of this study is to explore the application of a neural network in data assimilation by making use of the convenience in obtaining the tangent linear and adjoint (TL/AD) of a neural network (NN) and formulating a NN-based four-dimensional variational (4D-Var) DA system. A NN-based shallow water (SW) model is developed in this study. The NN model consists of three layers. The weights and biases in the NN-based SW model are trained with 60 years of hourly ERA5 geopotentials and wind field at 500 hPa as initial conditions and the corresponding 12-h forecasts by Model for Prediction Across Scales (MPAS)-SW, in total of 534,697 sets of samples. The 12-h forecasts from independent dates made by NN-based SW prove to closely emulate the simulations by the actual MPAS-SW model. This study further shows that the TL/AD of an NN model can be easily developed and validated. The ease of obtaining the TL/AD makes NN conveniently applicable in various aspects within a data assimilation (DA) system. To demonstrate such, a continuous 4D-Var DA system is also developed with the forward NN and its adjoint. To demonstrate the functionality of the NN-based 4D-Var DA system, the results from a higher resolution simulation will be treated as observations and assimilated to analyze the low resolution initial conditions. The forecasts starting from the analyzed initial conditions will be compared with those without assimilation to demonstrate improvements.

Machine Learning-Based Control for Fuel Cell Hybrid Buses: From Average Load Power Prediction to Energy Management

In this work, a machine learning-based energy management system is developed using a long short-term memory (LSTM) network for fuel cell hybrid buses. The neural network implicitly learns the complex relationship between various factors and the optimal power control from massive data. The selection of the neural network inputs is inspired by the adaptive Pontryagin’s minimum principle (APMP) strategy. Since an estimated value of the global average fuel cell power is required in the machine learning-based energy management strategy (EMS), some global features of driving cycles are extracted and then applied in a feedforward neural network to predict the average fuel cell power appropriately. The effectiveness of the machine learning-based energy management, with the integration of the mechanism of estimating the average fuel cell power based on the forward neural network, is tested under two different driving cycles from the training environment, with comparisons to a commercially used rule-based strategy. Based on the simulation results, the learning-based strategy outperforms the rule-based strategy regarding the charge-sustaining mode conditions and fuel economy. Moreover, compared to the best offline hydrogen consumption, the machine learning-based strategy consumed 0.58% and 0.36% more than the best offline results for both driving cycles. In contrast, the rule-based strategy consumed 1.80% and 0.96% more than optimal offline results for the two driving cycles, respectively. Finally, simulations under battery and fuel cell aging conditions show that the fuel economy of the machine learning-based strategy experiences no performance degradation under components aging compared to offline strategies.

AMResNet: An automatic recognition model of bird sounds in real environment

Birds are biological indicators reflecting environmental quality and its changes, so ecologists devote a great deal of attention to monitor their population trends. Automated acoustic recognition is regarded as an important technology to support bird monitoring and conservation activities. However, traditional bird sounds recognition methods have problems such as high cost, long time consuming and low precision. To address these problems, a novel automatic recognition model of bird sounds was proposed. The proposed model (AMResNet) was based on the combination of attentional mechanism and residual networks, which can automatically extract and select high-dimensional features to achieve a high classification accuracy. Attentional mechanism can improve the recognition efficiency by assigning appropriate weights to channels and space. Residual network can alleviate the problem of gradient disappearance and increase the information flow by skip connections. More importantly, an efficient combined feature was adopted in this paper, to give a more comprehensive representation of bird sounds. By trained and tested by 10-fold cross-validation in 12651 bird sound samples from the real environment, the AMResNet with combined features was proved to be appropriate for bird sound taxonomic problems, and dramatically outperform over other eight models, which contains two traditional models, a forward neural network, four main deep learning models and a latest vision transform model. The proposed model achieved a classification accuracy of 92.6%, which was 3.1% higher than the highest of other eight models, and it also achieved the best results in precision (97.6%), recall (97.3%) and F1-score (97.1%) across the 19 species.

A content distribution method of internet of vehicles based on edge cache and immune cloning strategy

In order to better reduce the delay of content distribution and the communication cost, this paper proposes a content distribution method of Internet of Vehicles based on edge cache and immune cloning strategy. Firstly,in the data pre-storage stage, the RSU uses the historical request data of the vehicle node to predict the content popularity in the next moment through the forward neural network, and actively caches some content with high popularity locally from the BS, further reducing the content request delay and improving the hit rate. In the content distribution stage, the requesting node can make different content source choices to obtain the content, so as to maximize the system utility. Furthermore, the problem is modeled as an optimization problem, and a decision algorithm based on immune clone strategy is proposed to obtain the optimal solution. The experimental results show that the content distribution method proposed in this paper not only improves the hit rate, but also effectively reduces the delay and network energy consumption caused by content distribution, and better meets the needs of vehicle user nodes.

Discovering Pareto-Optimal Magnetic-Design Solutions via a Generative Adversarial Network

In the framework of induction hardening, the coil design task is particularly suitable to be formulated as a multi-objective optimization problem. In fact, the Pareto front estimation raises the issue of guaranteeing a satisfactory diversity and number of non-dominated solutions to be provided to the decision maker (DM). In this article, a generative adversarial network (GAN) and a forward neural network (FNN), which is cascade connected to the GAN generator, produce additional Pareto optimal solutions starting from the results of a genetic algorithm [non-dominated sorting genetic algorithm (NSGA II)] used as a training set. The FNN ensures an accurate prediction of the objectives of the added solutions, removing the need for further field analyses. This method is first tested against two analytical problems and subsequently validated on a three-objective coil design task to illustrate its utility for a real-world case.

Multi-Parameter Inversion of AIEM by Using Bi-Directional Deep Neural Network

A novel multi-parameter inversion method is proposed for the Advanced Integral Equation Model (AIEM) by using bi-directional deep neural network. There is a very complex nonlinear relationship between the surface parameters (dielectric constant and roughness) and radar backscattering coefficient. The traditional inverse neural network, which is constructed by using the backscattering coefficients as the input and the surface parameters as the output, leads to bad convergence and wrong results. This is because many sets of surface parameters can get the same backscattering coefficient. Therefore, the proposed bi-directional deep neural network starts with building an AIEM-based forward deep neural network (AIEM-FDNN), whose inputs are the surface parameters and outputs are the backscattering coefficients. In this way, the weights and biases of the forward deep neural network can be optimized and predicted, which can be used for the backward deep neural network (AIEM-BDNN). Then, the multi-parameters are updated by minimizing the loss between the output backscattering coefficients with the measured ones. By inserting a sigmoid function between the input and the first hidden layer, the input multi-parameters can be efficiently approximated and continuously updated. As a result, both the forward and backward deep neural networks can be built with these weights and biases. By sharing the weights and biases of the forward network, the training of the inverse network is avoided. The bi-directional deep neural network can not only predict the backscattering coefficient but can also inverse the surface parameters. Numerical results are given to demonstrate that the RMSE of the backscattering coefficients calculated by the proposed bi-directional neural network can be reduced to 0.1%. The accuracy of the inversion parameters, including the real and imaginary parts of the dielectric constant, the root mean square height and the correlation length, can be improved to 97.56%, 91.14%, 99.04% and 98.45%, respectively. At the same time, the bi-directional neural network also has good accuracy for the inversion of the POLARSCAT measured data.

An Efficient Knowledge-Based Artificial Neural Network for the Design of Circularly Polarized 3-D-Printed Lens Antenna

An efficient knowledge-based artificial neural network (KBANN) is proposed, and it is used for the design of circularly polarized (CP) lens antenna in this article. In this KBANN, forward neural network (FNN) and inverse neural network (INN) are included. In this model, INN is the major component to predict the antenna structure parameters. As multiple performance indices are required, INN requires a large number of training samples to deduce complex mapping relationship. To solve this problem, FNN is introduced to provide prior knowledge for INN. FNN generates a huge training dataset for INN training, and then the trained INN can directly output the geometric parameters by feeding the target electromagnetic responses as input. This article solves the problem of multiple performance indices in antenna design, and a CP lens antenna with wideband, good axial ratio, and high gain is designed and fabricated to verify the effectiveness of the KBANN model.

A CT-based radiomics model for predicting renal capsule invasion in renal cell carcinoma

BackgroundRenal cell carcinoma (RCC) is a heterogeneous group of kidney cancers. Renal capsule invasion is an essential factor for RCC staging. To develop radiomics models from CT images for the preoperative prediction of capsule invasion in RCC patients.MethodsThis retrospective study included patients with RCC admitted to the Chongqing University Cancer Hospital (01/2011–05/2019). We built a radiomics model to distinguish patients grouped as capsule invasion versus non-capsule invasion, using preoperative CT scans. We evaluated effects of three imaging phases, i.e., unenhanced phases (UP), corticomedullary phases (CMP), and nephrographic phases (NP). Five different machine learning classifiers were compared. The effects of tumor and tumor margins are also compared. Five-fold cross-validation and the area under the receiver operating characteristic curve (AUC) are used to evaluate model performance.ResultsThis study included 126 RCC patients, including 46 (36.5%) with capsule invasion. CMP exhibited the highest AUC (AUC = 0.81) compared to UP and NP, when using the forward neural network (FNN) classifier. The AUCs using features extracted from the tumor region were generally higher than those of the marginal regions in the CMP (0.81 vs. 0.73) and NP phase (AUC = 0.77 vs. 0.76). For UP, the best result was obtained from the marginal region (AUC = 0.80). The robustness analysis on the UP, CMP, and NP achieved the AUC of 0.76, 0.79, and 0.77, respectively.ConclusionsRadiomics features in renal CT imaging are associated with the renal capsule invasion in RCC patients. Further evaluation of the models is warranted.

PREDICTING SOCIAL NETWORK ADDICTION USING VARIANT SIGMOID TRANSFER FEED-FORWARD NEURAL NETWORKS (FNN-SNA)

Researchers have reflected on personal traits that may predict Social Networking Sites (SNS) addiction. However, most of the researchers involved in the findings of personality traits predictor for social networking addiction either postulate or based their conclusions on analytical tools. Moreso, a review of the literature reveals that the prediction of social networking addiction using classifiers have not been well researched. We examined the prediction of SNS addiction from a well-structured questionnaire consisting of sixteen (16) personality traits. The questionnaire was administered on the google form with a response rate of 95% out of the 102-sample size. Additionally, a three (3) variant sigmoid transfer feed- forward neural networks was developed for the prediction of SNS addiction. Result indicated that pertinence (β = 0.251, p 0.01) was the most powerful predictor of social networking addiction in general and less obscurity addiction (β = 0.244, p 0.01). Experimental results also showed that the developed classifier correctly predict SNS addiction with 98% accuracy compared to similar classifiers.
 
 

Inverse design of organic light-emitting diode structure based on deep neural networks

Abstract The optical properties of thin-film light emitting diodes (LEDs) are strongly dependent on their structures due to light interference inside the devices. However, the complexity of the design space grows exponentially with the number of design parameters, making it challenging to optimize the optical properties of multilayer LEDs with rigorous electromagnetic simulations. In this work, we demonstrate an artificial neural network that can predict the light extraction efficiency of an organic LED structure in 30 ms, which is ∼103 times faster than the rigorous simulation in a single-treaded execution with root-mean-squared error of 1.86 × 10−3. The effective inference time per structure is brought down to ∼0.6 μs with unaltered error rate with parallelization. We also show that our neural networks can efficiently solve the inverse problem – finding a device design that exhibits the desired light extraction spectrum – within the similar time scale. We investigate the one-to-many mapping issue of the inverse problem and find that the degeneracy can be lifted by incorporating additional emission spectra at different observing angles. Furthermore, the forward neural network is combined with a conventional genetic algorithm to address additional large-scale optimization problems including maximization of light extraction efficiency and minimization of angle dependent color shift. Our approach establishes a platform for tackling computation-heavy optimization tasks with one-time computational cost.

An Anatomy of a Hybrid Color Descriptor with a Neural Network Model to Enhance the Retrieval Accuracy of an Image Retrieval System

Background: In order to retrieve a particular image from vast repository of images, an efficient system is required and such an eminent system is well-known by the name Content-Based Image Retrieval (CBIR) system. Color is indeed an important attribute of an image and the proposed system consist of a hybrid color descriptor which is used for color feature extraction. Deep learning has gained a prominent importance in the current era. So, the performance of this fusion based color descriptor is also analyzed in the presence of Deep learning classifiers. Method: This paper describes a comparative experimental analysis on various color descriptors and the best two are chosen to form an efficient color based hybrid system denoted as combined color moment-color autocorrelogram (Co-CMCAC). Then, to increase the retrieval accuracy of the hybrid system, a Cascade forward back propagation neural network (CFBPNN) is used. The classification accuracy obtained by using CFBPNN is also compared to Patternnet neural network. Results: The results of the hybrid color descriptor depict that the proposed system has superior results in the order 95.4%, 88.2%, 84.4% and 96.05% for Corel-1K, Corel-5K, Corel-10K and Oxford flower benchmark datasets respectively as compared to many state-of-the-art related techniques. Conclusion: This paper depicts an experimental and analytical analysis on different color feature descriptors namely, Color Moment (CM), Color Auto-Correlogram (CAC), Color Histogram (CH), Color coherence vector (CCV) and Dominant Color Descriptor (DCD). The proposed hybrid Color Descriptor (Co-CMCAC) is utilized for the withdrawal of color features with Cascade Forward Back Propagation Neural Network (CFBPNN) used as a classifier on four benchmark datasets namely Corel-1K, Corel-5K and Corel-10K and Oxford flower.

Predicting the success rate of a start-up using LSTM with a swish activation function

The researchers emphasised more small-scale start-ups which required improvement in the economy decreased the failure rate by avoiding the valuable resources. The resources were required to be used without wasting them lead to the development of the economy and reduces the rate of unemployment. Therefore, to minimise resource wastage and to avoid the risk of failure the researchers considered certain factors that affected the failure and success of small-scale industries. The present research uses the Crunch Base dataset that predicts the success or failure of a start-up by using the Long Short Term Memory (LSTM). The LSTM unit Recurrent Neural Network (RNN) uses the Swish activation function in Feed Forward Neural (FFN) Network for the classification. The proposed LSTM obtained better accuracy of 71.64% when compared with existing methods such as RNN that attained 65.67% and Artificial Neural Network (ANN) of 69.7%.

A Novel Constitutive Parameters Identification Procedure for Hyperelastic Skeletal Muscles Using Two-Way Neural Networks

Muscle soreness can occur after working beyond the habitual load, especially for people engaged in high-intensity work load. Prediction of hyperelastic material parameters is essentially an inverse process, which possesses challenges. This work presents a novel procedure that combines nonlinear finite element method (FEM), two-way neural networks (NNs) together with experiments, to predict the hyperelastic material parameters of skeletal muscles. FEM models are first established to simulate nonlinear deformation of skeletal muscles subject to compressions. A dataset of nonlinear relationship between nominal stress and principal stretch of skeletal muscles is created using our FEM models. The dataset is then used to establish two-way NNs, in which a forward NN is trained and it is in turn used to train the inverse NN. The inverse NN is used to predict the hyperelastic material parameters of skeletal muscles. Finally, experiments are carried out using fresh skeletal muscles to validate the predictions in great detail. In order to examine the accuracy of the two-way NNs predicted values against the experimental ones, a decision coefficient [Formula: see text] with penalty factor is introduced to evaluate the performance. Studies have also been conducted to compare the present two-way NNs approach with the other existing methods, including the directly (one-way) inverse problem NN, and improved niche genetic algorithm (INGA). The comparison results show that two-way NNs model is an accurate approach to identify the hyperelastic parameters of skeletal muscles. The present two-way NNs method can be further expanded to the predictions of constitutive parameters of other type of nonlinear materials.