Two-layer Neural Network Model Research Articles

Innovative integrated workflow for data-driven production forecasting and well completion optimization: A Montney Formation case study

With the use of advanced data analytics and machine learning (ML) techniques, the oil and gas industry has transformed significantly in recent years. These innovations have opened new opportunities in reservoir engineering, allowing engineers to leverage data for better decision-making and well completion optimization. A key challenge in hydrocarbon exploration and production is forecasting gas production rates from unconventional sources. This is a crucial factor that affects how resources are allocated and decisions are made, requiring the application of novel techniques to enhance hydrocarbon production. Here, a novel and comprehensive automated workflow is introduced covering data collection, data preparation, feature selection, ML model development, and operational parameter optimization. The workflow uses various ML and optimization methods, including a modified Differential Evolution (MDE) algorithm, to improve gas production forecasting and optimize well completion parameters. We applied the proposed workflow to 3144 horizontal gas wells in the Montney Formation, British Columbia (BC), Canada, to assess its performance. The results showed the advantages of incorporating the initial 24 months of cumulative gas production data alongside well and completion data. Also, following the fine-tuning of hyperparameters for the ML algorithms, we identified the most effective model for forecasting cumulative gas production within the Montney Formation as a two-layer Artificial Neural Network (ANN) model. Moreover, using the MDE algorithm, the optimization results revealed the possibility of increasing gas production by more than 10% for about 47% of the wells. This research makes a significant contribution to the field of unconventional gas exploration by introducing an automated workflow that simplifies data processing, model development, and decision-making processes. It also offers valuable insights for optimizing unconventional gas production in the Montney Formation and similar reservoirs, as the energy industry moves towards more sustainable and efficient practices.

Novel Advanced Artificial Neural Network-Based Online Stator and Rotor Resistance Estimator for Vector-Controlled Speed Sensorless Induction Motor Drives

This paper presents a new approach for the online estimation of stator and rotor resistance of induction motors for speed sensorless vector-controlled drives, using feed-forward artificial neural networks with advanced adaptive learning rates. For the rotor resistance estimation, a neural network model based on rotor speed and stator currents is developed. The rotor flux linkages acquired from the voltage model are compared with the neural network model. The feed-forward neural network employs an adaptive learning rate as the function of the obtained error during training for quick convergence with minimal estimation error. A two-layered neural network model based on the stator voltage and current equations is developed for the stator resistance estimation. The d-q axes stator currents obtained from the developed model are compared with the acquired d-q axes stator currents. For the fast convergence with minimal estimation error, an adaptive learning rate as the function of error is adopted during training. Furthermore, the neural network estimates the induction motor’s speed. The simulation and experimental results justify that the developed algorithms track variation in the resistances quickly and precisely along with the speed as compared with the conventional constant learning rate algorithm, leading to reliable operation of the drive.

Study on predicting the radiant heat flow rate of floor surface of radiant floor heating

Radiant heat flow rate of radiant surface is crucial for radiant floor heating design and terminal selection. We found that the present empirical formula to predict radiant heat flow rate of radiant surface has limits under varied room size and surface emissivity circumstances. Wall insulation conditions also have substantial influences on the operating efficiency of floor radiant heating. Adopting machine learning algorithms and backward selection method, a two-layer Neural Network model was demonstrated to have good accuracy and requisite relevant features for new empirical formula was identified. The new formula containing the information of room depth, weighted radiant surface area, insulation conditions, indoor air temperature and radiant surface temperature as independent variables exhibits great accuracy with R-squared of 0.97 and RMSE of 2.74. The substitution of AUST with indoor air temperature can increase the prediction accuracy. Analysis reveals structural pattern and indicates interaction between wall insulation and non-radiative surface with low emissivity. We propose the use of non-radiant surfaces with low emissivity as a passive energy-saving technique for radiant floor heating. And the updated empirical formula can increase its application scenarios, and aid promote application of FRH and new passive technique.

Performance simulation and diagnosis of faulty states in air-cycle refrigeration systems in civil aircrafts

In the present study, performance simulation models and diagnosis method of faulty states in air-cycle refrigeration systems in civil aircrafts are developed, and the effects of the training data set on the diagnosis accuracy were analyzed. A base model for normal state simulation of air-cycle refrigeration system (ACS) was established based on the existing method with several modifications to improve accuracy and performance. Dedicated models for faulty states were developed according to the causes of fault. Based on the established models, a data set was generated. Preliminary training of multiple two-layer neural network models is performed with the existing data set, and the trained model achieved a validation accuracy of 99.00 % which confirms the feasibility of using double-layer neural network for fault identification and classification based on simulation data. The effects of the training data set to the model are analyzed, and it was concluded that: a data set of about 50,000 entries can be used to train a satisfactory diagnostic model; the proportion of normal state data needs to be reduced appropriately to improve the sensitivity of the model to faulty states; it is possible to largely reduce the number of sensors with smaller effect to the diagnosis accuracy.

Prediction Model of Fouling Thickness of Heat Exchanger Based on TA-LSTM Structure

Heat exchangers in operation often experience scaling, which can lead to a decrease in heat exchange efficiency and even safety accidents when fouling accumulates to a certain thickness. To address this issue, manual intervention is currently employed to monitor fouling thickness in advance. In this study, we propose a two-layer LSTM neural network model with an attention mechanism to effectively learn fouling thickness data under different working conditions. The model accurately predicts the scaling thickness of the heat exchanger during operation, enabling timely human intervention and ensuring that the scaling remains within a safe range. The experimental results demonstrate that our proposed neural network model (TA-LSTM) outperforms both the traditional BP neural network model and the LSTM neural network model in terms of accuracy and stability. Our findings provide valuable technical support for future research on heat exchanger descaling and fouling growth detection.

Prognosis of LED lumen degradation using Bayesian optimized neural network approach

Light Emitting Diodes (LEDs) are among the most widely used electronic devices for everyday lighting applications due to their durability over incandescent lamps. However, there are no standardized approaches to predict the reliability of LEDs as they are manufactured and tested as per the user requirements and applications. This dramatically limits developing generic prognostic algorithms pertaining to predicting the remaining useful life (RUL) of LEDs. In this study, we propose a Bayesian optimized neural network approach to predict the lumen degradation trends of LEDs. The proposed method does not require an accurate physical model representing the LED degradation behavior and does not require a large amount of degradation data. We have used a particle filter algorithm to train a simple two-layer feedforward neural network model and use the trained model to predict the lumen degradation of LEDs. Also, the weight decay issues commonly encountered in particle filter algorithm are addressed using three different resampling strategies and particle roughening method. To evaluate the effectiveness of the proposed approach, Root Mean Squared Error (RMSE) and Relative Accuracy (RA) were used as the prognostic metrics.

Aberrant circulating tumour DNA methylations as biomarkers for early detection of pancreatic ductal adenocarcinoma: a retrospective study

Pancreatic ductal adenocarcinoma is a cancer with high mortality and low survival, the early detection of which is hampered by the absence of specific symptoms until an advanced stage is reached. No reliable early screening tool for pancreatic ductal adenocarcinoma is currently available. Circulating tumour DNA (ctDNA) methylation has emerged as a promising new type of biomarker for blood-based early detection of multiple types of cancer. As a part of the PanSeerX study, which aims to discover robust ctDNA methylation markers for multicancer early screening, a customised pancreatic ductal adenocarcinoma-specific targeted methylation sequencing panel has been preliminarily developed and validated for its accuracy in classifying pancreatic ductal adenocarcinoma from retrospective clinical plasma samples. We led a single-centre retrospective pilot study at the Changhai Hospital, Navy Medical University, Shanghai, China, where 62 adult patients with pancreatic ductal adenocarcinoma and 393 age-matched and sex-matched healthy controls were enrolled after written consent was obtained from each participant. Individuals with previous cancer diagnosis were excluded. Blood samples were drawn from all participants. A methylation target sequencing panel covering 1601 potential pancreatic ductal adenocarcinoma methylation markers, which were collected from our previous PanSeer study and other related studies, identified by mining the public methylomic datasets The Cancer Genome Atlas and Gene Expression Omnibus, or discovered from the analysis of in-house reduced-representation bisulfite sequencing data, was designed and technically validated. The panel was then applied to test the collected plasma samples at mean a sequencing depth of 1000× per target to quantify the methylation levels and patterns on the targets. A two-layer deep neural network model was built to classify cancer and healthy samples from the sequencing data. The robustness of entire approach was verified with a 3× cross-validation by randomly splitting samples into training set and test set at a 2:1 ratio. This study was approved by the Changhai Hospital, Navy Medical University (reference number CHEC2021-165). All blood samples were used in the development and validation of the pancreatic ductal adenocarcinoma panel. The average area under the curve of the training dataset during the 3× validation was 1·000 (95% CI 0·999-1·000; sensitivity 100·0% [95% CI 100·0-100·0; specificity 99·0%, [98·6-99·5]). The average area under the curve of the testing dataset during the 3× validation was 0·987 (95% CI 0·971-0·996; sensitivity 89·0% [95% CI 75·0-100·0]; specificity 96·0% [92·3-100·0]). Notably, this model is highly sensitive to pancreatic ductal adenocarcinoma of stages I and II, achieving a sensitivity of 81% for stage I and 88% for stage II, showing its ability to detect early stage pancreatic ductal adenocarcinoma. Our preliminary results show that our pancreatic ductal adenocarcinoma-detecting panel is highly accurate in classifying pancreatic ductal adenocarcinoma plasma from healthy controls using ctDNA methylation markers. Its high sensitivity for stages I and II pancreatic cancer is especially promising for further optimisation into diagnostics for blood-based, early pancreatic ductal adenocarcinoma screening. On the basis of these results, a larger, multicentre study is currently underway, which not only enrolled a higher number of patients with pancreatic ductal adenocarcinoma cases and healthy controls, but also included samples from acute and chronic pancreatitis to comprehensive evaluate our pancreatic ductal adenocarcinoma-detecting panel's accuracy in classifying pancreatic ductal adenocarcinoma plasma against non-malignant controls. The results of this study are expected to be reported later in 2022. The National Key Research and Development Project of China (grant 2019YFC1315904), the 234 Discipline Climbing Plan Project of the First Affiliated Hospital of Naval Military Medical University (grant 2019YXK033), and the Shanghai Science and Technology Committee.

An $L^2$ Analysis of Reinforcement Learning in High Dimensions with Kernel and Neural Network Approximation

Reinforcement learning (RL) algorithms based on high-dimensional function approximation have achieved tremendous empirical success in large-scale problems with an enormous number of states. However, most analysis of such algorithms gives rise to error bounds that involve either the number of states or the number of features. This paper considers the situation where the function approximation is made either using the kernel method or the two-layer neural network model, in the context of a fitted Q-iteration algorithm with explicit regularization. We establish an $\tilde{O}(H^3|\mathcal {A}|^{\frac14}n^{-\frac14})$ bound for the optimal policy with $Hn$ samples, where $H$ is the length of each episode and $|\mathcal {A}|$ is the size of action space. Our analysis hinges on analyzing the $L^2$ error of the approximated Q-function using $n$ data points. Even though this result still requires a finite-sized action space, the error bound is independent of the dimensionality of the state space.

Understanding the Phonetic Characteristics of Speech Under Uncertainty-Implications of the Representation of Linguistic Knowledge in Learning and Processing.

The uncertainty associated with paradigmatic families has been shown to correlate with their phonetic characteristics in speech, suggesting that representations of complex sublexical relations between words are part of speaker knowledge. To better understand this, recent studies have used two-layer neural network models to examine the way paradigmatic uncertainty emerges in learning. However, to date this work has largely ignored the way choices about the representation of inflectional and grammatical functions (IFS) in models strongly influence what they subsequently learn. To explore the consequences of this, we investigate how representations of IFS in the input-output structures of learning models affect the capacity of uncertainty estimates derived from them to account for phonetic variability in speech. Specifically, we examine whether IFS are best represented as outputs to neural networks (as in previous studies) or as inputs by building models that embody both choices and examining their capacity to account for uncertainty effects in the formant trajectories of word final [ɐ], which in German discriminates around sixty different IFS. Overall, we find that formants are enhanced as the uncertainty associated with IFS decreases. This result dovetails with a growing number of studies of morphological and inflectional families that have shown that enhancement is associated with lower uncertainty in context. Importantly, we also find that in models where IFS serve as inputs—as our theoretical analysis suggests they ought to—its uncertainty measures provide better fits to the empirical variance observed in [ɐ] formants than models where IFS serve as outputs. This supports our suggestion that IFS serve as cognitive cues during speech production, and should be treated as such in modeling. It is also consistent with the idea that when IFS serve as inputs to a learning network. This maintains the distinction between those parts of the network that represent message and those that represent signal. We conclude by describing how maintaining a “signal-message-uncertainty distinction” can allow us to reconcile a range of apparently contradictory findings about the relationship between articulation and uncertainty in context.

An Improved Whale optimization Algorithm for Cross layer Neural Connection Network of MANET

The connecting of numerous remote mobile nodes is known as a mobile ad hoc network. These networks are dynamic and self-contained, allowing them to move about freely. It is referred to as a structure-less network since it lacks a central controller. MANET(Mobile ad hoc network) is one of the most recent developing technologies to gain popularity. This research presents a improved Whale Optimization is enabled for the best feasible solution of the CNCN.The improved whale optimization uses a probability function to determine the best communication path in the network. According to a comparative examination of research, Improved WOA gives significant performance. A two-layer Neural Connection Network model with a cross-layer structure. Physical layer and data link layer. Then in the Physical Layer the load balancing as well as the packet specification is happensso we go for optimization technique. In Data Link Layer the Packet with huge amount of network path is enabled and the packets are delivered with the help of the Connector. The improved whale optimization is enabled in order to achieve the highest level of overall performance such as Waiting time, reliability, failure probability, throughput, and Instantaneous Throughput.

A PRELIMINARY STUDY ON UNDERSTANDING THE CONSUMPTIONS OF THERAPEUTIC ESSENTIAL OILS DURING COVID-19 PANDEMIC AMONG ADULTS USING ANN

The COVID-19 pandemic has emphasized the significance of utilizing essential oils (EO) as one of the holistic ways of supporting and enhancing health. As a consequence of growing knowledge of connected health concerns, people all over the world are looking for natural ways to avoid different ailments. It has been proven that excellent health and psychological awareness increase the human body's immune response, therefore boosting disease resistance. Essential oils are derived in a number of ways from valued plants containing active chemicals with medicinal qualities. In Malaysia, many have used EO in their daily lives. This paper identifies the hierarchy of importance among factors which contribute towards the usage frequency of essential oils in Malaysia using an artificial neural network. Two-layer neural network (NN) models have been applied, which are multilayer perceptron (MLP) and radial basis function (RBF). Based on the analysis done, RBF-NN performed the best with SSE=4.436 and RE=0.548. It can be concluded that, based on sensitivity analysis, the top five factors toward usage frequency are consumption, age, external use, clinic visit, and occasion, with normalized importance of 100%, 90.8%, 89.3%, 68.2%, and 42.2% respectively.

Estimation of Maximum Hail Diameters from FY-4A Satellite Data with a Machine Learning Method

The magnitude of damage caused by hail depends on its size; however, direct observation or indirect estimation of hail size remains a significant challenge. One primary reason for estimations by proxy, such as through remote sensing methods, is that empirical relationships or statistical models established in one region may not apply to other areas. This study employs a machine learning method to build a hail size estimation model without assuming relations in advance. It uses FY-4A AGRI data to provide cloud-top information and ERA5 data to add vertical environment information. Before training the model, we conducted a principal component analysis (PCA) to analyze the highly influential factors on hail sizes. A total of 18 features, composed of four groups, namely brightness temperature (BT), the difference in BT (BTD), thermodynamics, and dynamics groups, were chosen from 29 original features. Dynamic and BTD features show superior performance in identifying large hail. Although the selected features are more closely correlated to hail sizes than unselected ones, the relationships are complicated and nonlinear. As a result, a two-layer regression back propagation neural network (BPNN) model with powerful fitting ability is trained with selected features to predict maximum hail diameter (MHD). The linear fitting R2 between predicted and observed MHDs is 0.52 on the test set, which signifies that our model performs well compared with other hail size estimation models. We also examine the model concerning all three hail cases in Shanghai, China, between 2019 and 2021. The model attained more satisfactory results than the radar-based maximum estimated hail size (MEHS) method, which overestimates the MHDs, thus further supporting the operational applications of our model.

The Barron Space and the Flow-Induced Function Spaces for Neural Network Models

One of the key issues in the analysis of machine learning models is to identify the appropriate function space and norm for the model. This is the set of functions endowed with a quantity which can control the approximation and estimation errors by a particular machine learning model. In this paper, we address this issue for two representative neural network models: the two-layer networks and the residual neural networks. We define the Barron space and show that it is the right space for two-layer neural network models in the sense that optimal direct and inverse approximation theorems hold for functions in the Barron space. For residual neural network models, we construct the so-called flow-induced function space and prove direct and inverse approximation theorems for this space. In addition, we show that the Rademacher complexity for bounded sets under these norms has the optimal upper bounds.

Theory and Application of Weak Signal Detection Based on Stochastic Resonance Mechanism

Stochastic resonance is a new type of weak signal detection method. Compared with traditional noise suppression technology, stochastic resonance uses noise to enhance weak signal information, and there is a mechanism for the transfer of noise energy to signal energy. The purpose of this paper is to study the theory and application of weak signal detection based on stochastic resonance mechanism. This paper studies the stochastic resonance characteristics of the bistable circuit and conducts an experimental simulation of its circuit in the Multisim simulation environment. It is verified that the bistable circuit can achieve the stochastic resonance function very well, and it provides strong support for the actual production of the bistable circuit. This paper studies the stochastic resonance phenomenon of FHN neuron model and bistable model, analyzes the response of periodic signals and nonperiodic signals, verifies the effect of noise on stochastic resonance, and lays the foundation for subsequent experiments. It proposes to feedback the link and introduces a two-layer FHN neural network model to improve the weak signal detection performance under a variable noise background. The paper also proposes a multifault detection method based on the total empirical mode decomposition of sensitive intrinsic mode components with variable scale adaptive stochastic resonance. Using the weighted kurtosis index as the measurement index of the system output can not only maintain the similarity between the system output signal and the original signal but also be sensitive to impact characteristics, overcoming the missed or false detection of the traditional kurtosis index. Experimental research shows that this method has better noise suppression ability and a clear reproduction effect on details. Especially for images contaminated by strong noise (D = 500), compared with traditional restoration methods, it has better performance in subjective visual effects and signal-to-noise ratio evaluation.

Predicting vibrancy of metro station areas considering spatial relationships through graph convolutional neural networks: The case of Shenzhen, China

Vibrancy is one of the most desirable outcomes of transit-oriented development (TOD). The vibrancy of a metro station area (MSA) depends partially on the MSA’s built-environment features. Predicting an MSA’s vibrancy with its built-environment features is of great interest to decision makers as these features are often modifiable by public interventions. However, little has been done on MSAs’ vibrancy in existing studies. On the one hand, seldom has the vibrancy of MSAs been explicitly explored, and measuring the vibrancy is essential. On the other hand, because MSAs are interconnected, one MSA’s vibrancy depends on the MSA’s features and those of relevant MSAs. Hence, selecting a suitable metric that quantifies spatial relationships between MSAs can better predict MSAs’ vibrancy. In this study, we identify four single-dimensional vibrancy proxies and fuse them into an integrated index. Moreover, we design a two-layer graph convolutional neural network model that accounts for both the built-environment features of MSAs and spatial relationships between MSAs. We employ the model in an empirical study in Shenzhen, China, and illustrate (1) how different metrics of spatial relationships influence the prediction of MSAs’ vibrancy; (2) how the predictability varies across single-dimensional and integrated proxies of MSAs’ vibrancy; and (3) how the findings of this study can be used to enlighten decision makers. This study enriches our understandings of spatial relationships between MSAs. Moreover, it can help decision makers with targeted policies for developing MSAs towards TOD.

Cognitive Impairment and Dementia in Primary Care: Current Knowledge and Future Directions Based on Findings From a Large Cross-Sectional Study in Crete, Greece.

Introduction: Dementia severely affects the quality of life of patients and their caregivers; however, it is often not adequately addressed in the context of a primary care consultation, especially in patients with multi-morbidity.Study Population and Methods: A cross-sectional study was conducted between March-2013 and December-2014 among 3,140 consecutive patients aged >60 years visiting 14 primary health care practices in Crete, Greece. The Mini-Mental-State-Examination [MMSE] was used to measure cognitive status using the conventional 24-point cut-off. Participants who scored low on MMSE were matched with a group of elders scoring >24 points, according to age and education; both groups underwent comprehensive neuropsychiatric and neuropsychological assessment. For the diagnosis of dementia and Mild-Cognitive-Impairment (MCI), the Diagnostic and Statistical Manual-of-Mental-Disorders (DSM-IV) criteria and the International-Working-Group (IWG) criteria were used. Chronic conditions were categorized according to ICD-10 categories. Logistic regression was used to provide associations between chronic illnesses and cognitive impairment according to MMSE scores. Generalized Linear Model Lasso Regularization was used for feature selection in MMSE items. A two-layer artificial neural network model was used to classify participants as impaired (dementia/MCI) vs. non-impaired.Results: In the total sample of 3,140 participants (42.1% men; mean age 73.7 SD = 7.8 years), low MMSE scores were identified in 645 (20.5%) participants. Among participants with low MMSE scores 344 (54.1%) underwent comprehensive neuropsychiatric evaluation and 185 (53.8%) were diagnosed with Mild-Cognitive-Impairment (MCI) and 118 (34.3%) with dementia. Mental and behavioral disorders (F00-F99) and diseases of the nervous system (G00-G99) increased the odds of low MMSE scores in both genders. Generalized linear model lasso regularization indicated that 7/30 MMSE questions contributed the most to the classification of patients as impaired (dementia/MCI) vs. non-impaired with a combined accuracy of 82.0%. These MMSE items were questions 5, 13, 19, 20, 22, 23, and 26 of the Greek version of MMSE assessing orientation in time, repetition, calculation, registration, and visuo-constructive ability.Conclusions: Our study identified certain chronic illness-complexes that were associated with low MMSE scores within the context of primary care consultation. Also, our analysis indicated that seven MMSE items provide strong evidence for the presence of dementia or MCI.

Discrimination of cycling patterns using accelerometric data and deep learning techniques

The monitoring of physical activities and recognition of motion disorders belong to important diagnostical tools in neurology and rehabilitation. The goal of the present paper is in the contribution to this topic by (1) analysis of accelerometric signals recorded by wearable sensors located at specific body positions and by (2) implementation of deep learning methods to classify signal features. This paper uses the general methodology to analysis of accelerometric signals acquired during cycling at different routes followed by the global positioning system. The experimental dataset includes 850 observations that were recorded by a mobile device in the spine area (L3 verterbra) for cycling routes with the different slope. The proposed methodology includes the use of deep learning convolutional neural networks with five layers applied to signal values transformed into the frequency domain without specification of any signal features. The accuracy of discrimination between different motion patterns for the uphill and downhill cycling and recognition of 4 classes associated with different route slopes was 96.6% with the loss criterion of 0.275 for sigmoidal activation functions. These results were compared with those evaluated for selected sets of features estimated for each observation and classified by the support vector machine, Bayesian methods, and the two-layer neural network. The best cross-validation error of 0.361 was achieved for the two-layer neural network model with the sigmoidal and softmax transfer functions. Our methodology suggests that deep learning neural networks are efficient in the assessment of motion activities for automated data processing and have a wide range of applications, including rehabilitation, early diagnosis of neurological problems, and possible use in engineering as well.

Machine learning from a continuous viewpoint, I

We present a continuous formulation of machine learning, as a problem in the calculus of variations and differential-integral equations, in the spirit of classical numerical analysis. We demonstrate that conventional machine learning models and algorithms, such as the random feature model, the two-layer neural network model and the residual neural network model, can all be recovered (in a scaled form) as particular discretizations of different continuous formulations. We also present examples of new models, such as the flow-based random feature model, and new algorithms, such as the smoothed particle method and spectral method, that arise naturally from this continuous formulation. We discuss how the issues of generalization error and implicit regularization can be studied under this framework.

Modeling and optimizing the removal of cadmium by Sinapis alba L. from contaminated soil via Response Surface Methodology and Artificial Neural Networks during assisted phytoremediation with sewage sludge

The study was aimed to model and optimize the removal of cadmium from contaminated post-industrial soil via Sinapis alba L. by comparing two modeling approaches: Response Surface Methodology (RSM) and Artificial Neural Networks (ANN). The experimental design was done using the Box–Behnken Design method. In the RSM model, the quadratic model was shown to predict the closest results in comparison to our experimental data. For ANN approach, a two-layer Feed-Forward Back-Propagation Neural Network model was designed. The results showed that sewage sludge supplementation increased the efficiency of the Sinapis alba plant in removing Cd from the soil. After 28 days of exposure, the removal rate varied from 10.96% without any supplementation to 65.9% after supplementation with the highest possible (law allowed) dose of sewage sludge. The comparison proved that the prediction capability of the ANN model was much higher than that of the RSM model (adjusted R-square: 0.98, standard error of the Cd prediction removal: 0.85 ± 0.02). Thus, the ANN model could be used for the prediction of heavy metal removal during assisted phytoremediation with sewage sludge. Moreover, such approach could also be used to determinate the dose of sewage sludge that will ensure highest process efficiency.

HieNN-DWE: A hierarchical neural network with dynamic word embeddings for document level sentiment classification

The recent rapid developments of neural networks have stimulated significant performance progress on document level sentiment classification. The existing researches focus on extracting long-distance features from documents, which fail to exploit the deep semantic information of documents. Besides, few of them have taken the representations of polysemous words into consideration, such that the semantic information of documents are not precisely exploited. To tackle these issues, we propose a novel hierarchical neural network model based on dynamic word embeddings (HieNN-DWE) for document level sentiment classification. Specifically, the dynamic word embeddings are produced by ELMo (Embedding from Language Models), which can deeply exploit the semantic information for polysemous words. The embeddings are then fed into HieNN-DWE which is a two-layer hierarchical neural network model. Note that the first layer utilizes bidirectional gated recurrent unit (BiGRU) as well as attention mechanism to encode sentences. The second layer uses both BiGRU and convolutional neural network (CNN) to capture global and local features from sentence representations. Thus, the final document representations are constructed by concatenating the outputs from BiGRU and CNN. Moreover, experimental results have shown that HieNN-DWE outperforms existing competitive methods on four public datasets for document level sentiment classification.