Linear Neural Network Models Research Articles

Combined sentiment score and star rating analysis of travel destination prediction based on user preference using morphological linear neural network model with correlated topic modelling approach

Combined sentiment score and star rating analysis of travel destination prediction based on user preference using morphological linear neural network model with correlated topic modelling approach

Practical aspects of application of artifi cial intelligence in metrology

Artifi cial intelligence as one of the main elements of digital transformation and promising directions of its application in metrology are considered. The main attention is paid to the application of artifi cial neural networks as a part of measuring instruments and measurement systems for obtaining measurement results in cases when the measurement function is unknown, not suffi ciently defi ned or too complicated for algorithmic formalization. More often in practice we meet problems with partially uncertain function, when in addition to the deterministic basis there is an additional unknown component that has a signifi cant impact on the measurement result. A simulation experiment was conducted to solve such a measurement problem using a neural network model. In the experiment, a measurement function with a linear deterministic basis and an additional nonlinear component of about 10 % of the relative standard deviation, that was implied by the unknown when the model was created, was used. The results of the experiment confi rmed the practical possibility and high effi ciency of using artifi cial neural networks to solve such measurement problems. The neural network model in the conditions of a noisy training sample, corresponding to real measurement conditions, almost completely restored the measurement function despite the fact that a linear neural network model was used, and the additional component of the measurement function was nonlinear. In this particular experiment, due to the use of neural network, the accuracy of measurements was improved by about an order of magnitude. Access to the machine code implementing this simulation experiment is provided.

Physics Informed Piecewise Linear Neural Networks for Process Optimization

Constructing first-principles models is usually a challenging and time-consuming task due to the complexity of real-life processes. On the other hand, data-driven modeling, particularly a neural network model, often suffers from overfitting and lack of useful and high-quality data. At the same time, embedding trained machine learning models directly into the optimization problems has become an effective and state-of-the-art approach for surrogate optimization, whose performance can be improved by physics-informed machine learning. This study proposes using piecewise linear neural network models with physics-informed knowledge for optimization problems with neural network models embedded. In addition to using widely accepted and naturally piecewise linear rectified linear unit (ReLU) activation functions, this study also suggests piecewise linear approximations for the hyperbolic tangent activation function to widen the domain. Optimization of three case studies, a blending process, an industrial distillation column, and a crude oil column are investigated. Physics-informed trained neural network-based optimal results are closer to global optimality for all cases. Finally, associated CPU times for the optimization problems are much shorter than the standard optimization results.

PREDICTION OF THE DYSBIOSIS DEVELOPMENT RISK ACCORDING TO HUMORAL IMMUNITY INDICATORS

Diseases associated with bacterial vaginosis lead to chronic inflammatory processes of the internal genitals, the development of adhesions of the pelvic organs, infertility, spontaneous abortion at different times, as well as the development of malignant neoplasms. Vaginal microflora is an indicator of a woman’s health, which can form changing in hormonal and immunological status during various pathological conditions. The aim of the study was to create a system for prediction of the dysbiosis development according to the levels of nonspecific humoral factor of immune defense. Materials and methods. The study was performed in 298 women aged 16 to 64 years, 53 of whom were diagnosed with normocenosis, and 245 have dysbiosis. Women were divided into 3 groups according to age. Regression analysis was used. Research results and their discussion. Our previous research have shown a correlation between increased levels of anti-inflammatory cytokines in the blood and vaginal secretions with the stage of dysbiosis. A logistic regression model was constructed during the study, which showed that the risk of developing dysbiosis in terms of normobiota increases with increasing levels of interleukin 2 in the blood, tumor necrosis factor α. Significant features of the three-factor model for predicting the risk of developing dysbiosis (IL2, IL4 and TNFα) were selected by the method of genetic algorithm. The levels of these indicators in the blood were related to the severity of dysbiosis according to the results of discriminant analysis. Conclusions. Thus, a linear neural network model was developed for determination of dysbiosis severity according to the levels of nonspecific humoral factors of immune defense such as the C4 component of the complement system and γ-interferon in vaginal secretions, as well as the amount of circulating immune complexes and tumor necrosis factor α in the blood. Kappa Cohen’s agreement for this model on the training set was 0.87 (95% CI 0.82-0.91), and on the confirmatory set was 0.89 (95% CI 0.77-1.00). These indicators show the adequacy of the constructed model. The interface of the expert system for the dysbiosis severity prediction has been created. Key words: normobiota, bacterial vaginosis, linear neural network model for determination of the dysbiosis severity, humoral factor of immune defense.

Humoral immunity indicators as prediction factors for dysbiosis development risk

Diseases associated with bacterial vaginosis lead to chronic inflammatory processes of the internal genitals, the development of adhesions of the pelvic organs, infertility, spontaneous abortion at different times, as well as the development of malignant neoplasms. Vaginal microflora is an indicator of a woman's health which can form changing in hormonal and immunological status during various pathological conditions. The aim of the study was to create a system for prediction of the dysbiosis development according to the levels of nonspecific humoral factor of immune defence.
 The study was performed in 298 women aged 16 to 64 years, 53 of whom were diagnosed with normocenosis, and 245 have dysbiosis. Women were divided into 3 groups according to age. Regression analysis was used.
 Our previous researches have shown a correlation between increased levels of anti-inflammatory cytokines in the blood and vaginal secretions with the stage of dysbiosis. A logistic regression model was constructed during the study, which showed that the risk of developing dysbiosis in terms of normobiota increases with increasing levels of interleukin 2 in the blood, tumor necrosis factor α. Significant features of the three-factor model for predicting the risk of developing dysbiosis (IL2, IL4 and TNFα) were selected by the method of genetic algorithm. The levels of these indicators in the blood were related to the severity of dysbiosis according to the results of discriminant analysis. Thus, a linear neural network model was developed for determination of dysbiosis severity according to the levels of nonspecific humoral factors of immune defence such as the C4 component of the complement system and γ-interferon in vaginal secretions, as well as the amount of circulating immune complexes and tumor necrosis factor α in the blood. Kappa Cohen's agreement for this model on the training set was 0.87 (95% CI 0.82-0.91), and on the confirmatory set was 0.89 (95% CI 0.77-1.00). These indicators show the adequacy of the constructed model. The interface of the expert system for the dysbiosis severity prediction has been created.

Risk assessment of horizontal transport system in a copper mine

The risk assessment determines threats that could appear during the execution of production system goals. Thus, manufacturing companies need to evaluate and react to the risk as well as it is possible. Due to the complexity and variable character of the mining system, as well as different types of parameters obtained at subsequent stages of the horizontal transport process, the risk was assessed according to various methods. Three stages were distinguished in the horizontal transport system of copper ore studied in the paper: tyre haulage of the shot copper ore to the transfer point, the so-called grate, transferring the output to belt conveyors and crushing solid rock into smaller pieces; transport on belt conveyors. Risk factors have been characterized for each stage. To assess the risk of the loading and haulage processes, an ANN model was built to predict the amount of ore extracted. A general linear neural network model was also built to confirm the results of the correlation analysis. One of the methods recommended for risk assessment in manufacturing companies is the Failure Mode and Effects Analysis (FMEA), which allows for calculating the risk and prioritizing it. The FMEA method allows identifying only these elements of the production system that are the most sensitive to the impact of risk factors. For the next stages of the horizontal transport process, the risk was assessed according to the FMEA method. The risk levels at each stage were different. Production systems are composed of many elements that create the reliability structure of this system. Unfortunately, the FMEA method does not allow for analyzing the reliability structure of the production system. Therefore, the new idea presented in this paper is a method of total risk assessment of the horizontal transport system of copper ore, which does take its reliability structure into account. The currently used methods of risk analysis and assessment do not take into consideration the reliability structure of the production system. The proposed method can be applied for risk assessment in other production systems characterized by a diversified structure.

DNA mixture interpretation using linear regression and neural networks on massively parallel sequencing data of single nucleotide polymorphisms

ABSTRACT Massively parallel sequencing (MPS) enables concurrent analysis of multiple single nucleotide polymorphisms (SNPs) and detection of alleles from minor contributors to extremely imbalanced DNA mixtures. To interpret the complex MPS data obtained from DNA mixtures, EuroForMix, linear regression, and neural network models were employed. Data of 960 autosomal SNPs sequenced through MPS of 10 single-source DNA samples, 26 nondegraded DNA mixtures from nonrelative (mixture ratio 1:29–1:99), 16 nondegraded DNA mixtures from relatives (1:29–1:99), 8 degraded DNA mixtures from nonrelatives (1:29–1:99), and 16 degraded DNA mixtures of relatives (1:29–1:99) were analysed. In total, 89.4% (59/66), 93.9% (62/66), and 93.9% (62/66) of the minor contributors to DNA mixtures could be correctly inferred using EuroForMix, linear regression, and a neural network, respectively. In conclusion, the linear regression and neural network models outperformed EuroForMix in determining the minor contributor to DNA mixtures from MPS data.

Comparison of Multiple Linear Regression and Neural Network Models in Bank Performance Prediction in Botswana

Comparison of Multiple Linear Regression and Neural Network Models in Bank Performance Prediction in Botswana

GDP forecast of accommodation and catering industry in X city based on multi variable grey model —— Energy saving development of accommodation and catering industry

As the GDP of lodging and catering industry is generally quarterly or annual, the data volume is very small, so it is difficult to accurately predict the GDP of lodging and catering industry in this city for a city. However, a city can accurately predict the GDP development of the lodging and catering industry through electricity consumption. And the local government can formulate relevant policies to promote the stable development of lodging and catering industry and economy, and find the relationship between electricity consumption and GDP development of this industry, so as to achieve the purpose of low energy consumption and fast economic growth.t. This paper first conducted desensitization and normalization processing on the data, and then used Matlab to forecast the GDP of the lodging and catering industry in X City respectively by using GM (1, N) grey model, multiple linear regression model and BP neural network model. The experimental results show that the GM (1, N) grey model is more accurate and reliable than the other two methods in the case of less data. This paper solves the problem that the GDP of accommodation and catering industry in a single city is inaccurate.

The Loss Surface Of Deep Linear Networks Viewed Through The Algebraic Geometry Lens

By using the viewpoint of modern computational algebraic geometry, we explore properties of the optimization landscapes of deep linear neural network models. After providing clarification on the various definitions of "flat" minima, we show that the geometrically flat minima, which are merely artifacts of residual continuous symmetries of the deep linear networks, can be straightforwardly removed by a generalized L2-regularization. Then, we establish upper bounds on the number of isolated stationary points of these networks with the help of algebraic geometry. Combining these upper bounds with a method in numerical algebraic geometry, we find all stationary points for modest depth and matrix size. We demonstrate that, in the presence of the non-zero regularization, deep linear networks can indeed possess local minima which are not global minima. Finally, we show that even though the number of stationary points increases as the number of neurons (regularization parameters) increases (decreases), higher index saddles are surprisingly rare.

PSIII-5 Comparison of Bivariate Machine Learning and Linear Model for Genomic Prediction with Different Heritability, QTL and SNP Panel Scenarios

Abstract This simulation study used actual SNP genotypes on the first chromosome of Brangus beef cattle to simulate 0.50 genetically correlated two traits with heritabilities of 0.25 and 0.50 determined either by 50, 100, 250 or 500 QTL and then aimed to compare the accuracies of genomic prediction from bivariate linear and artificial neural network with 1 to 10 neurons models based on G genomic relationship matrix. QTL effects of 50, 100, 250 and 500 SNPs from the 3361 SNPs of 719 animals were sampled from a bivariate normal distribution. In each QTL scenario, the breeding values (Σgijβj) of animal i for two traits were generated by using genotype (gij) of animal i at QTL j and the effects (βj) of QTL j from a bivariate normal distribution. Phenotypic values of animal i for traits were generated by adding residuals from a bivariate normal distribution to the breeding values of animal i. Genomic predictions for traits were carried out by bivariate Feed Forward MultiLayer Perceptron ANN-1–10 neurons and linear (GBLUP) models. Three sets of SNP panels were used for genomic prediction: only QTL genotypes (Panel1), all SNP markers, including the QTL (Panel2), and all SNP markers, excluding the QTL (Panel3). Correlations from 10-fold cross validation for traits were used to assess predictive ability of bivariate linear (GBLUP) and artificial neural network models based on 4 QTL scenarios with 3 Panels of SNP panels. Table 1 shows that the trait with high heritability (0.50) resulted in higher correlation than the trait with low heritability (0.25) in bivariate linear (GBLUP) and artificial neural network models. However, bivariate linear (GBLUP) model produced higher correlation than bivariate neural network. Panel1 performed the best correlations for all QTL scenarios, then Panel2 including QTL and SNP markers resulted in better prediction than Panel3.

The vaginal bacterial dysbiosis severity predicting model according to the normobiota index

The local and systemic immunodeficiency is the main mechanism for vaginal bacterial dysbiosis and its extreme manifestation – bacterial vaginosis (BV) development. The complex immune response study and the establishment of the main mechanisms and factors, reflecting it and corresponding to the microbiocenosis severity disorder are relevant. Aim – to develop a neural network model of the severity of vaginal bacterial dysbiosis based on the assessment of normobiota. Divided into the following groups according to the Conditionally pathogenic microflora index (CPMI) and normobiota index (NBI): normocenosis (n=53), dysbiosis I (n=128) and II degree (n=117) among the latter 83 patients with PNB>1 lg GE/sample were identified, in whom BV was established. Molecular genetic studies of the epithelium scraping from the vagina posterolateral wall were carried out by Polymerase chain reaction (“DNK-Technologiia” LLC, RF). Facultative and obligate anaerobes, myco- and ureplasmas, and yeast-like fungi were quantified. The content of immunoglobulins, lysozyme, cytokines, complement, phagocytosis activity of leukocytes, hormones, the number of lymphocytes and their fractions, as well as the vaginal discharge pH (a total of 58 indexes) were identified in blood and vaginal discharge. For statistical and mathematical analysis, the Statistica 10 software (StatSoft, Inc., USA) was used. Using neural network modeling, it was revealed that among all the factorial signs for determining the bacterial dysbiosis degree, the complement component C4 and γ-INF content in the vaginal discharge and circulating immune complexes (CIC) and TNFα in the blood were important. A linear neural network model was built on the selected set of factor signs (the Cohen’s kappa coefficient consent index on the training set was k=0.87 (95% CI 0.82-0.91) for confirming plurality – k=0.89 (95% CI 0.77-1.00). With normocenosis, the complement activation PNB was decreased, and γ-INF and TNFα content was increased. The CIC levels blood increase corresponded to the opportunistic microflora growth and reflected the humoral immune response activation, which suggests that this indicator is an early dysbiosis marker. With I degree dysbiosis all factors had positive relationship with NBI, which reflected the immune system stress state. In case of II degree dysbiosis, NBI had a negative relationship with γ-INF content in the vaginal discharge, and CIC in the blood, while positive – with C4 content in vaginal discharge, and TNFα in the blood, which proved the immune system dysregulation and caused its further suppression with the BV-association immunodeficiency development. The immune system reaction during the BV development evolved from non-specific resistance reactions to cytokine-induced reactions of specific humoral immunity in response to the BV-associated microbiota growth, which subsequently experienced depletion and loss of immune control.

Estimation and demographic analysis of COVID-19 infections with respect to weather factors in Europe

ABSTRACT The main objective of this study is to investigate the relationship between the COVID-19 and the weather factors of the most populated and industrialised countries in Europe and propose the best mathematical model to forecast the daily number of COVID-19 cases. To find the relationship between the COVID-19 and the weather factors of absolute humidity and temperature in Spain, France, Italy, Germany, and the United Kingdom, we conducted a Poisson analysis. We also used the General Linear Neural Network (GRNN) model to forecast the trend and number of daily COVID-19 cases in these European countries. The results reveal a statistically significant negative relationship between the number of COVID-19 infections and weather factors of temperature & absolute humidity. Furthermore, the results show a stronger negative relationship between COVID-19 and absolute humidity than temperature. In our proposed GRNN method, we find better compatibility for the COVID-19 cases in Italy relative to the other European countries in this study.

The research on train resistance prediction of low vacuum pipeline based on different neural network algorithms

In order to obtain the maximum aerodynamic resistance of low-vacuum pipeline train under different working conditions, this paper uses different neural network algorithms to predict the maximum aerodynamic resistance. First, it calculates 85 groups of maximum resistance values of trains under different operation speeds, pipeline pressure and blocking ratios. Then, it takes 81 groups of data as training samples, establishing and training RBF neural network models, which are based on three different functions, and a linear neural network model as a comparison. Finally, it verifies those models with four groups of randomly selected verification data. The results show that the RBF network prediction model based on Newrbe function has the best prediction effect. It is superior to the other two RBF models in prediction accuracy. The prediction error of the linear neural network model for the maximum resistance of the train is large, and the prediction accuracy is far lower than that of the radial basis function neural network model.

Prediction of the inhibitory concentrations of chloroquine derivatives using deep neural networks models

In recent years, deep neural networks have begun to receive much attention, which has obvious advantages in feature extraction and modeling. However, in the using of deep neural networks for the QSAR modeling process, the selection of various parameters (number of neurons, hidden layers, transfer functions, data set partitioning, number of iterations, etc.) becomes difficult. Thus, we proposed a new and easy method for optimizing the model and selecting Deep Neural Networks (DNN) parameters through uniform design ideas and orthogonal design methods. By using this approach, 222 chloroquine (CQ) derivatives with half maximal inhibitory concentration values reported in different kinds of literature were selected to establish DNN models and a total number of 128,000 DNN models were built to determine the optimized parameters for selecting the better models. Comparing with linear and Artificial Neural Network (ANN) models, we found that DNN models showed better performance.Communicated by Ramaswamy H. Sarma

Data-Driven Air-Cooled Condenser Performance Assessment: Model and Input Variable Selection Comparison

This paper presents a data–driven model for the estimation of the performance of an aircooled steam condenser (ACC) with the aim to develop an efficient online monitoring, summarized by the condenser pressure (or vacuum) as Key Performance Indicator. The estimation of the ACC performance model was based on different dataset from three different combined cycle power plants with a gross power of above 380 MWe each, focusing on stationary condition of the steam turbine. The datasets include both boundary (e.g. Ambient Temperature, Wind Speed) and operative parameters (e.g. steam mass flow rate, Steam turbine power, electrical load of the ACC fans) acquired from the power plants and some derived variable as the incondensable fraction, which calculation is here proposed as additional parameter. After a preliminary sensitivity analysis on data correlation, the paper focuses on the evaluation of different ACC Condenser models: Semi-Empirical model is described trough curves typically based on steam mass flow rate (or condenser load) and the ambient temperature as main parameters. Since monitoring based on ACC design curves Semi-Empirical models, provides biased poor results, with an error of about 15%, the curves parameters were estimated basing on training data set. Other two data driven models were presented, basing on a neural network modelling and multi linear regression technique and compared on the base of the reduced number of input at first and then including aldo the other process variables in the prediction of the condenser back pressure. Estimate the parameters of the Semi-Empirical model, results in a better prediction if just steam mass flow rate and ambient temperature are available, with an error of the 7%, thanks to the knowledge contained within the “curves shapes”, with respect to linear regression (8.3%) and Neural Network models (7.6%). Higher accuracy can be then obtained by considering a larger number of operative parameters and exploiting more complex data-driven model. With a higher number of features, the neural network model has proved a higher accuracy than the linear regression model. In fact, the mean percentage error of the NN model (2.6%), in all plant operating conditions, is slightly lower than the error of the linear regression model, but presents and much lower than the mean error of the Semi-Empirical model thanks to the additional data-based knowledge.

Expert system for predicting the stage of diabetic retinopathy based on the analysis of platelet dysfunction

Aim: to develop an expert system for predicting the stage of diabetic retinopathy (DR) on the basis of determination of the platelet dysfunction using the methods of multi-factor neural network and logistic regression modelling at the time of the initial examination of the patient. Material and methods. The model was built (Statistica Neural Network v.4.0B) based on the results of clinical-laboratory study of 99 patients (99 eyes) with type 2 diabetes and DR. The evaluation of aggregation was carried out by spectrophotometric method on a ChronoLog (USA) aggregometer. The study used agonists (Sigma, USA): adenosine diphosphate (ADP, 2.5 μM); adrenaline (2.5 μM); angiotensin-2 (An-2, 1 μM); the platelet activation factor (PAF, 75 μM) and collagen (2.0 mg/ml). Agonists were used in an effective concentration (EC50), which caused the platelet aggregation at 50±5% in healthy individuals (10 donors). Results and discussion. To determine the leading determinants of the development of the DR stages, a mathematical analysis was carried out using the methods of constructing multi-factor neural network and logistic regression models. The prediction of the DR stage in a two-factor linear neural network model was based on platelet aggregation induced by ADP and collagen; the forecast accuracy was 81.8% (95% CI 73.5-88.8%). Four-factor nonlinear neural network MLP-model (An-2, ADP, adrenaline and collagen induced aggregation) was constructed, which allowed to increase the prediction accuracy of the DR stage to 93.9% (95% CI 88.3-97.8%). Conclusions. For the first time, on the basis of the definition of platelet aggregation, an expert system based on the construction of neural network models is proposed. With a high accuracy of prediction of the model, it is possible to predict the development of the DR stage even at the initial treatment of the patient.

Data‐Driven Hybrid Internal Temperature Estimation Approach for Battery Thermal Management

Temperature is a crucial state to guarantee the reliability and safety of a battery during operation. The ability to estimate battery temperature, especially the internal temperature, is of paramount importance to the battery management system for monitoring and thermal control purposes. In this paper, a data‐driven approach combining the RBF neural network (NN) and the extended Kalman filter (EKF) is proposed to estimate the internal temperature for lithium‐ion battery thermal management. To be specific, the suitable input terms and the number of hidden nodes for the RBF NN are first optimized by a two‐stage stepwise identification algorithm (TSIA). Then, the teaching‐learning‐based optimization (TLBO) algorithm is developed to optimize the centres and widths in every neuron of basis function. After optimizing the RBF NN model, a battery lumped thermal model is adopted as the state function with the EKF to filter out the outliers of the RBF model and reduce the estimation error. This data‐driven approach is validated under four different conditions in comparison with the linear NN models. The experimental results demonstrate that the proposed RBF data‐driven approach outperforms the other approaches and can be extended to other types of batteries for thermal monitoring and management.

An Adaptive Neural Network model for thermal characterization of building components

Building materials are usually characterized in stationary or almost-stationary conditions and mono dimensional heat flow regime. The existing standards (such as ISO 9869 or EN ISO 6946, EN 12664, EN 12667, ISO 8302 etc), require experiments carried out in steady-state conditions, with a very fine control of the measuring parameters with the aim to apply a simple and reproducible procedure for the determination of thermal properties. However, the thermodynamic conditions that lead to a steady-state operating mode and mono dimensional flow are very difficult to obtain (in real conditions) or very expensive and time consuming (in climate chambers). In this paper the authors present the development of a method for thermal characterization of building components, inferring the steady-state conditions, when only measures in transient conditions are available. The method, based on an adaptive linear neural network (ALNN) model also could be have the potentialities to determine the thermal diffusivity from a significant transient behavior ad hoc imposed. The study targets multilayered walls homogeneous and the results are compared with the experimental data measured by a climate chamber that operate according to the standard EN 12667

Estimating Inside Air Temperature of a Glasshouse Using Statistical Models

The efficiency of applying linear regression (LR) and artificial neural network (ANN) models to estimate inside air temperature (T) of a glasshouse (37o48΄20΄΄N, 23o57΄48΄΄E), Lavreotiki, was investigated in the present work. The T data from an urban meteorological station (MS) at 37058΄55΄΄N, 23o32΄14΄΄E, Athens, Attica, Greece, about 30 Km away from the glasshouse, were used as predictor variable, taking into account the actual time of measurement (ATM) and two hours earlier (ATM-2), depending on the case. Air temperature data were monitored in each examined area (glasshouse and MS) for four successive months (July-October) and averages on a two-hour basis were used for the aforementioned estimation. Results showed that ANN were better than LR models, considering their better performance as shown in the scatterplots of the distribution of observed versus estimated inside T data of the glasshouse, in terms of both higher coefficient of determination (R2) and lower mean absolute error (MAE). The best ANN model (highest R2 and lowest MAE) was achieved by using as predictor variables the T at ATM and the T at ATM-2 from MS. The findings of our study may be a first step towards the estimation of inside T of a glasshouse in Greece, from outside T data of a remote MS. Thus, the operation of the glasshouse could be improved noticeably.