Types Of Neural Network Models Research Articles

A video monitoring and computational system for estimating migratory juvenile fish abundance in river systems

AbstractDiadromous fishes migrate between marine and fresh waters for reproduction. For anadromous species, which spawn in freshwater, improved access to freshwater spawning and nursery habitats and ability of juveniles to emigrate to the ocean may support population recovery. Despite the potentially enormous influence of early life stage survival on adult population size, managers and scientists have limited capacity to assess numbers of juvenile anadromous fishes leaving freshwater ecosystems. Such data are critical for evaluating reproductive success and habitat suitability and have been identified as a top priority in anadromous fish research and management. We developed a state‐of‐the‐art underwater video and computational system to collect videos to estimate abundances and migration timing for juvenile river herring (Alosa pseudoharengus; Alosa aestivalis). We collected continuous video in the Monument River (Bourne, Massachusetts, USA) from June to November 2017. We trained three types of neural network models to detect and count fish in video frames and evaluated model performance by comparing human counts to model outputs. Our top model assessed presence and absence (F1 = 87%) and counted fish (counting error 9.4%) with an accuracy comparable to human counters (F1 = 88%). Our system's capability to collect accurate counts of emigrating juveniles will provide critical information that could be related to the numbers of spawning adults, system‐specific productivity, and spawning and nursery habitat suitability. Both the video collection system and computational model may be transferrable to other sites and for other species where tracking juvenile emigration may inform management efforts.

Improving Classification Performance in Dendritic Neuron Models through Practical Initialization Strategies.

A dendritic neuron model (DNM) is a deep neural network model with a unique dendritic tree structure and activation function. Effective initialization of its model parameters is crucial for its learning performance. This work proposes a novel initialization method specifically designed to improve the performance of DNM in classifying high-dimensional data, notable for its simplicity, speed, and straightforward implementation. Extensive experiments on benchmark datasets show that the proposed method outperforms traditional and recent initialization methods, particularly in datasets consisting of high-dimensional data. In addition, valuable insights into the behavior of DNM during training and the impact of initialization on its learning performance are provided. This research contributes to the understanding of the initialization problem in deep learning and provides insights into the development of more effective initialization methods for other types of neural network models. The proposed initialization method can serve as a reference for future research on initialization techniques in deep learning.

Evolutionary RNN framework for Precise Lung Nodule Detection from CT Scans

Radiologists find it challenging and time-consuming to recognize and evaluate nodules of lung using CT scans that are malignant. Because of this, early lung growth prediction is necessary for the inquiry technique, which raises the likelihood that the treatment will be successful. Computer-aided diagnostic (CAD) tools have been used to help with this issue. The primary goal of the work is to identify if the nodules are cancerous or not and to deliver more accurate results. The RNN [Recurrent] which is a type of neural network model that includes a feedback loop. In this paper, evolutionary algorithms are examined using the MATLAB Tool, including the Grey Wolf Optimization Algorithm and Recurrent Neural Network (RNN) Techniques. Additionally, statistical characteristics are generated and examined in comparison to other RNNs using Particle Swarm Optimization (PSO) and Genetic Algorithm (GA) combinations. Comparing the suggested approach to other state-of-the-art techniques, it yielded results with extremely high accuracy, sensitivity, specificity, and precision. In the past few years, there has been a substantial increase for evolutionary algorithms in the field of feature selection due to their simplicity and potential for global search capabilities. The suggested solutions have outperformed classical approaches employed across various fields, showing excellent results. Determining whether lung nodules will become malignant or not will be made easier with early identification.

Determination the parameters of model the technological process of diffusion in sugar production on the basis of neuro network identification

The article considers the sugar industry's importance in the country's food supply. The complexity of technological processes characterizes sugar factories. A system analysis of the technical operation of diffusion and an analysis of indicators of the diffusion process in the diffusion equipment were carried out. An analysis of the mathematical support of automated control systems of complex technological processes is given. The assessment of material, quality, and energy indicators was carried out, as well as the formation of the necessary information variables, which makes it possible to determine the structure of the technological process control system. To effectively manage the complex dynamic technological process of diffusion, it is proposed to use neurocontrol. The use of linear regression models was considered to determine the type of mathematical models in the system of automated control of technological processes in the diffusion department of the sugar factory. When building regression models, a significant number of input and output components of the technology diffusion process were considered. Two variants of regression models of the diffusion process were considered. The obtained regression models were analyzed, and the significance of the indicators of the diffusion process was determined. For an adequate description of the complex diffusion process in an automated control system, it is proposed to use models based on neural network identification. Significant indicators of regression models were taken as the main indicators of the process. The construction of mathematical models that make it possible to respond to changes in the technological process adequately boils down to the issue of building neural network models based on a multilayer perceptron and a radial base network. The most effective model structures were determined from various types of neural network models. The control activation time when using the proposed model was 3 minutes. The error of identification of the received models was 5-7%. Keywords: diffusion equipment, mathematical model, neural network, regression equation, control, identification.

Review on the Application of Photovoltaic Forecasting Using Machine Learning for Very Short- to Long-Term Forecasting

Advancements in renewable energy technology have significantly reduced the consumer dependence on conventional energy sources for power generation. Solar energy has proven to be a sustainable source of power generation compared to other renewable energy sources. The performance of a photovoltaic (PV) system is highly dependent on the amount of solar penetration to the solar cell, the type of climatic season, the temperature of the surroundings, and the environmental humidity. Unfortunately, every renewable’s technology has its limitation. Consequently, this prevents the system from operating to a maximum or optimally. Achieving a precise PV system output power is crucial to overcoming solar power output instability and intermittency performance. This paper discusses an intensive review of machine learning, followed by the types of neural network models under supervised machine learning implemented in photovoltaic power forecasting. The literature of past researchers is collected, mainly focusing on the duration of forecasts for very short-, short-, and long-term forecasts in a photovoltaic system. The performance of forecasting is also evaluated according to a different type of input parameter and time-step resolution. Lastly, the crucial aspects of a conventional and hybrid model of machine learning and neural networks are reviewed comprehensively.

Empirical analysis of machine learning techniques for prediction of indian exchange rate

Throughout the past few decades, there has been a dramatic surge in the currency market. The changes play an important role in balancing the market’s characteristics. As a result, accurate change price forecasting is essential to improve the success rate of many businesses and fund managers. Despite the fact that the market is renowned for its erratic behavior and volatility, there are organizations like agencies, banks, and others. In order to estimate the extraneous interchange rate of the dollar against the rupee with a high degree of accuracy, we used three distinct types of methodologies in this article. This research uses three different types of neural network models: ANNs (Artificial Neural Networks), LSTMs (Long Short-Term Memory Networks), and GRUs (Gated Recurring Units). The results depict that GRU’s model is outperforming the other two models.

Neural Network-Based State of Charge Estimation Method for Lithium-ion Batteries Based on Temperature

Lithium-ion batteries are commonly used in electric vehicles, mobile phones, and laptops. These batteries demonstrate several advantages, such as environmental friendliness, high energy density, and long life. However, battery overcharging and overdischarging may occur if the batteries are not monitored continuously. Overcharging causes fire and explosion casualties, and overdischarging causes a reduction in the battery capacity and life. In addition, the internal resistance of such batteries varies depending on their external temperature, electrolyte, cathode material, and other factors; the capacity of the batteries decreases with temperature. In this study, we develop a method for estimating the state of charge (SOC) using a neural network model that is best suited to the external temperature of such batteries based on their characteristics. During our simulation, we acquired data at temperatures of 25°C, 30°C, 35°C, and 40°C. Based on the temperature parameters, the voltage, current, and time parameters were obtained, and six cycles of the parameters based on the temperature were used for the experiment. Experimental data to verify the proposed method were obtained through a discharge experiment conducted using a vehicle driving simulator. The experimental data were provided as inputs to three types of neural network models: multilayer neural network (MNN), long short-term memory (LSTM), and gated recurrent unit (GRU). The neural network models were trained and optimized for the specific temperatures measured during the experiment, and the SOC was estimated by selecting the most suitable model for each temperature. The experimental results revealed that the mean absolute errors of the MNN, LSTM, and GRU using the proposed method were 2.17%, 2.19%, and 2.15%, respectively, which are better than those of the conventional method (4.47%, 4.60%, and 4.40%). Finally, SOC estimation based on GRU using the proposed method was found to be 2.15%, which was the most accurate.

A Hybrid Neural Network-Based Intelligent Forecasting Approach for Capacity of Photovoltaic Electricity Generation

In recent years, photovoltaic power generation technology has become the key planning direction of the country. It is important to effectively predict photovoltaic (PV) electricity generation capacity, so that the administrators can well schedule resource allocation. Currently, most of the photovoltaic electricity generation forecasting models took meteorological data as the input parameters of neural network. However, the input parameters and redundant data cause neural network to converge difficultly. Besides, single types of neural network models cannot well capture the comprehensive characteristics, which may influence forecasting effect in evolving process. As a result, we propose a hybrid neural network-based intelligent forecasting approach for PV electricity generation capacity. First, convolution neural network (CNN) is adopted to extract the connection between features and data from the perspective of convolution operations. And then, the extracted feature vector of time series is sent into the long short-term memory (LSTM) model. Finally, the forecasting values are predicated by training the outlined LSTM network. The experimental results indicate that such a hybrid CNN-LSTM model can significantly improve the precision of PV electricity generation prediction and provide an effective way to forecast generation power of PV system.

The contribution of statistical models in the field of real estate valuation

Abstract Testing a model in property evaluation can be a difficult task due to the large variety of these models. The most popular models used in valuation are regression and neural networks. This paper applied a systematic review study and presents 11 types of regression models and 9 types of neural network models applied in real estate valuation. Our aim is to provide a tool for model selection applied in real estate valuation. The selection criteria were based on their applicability, user preferences and price estimation performance. The findings were slightly different from our expectations. Multi-Layer Perceptron (MLP) and Multiple Linear Regression (GLM) are the most applied and popular models in valuation.

Earthquake Event Recognition on Smartphones Based on Neural Network Models.

Using sensors embedded in smartphones to study earthquake early warning (EEW) technology can effectively reduce the high construction and maintenance costs of traditional EEW systems. However, due to the impact of human activities, it is very difficult to accurately detect seismic events recorded on mobile phones. In this paper, to improve the detection accuracy of earthquakes on mobile phones, we investigated the suitability of different types of neural network models in seismic event detection. Firstly, we collected three-component acceleration records corresponding to human activities in various scenarios such as walking, running, and cycling through our self-developed mobile application. Combined with traditional strong-motion seismic event records fusing typical mobile phone accelerometer self-noise, all records were used for establishing the training and testing dataset. Finally, two types of neural network models, fully connected and convolutional neural networks, were trained, validated, and tested. The results showed that the accuracy rates of the neural network models were all over 98%, and the precision rate for seismic events and the recall rate for non-earthquake events could both reach 99%, indicating that the introduction of neural networks into the earthquake recognition on smartphones can significantly enhance the accuracy of seismic event recognition. Therefore, we can exceedingly reduce the amount of data transmitted to the processing server, further lowering the load on the server processor and effectively increasing the lead time at each target site for an EEW system.

A NEURAL NETWORK MODEL FOR HUMAN FACE BOUNDARY DETECTION

The relevance of the implementation of means of recognition of the emotional state by the image of the face into the personnel management system is well-founded. It is shown that the implementation of such tools leads to the need to adapt the values of architectural parameters of neural network models for detecting the boundaries of target objects on bitmap images to the expected conditions of use. An approach to determining the most effective type of neural network model is proposed, which involves expert evaluation of the effectiveness of acceptable types of models and conducting computer experiments to make a final decision. As a result of the conducted research, it was determined that among the types of neural network models tested in the task of segmentation of raster images, the U-Net model is the most effective for detecting facial borders on small raster images. Using this neural network model provides a mask selection accuracy of 0.88. At the same time, the necessity of improving the mathematical support, which is used to determine the accuracy of face border detection, is determined. It is also advisable to correlate the ways of further research with the correction of typical shortcomings associated with the incorrect marking of the boundaries of various objects that are perceived by the neural network model as a human face.

Unbalanced network attack traffic detection based on feature extraction and GFDA-WGAN

Detecting various types of attack traffic is critical to computer network security . The current detection methods require massive amounts of data to detect attack traffic. However, in most cases, the attack traffic samples are unbalanced. A typical neural network model cannot detect such unbalanced attack traffic. Additionally, malicious network noise traffic has a detrimental effect on the detection stability. Very few effective methods exist to detect unbalanced attack traffic. In this paper, we develop a method to detect unbalanced attack traffic. A dynamic chaotic cross-optimized bidirectional residual-gated recurrent unit (DCCSO-Res-BIGRU) and an adaptive Wasserstein generative adversarial network with generated feature domains (GFDA-WGAN) are proposed. First, feature extraction is achieved using the DCCSO-Res-BIGRU. The GFDA-WGAN can then be used to detect the unbalanced attack traffic. We use a conditional WGAN network to generate the pseudo-sample features of the invisible classes. A GFDA strategy for conditional WGAN optimization is also proposed. Furthermore, we use an invisible sample and supervised learning to detect unbalanced attack traffic. Finally, the performance of the proposed method is validated using four network datasets. According to the experimental results, the proposed method significantly improves sample convergence and generation. It has a higher detection accuracy with respect to detecting unbalanced attack traffic. Furthermore, it provides the most powerful and effective visual classification . When noise is added, it outperforms all other conventionally used methods. Real-time traffic detection is also possible using this method.

An open-source Python library for self-organizing-maps

Organizations have realized the importance of data analysis and its benefits. This in combination with Machine Learning algorithms has allowed to solve problems more easily, making these processes less time-consuming. Neural networks are the Machine Learning technique that is recently obtaining very good best results. This paper describes an open-source Python library called GEMA developed to work with a type of neural network model called Self-Organizing-Maps. GEMA is freely available under GNU General Public License at GitHub (https://github.com/ufvceiec/GEMA). The library has been evaluated in different a particular use case obtaining accurate results.

Extraction of entity relations from Chinese medical literature based on multi-scale CRNN.

BackgroundEntity relation extraction technology can be used to extract entities and relations from medical literature, and automatically establish professional mapping knowledge domains. The classical text classification model, convolutional neural networks for sentence classification (TEXTCNN), has been shown to have good classification performance, but also has a long-distance dependency problem, which is a common problem of convolutional neural networks (CNNs). Recurrent neural networks (RNN) address the long-distance dependency problem but cannot capture text features at a specific scale in the text.MethodsTo solve these problems, this study sought to establish a model with a multi-scale convolutional recurrent neural network for Sentence Classification (TEXTCRNN) to address the deficiencies in the 2 neural network structures. In entity relation extraction, the entity pair is generally composed of a subject and an object, but as the subject in the entity pair of medical literature is always omitted, it is difficult to use this coding method to obtain general entity position information. Thus, we proposed a new coding method to obtain entity position information to re-establish the relationship between subject and object and complete the entity relation extraction.ResultsBy comparing the benchmark neural network model and 2 typical multi-scale TEXTCRNN models, the TEXTCRNN [bidirectional long- and short-term memory (BiLSTM)] and TEXTCRNN [double-layer stacking gated recurrent unit (GRU)], the results showed that the multi-scale CRNN model had the best F1 value performance, and the TEXTCRNN (double-layer stacking GRU) was more capable of entity relation classification when the same entity word did not belong to the same entity relation.ConclusionsThe experimental results of the entity relation extraction from Pharmacopoeia of the People’s Republic of China—Guidelines for Clinical Drug Use—Volume of Chemical Drugs and Biological Products showed that entity relation extraction could effectively proceed using the new labeling method. Additionally, compared to typical neural network models, including the TEXTCNN, GRU, and BiLSTM, the multi-scale convolutional recurrent neural network structure had advantages across several evaluation indicators.

Comparison of Tree-Structured Parzen Estimator Optimization in Three Typical Neural Network Models for Landslide Susceptibility Assessment

Landslides pose a constant threat to the lives and property of mountain people and may also cause geomorphological destruction such as soil and water loss, vegetation destruction, and land cover change. Landslide susceptibility assessment (LSA) is a key component of landslide risk evaluation. There are many related studies, but few analyses and comparisons of models for optimization. This paper aims to introduce the Tree-structured Parzen Estimator (TPE) algorithm for hyperparameter optimization of three typical neural network models for LSA in Shuicheng County, China, as an example, and to compare the differences of predictive ability among the models in order to achieve higher application performance. First, 17 influencing factors of landslide multiple data sources were selected for spatial prediction, hybrid ensemble oversampling and undersampling techniques were used to address the imbalanced sample and small sample size problem, and the samples were randomly divided into a training set and validation set. Second, deep neural network (DNN), recurrent neural network (RNN), and convolutional neural network (CNN) models were adopted to predict the regional landslides susceptibility, and the TPE algorithm was used to optimize the hyperparameters respectively to improve the assessment capacity. Finally, to compare the differences and optimization effects of these models, several objective measures were applied for validation. The results show that the high-susceptibility regions mostly distributed in bands along fault zones, where the lithology is mostly claystone, sandstone, and basalt. The DNN, RNN, and CNN models all perform well in LSA, especially the RNN model. The TPE optimization significantly improves the accuracy of the DNN and CNN (3.92% and 1.52%, respectively), but does not improve the performance of the RNN. In summary, our proposed RNN model and TPE-optimized DNN and CNN model have robust predictive capability for landslide susceptibility in the study area and can also be applied to other areas containing similar geological conditions.

Motion control of deep sea vehicle ‘OTOHIME’: modeling with neural network

Deep sea survey vehicle ‘OTOHIME’ is one of the underwater vehicle owned in JAMSTEC and its automatic navigation system is developing now. To construct the control system, vehicle motion should be modeled at first. In this study, we treated a dynamic equation model and two types of neural network models and compared their performance. The latter showed rather better results than the former. Then we decide to use the neural network model to design the control system as the next step.

Application of neural network autoencoder algorithm in the cancer informatics research

Cancers have been widely recognized as highly heterogeneous diseases, and early diagnosis and prognosis of cancer types have become the focus of cancer research. In the era of big data, efficient mining of massive biomedical data has become a grand challenge for bioinformatics research. As a typical neural network model, the autoencoder is able to efficiently learn the features of input data by unsupervised training method and further help integrate and mine the biological data. In this article, the primary structure and workflow of the autoencoder model are introduced, followed by summarizing the advances of the autoencoder model in cancer informatics using various types of biomedical data. Finally, the challenges and perspectives of the autoencoder model are discussed.

Logic could be learned from images

Logic reasoning is a significant ability of human intelligence and also an important task in artificial intelligence. The existing logic reasoning methods, quite often, need to design some reasoning patterns beforehand. This has led to an interesting question: can logic reasoning patterns be directly learned from given data? The problem is termed as a data concept logic. In this study, a learning logic task from images, called a LiLi task, first is proposed. This task is to learn and reason the logic relation from images, without presetting any reasoning patterns. As a preliminary exploration, we design six LiLi data sets (Bitwise And, Bitwise Or, Bitwise Xor, Addition, Subtraction and Multiplication), in which each image is embedded with a n-digit number. It is worth noting that a learning model beforehand does not know the meaning of the n-digit numbers embedded in images and the relation between the input images and the output image. In order to tackle the task, in this work we use many typical neural network models and produce fruitful results. However, these models have the poor performances on the difficult logic task. For furthermore addressing this task, a novel network framework called a divide and conquer model by adding some label information is designed, achieving a high testing accuracy.

Graph embedding‐based intelligent industrial decision for complex sewage treatment processes

Intelligent algorithms-driven industrial decision systems have been a general demand for modeling complex sewage treatment processes (STP). Existing researches modeled complex STP with the use of various neural network models, yet neglecting the fact that latent and occasional relations exist inside complex STP. To deal with the challenge, this paper proposes graph embedding-based intelligent industrial decision for complex STP (GE-STP). The graph embedding (GE) scheme is employed to enhance feature extraction and neural computing structure is utilized to simulate uncertain biochemical transformation inside STP. The introduction of GE can not only improves the fineness of feature spaces, but also improves the representative ability of models towards complex industrial processes. On this basis, the GE-STP is evaluated on a real-world data set collected from a realistic sewage treatment plant equipped with a set of Internet of Things devices. And some typical neural network models that have been utilized for modeling complex STP, are selected as baseline methods. Three groups of experiments show that efficiency of the GE-STP exceeds baselines about 6%–12%, and that the GE-STP is not susceptible to parameter changing.

Analyzing the relationship between the cytokine profile of invasive breast carcinoma, its histopathological characteristics and metastasis to regional lymph nodes

This study was aimed at analyzing the relations of metastasis to regional lymph nodes (RLNs) with histopathological indicators of invasive breast carcinoma of no special type (IC-NST) and its cytokine profile. Enzyme-linked immunosorbent assays were performed to determine concentrations of IL-2, IL-6, IL-8, IL-10, IL-17, IL-18, IL-1β, IL-1Ra, TNF-α, IFN-γ, G-CSF, GM-CSF, VEGF-A, and MCP-1 in the culture supernatant of IC-NST samples from 48 female patients. Histopathological indicators (degree of tumor cell differentiation, mitoses, and others) and ER, PR, Her2/neu, Ki-67, and CD34 expression levels were determined. By means of three types of neural network models, it was shown that for different parameters of the output layer, different groups of parameters are involved that have predictive value regarding metastasis to RLNs. As a result of multi-dimensional cluster analysis, three clusters were formed with different cytokines profiles of IC-NST. Different correlations between indicators of cytokine production by IC-NST and its histopathological parameters were revealed in groups with different cytokine profiles. It was shown that at simultaneous evaluation of the production of even two cytokines, the importance of which relationship with metastasis was revealed by neural network modeling, can increase the probability of determining the presence of metastasis in the RLNs.