Learning Process Of Network Research Articles

Research on Economic Mathematical Analysis and Construction Model of Prefabricated Building Structure Based on Improved Neural Network Algorithm

In this article, a mathematical analysis model of economics of prefabricated building structure based on improved neural network algorithm is proposed in order to solve the low analysis accuracy in traditional methods. Firstly, by means of analyzing the costs of materials, labor, and equipment, the economic characteristics of the cost of fabricated building structures are determined. Secondly, the single neuron is analyzed and the weight coefficient is adjusted in accordance with the multilayer neural network structure, so as to minimize the construction error of the economic analysis model of the assembled building structure. Meanwhile, the weight vector is obtained, error-weighted square sum is calculated through choosing an adaptive filter and obtained, and the weight vector is updated by the least squares algorithm. Thirdly, the neural network algorithm training and learning process is designed and improved, the dependent variable is selected, the number of input points is determined, and then, the training and learning process of the improved neural network algorithm is completed. Finally, a fitness function is set to measure the authenticity of dataset, which is further defined as a combination of different weights to construct an economic mathematical analysis model. The experimental results indicate that the analysis results of this method can reach an accuracy up to 96%, so it has a broader application prospect in low-rise buildings.

A deep neural network-based transfer learning to enhance the performance and learning speed of BCI systems

ABSTRACT Brain–computer interfaces (BCIs) suffer from a lack of classification accuracy when the number of electroencephalography (EEG) trials is low. This is therefore during the learning of a BCI for a subject, there is no clear protocol to use the captured knowledge of other trained BCIs. To overcome this, we have proposed a new parallel deep neural structure containing long short-term memory and multi-layer perception. Furthermore, a subject-to-subject transfer learning is exploited to improve both performance and learning speed. First, the proposed combinatorial classifier is trained over different subjects, then for each new case, a copy of this learned network is adopted to be fine-tuned by the EEG features of the new subject. The proposed method is assessed on an EEG dataset of motor imagery movements and compared to the support vector machines. The proposed method provides superior classification results and significantly speed up the learning process of the deep network.

Reform and Innovation of Higher Fine Arts Distance Education under the Background of Big Data

The development direction of online teaching is personalized and customized teaching based on user needs. The application of big data(BD) technology to the art remote network teaching platform can effectively solve the problems of the huge and chaotic amount of teaching resources, the difficulty of user selection, the weak management of the network learning process, the inefficient use of teaching resources, and the lack of user role sense. The purpose of this article is to study the reform and innovation of higher art distance online education under the background of BD. Methods such as literature analysis and BD technology closely link BD with education reform and innovation. This article uses BD technology methods to discover and grasp the laws of higher art distance online education in colleges and universities(CAU), build a higher art distance online education innovation system in CAU, and then promote the innovation of college art distance online education. This paper proposes a distance education innovation system for CAU from the perspective of BD, and further adopts the method of structured system analysis to carry out an in-depth study of the innovation system of CAU distance education of fine arts from the perspective of BD. This article points out that the college network ideological and political education innovation system under the BD perspective is composed of internal and external elements such as ideas, mechanisms, paths and carriers. Based on the analysis of the realistic basis of higher art education innovation in CAU from the perspective of BD, this paper puts forward the realistic possibility of the innovation of higher art distance network education under the vision of BD. Experimental research shows that there are more than 700 million netizens in my country, among which netizens are mainly 20-29 years old, accounting for 30.30% of the total number of netizens. In addition, students are an important part of netizens, reaching 25.4%.

ПРИМЕНЕНИЕ НЕЙРОННОЙ СЕТИ ДЛЯ УПРАВЛЕНИЯ ТРАЛОВЫМ ПРОМЫСЛОМ

The article highlights the problems of managing trawl fishing, increasing the operation efficiency and reducing the influence of the human factor. There has been considered using 
 a neural network in combination with a mathematical model and BigData technologies for predictive modeling in the process of automatic control of trawl fishing in order to increase its efficiency (to reduce energy and labor costs, to increase fishing productivity). Advantages of the proposed approach are the possibility to account for the above factors neglected in the mathematical model due to the complexity of their mathematical description (e.g. time of the day, time of the year, weather conditions, density, congestion, availability and distribution of food resources, other aquatic species), as well as the possibility of collecting and accumulating data obtained in many fishing operations and from different fishers for their subsequent consideration in the fishery management in the future. There has been proposed a solution based on the corrected output data obtained from 
 a mathematical model and on the output data of a neural network. The weight coefficients of the neural network are extracted from a centralized database using BigData technologies before fishing with a selection criterion for the area and object of fishing. In the course of fishing the input data of the neural network and the final (adjusted) output data of the control are recorded. At the end of fishing the saved data is used in the process of training the neural network, followed by updating the weight coefficients in a centralized database. The neural network learning process occurs between the fisheries on a centralized shared neural network. The adjusted weight coefficients are updated in the general database of fishers.

Deep mixed precision for hyperspectral image classification

Hyperspectral images (HSIs) record scenes at different wavelength channels, providing detailed spatial and spectral information. How to storage and process this high-dimensional data plays a vital role in many practical applications, where classification technologies have emerged as excellent processing tools. However, their high computational complexity and energy requirements bring some challenges. Adopting low-power consumption architectures and deep learning (DL) approaches has to provide acceptable computing capabilities without reducing accuracy demand. However, most DL architectures employ single-precision (FP32) to train models, and some big DL architectures will have a limitation on memory and computation resources. This can negatively affect the network learning process. This letter leads these challenges by using mixed precision into DL architectures for HSI classification to speed up the training process and reduce the memory consumption/access. Proposed models are evaluated on four widely used data sets. Also, low and high-power consumption devices are compared, considering NVIDIA Jetson Xavier and Titan RTX GPUs, to evaluate the proposal viability in on-board processing devices. Obtained results demonstrate the efficiency and effectiveness of these models within HSI classification task for both devices. Source codes: https://github.com/mhaut/CNN-MP-HSI .

ALGORITHMIC STABILITY OF DEEP LEARNING NEURAL NETWORKS IN RECOGNIZING THE MICROSTRUCTURE OF MATERIALS

The division of data for training a neural network into training and test data in various proportions to each other is investigated. The question is raised about how the quality of data distribution and their correct annotation can affect the final result of constructing a neural network model. The paper investigates the algorithmic stability of training a deep neural network in problems of recognition of the microstructure of materials. The study of the stability of the learning process makes it possible to estimate the performance of a neural network model on incomplete data distorted by up to 10%. Purpose. Research of the stability of the learning process of a neural network in the classification of microstructures of functional materials. Materials and methods. Artificial neural network is the main instrument on the basis of which produced the study. Different subtypes of deep convolutional networks are used such as VGG and ResNet. Neural networks are trained using an improved backpropagation method. The studied model is the frozen state of the neural network after a certain number of learning epochs. The amount of data excluded from the study was randomly distributed for each class in five different distributions. Results. Investigated neural network learning process. Results of experiments conducted computing training with gradual decrease in the number of input data. Distortions of calculation results when changing data with a step of 2 percent are investigated. The percentage of deviation was revealed, equal to 10, at which the trained neural network model loses its stability. Conclusion. The results obtained mean that with an established quantitative or qualitative deviation in the training or test set, the results obtained by training the network can hardly be trusted. Although the results of this study are applicable to a particular case, i.e., microstructure recognition problems using ResNet-152, the authors propose a simpler technique for studying the stability of deep learning neural networks based on the analysis of a test, not a training set.

Development and Innovation of Music Course Teaching Mode Based on Big Data

The development direction of online teaching is personalized and customized teaching based on user needs. Applying big data technology to the online network teaching platform can effectively solve the problems of the huge and chaotic number of teaching resources, the difficulty of user selection, the weak management of the network learning process, the inefficient use of teaching resources, and the lack of user role sense. The purpose of this article is to explore the development and innovation of music curriculum teaching models based on big data. This paper studies the modern network music teaching platform based on big data, realizes the user demand discovery of music teaching platform and optimizes the push of music teaching resources, and realizes the full use of music teaching resources. Starting from five dimensions of user attributes, learning roles, learning process, learning objects and learning tasks, this paper analyzes the user portraits of modern online music teaching platforms based on big data and expresses them based on ontology technology. This article is based on current scholars’ theoretical foundation and practical application research on education big data and smart classroom model, through the previous literature review and analysis, research on the popularization of smart classroom model and actual observation of existing smart classroom models. For the verification of the practical effects of music teaching cases using this model, the author used questionnaire and interview methods to investigate teachers and students, and conducted investigations on teachers from basic information, classroom application and effects, and classroom satisfaction.. Research shows that 29% of teachers are very satisfied with the application of this model, and 16% of teachers are generally satisfied. It can be known that most teachers are quite satisfied with the “smart classroom” teaching model under the big data environment.

THE SPEED OF LEARNING CONVOLUTIONAL NEURAL NETWORKS ON THE GPU AND CPU TO DETECT SYNTHESIZED SPEECH USING SPECTROGRAMS

In this work has been investigated the possibility of using convolutional neural networks to detect synthesized speech. The Python programming language, the TensorFlow library in combination with the high-level Keras API and the ASVspoof 2019 audio database in flac format were used to create the software application. The voice signal of synthesized and natural speech was converted into mel-frequency spectrograms. The structure of a convolutional neural network with high indicators of recognition accuracy is proposed. The learning speed of neural networks on GPU and CPU is compared using the CUDA library. The influence of the batch size parameter on the accuracy of the neural network was investigated. The TensorBoard tool was used to monitor and profile the learning process of neural networks. Keywords: audio deepfake, mel-frequency sound spectrograms, convolutional neural networks, learning speed of neural networks.

CHAOTIC STATES OF A MULTILAYER NEURAL NETWORK

The study of the influence of learning speed η on the learning process of a multilayer neural network was studied. The program for the multilayer neural network was written in Python. We determined the speed of learning at which the best learning is observed. To analyze the impact of learning speed on the learning process, a logistic function was used, which describes the doubling of frequency. It is shown that the learning error function is characterized by bifurcation processes that lead to a chaotic state at η> 0.8. The optimal value of the learning speed is determined, which determines the appearance of the process of doubling the number of local minima. Increasing the number of hidden layers and the number of neurons in each layer does not lead to a radical change in the diagram of the logistics function, and hence the optimal value of the learning speed. It is shown that the increase in the number of hidden layers, as well as the number of neurons in them, with increasing speed of learning is accompanied by the appearance of a chaotic state. The chaotic state is characterized by the lack of a learning process. Using the Fourier transform of the error function, the bifurcation process depending on the learning speed was investigated. On the basis of these researches the program for definition of size of optimum speed of training of a neural network is developed. The text of the article adds program codes for a multilayer neural network with the ability to build a branching diagram and a program to determine the presence or absence of the learning process, taking into account the value of optimal learning speed and error. Key words : multilayer neural network, optimal learning speed, bifurcations, chaotic states.

Gradient-Based Algorithm for Tracking the Activity of Neural Network Weights Changing

The research conducted in this paper is in the field of machine learning. The main object of the research is the learning process of an artificial neural network in order to increase its efficiency. The algorithm based on the analysis of retrospective learning data. The dynamics of changes in the values of the weights of an artificial neural network during training is an important indicator of training efficiency. The algorithm proposed in this work is based on changing the weight gradients values. Changing of the gradients weights makes it possible to understand how actively the network weights change during training. This knowledge helps to diagnose the training process and makes an adjusting the training parameters. The results of the algorithm can be used to train an artificial neural network. The network will help to determine the set of measures (actions) needed to optimize the learning process by the algorithm results.

EEG channel selection strategy for deep learning in emotion recognition

Emotions play an important role in everyday life and contribute to physical and mental health. Emotional states can be detected by electroencephalography (EEG signals). Efficient information retrieval from the EEG sensors is a complex and challenging task. Therefore, deep learning methods for EEG signal analysis attract more and more attention. Many researchers emphasize automated feature learning as the motivation for using deep learning approaches. We propose using a limited number of EEG channels as an input for a deep neural network. In the research, we confirm that our electrode selection enhances the learning process of the convolutional neural network. The classification accuracy for the reduced subset of electrodes yields results comparable to the full dataset in a significantly shorter time—the average learning time 58% faster using our proposed strategy.

Activation Maximization with a Prior in Speech Data

Recently, more and more studies regarding neural networks have been done. However, the learning process of neural networks is often elusive to human beings, which leads to the advent of feature visualization techniques. Activation Maximization (AM) is one of the feature visualization techniques, originally designed for image data. In AM, the input data is optimized to find the data that activates the selected neuron. In this paper, the emotion recognizer’s output is selected as the neuron, and the latent code of a generator (of Generative Adversarial Networks) is optimized instead of the input raw data. The aim of this study is to apply AM to different representations of audio data (waveform-based data and mel-spectrogram-based data) and different model structures (CNN, WaveNet, LSTM), and to find out the most suitable condition for AM in audio domain data. Additionally, we have also tried to visualize the essential features of being a certain class for emotion classification in speech data, using 2 datasets: the Toronto emotional speech set (TESS) and the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). The mel-spectrogram-based models were found to be superior to the others, showing the distinctive features of selected emotions. More specifically, the CNN-mel-spectrogram-based model was the best in both qualitative and quantitative (FID score) results. Moreover, as demonstrated in this study, AM can also be employed as an output enhancer for generative models.

Impact of the compilation method on determining the accuracy of the error loss in neural network learning

In the field of NLP (Natural Language Processing) research, the use of a neural network has become important. The neural network is widely used in the semantic analysis of texts in different languages. In connection with the actualization of the processing of big data in the Kazakh language, a neural network was built for deep learning. In this study, the object is the learning process of a deep neural network, which evaluates the algorithm for constructing an LDA model. One of the most problematic places is determining the correct arguments, which, when compiling the model, will give an estimate of the algorithm’s performance. During the research, the compile () method from the Keras modular library was used, the main arguments of which are the loss function, optimizers, and metrics. The neural network is implemented in the Python programming language. The main arguments of the neural network deep learning compiler for evaluating the LDA model is the selection of arguments to obtain the correct evaluation of the algorithm of the constructed model using deep learning of the neural network. A corpus of text in the Kazakh language with no more than 8000 words is presented as learning data. Using the above methods, an experiment was carried out on the selection of arguments for the model compiler when learning a text corpus in the Kazakh language. As a result, the optimizer – SGD, the loss function – binary_crossentropy, and the estimation metric – ‘cosine_proximity’ were chosen as the optimal arguments, which, as a result of learning, showed a tendency to 0 loss (errors)=0.1984, and cosine_proximity (learning accuracy)=0.2239, which is considered acceptable learning measures. The results indicate the correct choice of compilation arguments. These arguments can be applied when conducting deep learning of a neural network, where the sample data is a pair of «topic and keywords».

Modeling of Solar Radiation Using the Wavelet Neural Network Model in Mataram City Lombok Island

Sunlight is a source of energy for living things in general. In reality, the intensity of solar radiation is an environmental parameter that has positive and negative impacts on human life in particular. Furthermore, the knowledge on the characteristics of solar radiation, including its distribution pattern, is considered by many circles, both policy-makers and researchers in the environmental field. This study aims to create a solar radiation model in response to meteorological factors such as wind speed, air pressure and temperature, humidity, and rainfall using the Wavelet Neural Network (WNN). The modeling of solar radiation in this study is carried out by simultaneously utilizing its advantages as a hybrid model that combines the neural network model and the wavelet method. These advantages through the learning process (supervised learning) are multiplied through the use of the wavelet transform as a pre-processing data method and two type wavelets function, B-spline and Morlet wavelets, as an activation function in the neural network learning process. The WNN model was analyzed in two cases of meteorological variables, which are with and without rainfall. The results based on the root of the mean square error (RMSE) indicator show that the WNN model in these two cases is quite accurate. Meanwhile, the other indicator shows that the interval of the data distribution from the model is within the actual range. This implies that the predicted intensity of the solar radiation will be in a safe position in its adverse effect when the model is used as a reference.

Search for the Global Extremum Using the Correlation Indicator for Neural Networks Supervised Learning

The article discusses the search for a global extremum in the training of artificial neural networks using a correlation indicator. A method based on a mathematical model of an artificial neural network presented as an information transmission system is proposed. Drawing attention to the fact that in information transmission systems widely used methods that allow effective analysis and recovery of useful signal against the background of various interferences: Gaussian, concentrated, pulsed, etc., it is possible to make an assumption about the effectiveness of the mathematical model of artificial neural network, presented as a system of information transmission. The article analyzes the convergence of training and experimentally obtained sequences based on a correlation indicator for fully-connected neural network. The possibility of estimating the convergence of the training and experimentally obtained sequences based on the joint correlation function as a measure of their energy similarity (difference) is confirmed. To evaluate the proposed method, a comparative analysis is made with the currently used indicators. The potential sources of errors in the least-squares method and the possibilities of the proposed indicator to overcome them are investigated. Simulation of the learning process of an artificial neural network has shown that the use of the joint correlation function together with the Adadelta optimizer allows us to get again in learning speed 2-3 times compared to CrossEntropyLoss.

Neural Networks in Forecasting Disease Dynamics

Introduction.In recent years, computer technologies are more and more widely used in medicine. Thus, medical neuro‑ informatics solves diagnostic and forecasting tasks using neural networks.Materials and methods. Using the example of erysipelas, the possibility of forecasting the course and outcome of the dis‑ ease is demonstrated. A retrospective study of the medical histories of patients treated for erysipelas at the Ufa Clinical Hospital No.8 during 2006–2015 was carried out. Modern statistical packages and the MATLAB environment were used.Results and discussion.The conducted comparative analysis showed a 3-layer recurring network of direct distribution to be the most suitable neural network architecture. The optimal structure of the neural network was found to be: 27–6–1 (i.e. 27 neurons are used at the entrance; 6 — in a hidden layer; 1 — in the output layer). The best convergence of the network learning process is provided by the quasi-Newton and conjugated gradient algorithms. In order to assess the effectiveness of the proposed neural network in predicting the dynamics of inflammation, a comparative analysis was carried out using a number of conventional methods, such as exponential smoothing, moving average, least squares and group data handling.Conclusion.The proposed neural network based on approximation and extrapolation of variations in the patient’s medi‑ cal history over fixed time window segments (within the ‘sliding time window’) can be successfully used for forecasting the development and outcome of erysipelas.

Optimization of Online Teaching Quality Evaluation Model Based on Hierarchical PSO-BP Neural Network

In the evaluation of teaching quality, aiming at the shortcomings of slow convergence of BP neural network and easy to fall into local optimum, an online teaching quality evaluation model based on analytic hierarchy process (AHP) and particle swarm optimization BP neural network (PSO-BP) is proposed. Firstly, an online teaching quality evaluation system was established by using the analytic hierarchy process to determine the weight of each subsystem and each index in the online teaching quality evaluation system and then combined with actual experience, the risk value of each index was constructed according to safety regulations. The regression model is established through BP neural network, and the weight and threshold of the model are optimized by the particle swarm algorithm. Based on the online teaching quality evaluation model of BP neural network, the parameters of the model are constantly adjusted, the appropriate function is selected, and the particle swarm algorithm which is used in the training and learning process of the neural network is optimized. The scientificity of the questionnaire was verified by reliability and validity test. According to the scoring results and combined with the weight coefficient of each indicator in the online course quality evaluation index system, the key factors affecting the quality of online courses were obtained. Based on the survey data, descriptive statistics, analysis of variance, and Pearson’s correlation coefficient method are used to verify the research hypothesis and obtain valuable empirical results. By comparing the model with the standard BP model, the results show that the accuracy of the PSO-BP model is higher than that of the standard BP model and PSO-BP effectively overcomes the shortcomings of the BP neural network.

Learned Model Composition With Critical Sample Look-Ahead for Semi-Supervised Learning on Small Sets of Labeled Samples

In this work, we propose to push the performance limit of semi-supervised learning on very small sets of labeled samples by developing a new method called learned model composition with critical sample look-ahead (LMCS). Training efficient deep neural networks on much smaller sets of labeled samples is a challenging problem. With a small labeled set, the initial network suffers from low accuracy. Based on this error-prone network, the subsequent semi-supervised learning process will be fragile and unstable. To address this issue, we propose to introduce a look-ahead master model to identify the correct direction of model evolution to effectively guide the semi-supervised learning process of the student model. Specifically, our proposed LMCS method explores two major ideas. First, it introduces a new learned model composition structure so that we can compose a more efficient master network from student models of past iterations through a network learning process. Second, we develop a new method, called confined maximum entropy search, to discover new critical samples near the model decision boundary and provide the master model with look-ahead access to these samples to enhance its guidance capability. Our extensive experimental results demonstrate that the proposed LMCS method outperforms the state-of-the-art semi-supervised learning methods, especially on small sets of labeled samples. For example, on the CIFAR-10 dataset, with a very small set of 80 labeled samples, our method outperforms Google's MixMatch method, reducing the error rate by more than 10%.

A proactive metaheuristic model for optimizing weights of artificial neural network

<span>This paper proposes the Particle Swarm Optimization model for enhancing the performance of an Artificial Neural Network. The learning process of Artificial Neural Network requires a long time to satisfy requirements because of processing complexity of the backpropagation algorithm that has been used in training Artificial Neural Network. It is a nonlinear complex model that can be used to configure and train an artificial neuron system. Both Artificial Neural Network and Particle Swarm Optimization model have been managed to solve and optimize several nonlinear models. Heuristic Optimization Weight of Artificial Neural Network (HNN) is a proactive metaheuristic model proposed to optimize the performance of Artificial Neural Network. The proposed system applies Particle Swarm Optimization to find the optimum weights of the Artificial Neural Network instead of using the Backpropagation algorithm. Experimentally, the proposed system has required less processing time (average of 76.91 Sec.) than Backpropagation (average of 93.32 Sec). Furthermore, It has provided better classification accuracy (start from 80% to 97.20%) comparing with Backpropagation (start from 75.32% to 94.32%).</span>

Blender as a tool for generating synthetic data

Acquiring data for neural network training is an expensive and labour-intensive task, especially when such data isdifficult to access. This article proposes the use of 3D Blender graphics software as a tool to automatically generatesynthetic image data on the example of price labels. Using the fastai library, price label classifiers were trained ona set of synthetic data, which were compared with classifiers trained on a real data set. The comparison of the resultsshowed that it is possible to use Blender to generate synthetic data. This allows for a significant acceleration of thedata acquisition process and consequently, the learning process of neural networks.