Supervised Training Algorithm Research Articles

Methodology based on spiking neural networks for univariate time-series forecasting

Spiking Neural Networks (SNN) are recognised as well-suited for processing spatiotemporal information with ultra-low energy consumption. However, proposals based on SNN for classification tasks are more common than for forecasting problems. In this sense, this paper presents a new general training methodology for univariate time-series forecasting based on SNN. The methodology is focused on one-step ahead forecasting problems and combines a PulseWidth Modulation based encoding–decoding algorithm with a Surrogate Gradient method as supervised training algorithm. In order to validate the generality of the presented methodology sine-wave, 3 UCI and 1 available real-world datasets are used. The results show very satisfactory forecasting results (MAE∈[0.0094,0.2891]) regardless of the characteristics of the dataset or the application field. In addition, weights can be initialised just once to achieve robust results, boosting the advantages of computational and energy cost of SNN.

Constructing a Guarantee System for Physical Education Teaching Reform and Innovation in the Digital Era

Abstract In this paper, the world coordinate system transformation in virtual reality technology is utilized to obtain the gesture information for estimating human movement in the process of physical education, and the dynamic sampling light projection algorithm is used to match the gestures of human sports movements. The selection algorithm based on the Q statistic defines the virtual reality technology difference as the error trend of the existence of the difference obtained by each classifier for the new data sports students, and at the same time, combines with the VRT method to construct the VR sports teaching model. In the digital teaching of sports, digital sports teaching materials can be produced by motion editing and video synthesis, and statistical analysis methods can be used to study and analyze the teaching of sports in the digital era. The results show that the highest value recognized by the supervised training algorithm based on JPCD+SRD is 0.92, while the highest value recognized by this paper’s algorithm is 1, i.e., it shows that the effect obtained by this paper’s algorithm is higher than that of supervised training methods, which makes the effect of action recognition in virtual reality technology further improved. The average scores of sit-ups of girls in the control class and the experimental class were improved by 4.03 points and 11.4 points, respectively, compared with those before the experiment, i.e., the use of Internet tools to assist teaching has a significant promotion effect on the sit-up scores of girls. This study facilitates teachers to innovate teaching methods and promotes the process of digital teaching reform in physical education.

Dynamic Q-Learning-Based Optimized Load Balancing Technique in Cloud

Cloud computing provides on-demand access to a shared puddle of computing resources, containing applications, storage, services, and servers above the internet. This allows organizations to scale their IT infrastructure up or down as needed, reduce costs, and improve efficiency and flexibility. Improving professional guidelines for social media interactions is crucial to address the wide range of complex issues that arise in today’s digital age. It is imperative to enhance and update professional guidelines regarding social media interactions in order to effectively tackle the multitude of intricate issues that emerge. In this paper, we propose a reinforcement learning (RL) method for handling dynamic resource allocation (DRA) and load balancing (LB) activity in a cloud environment and achieve good scalability and a significant improvement in performance. To address this matter, we propose a dynamic load balancing technique based on Q-learning, a reinforcement learning algorithm. Our technique leverages Q-learning to acquire an optimal policy for resource allocation in real-time based on existing workload, resource accessibility, and user preferences. We introduce a reward function that takes into account performance metrics such as response time and resource consumption, as well as cost considerations. We evaluate our technique through simulations and show that it outperforms traditional load balancing techniques in expressions of response time and resource utilization while also reducing overall costs. The proposed model has been compared with previous work, and the consequences show the significance of the proposed work. Our model secures a 20% improvement in scalability services. The DCL algorithm offers significant advantages over genetic and min-max algorithms in terms of training time and effectiveness. Through simulations and analysis on various datasets from the machine learning dataset repository, it has been observed that the proposed DCL algorithm outperforms both genetic and min-max algorithms. The training time can be reduced by 10% to 45%, while effectiveness is enhanced by 30% to 55%. These improvements make the DCL algorithm a promising option for enhancing training time and effectiveness in machine learning applications. Further research can be conducted to investigate the potential of combining the DCL algorithm with a supervised training algorithm, which could potentially further improve its performance and apply in real-world application.

Neuromorphic Speech Recognition With Photonic Convolutional Spiking Neural Networks

Spiking neural network (SNN) have attracted lots of attention due to its event-driven nature and powerful computation capability. However, it is still limited to simple task due to the training difficulty. In this work, we propose a hybrid architecture of photonic convolutional spiking neural network (PCSNN) to realize the speech recognition task. In the PCSNN, the feature extraction is realized by a convolution SNN with unsupervised learning algorithm, the classification is realized by a photonic SNN with modified time-based supervised training algorithm. The TIDIGITS dataset is used to test the speech recognition performance of the proposed PCSNN, and the highest testing accuracy is 93.75%. The proposed PCSNN provides a solution for architecture and algorithm co-design for the speech recognition task, which is helpful for extending the applications of photonic SNN.

Background-Mixed Augmentation for Weakly Supervised Change Detection

Change detection (CD) is to decouple object changes (i.e., object missing or appearing) from background changes (i.e., environment variations) like light and season variations in two images captured in the same scene over a long time span, presenting critical applications in disaster management, urban development, etc. In particular, the endless patterns of background changes require detectors to have a high generalization against unseen environment variations, making this task significantly challenging. Recent deep learning-based methods develop novel network architectures or optimization strategies with paired-training examples, which do not handle the generalization issue explicitly and require huge manual pixel-level annotation efforts. In this work, for the first attempt in the CD community, we study the generalization issue of CD from the perspective of data augmentation and develop a novel weakly supervised training algorithm that only needs image-level labels. Different from general augmentation techniques for classification, we propose the background-mixed augmentation that is specifically designed for change detection by augmenting examples under the guidance of a set of background changing images and letting deep CD models see diverse environment variations. Moreover, we propose the augmented & real data consistency loss that encourages the generalization increase significantly. Our method as a general framework can enhance a wide range of existing deep learning-based detectors. We conduct extensive experiments in two public datasets and enhance four state-of-the-art methods, demonstrating the advantages of our method. We release the code at https://github.com/tsingqguo/bgmix.

Analysis and Research on Technical and Tactical Action Recognition in Football Based on 3D Neural Network

Abstract The current action recognition analysis method is easily affected by factors such as background, illumination, and target angle, which not only has low accuracy, but also relies on prior knowledge. Research on the identification and analysis of technical and tactical movements in football. According to the characteristics of football video, a multi-resolution three-dimensional convolutional neural network is constructed by combining the convolutional neural network and the three-dimensional neural network. The supervised training algorithm is used to update the network weights and thresholds, and the video images are input into the input layer. After the convolutional layer, sub-sampling layer and fully connected layer and other network layers to obtain action recognition results. The principal component analysis method is used to reduce the dimension to process the action data set, and the Fourier transform method is used to filter out the principal component noise. The experimental results show that the method can effectively identify the technical and tactical movements of athletes from complex football game videos, and analyze the applied technical and tactical strategies. The average value of accuracy, recall and precision of technical and tactical analysis is as high as 0.96, 0.97, and 0.95, and the recognition and analysis effect has significant advantages.

Developing a Clinical Decision Support System for Prediction Postoperative Coronary Artery Bypass Grafting Infection in Diabetic Patients.

Postoperative infection in Coronary Artery Bypass Graft (CABG) is one of the most common complications for diabetic patients, due to an increase in the hospitalization and cost. To address these issues, it is necessary to apply some solutions. The study aimed to the development of a Clinical Decision Support System (CDSS) for predicting the CABG postoperative infection in diabetic patients. This developmental study is conducted on a private hospital in Tehran in 2016. From 1061 CABG surgery medical records, we selected 210 cases randomly. After data gathering, we used statistical tests for selecting related features. Then an Artificial Neural Network (ANN), which was a one-layer perceptron network model and a supervised training algorithm with gradient descent, was constructed using MATLAB software. The software was then developed and tested using the receiver operating characteristic (ROC) diagram and the confusion matrix. Based on the correlation analysis, from 28 variables in the data, 20 variables had a significant relationship with infection after CABG (P<0.05). The results of the confusion matrix showed that the sensitivity of the system was 69%, and the specificity and the accuracy were 97% and 84%, respectively. The Receiver Operating Characteristic (ROC) diagram shows the appropriate performance of the CDSS. The use of CDSS can play an important role in predicting infection after CABG in patients with diabetes. The designed software can be used as a supporting tool for physicians to predict infections caused by CABG in diabetic patients as a susceptible group. However, other factors affecting infection must also be considered for accurate prediction.

Lumbar Spine Computed Tomography to Magnetic Resonance Imaging Synthesis Using Generative Adversarial Network: Visual Turing Test.

(1) Introduction: Computed tomography (CT) and magnetic resonance imaging (MRI) play an important role in the diagnosis and evaluation of spinal diseases, especially degenerative spinal diseases. MRI is mainly used to diagnose most spinal diseases because it shows a higher resolution than CT to distinguish lesions of the spinal canals and intervertebral discs. When it is inevitable for CT to be selected instead of MR in evaluating spinal disease, evaluation of spinal disease may be limited. In these cases, it is very helpful to diagnose spinal disease with MR images synthesized with CT images. (2) Objective: To create synthetic lumbar magnetic resonance (MR) images from computed tomography (CT) scans using generative adversarial network (GAN) models and assess how closely the synthetic images resembled the true images using visual Turing tests (VTTs). (3) Material and Methods: Overall, 285 patients aged ≥ 40 years who underwent lumbar CT and MRI were enrolled. Based on axial CT and T2-weighted axial MR images from 285 patients, an image synthesis model using a GAN was trained using three algorithms (unsupervised, semi-supervised, and supervised methods). Furthermore, VTT to determine how similar the synthetic lumbar MR images generated from lumbar CT axial images were to the true lumbar MR axial images were conducted with 59 patients who were not included in the model training. For the VTT, we designed an evaluation form comprising 600 randomly distributed axial images (150 true and 450 synthetic images from unsupervised, semi-supervised, and supervised methods). Four readers judged the authenticity of each image and chose their first- and second-choice candidates for the true image. In addition, for the three models, structural similarities (SSIM) were evaluated and the peak signal to noise ratio (PSNR) was compared among the three methods. (4) Results: The mean accuracy for the selection of true images for all four readers for their first choice was 52.0% (312/600). The accuracies of determining the true image for each reader’s first and first + second choices, respectively, were as follows: reader 1, 51.3% and 78.0%; reader 2, 38.7% and 62.0%, reader 3, 69.3% and 84.0%, and reader 4, 48.7% and 70.7%. In the case of synthetic images chosen as first and second choices, supervised algorithm-derived images were the most often selected (supervised, 118/600 first and 164/600 second; semi-supervised, 90/600 and 144/600; and unsupervised, 80/600 and 114/600). For image quality, the supervised algorithm received the best score (PSNR: 15.987 ± 1.039, SSIM: 0.518 ± 0.042). (5) Conclusion: This was the pilot study to apply GAN to synthesize lumbar spine MR images from CT images and compare training algorithms of the GAN. Based on VTT, the axial MR images synthesized from lumbar CT using GAN were fairly realistic and the supervised training algorithm was found to provide the closest image to true images.

A Comparative Study on the Performance and Security Evaluation of Spiking Neural Networks

The brain-inspired Spiking neural networks (SNN) claim to present advantages for visual classification tasks in terms of energy efficiency and inherent robustness. In this work, we explore the impact on network inter-layer sparsity through neural coding schemes and the intrinsic structural parameters of Leaky Integrate-and-Fire (LIF) neurons, which can be a candidate metric for performance evaluation. Towards this, we perform a comparative study of four critical neural coding schemes: rate coding (poisson coding), latency coding, phase coding, and direct coding, as well as 6 LIF neuron intrinsic parameter options for a total of 24 combined parameter schemes. Specifically, the models were trained using a supervised training algorithm with a surrogate gradient, and two adversarial attacks, Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) were applied on a CIFAR10 dataset. We identified the sources of interlayer sparsity in SNN, and quantitatively analyzed the differences in sparsity caused by coding schemes, neuron leakage factors and thresholds. Various aspects of network performance were thoroughly considered in this paper, including inference accuracy, adversarial robustness, and energy efficiency. Our results show that latency coding is the optimum choice in achieving the highest adversarial robustness and energy efficient against low intensity attacks, while rate coding offers the best adversarial robustness against medium and high intensity attacks. The maximum deviations of robustness and efficiency between different coding schemes are 9.35% in VGG5 and 13.59% in VGG9. Increasing the sparsity of spike activity by improving the threshold can bring a short-lived adversarial robustness sweet spot, while excessive sparsity due to changes in threshold and leakage can instead reduce the adversarial robustness. The study reveals the advantages and disadvantages, and design space of SNN in various dimensions, allowing researchers to frame their neuromorphic systems in terms of the coding methods, neuron inherent structure, and model learning capabilities.

Modelagem chuva-vazão via redes neurais artificiais para simulação de vazões de uma bacia hidrográfica da Amazônia

The techniques of artificial neural networks (ANNs) have been used in the prediction of hydrological variables due to the ability to generalize information, which makes the implementation of models less costly and more agile. In this study, the phenomenon of converting rainfall into streamflows of a Guama River Hydrographic Sub-basin (GRHS) in the State of Para, Amazon, was simulated. The models are based on MLP (Multilayer Perceptron) and NARX (Nonlinear Autoregressive with Exogenous Inputs) ANNs, with Hyperbolic Tangent activation function in hidden layer neurons, being trained by the supervised training algorithm Levenberg-Marquardt. Comparing the proposed ANNs, the NARX-ANN models presented better performances compared to the MLP-ANN model. The best of the NARX-ANN models presented, on average, for the training, validation and test phases, R² equal to 0.9901, RMSE equal to 11.73 m3s-1 and MAPE equal to 5.94%. These results show the possibility of simulating the streamflow of small and medium hydrographic basins in the Amazon through the combination of NARX-ANNs, mainly those basins with no or limited rainfall-flow data.

Multi-Disease Classification Model Using Strassen’s Half of Threshold (SHoT) Training Algorithm in Healthcare Sector

In healthcare industry, Neural Network has attained a milestone in solving many real-life classification problems varies from very simple to complex and from linear to non-linear. To improve the training process by reducing the training time, Adaptive Skipping Training algorithm named as Half of Threshold (HOT) has been proposed. To perform the fast classification and also to improve the computational efficiency such as accuracy, error rate, etc., the highlighted characteristics of proposed HOT algorithm has been integrated with Strassen’s matrix multiplication algorithm and derived a novel, hybrid and computationally efficient algorithm for training and validating the neural network named as Strassen’s Half of Threshold (SHoT) Training Algorithm. The experimental outcome based on the simulation demonstrated that the proposed SHOT algorithm outperforms both BPN and HOT algorithm in terms of training time which is reduced with the range of 7% to 54% and its efficiency which is improved with the range of 3% to 15% on various dataset such as Hepatitis, SPeCT, Heart, Liver Disorders, Breast Cancer Wisconsin (Diagnostic), Drug Consumption, Cardiotocography, Splice-junction Gene Sequences and Thyroid Disease dataset that are extracted from Machine Learning Dataset Repository of UCI. It can be integrated with any type of supervised training algorithm.

SSLA BASED TRAFFIC SIGN AND LANE DETECTION FOR AUTONOMOUSCARS

The Self-Driving Cars are also known as Autonomous Vehicles. This Car has the ability to sense around the environment. These sensed parameters are processed and according to it the different actuators in the car will work without any human involvement. An Autonomous car work like a normal car but without any human driver. Autonomous cars rely on sensors, actuators, machine learning algorithms and Software to perform all the Automated Functions. The Software part is very important for Autonomous vehicles. The Software architecture acts as a bridge between Hardware Components and Application. The Standardized Software for Automotive cars is AUTOSAR. The AUTOSAR is a Standardized Architecture between Application Software and Hardware. This Standardized Architecture provide all Communication Interfaces, Device Drivers, Basic Software and Run-Time Environment. There are two important modules in Self-Driving Cars. They are Lane Detection and Traffic Signal detection which works automatically without any Human Intervention. A Machine Learning Algorithm is proposed in this paper. This Algorithm is mainly used to train the shape models and helps to detect the shape for Traffic Sign detection and Lane Detection. These both tasks are programmed using python with Open cv2 library file, numpy library file and Hough Detection technique is used to detect the appropriate circles of the traffic signals.By using all these tools, all the shape models are trained using Supervised training Algorithm and the detection is performed in such a way to help Autonomous cars to detect the lane and traffic Sign.

A New Supervised Learning Algorithm Based on Genetic Inheritance for Spiking Neural Networks

Spiking neural networks (SNNs) can perform complex spatio-temporal information computations in precise temporal coding. These networks differ from previous models in that spiking neurons convey information by time rather than rate of spikes. Most existing training algorithms are based on gradient descent with inherent defects, such as local optimum and over-fitting. In this paper, we investigate the performance of the Genetic Algorithm Involving Mechanism of Simulated Annealing, as a supervised training algorithm for SNNs. The key idea is to adopt global search, which effectively avoid local optima and over-fitting. According to the experiment results, this approach has higher accuracy than other learning algorithms on well-known classification problems.

Assessment of dysarthric speech using Elman back propagation network (recurrent network) for speech recognition

In this paper, the role of speech recognition system in the assessment of dysarthric speech based on a method called Elman back propagation network (EBN) is studied. Dysarthria is a neurological disability that damages the control of motor speech articulators. The persons who suffer from Dysarthria may have speech intelligibility rate which may vary from low (2 %) to high (95 %). EBN is a Recurrent network, here a fully connected neural network is built such that the speech characteristics are represented simultaneously by neuron activation states. It is an efficient self supervised training algorithm. For parametric representation of the speech signal, we used Glottal feature along with mel frequency cepstral coefficients. Then finally the output of both the features is compared after the evaluation process using different neural networks and modeling methods. Evaluation of the proposed method is done on the subset of the Universal Access Research database. The subset consists of 9 dysarthric speakers out of 19 speakers each uttering 100 words repeatedly 3 times. The promising performance of the proposed system can be successfully applied to help the people who work for the voice disorder persons.

An Efficient Supervised Training Algorithm for Multilayer Spiking Neural Networks.

The spiking neural networks (SNNs) are the third generation of neural networks and perform remarkably well in cognitive tasks such as pattern recognition. The spike emitting and information processing mechanisms found in biological cognitive systems motivate the application of the hierarchical structure and temporal encoding mechanism in spiking neural networks, which have exhibited strong computational capability. However, the hierarchical structure and temporal encoding approach require neurons to process information serially in space and time respectively, which reduce the training efficiency significantly. For training the hierarchical SNNs, most existing methods are based on the traditional back-propagation algorithm, inheriting its drawbacks of the gradient diffusion and the sensitivity on parameters. To keep the powerful computation capability of the hierarchical structure and temporal encoding mechanism, but to overcome the low efficiency of the existing algorithms, a new training algorithm, the Normalized Spiking Error Back Propagation (NSEBP) is proposed in this paper. In the feedforward calculation, the output spike times are calculated by solving the quadratic function in the spike response model instead of detecting postsynaptic voltage states at all time points in traditional algorithms. Besides, in the feedback weight modification, the computational error is propagated to previous layers by the presynaptic spike jitter instead of the gradient decent rule, which realizes the layer-wised training. Furthermore, our algorithm investigates the mathematical relation between the weight variation and voltage error change, which makes the normalization in the weight modification applicable. Adopting these strategies, our algorithm outperforms the traditional SNN multi-layer algorithms in terms of learning efficiency and parameter sensitivity, that are also demonstrated by the comprehensive experimental results in this paper.

Detection and Classification of Brain Tumor using Radial Basis Function

Aim: This paper proposes the automatic support system for detecting the tumor cells by analyzing the scalp EEG by means of RBF technique. Objectives: To acquire the EEG signal from the various electrodes. The artificial neural network will be focused to split up the EEG signal whether cyst i.e. tumor or regular. Methods: The EEG signal is been acquired from the subject using EEG scalp electrodes. The various features such as mean, variance, co-variance, Eigen values and Eigen vectors are extracted from those signals using the Principal Component Analysis. Radial Basis Function (RBF) networks are feed-forward networks which uses a supervised training algorithm are used for function approximation, time series prediction and system control. The RBF is used to train and classify the signal whether the subject is normal or suffering from abnormalities. Results: The features are been extracted using the Principal Component Analysis and the features are skilled. Thus the acquired signals are been classified as cyst or normal. Conclusion: Thus in this paper an automatic system is been developed for diagnosing the tumor cells by means of analyzing EEG signal which is non-invasive method. It can also extend for analyzing other diseases seizures of epilepsy, Alzheimer's disease.

Neuro-Fuzzy Supervised Training Algorithm for Varied Chicken Disease Recognition

Poultry farming is an integral part of human existence. It link or interlink most, if not all of human endeavours which provides food in terms of meat and other proteins ingredients for human consumption and existence. Diseases on the other hand, are usually a resultant of human interaction with the ecosystem which has affected poultry farming for centuries. Most of the approaches applied in chicken disease recognition is subjective (based on the experiences, skills, exposure and talents of a personnel) in nature at best. Fuzzy Supervised Neural Network Training Algorithm has been designed and implemented with Matrix Laboratory (MATLAB) and Hypertext Pre-processor as the simulation tools and language respectively. This paper demonstrates the practical application of algorithm techniques in medical

Control of a direct drive robot using fuzzy spiking neural networks with variable structure systems-based learning algorithm

In this work, a sliding mode theory based supervised training algorithm that implements fuzzy reasoning on a spiking neural network has been developed and tested on the trajectory control problem of a two-degrees-of-freedom direct drive robotic manipulator. To describe the generation of a new spike train from the incoming spike trains Spike Response Model has been utilized and the Lyapunov stability method has been adopted in the derivation of the update rules for the neurocontroller parameters. The results of the real-time experiments indicate that stable online tuning and fast learning speed are the prominent characteristics of the proposed algorithm.

A Novel Histogram-Based Multi-Threshold Searching Algorithm for Multilevel Colour Thresholding

Image segmentation is an important preliminary process required in object tracking applications. This paper addresses the issue of unsupervised multi-colour thresholding design for colour-based multiple objects segmentation. Most of the current unsupervised colour thresholding techniques require adopting a supervised training algorithm or a cluster-number decision algorithm to obtain optimal threshold values of each colour channel for a colour-of-interest. In this paper, a novel unsupervised multi-threshold searching algorithm is proposed to automatically search the optimal threshold values for segmenting multiple colour objects. To achieve this, a novel ratio-map image computation method is proposed to efficiently enhance the contrast between colour and non-colour pixels. The Otsu's method is then applied to the ratio-map image to extract all colour objects from the image. Finally, a new histogram-based multi-threshold searching algorithm is developed to search the optimal upper-bound and lower-bound threshold values of hue, saturation and brightness components for each colour object. Experimental results show that the proposed method not only succeeds in separating all colour objects-of-interest in colour images, but also provides satisfactory colour thresholding results compared with an existing multilevel thresholding method.

Substitution of G.728 vocoder’s codebook search module with SOM array trained by PSO-optimized supervised algorithm

Low delay-code excited linear prediction (LD-CELP) is an attractive algorithm in implementing vocoders in voice over Internet protocol networks. This algorithm has been proposed for the coding of speech at 16 kbps with toll quality. However, operation at transmission rates lower than 16 kbps is desirable, so that traffic can be accommodated during system overload conditions. In this paper, an array of self-organizing maps (SOMs) is employed instead of traditional codebook search module, recommended in ITU-T G.728, to determine the optimum index value of shape codebook. It is noted that a modified supervised training algorithm is used for SOMs in which some of the training parameters are optimized using particle swarm optimization (PSO) algorithm. Based on the occurrence frequency characteristics of codevectors, six bits for shape codebook and two bits for gain codebook are used in this work to produce a vocoder with lower bit rate as compared with traditional ITU-T G.728 vocoder. The performance comparison of the proposed SOM array trained by PSO-optimized supervised algorithm as the codebook search module in the structure of LD-CELP with a conventional implementation of LD-CELP coder shows that execution time of the algorithm is reduced up to 44 %. However, the degradation of voice quality in terms of mean opinion score, perceived evaluation of speech quality and segmental signal-to-noise ratio (SNRseg) is acceptable.