Neural Network Pattern Recognition Research Articles

A Novel Implementation of Siamese Type Neural Networks in Predicting Rare Fluctuations in Financial Time Series

Stock trading has tremendous importance not just as a profession but also as an income source for individuals. Many investment account holders use the appreciation of their portfolio (as a combination of stocks or indexes) as income for their retirement years, mostly betting on stocks or indexes with low risk/low volatility. However, every stock-based investment portfolio has an inherent risk to lose money through negative progression and crash. This study presents a novel technique to predict such rare negative events in financial time series (e.g., a drop in the S&P 500 by a certain percent in a designated period of time). We use a time series of approximately seven years (2517 values) of the S&P 500 index stocks with publicly available features: the high, low and close price (HLC). We utilize a Siamese type neural network for pattern recognition in images followed by a bootstrapped image similarity distribution to predict rare events as they pertain to financial market analysis. Extending on literature about rare event classification and stochastic modeling in financial analytics, the proposed method uses a sliding window to store the input features as tabular data (HLC price), creates an image of the time series window, and then uses the feature vector of a pre-trained convolutional neural network (CNN) to leverage pre-event images and predict rare events. This research does not just indicate that our proposed method is capable of distinguishing event images from non-event images, but more importantly, the method is effective even when only limited and strongly imbalanced data is available.

Neural Network Pattern Recognition Experiments Toward a Fully Automatic Detection of Anomalies in InSAR Time Series of Surface Deformation

We present a neural network-based method to detect anomalies in time-dependent surface deformation fields given a set of geodetic images of displacements collected from multiple viewing geometries. The presented methodology is based on a supervised classification approach using combinations of line of sight multitemporal, multi-geometry interferometric synthetic aperture radar (InSAR) time series of displacements. We demonstrate this method with a set of 170 million time series of surface deformation generated for the entire Italian territory and derived from ERS, ENVISAT, and COSMO-SkyMed Synthetic Aperture Radar satellite constellations. We create a training dataset that has been compared with independently validated data and current state-of-the-art classification techniques. Compared to state-of-the-art algorithms, the presented framework provides increased detection accuracy, precision, recall, and reduced processing times for critical infrastructure and landslide monitoring. This study highlights how the proposed approach can accelerate the anomalous points identification step by up to 147 times compared to analytical and other artificial intelligence methods and can be theoretically extended to other geodetic measurements such as GPS, leveling data, or extensometers. Our results indicate that the proposed approach would make the anomaly identification post-processing times negligible when compared to the InSAR time-series processing.

Odor pattern recognition of a novel bio-inspired olfactory neural network based on kernel clustering

Olfactory system is an important component in sensory nervous system. When an olfactory receptor receives odor stimulation, it transfers chemical signals into electrical signals, and delivers to the olfactory bulb, where integrates and codes the olfactory information, further to the cerebral olfactory cortex to generate olfaction. Establishment of a novel bio-inspired olfactory neural network for the study of olfactory information processing is helpful to understand how olfactory system can effectively differentiate different types and concentrations of odor. Based on a traditional olfactory bulb model composed of mitral cells, granule cells, and periglomerular cells, olfactory cortex was presented to establish a novel bio-inspired olfactory neural network model. Meanwhile, inhibitory synaptic plasticity was considered when network was receiving stimulation. The results of simulation indicated that inhibitory synaptic plasticity could balance excitatory and inhibitory synaptic currents in the olfactory cortex with specific firing patterns under odor stimulation. Olfactory cortex showed different firing patterns with different odor stimulations, and also presented similar firing patterns and different firing strengths for the same type of odor at different concentrations. Meanwhile, based on hierarchical clustering and fuzzy clustering, recognition of pure odors and mixed odors could be realized.

Particle swarm optimization pattern recognition neural network for transmission lines faults classification

The operating conditions of the transmission lines can be assessed through the information measured by the smart meters in the power supply bureau. Accurate classification of transmission line faults can be helpful to improve the maintenance strategy of smart grids. This paper analyzes the mechanism of the voltage loss and the phase fault of the transmission line by using the operation data collected by the smart meters from three power supply bureaus (named Bureau A, B and C), where the faults are labeled by expert systems. In this work, a novel Particle Swarm Optimization Pattern Recognition Neural Network (PSO-PRNN) classifier is built to accurately categorize the faults and its classification performance is compared with the ones of traditional K-Nearest Neighbor (KNN), Decision Tree (DT), PSO-KNN and PSO-DT classifiers. The results show that the classification accuracy of PSO-PRNN outperforms traditional classifiers when being applied to the data collected from all three bureaus. In the A power supply bureau are 83.0%, 88.7%, 82.0%, 86.9% and 96.1%, and the classification accuracy rates are 55.7%, 68.7%, 56.6%, 68.7% and 82.5%, when used to process the data of the bureau B. The classification accuracy is 57.1%, 66.4%, 57.2%, 69.0% and 82.1%, when processing the data of bureau C. The results show that the PSO-PRNN classifier is superior to the others in terms of accuracy and applicability.

Research on control system of small intelligent drilling rig based on lithology identification

Intelligent oil drilling rig can automatically complete the whole drilling process without human intervention, and can integrate big data with drilling, geology, logging and other disciplines. Aiming at the problem of insufficient rock breaking capacity and complex and changeable drilling process, a small intelligent drilling rig control system is designed. The structural drilling parameter samples such as WOB and rotating speed obtained by the sensor are used for lithology identification in the software MATLAB by using the neural network pattern recognition toolbox and GRNN generalized regression neural network respectively, and the control block diagram of the automatic drilling system is designed in Matlab / Simulink. The neural network control algorithm is used to compare with the classical PID control, The results show that the overshoot and rise time are significantly reduced, and the effect is good. When applied to the oil rig to realize constant WOB drilling, it is conducive to reasonably select the bit type, timely adjust the drilling parameters and improve the drilling efficiency, so as to realize the real intelligence of the oil rig.

Rotating Machinery Fault Diagnosis Method Based on Improved Semisupervised Generative Confrontation Network

Error diagnosis and detection have become important in modern production due to the importance of spinning equipment. Artificial neural network pattern recognition methods are widely utilized in rotating equipment fault detection. These methods often need a large quantity of sample data to train the model; however, sample data (especially fault samples) are uncommon in engineering. Preliminary work focuses on dimensionality reduction for big data sets using semisupervised methods. The rotary machine’s polar coordinate signal is used to build a GAN network structure. ANN and tiny samples are utilized to identify DCGAN model flaws. The time-conditional generative adversarial network is proposed for one-dimensional vibration signal defect identification under data imbalance. Finally, auxiliary samples are gathered under similar conditions, and CCNs learn about target sample characteristics. Convolutional neural networks handle the problem of defect identification with small samples in different ways. In high-dimensional data sets with nonlinearities, low fault type recognition rates and fewer marked fault samples may be addressed using kernel semisupervised local Fisher discriminant analysis. The SELF method is used to build the optimum projection transformation matrix from the data set. The KNN classifier then learns low-dimensional features and detects an error kind. Because DCGAN training is unstable and the results are incorrect, an improved deep convolutional generative adversarial network (IDCGAN) is proposed. The tests indicate that the IDCGAN generates more real samples and solves the problem of defect identification in small samples. Time-conditional generation adversarial network data improvement lowers fault diagnosis effort and deep learning model complexity. The TCGAN and CNN are combined to provide superior fault detection under data imbalance. Modeling and experiments demonstrate TCGAN’s use and superiority.

Application of a Pattern-Recognition Neural Network for Detecting Analog Electronic Circuit Faults

In this study, machine learning techniques based on the development of a pattern–recognition neural network were used for fault diagnosis in an analog electronic circuit to detect the individual hard faults (open circuits and short circuits) that may arise in a circuit. The ability to determine faults in the circuit was analyzed through the availability of a small number of measurements in the circuit, as test points are generally not accessible for verifying the behavior of all the components of an electronic circuit. It was shown that, despite the existence of a small number of measurements in the circuit that characterize the existing faults, the network based on pattern-recognition functioned adequately for the detection and classification of the hard faults. In addition, once the neural network has been trained, it can be used to analyze the behavior of the circuit versus variations in its components, with a wider range than that used to develop the neural network, in order to analyze the ability of the ANN to predict situations different from those used to train the ANN and to extract valuable information that may explain the behavior of the circuit.

Comparative numerical analysis of Bayesian decision rule and probabilistic neural network for pattern recognition

At present, neural networks are increasingly used to solve many problems instead of traditional methods for solving them. This involves comparing the neural network and the traditional method for specific tasks. In this paper, computer modeling of the Bayesian decision rule and the probabilistic neural network is carried out in order to compare their operational characteristics for recognizing Gaussian patterns. Recognition of four and six images (classes) with the number of features from 1 to 6 was simulated in cases where the images are well and poorly separated. The sizes of the training and test samples are chosen quiet big: 500 implementations for each image. Such characteristics as training time of the decision rule, recognition time on the test sample, recognition reliability on the test sample, recognition reliability on the training sample were analyzed. In framework of these conditions it was found that the recognition reliability on the test sample in the case of well separated patterns and with any number of the instances is close to 100 percent for both decision rules. The neural network loses 0,1–16 percent to Bayesian decision rule in the recognition reliability on the test sample for poorly separated patterns. The training time of the neural network exceeds the training time of the Bayesian decision rule in 4–5 times and the recognition time – in 4–6 times. As a result, there are no obvious advantages of the probabilistic neural network over the Bayesian decision rule in the problem of Gaussian pattern recognition. The existing generalization of the Bayesian decision rule described in the article is an alternative to the neural network for the case of non-Gaussian patterns.

An Intelligent Multimode Clustering Mechanism Using Driving Pattern Recognition in Cognitive Internet of Vehicles.

Connected autonomous vehicles can leverage communication and artificial intelligence technologies to effectively overcome the perceived limitations of individuals and enhance driving safety and stability. However, due to the high dynamics of the vehicular network and frequent interruptions and handovers, it is still challenging to provide stable communication connections between vehicles, which is likely to cause disasters. To address this issue, in this paper, we propose an intelligent clustering mechanism based on driving patterns in heterogeneous Cognitive Internet of Vehicles (CIoVs). In the proposed approach, we analyze the driving mode containing multiple feature parameters to accurately capture the driving characteristics. To ensure the accuracy of pattern recognition, a genetic algorithm-based neural network pattern recognition algorithm is proposed to support the reliable clustering of connected autonomous vehicles. The cognitive engines recognize the driving modes to group vehicles with a similar driving mode into a relatively stable cluster. In addition, we formulate the stability and survival time of clusters and analyze the communication performance of the clustering mechanism. Simulation results show that the proposed mechanism improves the reliable communication throughput and average cluster lifetime by approximately 14.4% and 11.5% respectively compared to the state-of-the-art approaches.

Classification of catchments for nitrogen using Artificial Neural Network Pattern Recognition and spatial data

In hydrological modelling, classification of catchments is a fundamental task for overcoming deficits in observational datasets. Most attention on this issue has focussed on identifying the catchments with similar hydrological responses for streamflow. Yet, effective methods for catchment classification are currently lacking in respect to Dissolved Inorganic Nitrogen (DIN), a water quality constituent that, at increasing concentrations, is threatening nutrient sensitive environments. Pattern recognition, using standard Artificial Neural Network algorithm is applied, as a novel approach to classify datasets that are considered to be suitable proxies for biological and anthropogenic drivers of observed DIN releases. Eleven gauged Great Barrier Reef (GBR) catchments within Queensland Australia are classified using spatial datasets extracted from ecosystem (e.g. original ecosystem responses to biogeographic, land zone, land form, and soil type attributes) and land use maps. To evaluate the performance of the examined spatial datasets as a proxy for deductive classification, the classification process is repeated inductively, using observed DIN and streamflow data from gauging stations. The ANN-PR method is seen to generate the same classification score format for the differing dataset types, and this facilitates a direct comparison for model output for observed data corroborations. The Kruskal-Wallis test for independence, at p > 0.05, identifies the deductive classification approach as a predictor for classification using DIN observations, which lacks an independence from each other at a p value of 0.01 and 0.02. This study concludes that an ANN-PR method can integrate the ecosystem and land use mapping data to deductively classify the GBR catchments into four regions that also have similar patterns of DIN concentrations. Due to the uniform availability of the mapping data, the findings provide a sound basis for further investigations into the transposing of knowledge from gauged catchments to ungauged areas.

Enhancing Thermal Performance and Lifetime Cycles of Li-ion Battery in Electric Vehicles

Hybrid energy storage system has essential priority in Electric Vehicle applications. Therefore, the design of an appropriate power sharing algorithm among energy storage components is necessary to improve battery thermal performance and provide extra extension of battery lifetime cycles. This paper presents an analytical study on the effect of using wavelet decomposition-based power sharing algorithm to force the high frequency component to be fed by the supercapacitor and accordingly reduces the thermal stress on the battery. The proposed approach was investigated by applying it on electric vehicle model in ADVISOR Tool/MATLAB using different driving profiles such as Urban Dynamometer Driving Schedule profile, Highway Fuel Economy Test, New York City Cycle, Los Angeles 1992 and new European driving cycle. The results declare that by using proposed power sharing algorithm, the working temperature of lithium battery decreases significantly while battery lifetime cycles increase, apparently. For urban dynamometer driving schedule, the operating temperature of lithium battery is improved much at maximum decomposition levels reaching to only 25.6 °C compared to 35 °C. In addition, the battery lifetime cycles increased from 2213 to 2585 cycles. Neural Networks pattern recognition tool is also applied to classify the driving cycle to the nearest reference cycles chosen to represent the different driving conditions which help to detect the appropriate wavelet decomposition level, achieving better battery thermal performance and battery lifetime cycles.

A Data-Driven Method for Vehicle Speed Estimation

This paper presents a method based on Artificial Neural Networks for estimation of the vehicle speed. The technique exploits the combination of two tasks: a) speed estimation by means of regression neural networks dedicated to different road conditions (dry, wet and icy); b) identification of the road condition with a pattern recognition neural network. The training of the networks is conducted with experimental datasets recorded during the driving sessions performed with a vehicle on different tracks. The effectiveness of the proposed approach is validated experimentally on the same car by deploying the algorithm on a dSPACE computing platform. The estimation accuracy is evaluated by comparing the obtained results to the measurement of an optical sensor installed on the vehicle and to the output of another estimation method, based on the mean value of velocity of the four wheels.

A Comparative Study of Handwritten Character Recognition by using Image Processing and Neural Network Techniques

This study aims to analyze the effects of noise, image filtering, and edge detection techniques in the preprocessing phase of character recognition by using a large set of character images exported from MNIST database trained with various sizes of neural networks. Canny edge detection algorithm was deployed to smooth the edges of the images while the Sobel edge detection algorithm was used to detect the edges of the images. Skeletonization algorithm was applied to re-shape the structural shapes. In the context of the image filtering, the Laplacian filter was utilized to enhance the images and High pass filtering was used to highlight the fine details in blurred images. Gaussian noise, image noise with Gaussian intensity, function in Matlab with the probability density function P was deployed on character images of MINST. Pattern recognition neural networks are widely used in optical character recognition. Feedforward neural networks are deployed in this study. A comprehensive analysis of the above-mentioned image processing techniques is included during character recognition. Improved accuracy is observed with character recognition during the prediction phase of the neural networks. A sample of unknown characters is tested with the application of High pass filtering + feedforward neural network and 89%, the highest, average output prediction accuracy was obtained. Other prediction accuracies were also tabulated for the reader’s attention.

Estimation of ground-level particulate matter concentrations based on synergistic use of MODIS, MERRA-2 and AERONET AODs over a coastal site in the Eastern Mediterranean

Satellite-derived aerosol optical depth (AOD) products are widely used to estimate the spatial and temporal characteristics of ground-level particulate matters (PM). Satellite-based approaches in the operational use of PM estimation models have limitations such as missing values in satellite-based variables due to the impact of cloud cover, relative humidity, and aerosol vertical distribution. Therefore, the spatial-temporal resolution of the estimation models is forced to be improved by using AOD products having different temporal resolutions from different platforms. In this study, ground-based PM10 concentrations were aimed to be estimated using different gap-filled AOD datasets between 2008 and 2016 over a coastal site in the Eastern Mediterranean. For the estimation of PM10 concentrations, daily AOD data were mainly used from MODIS. Then, two different gap-filled MODIS AOD datasets with both AERONET and MERRA-2 AODs were also examined separately on both multi-annual and multi-seasonal basis since the number of MODIS AOD data was temporally limited in the region. A pattern recognition neural network (PRNN) model was used for the estimation of PM10 concentrations. AOD with several meteorological parameters from an on-site meteorological station and aerosol diagnostic products from MERRA-2 were used as inputs to the estimation model. The most significant variables for the model were identified in 23 independent variables using the multiple linear regression (MLR). The results indicated that the best estimation (R = 0.74) was obtained with the gap-filled AODMODIS+MERRA dataset for the period covering all years whereas the AODMODIS dataset alone showed the poorest performance (R = 0.62). However, the performance of the gap-filled datasets varied with the seasons. For example, the AODMODIS dataset alone showed the best estimation performance (R = 0.67) in the summer whereas the gap-filled AODMODIS+AERONET dataset had the best performance (R = 0.59) in the winter. Overall results suggest that for estimating ground-level PM concentrations, an approach of gap-filled AOD dataset usage for estimation models is useful especially in rainy seasons such as the winter and autumn where MODIS AOD retrievals are limited.

Neural Network Pattern Recognition of Ultrasound Image Gray Scale Intensity Histograms of Breast Lesions to Differentiate Between Benign and Malignant Lesions: Analytical Study.

Ultrasound-based radiomic features to differentiate between benign and malignant breast lesions with the help of machine learning is currently being researched. The mean echogenicity ratio has been used for the diagnosis of malignant breast lesions. However, gray scale intensity histogram values as a single radiomic feature for the detection of malignant breast lesions using machine learning algorithms have not been explored yet. This study aims to assess the utility of a simple convolutional neural network in classifying benign and malignant breast lesions using gray scale intensity values of the lesion. An open-access online data set of 200 ultrasonogram breast lesions were collected, and regions of interest were drawn over the lesions. The gray scale intensity values of the lesions were extracted. An input file containing the values and an output file consisting of the breast lesions' diagnoses were created. The convolutional neural network was trained using the files and tested on the whole data set. The trained convolutional neural network had an accuracy of 94.5% and a precision of 94%. The sensitivity and specificity were 94.9% and 94.1%, respectively. Simple neural networks, which are cheap and easy to use, can be applied to diagnose malignant breast lesions with gray scale intensity values obtained from ultrasonogram images in low-resource settings with minimal personnel.

Anisotropic photoresponse of layered rhenium disulfide synaptic transistors

Layered ReS2 with direct bandgap and strong in-plane anisotropy shows great potential to develop high-performance angle-resolved photodetectors and optoelectronic devices. However, systematic characterizations of the angle-dependent photoresponse of ReS2 are still very limited. Here, we studied the anisotropic photoresponse of layered ReS2 phototransistors in depth. Angel-resolved Raman spectrum and field-effect mobility are tested to confirm the inconsistency between its electrical and optical anisotropies, which are along 120° and 90°, respectively. We further measured the angle-resolved photoresponse of a ReS2 transistor with 6 diagonally paired electrodes. The maximum photoresponsivity exceeds 0.515 A⋅W−1 along b-axis, which is around 3.8 times larger than that along the direction perpendicular to b axis, which is consistent with the optical anisotropic directions. The incident wavelength- and power-dependent photoresponse measurement along two anisotropic axes further demonstrates that b axis has stronger light–ReS2 interaction, which explains the anisotropic photoresponse. We also observed angle-dependent photoresistive switching behavior of the ReS2 transistor, which leads to the formation of angle-resolved phototransistor memory. It has simplified structure to create dynamic optoelectronic resistive random access memory controlled spatially through polarized light. This capability has great potential for real-time pattern recognition and photoconfiguration of artificial neural networks (ANN) in a wide spectral range of sensitivity provided by polarized light.

Rough Set Neural Network Feature Extraction and Pattern Recognition of Shaft Orbits Based on the Zernike Moment

In the shaft axis monitoring of hydrogenerating unit condition monitoring and fault diagnosis, the shaft orbit is intuitive and comprehensively reflects the unit operation state, and different shaft orbits correspond to different fault types, which can accurately indicate a system vibration fault. Shaft orbit identification has important significance for vibration fault diagnosis. In getting the feature extraction and pattern recognition of a shaft orbit, the Zernike moment is better than the Hu moment; it has the advantages of a small calculation error and a high recognition rate. A rough set neural network (RS-BP hybrid model) of shaft orbit recognition is established, which uses just 13 moment eigenvalues reserved by the rough set feature selection algorithm as input variables; it has the same calculation error and recognition rate and reduces the calculation time step. The simulation of the recognition of shaft orbits shows that the hybrid model has achieved good results in the identification of shaft orbits.

Integrated CWT-CNN for Epilepsy Detection Using Multiclass EEG Dataset

Electroencephalography is a common clinical procedure to record brain signals generated by human activity. EEGs are useful in Brain controlled interfaces and other intelligent Neuroscience applications, but manual analysis of these brainwaves is complicated and time-consuming even for the experts of neuroscience. Various EEG analysis and classification techniques have been proposed to address this problem however, the conventional classification methods require identification and learning of specific EEG characteristics beforehand. Deep learning models can learn features from data without having in depth knowledge of data and prior feature identification. One of the great implementations of deep learning is Convolutional Neural Network (CNN) which has outperformed traditional neural networks in pattern recognition and image classification. Continuous Wavelet Transform (CWT) is an efficient signal analysis technique that presents the magnitude of EEG signals as time-related Frequency components. Existing deep learning architectures suffer from poor performance when classifying EEG signals in the Time-frequency domain. To improve classification accuracy, we propose an integrated CWT and CNN technique which classifies five types of EEG signals using. We compared the results of proposed integrated CWT and CNN method with existing deep learning models e.g., GoogleNet, VGG16, AlexNet. Furthermore, the accuracy and loss of the proposed integrated CWT and CNN method have been cross validated using Kfold cross validation. The average accuracy and loss of Kfold cross-validation for proposed integrated CWT and CNN method are, 76.12% and 56.02% respectively. This model produces results on a publicly available dataset: Epilepsy dataset by UCI (Machine Learning Repository).

Two-Level Genetic Algorithm for Evolving Convolutional Neural Networks for Pattern Recognition

The aim of Neuroevolution is to find neural networks and convolutional neural network (CNN) architectures automatically through evolutionary algorithms. A crucial problem in neuroevolution is search time, since multiple CNNs must be trained during evolution. This problem has led to fitness acceleration approaches, generating a trade-off between time and fitness fidelity. Also, since search spaces for this problem usually include only a few parameters, this increases the human bias in the search. In this work, we propose a novel two-level genetic algorithm (GA) for addressing the fidelity-time trade-off problem for the fitness computation in CNNs. The first level evaluates many individuals quickly, and the second evaluates only those with the best results more finely. We also propose a search space with few restrictions, and an encoding with unexpressed genes to facilitate the crossover operation. This search space allows CNN architectures to have any sizes, shapes, and skip-connections among nodes. The two-level GA was applied to the pattern recognition problem on seven datasets, five MNIST-Variants, Fashion-MNIST, and CIFAR-10, achieving significantly better results than all those previously published. Our results show an improvement of 39.89% (4.2% error reduction) on the most complex dataset of MNIST (MRDBI), and on average 30.52% (1.35% error reduction) on all the five datasets. Furthermore, we show that our algorithm performed as well as precise-training GA, but took only the time of a fast-training GA. These results can be relevant and useful not only for image classification problems but also for GA-related problems.

Rule extraction from the Artificial Neural Network

Applying machine learning techniques to various critical decision-making domains implies the need to explain why a machine learning algorithm makes certain conclusions. Artificial neural networks (ANNs) have achieved high classification accuracy for many classification problems, but their results can’t be interpreted, which is why ANNs are often considered as «black box». To interpret the results of neural network classification, the paper presents a method for extracting rules from ANN. The proposed method is based on the structuring of information flows processed in the ANN information field through transforming complex multidimensional data into a simpler structure of a lower dimension, followed by a trivial transformation of the results into a set of fuzzy rules of a certain type. The implementation of the proposed method provides for the construction of the hybrid ANN configuration based on self-organizing and multilayer ANNs. The results of an experimental research of the proposed method are presented on the example of solving the well-known problem of multiparameter classification. The results obtained confirm the adequacy of the proposed method, which can be used both in independent neural network pattern recognition systems and within decision support systems.