Perform Pattern Recognition Research Articles

Interpolating between boolean and extremely high noisy patterns through minimal dense associative memories

Recently, Hopfield and Krotov introduced the concept of dense associative memories [DAM] (close to spin-glasses with P-wise interactions in a disordered statistical mechanical jargon): they proved a number of remarkable features these networks share and suggested their use to (partially) explain the success of the new generation of Artificial intelligence. Thanks to a remarkable ante-litteram analysis by Baldi & Venkatesh, among these properties, it is known these networks can handle a maximal amount of stored patterns K scaling as .In this paper, once introduced a minimal dense associative network as one of the most elementary cost-functions falling in this class of DAM, we sacrifice this high-load regime -namely we force the storage of solely a linear amount of patterns, i.e. (with )- to prove that, in this regime, these networks can correctly perform pattern recognition even if pattern signal is and is embedded in a sea of noise , also in the large N limit. To prove this statement, by extremizing the quenched free-energy of the model over its natural order-parameters (the various magnetizations and overlaps), we derived its phase diagram, at the replica symmetric level of description and in the thermodynamic limit: as a sideline, we stress that, to achieve this task, aiming at cross-fertilization among disciplines, we pave two hegemon routes in the statistical mechanics of spin glasses, namely the replica trick and the interpolation technique.Both the approaches reach the same conclusion: there is a not-empty region, in the noise-T versus load- phase diagram plane, where these networks can actually work in this challenging regime; in particular we obtained a quite high critical (linear) load in the (fast) noiseless case resulting in .

MCENN: A variant of extended nearest neighbor method for pattern recognition

Recent studies have shown that extended nearest neighbor (ENN) method is able to improve the classification performance over traditional k-nearest neighbor (KNN) methods, due to its novel “two-way communication” decision making process. The ENN method classifies a test data sample by maximizing the intra-class coherence gain over the whole data set. However, the original ENN method, like KNN, is a type of instance-based learning algorithm without a training stage. This paper presents a variant of ENN method, called Maximum intra-class Coherence Extended Nearest Neighbor (MCENN), which incorporates distances between individual data sample and its nearest neighbors into the decision making process and introduces a novel distance metric learning algorithm as a training process to learn an optimal linear transformation that maximizes the overall intra-class coherence of training data. We demonstrate that the proposed MCENN approach is able to improve the discriminative performance for pattern recognition. Experimental results on real-life data sets demonstrate the effectiveness of the proposed approaches.

Neural Networks with a Redundant Representation: Detecting the Undetectable.

We consider a three-layer Sejnowski machine and show that features learnt via contrastive divergence have a dual representation as patterns in a dense associative memory of order P=4. The latter is known to be able to Hebbian store an amount of patterns scaling as N^{P-1}, where N denotes the number of constituting binary neurons interacting P wisely. We also prove that, by keeping the dense associative network far from the saturation regime (namely, allowing for a number of patterns scaling only linearly with N, while P>2) such a system is able to perform pattern recognition far below the standard signal-to-noise threshold. In particular, a network with P=4 is able to retrieve information whose intensity is O(1) even in the presence of a noise O(sqrt[N]) in the large N limit. This striking skill stems from a redundancy representation of patterns-which is afforded given the (relatively) low-load information storage-and it contributes to explain the impressive abilities in pattern recognition exhibited by new-generation neural networks. The whole theory is developed rigorously, at the replica symmetric level of approximation, and corroborated by signal-to-noise analysis and MonteCarlo simulations.

Automated Recognition of Hand Grasps Using Electromyography Signal Based on LWT and DTCWT of Wavelet Energy

This paper presents a novel framework that automatically classifies hand grasps using Electromyogram (EMG) signals based on advanced Wavelet Transform (WT). This method is motivated by the observation that there lies a unique correlation between different samples of the signal at various frequency levels obtained by Discrete WT. In the proposed approach, EMG signals captured from the subjects are subjected to denoising using symlet wavelets, followed by Principal Component Analysis (PCA) for dimensionality reduction. Further, the important attributes of the signal are extracted using Lifting Wavelet Transform (LWT) and Dual Tree Complex WT (DTCWT). Multiple classifiers such as Feed Forward Neural Networks (FFNN), Cascaded Feed Forward Neural Networks (CFNN), Support Vector Machine (SVM) and Deep Learning Neural Network (DLNN) are used for classification. The simulation results are compared with various training algorithms and it is observed that DTCWT features combined with CFNN and trained with Gradient Descent with Adaptive Back Propagation (GDABP) algorithm achieved the best performance. The advantages of the proposed method were proved by comparing with the earlier conventional methods, in terms of recognition performance. These experimental results prove that the proposed method gives a potential performance in the recognition of hand grasps using EMG signals. In addition, the proposed method supports clinicians to improve the performance of myoelectric pattern recognition.

A feature­based approach to fractal image compression

Recently a number of researches have demonstrated performance improvement in the video fractal compression compared to the current video transmission standards (MPEG, H.263, H.264). This article describes a current problem of relatively low fractal encoding speed. Indeed, high computational complexity is a sore point of fractal compression approach. It seems almost every paper on this subject touches the problem of encoding speed. Productive ideas and algorithms can be borrowed from the pattern recognition problem. In the course of recent decades feature points approach in computer vision has been demonstrating good performance in SLAM and pattern recognition. Technology of feature detection, description and tracking is being developed successfully and has effective applications in augmented reality like Android ARCore and IOS ARKit frameworks that are real-time engines. Similarities among parts of video frames are analyzed and used for both image registration and visual scene tracking therefore it fits highly to block matching task. Statistic properties for domain/range blocks matching has been analyzed on the basis of previous investigation for fractal compression. As a result, a simple algorithm is proposed based on computer vision approach. The approach includes a visual feature points extraction, feature descriptors calculation and fast NN-search in descriptor space. The key idea of the proposed approach is as follows. Only a limited number of domain blocks around the most salient points are subject to selection. Other blocks are not essential for matching as transforms would have big Lipschitz constant and will have worse contractive properties. Salient points should be unique as well. Further the descriptors for feature point are calculated. The algorithm has O(N log N) complexity for pixel number in the frame image, however if the number of domain blocks is limited the complexity could be almost linear. Python program for the algorithm test has been developed and shows that reconstruction result is acceptable in terms of encoding speed (< 2 s on 2 GHz CPU) and quality (PSNR) ~25 dB. The result of the proposed approach could be interesting for further improvement both for image and video compression. Further steps for quality increasing are also described.

XNORCONV: CNNs accelerator implemented on FPGA using a hybrid CNNs structure and an inter‐layer pipeline method

Nowadays, convolutional neural networks (CNNs) have become a research hotspot because of their high performance in computer vision and pattern recognition. However, as the high energy consumption of traditional graphic processing units-based CNNs, it is difficult to deploy them into portable devices. To deal with this problem, a hybrid CNN structure (XNORCONV) was proposed and implemented on field-programmable gate array (FPGA) in this study. Two improvements have been applied in XNORCONV. Firstly, the multiplications in the convolutional layer (CONV) were replaced by XNOR operations to save the multiplier and reduce computational complexity. Secondly, an inter-layer pipeline was designed to further accelerate the calculation. XNORCONV was implemented on Xilinx Zynq-7000 xc7z020clg400-1 under the clock frequency of 150 MHz and tested with MNIST dataset. The results of the experiment show that XNORCONV can classify each picture from MNIST in , and achieve 98.4% recognition accuracy. Compared with traditional Lenet-5 on different platforms, XNORCONV reduced multiplication by 85.6% with only 0.4% accuracy loss.

Unsupervised discriminative feature representation via adversarial auto-encoder

Feature representation is generally applied to reducing the dimensions of high-dimensional data to accelerate the process of data handling and enhance the performance of pattern recognition. However, the dimensionality of data nowadays appears to be a rapidly increasing trend. Existing unsupervised feature representation methods are susceptible to the rapidly increasing dimensionality of data, which may result in learning a meaningless feature that in turn affect their performance in other applications. In this paper, an unsupervised adversarial auto-encoder network is studied. This network is a probability model that combines generative adversarial networks and variational auto-encoder to perform variational inference and aims to generate reconstructed data similar to original data as much as possible. Due to its adversarial training, this model is relatively robust in feature learning compared with other methods. First, the architecture and training strategy of adversarial auto-encoder are presented. We attempt to learn a discriminative feature representation for high-dimensional image data via adversarial auto-encoder and take its advantage into image clustering, which has become a difficult computer vision task recently. Then amounts of comparative experiments are carried out. The comparison contains eight feature representation methods and two recently proposed deep clustering methods performed on eight different publicly available image data sets. Finally, to evaluate their performance, we utilize a K-means clustering on the low-dimensional feature learned from each feature representation algorithm, and select three evaluation metrics including clustering accuracy, adjusted rand index and normalized mutual information, to provide a comparison. Comprehensive experiments prove the usefulness of the learned discriminative feature via adversarial auto-encoder in the tested data sets.

Unsupervised Online Learning With Multiple Postsynaptic Neurons Based on Spike-Timing-Dependent Plasticity Using a Thin-Film Transistor-Type NOR Flash Memory Array.

We present a two-layer fully connected neuromorphic system based on a thin-film transistor (TFT)-type NOR flash memory array with multiple postsynaptic (POST) neurons. Unsupervised online learning by spike-timing-dependent plasticity (STDP) on the binary MNIST handwritten datasets is implemented, and its recognition result is determined by measuring firing rate of POST neurons. Using a proposed learning scheme, we investigate the impact of the number of POST neurons in terms of recognition rate. In this neuromorphic system, lateral inhibition function and homeostatic property are exploited for competitive learning of multiple POST neurons. The simulation results demonstrate unsupervised online learning of the full black-and-white MNIST handwritten digits by STDP, which indicates the performance of pattern recognition and classification without preprocessing of input patterns.

Label Self-Advised Support Vector Machine (LSA-SVM)-Automated Classification of Foot Drop Rehabilitation Case Study.

Stroke represents a major health problem in our society. One of the effects of stroke is foot drop. Foot drop (FD) is a weakness that occurs in specific muscles in the ankle and foot such as the anterior tibialis, gastrocnemius, plantaris and soleus muscles. Foot flexion and extension are normally generated by lower motor neurons (LMN). The affected muscles impact the ankle and foot in both downward and upward motions. One possible solution for FD is to investigate the movement based on the bio signal (myoelectric signal) of the muscles. Bio signal control systems like electromyography (EMG) are used for rehabilitation devices that include foot drop. One of these systems is function electrical stimulation (FES). This paper proposes new methods and algorithms to develop the performance of myoelectric pattern recognition (M-PR), to improve automated rehabilitation devices, to test these methodologies in offline and real-time experimental datasets. Label classifying is a predictive data mining application with multiple applications in the world, including automatic labeling of resources such as videos, music, images and texts. We combine the label classification method with the self-advised support vector machine (SA-SVM) to create an adapted and altered label classification method, named the label self-advised support vector machine (LSA-SVM). For the experimental data, we collected data from foot drop patients using the sEMG device, in the Metro Rehabilitation Hospital in Sydney, Australia using Ethical Approval (UTS HREC NO. ETH15-0152). The experimental results for the EMG dataset and benchmark datasets exhibit its benefits. Furthermore, the experimental results on UCI datasets indicate that LSA-SVM achieves the best performance when working together with SA-SVM and SVM. This paper describes the state-of-the-art procedures for M-PR and studies all the conceivable structures.

Physics Based Deep Learning Technique for Prognostics

Remaining useful life (RUL) estimation is one of the most important aspects of prognostics and health management (PHM). Various deep learning (DL) based techniques have been developed and applied for the purposes of RUL estimation. One limitation of DL is the lack of physical interpretations as they are purely data driven models. Another limitation is the need for an exceedingly large amount of data to arrive at an acceptable pattern recognition performance for the purposes of RUL estimation. This research is aimed to overcome these limitations by developing physics based DL techniques for RUL prediction and validate the method with real run-to-failure datasets. The contribution of the research relies on creating hybrid DL based techniques as well as combining physics based approaches with DL techniques for effective RUL prediction.

A robust cutting pattern recognition method for shearer based on Least Square Support Vector Machine equipped with Chaos Modified Particle Swarm Optimization and Online Correcting Strategy

Accurate cutting pattern recognition method for shearer in coal mining process has drawn more and more attention over the past decades due to its important role in guaranteeing the steady operation of the equipment, which, however, remains challenging caused by the mismatch of cutting pattern recognition especially for dynamic uncertainty of future sampled data. Therefore, a novel approach for cutting pattern recognition with an optimal Online Correcting Strategy (OCS) combined with Least Square Support Vector Machine (LSSVM) and Chaos Modified Particle Swarm Optimization (CMPSO) algorithm, named OCS-CMPSO-LSSVM, is proposed, where LSSVM models the functional relationship between input and output of the system, CMPSO optimizes the parameters of LSSVM, and OCS modifies the model to reduce its mismatch as the system runs, respectively. The performance of the proposed model is demonstrated with a simulation experiment and compared with the existing methods reported in the literature in detail. The experimental results reveal that the proposed models can achieve better cutting pattern recognition performance and higher robustness.

Application of graphics processing unit parallel computing in pattern recognition for vibration events based on a phase-sensitive optical time domain reflectometer.

The technology of pattern recognition for vibration events based on the phase-sensitive optical time domain reflectometer (Φ-OTDR) has been significantly improved, thanks to plenty of valuable research in recent years. However, it remains challenging to develop an efficient algorithm for it with computing resources that are simpler to achieve at lower cost. To the best of our knowledge, this paper, for the first time, analyzes the superiority of using graphics processing unit (GPU) parallel computing to improve time-consuming performance in pattern recognition for vibration events based on Φ-OTDR. And the pattern-recognition algorithm, including spectral subtraction and artificial neural networks, is implemented by CPU and GPU, respectively. Then, the time consumption of the CPU-based method and the time consumption of the GPU-based method are, respectively, recorded and compared. As a result of our experiments, we concluded that using GPU parallel computing can develop an efficient algorithm with a computing resource that is simpler to achieve at a lower cost.

A research on fatigue crack growth monitoring based on multi-sensor and data fusion

Fatigue crack propagation is one of the main problems in structural health monitoring. For the safety and operability of the metal structure, it is necessary to monitor the fatigue crack growth process of the structure in real time. In order to more accurately monitor the expansion of fatigue cracks, two kinds of sensors are used in this article: strain gauges and piezoelectric transducers. A model-based inverse finite element model algorithm is proposed to perform pattern recognition of fatigue crack length, and the fatigue crack monitoring experiment is carried out to verify the algorithm. The strain spectra of the specimen under cyclic load in the simulation and experimental crack propagation are obtained, respectively. The active lamb wave technique is also used to monitor the crack propagation. The relationship between the crack length and the lamb wave characteristic parameter is established. In order to improve the recognition accuracy of the crack propagation mode, the random forest and inverse finite element model algorithms are used to identify the crack length, and the Dempster–Shafer evidence theory is used as data fusion to integrate the conclusion of the two algorithms to make a more accountable and correct judge of the crack length. An experiment has been conducted to demonstrate the effectiveness of the method.

Research and optimization of intelligent diagnosis algorithm based on rope tension

For less monitor and poor performance in the intelligence of the existing fault diagnosis system based on wire rope tension, an optimized intelligent diagnosis algorithm is proposed to diagnose the faults. These faults are difficult to monitor in the past, such as blocked cage, over-wind and slipped rope. Selecting the radial basis function (RBF) as the kernel function, two parameters of penalty factor and radial basis kernel parameter in the least squares support vector machine (LSSVM) are further optimized by artificial bee colony (ABC) algorithm. The results show that the LSSVM algorithm does not need a large number of original data, and has no overfitting and generalization ability. The prediction accuracy and the mean square error of the ABC-LSSVM algorithm are improved. It shows better pattern recognition performance, which can be used as a kind of intelligent diagnosis algorithm for the design of the rope tension fault diagnosis system.

Prediction of the Weld Qualities Using Surface Appearance Image in Resistance Spot Welding

The quality of the resistance spot weld is predicted qualitatively using information from the weld’s external apparent image. The predicting tool used for weld qualities was a convolution neural network (CNN) algorithm with excellent performance in pattern recognition. A heat trace image of the weld surface was used as information on the external apparent image of welds. The materials used in the experiment were advanced high strength steel (AHSS) with 980 MPa strength, and uncoated cold-rolled (CR) steel sheets and galvannealed (GA) steel sheets were used. The quantitatively predicted weld quality information contained tensile shear strength, nugget diameter, fracture mode of welds, and expulsion occurrence. The predicted performance of the verification step of the model determined through the learning process was as follows; the predicted error rate for tensile shear strength and nugget diameter were 2.2% and 2.6%, respectively. And the predicted accuracy on fracture mode and expulsion occurrence was 100%.

Study of the retina algorithm on FPGA for fast tracking

Real-time track reconstruction in high-energy physics experiments at colliders running at high luminosity is very challenging for trigger systems. To perform pattern recognition and track fitting, artificial retina or Hough transformation algorithms have been introduced to the field typically implemented on state-of-the-art field programmable gate array (FPGA) devices. In this paper, we report on two FPGA implementations of the retina algorithm: one using a mixed Floating-Point core and the other using Fixed-Point and Look-Up Table, and detailed measurements of the retina performance are investigated and compared. So far, the retina has mainly been used in a detector configuration comprising parallel planes, and the goal of our work is to study the hardware implementation of the retina algorithm and estimate the possibility of using such a method in a real experiment.

A Fractional Fourier Transform-Based Approach for Gas-Insulated Switchgear Partial Discharge Recognition

Partial discharge (PD) is very likely to deteriorate the insulating materials in power facilities. For this sake, PD detection and pattern recognition are treated as an effective approach to diagnosing the high-voltage insulation of power systems. A novel PD feature extraction approach for defect pattern recognition of a gas-insulated switchgear (GIS) is developed and then presented in this paper, using fractional Fourier transform (FrFT). A PD detector is utilized to measure the PD signal of the GIS using an ultra high frequency sensor, which is first transformed to its FrFT as the order ranges from 0 to 1. A 3D characteristic spectrum is then constructed, according to which the fractal parameters are extracted as the constituents of the cluster domains of defect types. Extension matter-element models of the PD defect types are then established according to the experimentally derived PD features. Subsequently, the PD defect type can be directly identified using the correlation degree between a tested pattern and the matter-element models. The pattern recognition performance is investigated on 80 sets of PD patterns, and the performance of the proposed approach is demonstrated accordingly. High accuracy, accompanied with high tolerance, was reached in the presence of noise interference using FrFT.

Selective Poisoning Attack on Deep Neural Networks †

Studies related to pattern recognition and visualization using computer technology have been introduced. In particular, deep neural networks (DNNs) provide good performance for image, speech, and pattern recognition. However, a poisoning attack is a serious threat to a DNN’s security. A poisoning attack reduces the accuracy of a DNN by adding malicious training data during the training process. In some situations, it may be necessary to drop a specifically chosen class of accuracy from the model. For example, if an attacker specifically disallows nuclear facilities to be selectively recognized, it may be necessary to intentionally prevent unmanned aerial vehicles from correctly recognizing nuclear-related facilities. In this paper, we propose a selective poisoning attack that reduces the accuracy of only the chosen class in the model. The proposed method achieves this by training malicious data corresponding to only the chosen class while maintaining the accuracy of the remaining classes. For the experiment, we used tensorflow as the machine-learning library as well as MNIST, Fashion-MNIST, and CIFAR10 as the datasets. Experimental results show that the proposed method can reduce the accuracy of the chosen class by 43.2%, 41.7%, and 55.3% in MNIST, Fashion-MNIST, and CIFAR10, respectively, while maintaining the accuracy of the remaining classes.

Relationship Between Offline and Online Metrics in Myoelectric Pattern Recognition Control Based on Target Achievement Control Test.

Offline classification accuracy (CA) is a widely accepted measure to evaluate the performance in pattern recognition based myoelectric scheme. However, whether offline metrics are able to be transferred to evaluate or predict online performance is still unclear. In this study, the relationship between offline metrics and online metrics are analyzed. In offline scenario, global CA is biased, thus class-wise accuracy standard deviation (std) is proposed as a supplement. Target Achievement Control Test (TAC Test) is adopted in online scenario. Completion rate, completion time and path efficiency are considered as online metrics. Our results demonstrate that online completion rate is strongly correlated with offline global CA and class-wise accuracy std. The correlation between offline and online performance metrics indicates it is reasonable to develop efficient algorithm in offline scenario if both global CA and class-wise accuracy are considered.

Featureless EMG pattern recognition based on convolutional neural network

In this paper, the performance of featureless EMG pattern recognition in classifying hand and wrist movements are presented. The time-frequency distribution (TFD), spectrogram is employed to transform the raw EMG signals into time-frequency representation (TFR). The TFRs or spectrogram images are then directly fed into convolutional neural network (CNN) for classification. Two CNN models are proposed to learn the features automatically from the images without the need of manual feature extraction. The performance of CNN with different number of convolutional layers is examined. The proposed CNN models are evaluated using the EMG data from 10 intact and 11 amputee subjects through the publicly access NinaPro database. Our results show that CNN classifier offered the best mean classification accuracy of 88.04% in recognizing hand and wrist movements.