Digit Recognition Application Research Articles

Pulse-stream impact on recognition accuracy of reservoir computing from SiO2-based low power memory devices

Reservoir computing (RC)-based neuromorphic applications exhibit extremely low power consumption, thus challenging the use of deep neural networks in terms of both consumption requirements and integration density. Under this perspective, this work focuses on the basic principles of RC systems. The ability of self-selective conductive-bridging random access memory devices to operate in two modes, namely, volatile and non-volatile, by regulating the applied voltage is first presented. We then investigate the relaxation time of these devices as a function of the applied amplitude and pulse duration, a critical step in determining the desired non-linearity by the reservoir. Moreover, we present an in-depth study of the impact of selecting the appropriate pulse-stream and its final effects on the total power consumption and recognition accuracy in a handwritten digit recognition application from the National Institute of Standards and Technology dataset. Finally, we conclude at the optimal pulse-stream of 3-bit, through the minimization of two cost criteria, with the total power remaining at 287 µW and simultaneously achieving 82.58% recognition accuracy upon the test set.

Application of digit and speech recognition in food delivery robot

Application of digit and speech recognition in food delivery robot

A Fast Spiking Neural Network Accelerator based on BP-STDP Algorithm and Weighted Neuron Model

Spiking neural networks (SNNs) are inspired from biological brains and have demonstrated great energy efficiency on hardware computing platforms. However, it is a challenge to implement an online training algorithm on SNN hardware to adapt to the realistic cognitive applications. Besides, the accuracies of SNN adopted rate-based coding scheme are highly depend on the number of coding timesteps, resulting in the long data processing time and massive power consumption. In this brief, a multilayer online training SNN accelerator based on spike-timing-dependent plasticity based back-propagation (BP-STDP) algorithm is carried out. To speed up the spike processing, a weighted neuron model is proposed according to the characteristics of BP-STDP training algorithm. In addition, the training module can reuse the integrate and fire arithmetic unit circuits in hidden layer neuron module, which can significantly reduce the hardware overheads. The proposed accelerator is verified on the Xilinx Virtex-7 FPGA under 100Mhz clock frequency for the application of handwritten digit recognition. The proposed accelerator achieves a 95.3% recognition accuracy rate while consuming 0.27 ms for recognizing one image, and 1.22 ms for training one image. Compared with the software implementation, the proposed accelerator demonstrates <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$41.87\times $ </tex-math></inline-formula> training speedup and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$56.96\times $ </tex-math></inline-formula> recognizing speedup.

Efficient Majority Voting in Digital Hardware

In recent years, machine learning methods became increasingly important for a manifold number of applications. However, they often suffer from high computational requirements impairing their efficient use in real-time systems, even when employing dedicated hardware accelerators. Ensemble learning methods are especially suitable for hardware acceleration since they can be constructed from individual learners of low complexity and thus offer large parallelization potential. For classification, the outputs of these learners are typically combined by majority voting, which often represents the bottleneck of a hardware accelerator for ensemble inference. In this brief, we present a novel architecture that allows obtaining a majority decision in a number of clock cycles that is logarithmic in the number of inputs. We show, that for the example application of handwritten digit recognition a random forest processing engine employing this majority decision architecture implemented on an FPGA allows the classification of more than seven million images per second, resulting in a speed-up factor of more than 29 compared to the fastest state-of-the-art implementation considered.

Energy efficient approximate booth multipliers using compact error compensation circuit for mitigation of truncation error

AbstractArtificial neural networks and image processing are the error tolerant applications that require massive workloads to be executed within certain power limits. Energy efficient multipliers on an embedded processor are crucial for such applications. In this paper, a highly diminished yet accurate design of an approximate truncated multiplier is presented. A compact error compensation circuit is presented that aims to mitigate the truncation error. To reduce the energy consumption further, the voltage overscaling method is used. The performance of the circuit is optimized by adjusting the two approximation knobs of circuit pruning and voltage scaling. Simulation results for a truncation factor of 8 with an MRED of 0.01 demonstrate an 11.4% reduction in energy consumption compared to that of the 16‐bit radix‐4 exact multiplier. Application based evaluation is performed by using the approximate multipliers on image multiplication application and a high value of PSNR (43.17 dB) is obtained. When evaluated using a handwritten digit recognition application based on convolutional neural networks (CNN), an accuracy of 97.85% is obtained.

A Novel Twin Support Vector Machine with Generalized Pinball Loss Function for Pattern Classification

We introduce a novel twin support vector machine with the generalized pinball loss function (GPin-TSVM) for solving data classification problems that are less sensitive to noise and preserve the sparsity of the solution. In addition, we use a symmetric kernel trick to enlarge GPin-TSVM to nonlinear classification problems. The developed approach is tested on numerous UCI benchmark datasets, as well as synthetic datasets in the experiments. The comparisons demonstrate that our proposed algorithm outperforms existing classifiers in terms of accuracy. Furthermore, this employed approach in handwritten digit recognition applications is examined, and the automatic feature extractor employs a convolution neural network.

Application of digit and speech recognition in food delivery robot

Application of digit and speech recognition in food delivery robot

Energy-Efficient All-Spin BNN Using Voltage-Controlled Spin-Orbit Torque Device for Digit Recognition

Artificial intelligence has been demonstrated for numerous applications including image recognition and processing, internet-of-things (IoT), and speech recognition. Recent neural network (NN) architectures escalating to perform these functionalities require deep-learning and deep NN (DNN) implementation. However, the present algorithm and circuit of DNN suffer from area and energy-efficiency. The usage of an energy-efficient and cost-effective binary NN (BNN) can mitigate the performance bottleneck. The basic component of BNN (the XNOR and accumulate module) is capable of replacing the conventional multiply-and-accumulate (MAC) operation. Among various emerging nonvolatile memories (NVMs), spintronics-based devices are attracting attention for the implementation of the aforementioned architectures. Multilevel voltage-controlled spin-orbit torque -based magnetic memory (MV-SOTM) spin device as synapse and voltage-controlled SOTM (V-SOTM) device as spin-neuron are capable of producing an all-spin NN. This article presents advanced XNOR operation using a computing-in-memory (CiM) mechanism wherein both V-SOTM and MV-SOTM (using series and parallel configuration) based synapses are compared. The MV-SOTM design dissipates 35.51% lesser energy as compared to the 1-bit V-SOTM. Further, the 8-bit synaptic crossbar array is implemented using both the devices with two output spin-neurons. These series and parallel MV-SOTM-based synaptic arrays consume 42.22% and 45.12% lesser power, respectively, as compared to 1-bit V-SOTM based synaptic array. Furthermore, the all-spin BNN design demonstrates a handwritten digit recognition application.

Design and Analysis of Energy-Efficient Dynamic Range Approximate Logarithmic Multipliers for Machine Learning

Approximate computing provides an emerging approach to design high performance and low power arithmetic circuits. The logarithmic multiplier (LM) converts multiplication into addition and has inherent approximate characteristics. In this article, dynamic range approximate LMs (DR-ALMs) for machine learning applications are proposed; they use Mitchell’s approximation and a dynamic range operand truncation scheme. The worst case (absolute and relative) errors for the proposed DR-ALMs are analyzed. The accuracy and the hardware overhead of these designs are provided to select the best approximate scheme according to different metrics. The proposed DR-ALMs are compared with the conventional LM with exact operands and previous approximate multipliers; the results show that the power-delay product (PDP) of the best proposed DR-ALM (DR-ALM-6) are decreased by up to 54.07 percent with the mean relative error distance (MRED) decreasing by 21.30 percent compared with 16-bit conventional design. Case studies for three machine learning applications show the viability of the proposed DR-ALMs. Compared with the exact multiplier and its conventional counterpart, the back-propagation classifier with DR-ALMs with a truncation length larger than 4 has a similar classification result for the three datasets; the K-means clustering application with all DR-ALMs has a similar clustering result for four datasets; and the handwritten digit recognition application with DR-ALM-5 or DR-ALM-6 for LeNet-5 achieves similar or even slightly higher recognition rate.

High-Quality Wavelets Features Extraction for Handwritten Arabic Numerals Recognition

Arabic handwritten digit recognition is the science of recognition and classification of handwritten Arabic digits. It has been a subject of research for many years with rich literature available on the subject. Handwritten digits written by different people are not of the same size, thickness, style, position or orientation. Hence, many different challenges have to overcome for resolving the problem of handwritten digit recognition. The variation in the digits is due to the writing styles of different people which can differ significantly. Automatic handwritten digit recognition has wide application such as automatic processing of bank cheques, postal addresses, and tax forms. A typical handwritten digit recognition application consists of three main stages namely features extraction, features selection, and classification. One of the most important problems is feature extraction. In this paper, a novel feature extraction approach for off-line handwritten digit recognition is presented. Wavelets-based analysis of image data is carried out for feature extraction, and then classification is performed using various classifiers. To further reduce the size of training data-set, high entropy subbands are selected. To increase the recognition rate, individual subbands providing high classification accuracies are selected from the over-complete tree. The features extracted are also normalized to standardize the range of independent variables before providing them to the classifier. Classification is carried out using k-NN and SVMs. The results show that the quality of extracted features is high as almost equivalently high classification accuracies are acquired for both classifiers, i.e. k-NNs and SVMs.

Proposal for a Leaky-Integrate-Fire Spiking Neuron Based on Magnetoelectric Switching of Ferromagnets

The efficiency of the human brain in performing classification tasks has attracted considerable research interest in brain-inspired neuromorphic computing. The spiking neuromorphic architectures attempt to mimic the computations performed in the brain through a dense interconnection of the neurons and synaptic weights. A leaky-integrate-fire (LIF) spiking model is widely used to emulate the dynamics of the biological neurons. In this paper, we propose a spin-based LIF spiking neuron using the magnetoelectric (ME) switching of ferromagnets. The voltage across the ME oxide exhibits a typical leaky-integrate behavior, which in turn switches an underlying ferromagnet. Due to the effect of thermal noise, the ferromagnet exhibits probabilistic switching dynamics, which is reminiscent of the stochasticity exhibited by biological neurons. The energy efficiency of the ME switching mechanism coupled with the intrinsic nonvolatility of ferromagnets results in lower energy consumption, when compared with a CMOS LIF neuron. A device to system-level simulation framework has been developed to investigate the feasibility of the proposed LIF neuron for a hand-written digit recognition application.

ASP: Learning to Forget With Adaptive Synaptic Plasticity in Spiking Neural Networks

A fundamental feature of learning in animals is the "ability to forget" that allows an organism to perceive, model and make decisions from disparate streams of information and adapt to changing environments. Against this backdrop, we present a novel unsupervised learning mechanism ASP (Adaptive Synaptic Plasticity) for improved recognition with Spiking Neural Networks (SNNs) for real time on-line learning in a dynamic environment. We incorporate an adaptive weight decay mechanism with the traditional Spike Timing Dependent Plasticity (STDP) learning to model adaptivity in SNNs. The leak rate of the synaptic weights is modulated based on the temporal correlation between the spiking patterns of the pre- and post-synaptic neurons. This mechanism helps in gradual forgetting of insignificant data while retaining significant, yet old, information. ASP, thus, maintains a balance between forgetting and immediate learning to construct a stable-plastic self-adaptive SNN for continuously changing inputs. We demonstrate that the proposed learning methodology addresses catastrophic forgetting while yielding significantly improved accuracy over the conventional STDP learning method for digit recognition applications. Additionally, we observe that the proposed learning model automatically encodes selective attention towards relevant features in the input data while eliminating the influence of background noise (or denoising) further improving the robustness of the ASP learning.

Energy efficient parallel neuromorphic architectures with approximate arithmetic on FPGA

In this paper, we present the parallel neuromorphic processor architectures for spiking neural networks on FPGA. The proposed architectures address several critical issues pertaining to efficient parallelization of the update of membrane potentials, on-chip storage of synaptic weights and integration of approximate arithmetic units. The trade-offs between throughput, hardware cost and power overheads for different configurations are thoroughly investigated. Notably, for the application of handwritten digit recognition, a promising training speedup of 13.5x and a recognition speedup of 25.8x are achieved by a parallel implementation whose degree of parallelism is 32. In spite of the 120MHz operating frequency, the 32-way parallel hardware design demonstrates a 59.4x training speedup over the single-thread software program running on a 2.2GHz general purpose CPU. Equally importantly, by leveraging the built-in resilience of the neuromorphic architecture we demonstrate the energy benefit resulted from the use of approximate arithmetic computation. Up to 20% improvement in energy consumption is achieved by integrating approximate multipliers into the system while maintaining almost the same level of recognition rate achieved using standard multipliers. To the best of our knowledge, it is the first time that the approximate computing and parallel processing are applied to FPGA based spiking neural networks. The influence of the parallel processing on the benefits of approximate computing is also discussed in detail.

Implementing Multilayer Neural Network Behavior Using Polychronous Wavefront Computation

Polychronous Wavefront Computation (PWC) is an abstraction of spiking neural networks that provides a potentially practical model for implementing neuromorphic computing systems. While it's has been shown to exhibit some basic computational capabilities, its use in complex neuro-computational models remains to be explored. The paper presents a model and approach for configuring PWC transponders to implement multilayer neural network behavior to provide a basis for more complex applications of the technology. The model uses a set of input transponders representing pattern features to stimulate hidden layer transponders that combine features and trigger output layer transponders to identify patterns. The input layer transponder geometry is selected to create wavefront intersections for all relevant feature combinations. Hidden layer transponders are positioned by solving the intersection of the circles equations defined by sets of input transponders. Output layer transponders are defined to collect complete sets of features for recognition based on the hidden layer transponder geometry. The approach uses the intersections of three wavefronts to maximize transponder selectivity and increase information density. The concept is experimentally demonstrated and analyzed with a 7-segment display digit recognition application which provides a simple but representative example of more complex pattern recognition problems.

RECOGNITION OF HANDWRITTEN DIGITS USING RBF NEURAL NETWORK

Pattern recognition is required in many fields for different purposes. Methods based on Radial basis function (RBF) neural networks are found to be very successful in pattern classification problems. Training neural network is in general a challenging nonlinear optimization problem. Several algorithms have been proposed for choosing the RBF neural network prototypes and training the network. In this paper RBF neural network using decoupling Kalman filter method is proposed for handwritten digit recognition applications. The efficacy of the proposed method is tested on the handwritten digits of different fonts and found that it is successful in recognizing the digits.

K-means clustering algorithm for multimedia applications with flexible HW/SW co-design

In this paper, we report a hardware/software (HW/SW) co-designed K-means clustering algorithm with high flexibility and high performance for machine learning, pattern recognition and multimedia applications. The contributions of this work can be attributed to two aspects. The first is the hardware architecture for nearest neighbor searching, which is used to overcome the main computational cost of a K-means clustering algorithm. The second aspect is the high flexibility for different applications which comes from not only the software but also the hardware. High flexibility with respect to the number of training data samples, the dimensionality of each sample vector, the number of clusters, and the target application, is one of the major shortcomings of dedicated hardware implementations for the K-means algorithm. In particular, the HW/SW K-means algorithm is extendable to embedded systems and mobile devices. We benchmark our multi-purpose K-means system against the application of handwritten digit recognition, face recognition and image segmentation to demonstrate its excellent performance, high flexibility, fast clustering speed, short recognition time, good recognition rate and versatile functionality.

An Accelerometer-Based Digital Pen With a Trajectory Recognition Algorithm for Handwritten Digit and Gesture Recognition

This paper presents an accelerometer-based digital pen for handwritten digit and gesture trajectory recognition applications. The digital pen consists of a triaxial accelerometer, a microcontroller, and an RF wireless transmission module for sensing and collecting accelerations of handwriting and gesture trajectories. The proposed trajectory recognition algorithm composes of the procedures of acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction. The algorithm is capable of translating time-series acceleration signals into important feature vectors. Users can use the pen to write digits or make hand gestures, and the accelerations of hand motions measured by the accelerometer are wirelessly transmitted to a computer for online trajectory recognition. The algorithm first extracts the time- and frequency-domain features from the acceleration signals and, then, further identifies the most important features by a hybrid method: kernel-based class separability for selecting significant features and linear discriminant analysis for reducing the dimension of features. The reduced features are sent to a trained probabilistic neural network for recognition. Our experimental results have successfully validated the effectiveness of the trajectory recognition algorithm for handwritten digit and gesture recognition using the proposed digital pen.

A genetic algorithm based region sampling for selection of local features in handwritten digit recognition application

Identification of local regions from where optimal discriminating features can be extracted is one of the major tasks in the area of pattern recognition. To locate such regions different kind of region sampling techniques are used in the literature. There is no standard methodology to identify exactly such regions. Here we have proposed a methodology where local regions of varying heights and widths are created dynamically. Genetic algorithm (GA) is then applied on these local regions to sample the optimal set of local regions from where an optimal feature set can be extracted that has the best discriminating features. We have evaluated the proposed methodology on a data set of handwritten Bangla digits. In the present work, we have randomly generated seven sets of local regions and from every set, GA selects an optimal group of local regions which produces best recognition performance with a support vector machine (SVM) based classifier. Other popular optimization techniques like simulated annealing (SA) and hill climbing (HC) have also been evaluated with the same data set and maximum recognition accuracies were found to be 97%, 96.7% and 96.7% for GA, SA and HC, respectively. We have also compared the performance of the present technique with those of other zone based techniques on the same database.

An Inertial-Measurement-Unit-Based Pen With a Trajectory Reconstruction Algorithm and Its Applications

This paper presents an inertial-measurement-unit-based pen (IMUPEN) and its associated trajectory reconstruction algorithm for motion trajectory reconstruction and handwritten digit recognition applications. The IMUPEN is composed of a triaxial accelerometer, two gyroscopes, a microcontroller, and an RF wireless transmission module. Users can hold the IMUPEN to write numerals or draw simple symbols at normal speed. During writing or drawing movements, the inertial signals generated for the movements are transmitted to a computer via the wireless module. A trajectory reconstruction algorithm composed of the procedures of data collection, signal preprocessing, and trajectory reconstruction has been developed for reconstructing the trajectories of movements. In order to minimize the cumulative errors caused by the intrinsic noise/drift of sensors, we have developed an orientation error compensation method and a multiaxis dynamic switch. The advantages of the IMUPEN include the following: 1) It is portable and can be used anywhere without any external reference device or writing ambit limitations, and 2) its trajectory reconstruction algorithm can reduce orientation and integral errors effectively and thus can reconstruct the trajectories of movements accurately. Our experimental results on motion trajectory reconstruction and handwritten digit recognition have successfully validated the effectiveness of the IMUPEN and its trajectory reconstruction algorithm.

Genetic optimization of GRNN for pattern recognition without feature extraction

This paper describes an approach for pattern recognition using genetic algorithm and general regression neural network (GRNN). The designed system can be used for both 3D object recognition from 2D poses of the object and handwritten digit recognition applications. The system does not require any preprocessing and feature extraction stage before the recognition. In GRNN, placement of centers has significant effect on the performance of the network. The centers and widths of the hidden layer neuron basis functions are coded in a chromosome and these two critical parameters are determined by the optimization using genetic algorithms. Experimental results show that the optimized GRNN provides higher recognition ability compared with that of unoptimized GRNN.