Handwritten Digit Recognition Application Research Articles

Pulse-stream impact on recognition accuracy of reservoir computing from SiO2-based low power memory devices

Reservoir computing (RC)-based neuromorphic applications exhibit extremely low power consumption, thus challenging the use of deep neural networks in terms of both consumption requirements and integration density. Under this perspective, this work focuses on the basic principles of RC systems. The ability of self-selective conductive-bridging random access memory devices to operate in two modes, namely, volatile and non-volatile, by regulating the applied voltage is first presented. We then investigate the relaxation time of these devices as a function of the applied amplitude and pulse duration, a critical step in determining the desired non-linearity by the reservoir. Moreover, we present an in-depth study of the impact of selecting the appropriate pulse-stream and its final effects on the total power consumption and recognition accuracy in a handwritten digit recognition application from the National Institute of Standards and Technology dataset. Finally, we conclude at the optimal pulse-stream of 3-bit, through the minimization of two cost criteria, with the total power remaining at 287 µW and simultaneously achieving 82.58% recognition accuracy upon the test set.

Energy efficient approximate booth multipliers using compact error compensation circuit for mitigation of truncation error

AbstractArtificial neural networks and image processing are the error tolerant applications that require massive workloads to be executed within certain power limits. Energy efficient multipliers on an embedded processor are crucial for such applications. In this paper, a highly diminished yet accurate design of an approximate truncated multiplier is presented. A compact error compensation circuit is presented that aims to mitigate the truncation error. To reduce the energy consumption further, the voltage overscaling method is used. The performance of the circuit is optimized by adjusting the two approximation knobs of circuit pruning and voltage scaling. Simulation results for a truncation factor of 8 with an MRED of 0.01 demonstrate an 11.4% reduction in energy consumption compared to that of the 16‐bit radix‐4 exact multiplier. Application based evaluation is performed by using the approximate multipliers on image multiplication application and a high value of PSNR (43.17 dB) is obtained. When evaluated using a handwritten digit recognition application based on convolutional neural networks (CNN), an accuracy of 97.85% is obtained.

A Novel Twin Support Vector Machine with Generalized Pinball Loss Function for Pattern Classification

We introduce a novel twin support vector machine with the generalized pinball loss function (GPin-TSVM) for solving data classification problems that are less sensitive to noise and preserve the sparsity of the solution. In addition, we use a symmetric kernel trick to enlarge GPin-TSVM to nonlinear classification problems. The developed approach is tested on numerous UCI benchmark datasets, as well as synthetic datasets in the experiments. The comparisons demonstrate that our proposed algorithm outperforms existing classifiers in terms of accuracy. Furthermore, this employed approach in handwritten digit recognition applications is examined, and the automatic feature extractor employs a convolution neural network.

Energy-Efficient All-Spin BNN Using Voltage-Controlled Spin-Orbit Torque Device for Digit Recognition

Artificial intelligence has been demonstrated for numerous applications including image recognition and processing, internet-of-things (IoT), and speech recognition. Recent neural network (NN) architectures escalating to perform these functionalities require deep-learning and deep NN (DNN) implementation. However, the present algorithm and circuit of DNN suffer from area and energy-efficiency. The usage of an energy-efficient and cost-effective binary NN (BNN) can mitigate the performance bottleneck. The basic component of BNN (the XNOR and accumulate module) is capable of replacing the conventional multiply-and-accumulate (MAC) operation. Among various emerging nonvolatile memories (NVMs), spintronics-based devices are attracting attention for the implementation of the aforementioned architectures. Multilevel voltage-controlled spin-orbit torque -based magnetic memory (MV-SOTM) spin device as synapse and voltage-controlled SOTM (V-SOTM) device as spin-neuron are capable of producing an all-spin NN. This article presents advanced XNOR operation using a computing-in-memory (CiM) mechanism wherein both V-SOTM and MV-SOTM (using series and parallel configuration) based synapses are compared. The MV-SOTM design dissipates 35.51% lesser energy as compared to the 1-bit V-SOTM. Further, the 8-bit synaptic crossbar array is implemented using both the devices with two output spin-neurons. These series and parallel MV-SOTM-based synaptic arrays consume 42.22% and 45.12% lesser power, respectively, as compared to 1-bit V-SOTM based synaptic array. Furthermore, the all-spin BNN design demonstrates a handwritten digit recognition application.

Design and Analysis of Energy-Efficient Dynamic Range Approximate Logarithmic Multipliers for Machine Learning

Approximate computing provides an emerging approach to design high performance and low power arithmetic circuits. The logarithmic multiplier (LM) converts multiplication into addition and has inherent approximate characteristics. In this article, dynamic range approximate LMs (DR-ALMs) for machine learning applications are proposed; they use Mitchell’s approximation and a dynamic range operand truncation scheme. The worst case (absolute and relative) errors for the proposed DR-ALMs are analyzed. The accuracy and the hardware overhead of these designs are provided to select the best approximate scheme according to different metrics. The proposed DR-ALMs are compared with the conventional LM with exact operands and previous approximate multipliers; the results show that the power-delay product (PDP) of the best proposed DR-ALM (DR-ALM-6) are decreased by up to 54.07 percent with the mean relative error distance (MRED) decreasing by 21.30 percent compared with 16-bit conventional design. Case studies for three machine learning applications show the viability of the proposed DR-ALMs. Compared with the exact multiplier and its conventional counterpart, the back-propagation classifier with DR-ALMs with a truncation length larger than 4 has a similar classification result for the three datasets; the K-means clustering application with all DR-ALMs has a similar clustering result for four datasets; and the handwritten digit recognition application with DR-ALM-5 or DR-ALM-6 for LeNet-5 achieves similar or even slightly higher recognition rate.

High-Quality Wavelets Features Extraction for Handwritten Arabic Numerals Recognition

Arabic handwritten digit recognition is the science of recognition and classification of handwritten Arabic digits. It has been a subject of research for many years with rich literature available on the subject. Handwritten digits written by different people are not of the same size, thickness, style, position or orientation. Hence, many different challenges have to overcome for resolving the problem of handwritten digit recognition. The variation in the digits is due to the writing styles of different people which can differ significantly. Automatic handwritten digit recognition has wide application such as automatic processing of bank cheques, postal addresses, and tax forms. A typical handwritten digit recognition application consists of three main stages namely features extraction, features selection, and classification. One of the most important problems is feature extraction. In this paper, a novel feature extraction approach for off-line handwritten digit recognition is presented. Wavelets-based analysis of image data is carried out for feature extraction, and then classification is performed using various classifiers. To further reduce the size of training data-set, high entropy subbands are selected. To increase the recognition rate, individual subbands providing high classification accuracies are selected from the over-complete tree. The features extracted are also normalized to standardize the range of independent variables before providing them to the classifier. Classification is carried out using k-NN and SVMs. The results show that the quality of extracted features is high as almost equivalently high classification accuracies are acquired for both classifiers, i.e. k-NNs and SVMs.

Mimicking Leaky-Integrate-Fire Spiking Neuron Using Automotion of Domain Walls for Energy-Efficient Brain-Inspired Computing

Although an average human brain might not be able to compete with modern day computers in performing arithmetic operations, when it comes to recognition and classification tasks, biological systems are clear winners in terms of performance and energy efficiency. Building blocks of all such biological systems are neurons and synapses . In order to exploit the benefits of such systems, novel devices are being explored to mimic the behavior of neurons and synapses. We propose a leaky-integrate-fire (LIF) neuron using the physics of automotion in magnetic domain walls (DWs). Due to the shape anisotropy in a high-aspect ratio magnet, DW has a tendency to move automatically, without any external driving force. This property can be exploited to mimic the realistic dynamics of spiking neurons, without any extra energy penalty. We analyze the dynamics of a DW under automotion and show that it can be approximated to mimic the LIF neuronal dynamics. We propose a compact, energy-efficient magnetic neuron, which can directly be cascaded to memristive crossbar array of synapses, thereby evading additional interfacing circuitry. Furthermore, we develop a device-to-system-level behavioral model to underscore the applicability of the proposal in a typical handwritten-digit recognition application.

Proposal for a Leaky-Integrate-Fire Spiking Neuron Based on Magnetoelectric Switching of Ferromagnets

The efficiency of the human brain in performing classification tasks has attracted considerable research interest in brain-inspired neuromorphic computing. The spiking neuromorphic architectures attempt to mimic the computations performed in the brain through a dense interconnection of the neurons and synaptic weights. A leaky-integrate-fire (LIF) spiking model is widely used to emulate the dynamics of the biological neurons. In this paper, we propose a spin-based LIF spiking neuron using the magnetoelectric (ME) switching of ferromagnets. The voltage across the ME oxide exhibits a typical leaky-integrate behavior, which in turn switches an underlying ferromagnet. Due to the effect of thermal noise, the ferromagnet exhibits probabilistic switching dynamics, which is reminiscent of the stochasticity exhibited by biological neurons. The energy efficiency of the ME switching mechanism coupled with the intrinsic nonvolatility of ferromagnets results in lower energy consumption, when compared with a CMOS LIF neuron. A device to system-level simulation framework has been developed to investigate the feasibility of the proposed LIF neuron for a hand-written digit recognition application.

RECOGNITION OF HANDWRITTEN DIGITS USING RBF NEURAL NETWORK

Pattern recognition is required in many fields for different purposes. Methods based on Radial basis function (RBF) neural networks are found to be very successful in pattern classification problems. Training neural network is in general a challenging nonlinear optimization problem. Several algorithms have been proposed for choosing the RBF neural network prototypes and training the network. In this paper RBF neural network using decoupling Kalman filter method is proposed for handwritten digit recognition applications. The efficacy of the proposed method is tested on the handwritten digits of different fonts and found that it is successful in recognizing the digits.

An Accelerometer-Based Digital Pen With a Trajectory Recognition Algorithm for Handwritten Digit and Gesture Recognition

This paper presents an accelerometer-based digital pen for handwritten digit and gesture trajectory recognition applications. The digital pen consists of a triaxial accelerometer, a microcontroller, and an RF wireless transmission module for sensing and collecting accelerations of handwriting and gesture trajectories. The proposed trajectory recognition algorithm composes of the procedures of acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction. The algorithm is capable of translating time-series acceleration signals into important feature vectors. Users can use the pen to write digits or make hand gestures, and the accelerations of hand motions measured by the accelerometer are wirelessly transmitted to a computer for online trajectory recognition. The algorithm first extracts the time- and frequency-domain features from the acceleration signals and, then, further identifies the most important features by a hybrid method: kernel-based class separability for selecting significant features and linear discriminant analysis for reducing the dimension of features. The reduced features are sent to a trained probabilistic neural network for recognition. Our experimental results have successfully validated the effectiveness of the trajectory recognition algorithm for handwritten digit and gesture recognition using the proposed digital pen.

A genetic algorithm based region sampling for selection of local features in handwritten digit recognition application

Identification of local regions from where optimal discriminating features can be extracted is one of the major tasks in the area of pattern recognition. To locate such regions different kind of region sampling techniques are used in the literature. There is no standard methodology to identify exactly such regions. Here we have proposed a methodology where local regions of varying heights and widths are created dynamically. Genetic algorithm (GA) is then applied on these local regions to sample the optimal set of local regions from where an optimal feature set can be extracted that has the best discriminating features. We have evaluated the proposed methodology on a data set of handwritten Bangla digits. In the present work, we have randomly generated seven sets of local regions and from every set, GA selects an optimal group of local regions which produces best recognition performance with a support vector machine (SVM) based classifier. Other popular optimization techniques like simulated annealing (SA) and hill climbing (HC) have also been evaluated with the same data set and maximum recognition accuracies were found to be 97%, 96.7% and 96.7% for GA, SA and HC, respectively. We have also compared the performance of the present technique with those of other zone based techniques on the same database.

An Inertial-Measurement-Unit-Based Pen With a Trajectory Reconstruction Algorithm and Its Applications

This paper presents an inertial-measurement-unit-based pen (IMUPEN) and its associated trajectory reconstruction algorithm for motion trajectory reconstruction and handwritten digit recognition applications. The IMUPEN is composed of a triaxial accelerometer, two gyroscopes, a microcontroller, and an RF wireless transmission module. Users can hold the IMUPEN to write numerals or draw simple symbols at normal speed. During writing or drawing movements, the inertial signals generated for the movements are transmitted to a computer via the wireless module. A trajectory reconstruction algorithm composed of the procedures of data collection, signal preprocessing, and trajectory reconstruction has been developed for reconstructing the trajectories of movements. In order to minimize the cumulative errors caused by the intrinsic noise/drift of sensors, we have developed an orientation error compensation method and a multiaxis dynamic switch. The advantages of the IMUPEN include the following: 1) It is portable and can be used anywhere without any external reference device or writing ambit limitations, and 2) its trajectory reconstruction algorithm can reduce orientation and integral errors effectively and thus can reconstruct the trajectories of movements accurately. Our experimental results on motion trajectory reconstruction and handwritten digit recognition have successfully validated the effectiveness of the IMUPEN and its trajectory reconstruction algorithm.

Genetic optimization of GRNN for pattern recognition without feature extraction

This paper describes an approach for pattern recognition using genetic algorithm and general regression neural network (GRNN). The designed system can be used for both 3D object recognition from 2D poses of the object and handwritten digit recognition applications. The system does not require any preprocessing and feature extraction stage before the recognition. In GRNN, placement of centers has significant effect on the performance of the network. The centers and widths of the hidden layer neuron basis functions are coded in a chromosome and these two critical parameters are determined by the optimization using genetic algorithms. Experimental results show that the optimized GRNN provides higher recognition ability compared with that of unoptimized GRNN.

Comparison of multilayer neural network and Nearest Neighbor Classifiers for handwritten digit recognition.

The basic Nearest Neighbor Classifier (NNC) is often inefficient for classification in terms of memory space and computing time needed if all training samples are used as prototypes. These problems can be solved by reducing the number of prototypes using clustering algorithms and optimizing the prototypes using a special neural network model. In this paper, we compare the performance of the multilayer neural network and an Optimized Nearest Neighbor Classifier (ONNC) for handwritten digit recognition applications. We show that an ONNC can have the same recognition performance as an equivalent neural network classifier. The ONNC can be efficiently implemented using prototype and variable ranking, partial summation and distance triangular inequality based strategies. It requires the same memory space as, but less, training time and classification time than the neural network.