Handwritten Digit Recognition Research Articles

Handwritten Digit Recognition System based on CNN and SVM

The recognition of handwritten digits has aroused the interest of the scientific community and is the subject of a large number of research works thanks to its various applications. The objective of this paper is to develop a system capable of recognizing handwritten digits using a convolutional neural network (CNN) combined with machine learning approaches to ensure diversity in automatic classification tools. In this work, we propose a classification method based on deep learning, in particular the convolutional neural network for feature extraction, it is a powerful tool that has had great success in image classification, followed by the support vector machine (SVM) for higher performance. We used the dataset (MNIST), and the results obtained showed that the combination of CNN with SVM improves the performance of the model as well as the classification accuracy with a rate of 99.12%.

Learning From a Complementary-Label Source Domain: Theory and Algorithms.

In unsupervised domain adaptation (UDA), a classifier for the target domain is trained with massive true-label data from the source domain and unlabeled data from the target domain. However, collecting true-label data in the source domain can be expensive and sometimes impractical. Compared to the true label (TL), a complementary label (CL) specifies a class that a pattern does not belong to, and hence, collecting CLs would be less laborious than collecting TLs. In this article, we propose a novel setting where the source domain is composed of complementary-label data, and a theoretical bound of this setting is provided. We consider two cases of this setting: one is that the source domain only contains complementary-label data [completely complementary UDA (CC-UDA)] and the other is that the source domain has plenty of complementary-label data and a small amount of true-label data [partly complementary UDA (PC-UDA)]. To this end, a complementary label adversarial network (CLARINET) is proposed to solve CC-UDA and PC-UDA problems. CLARINET maintains two deep networks simultaneously, with one focusing on classifying the complementary-label source data and the other taking care of the source-to-target distributional adaptation. Experiments show that CLARINET significantly outperforms a series of competent baselines on handwritten digit-recognition and object-recognition tasks.

Image recognition based on optical reservoir computing.

We propose an image recognition approach using a single physical node based optical reservoir computing. Specifically, an optically injected semiconductor laser with self-delayed feedback is used as the reservoir. We perform a handwritten-digit recognition task by greatly increasing the number of virtual nodes in delayed feedback using outputs from multiple delay times. Final simulation results confirm that the recognition accuracy can reach 99% after systematically optimizing the reservoir hyperparameters. Due to its simple architecture, this scheme may provide a resource-efficient alternative approach to image recognition.

Secure communication and implementation of handwritten digit recognition using deep neural network

Secure communication and implementation of handwritten digit recognition using deep neural network

Handwritten Geez Digit Recognition Using Deep Learning

Amharic language is the second most spoken language in the Semitic family after Arabic. In Ethiopia and neighboring countries more than 100 million people speak the Amharic language. There are many historical documents that are written using the Geez script. Digitizing historical handwritten documents and recognizing handwritten characters is essential to preserving valuable documents. Handwritten digit recognition is one of the tasks of digitizing handwritten documents from different sources. Currently, handwritten Geez digit recognition researches are very few, and there is no available organized dataset for the public researchers. Convolutional neural network (CNN) is preferable for pattern recognition like in handwritten document recognition by extracting a feature from different styles of writing. In this work, the proposed model is to recognize Geez digits using CNN. Deep neural networks, which have recently shown exceptional performance in numerous pattern recognition and machine learning applications, are used to recognize handwritten Geez digits, but this has not been attempted for Ethiopic scripts. Our dataset, which contains 51,952 images of handwritten Geez digits collected from 524 individuals, is used to train and evaluate the CNN model. The application of the CNN improves the performance of several machine-learning classification methods significantly. Our proposed CNN model has an accuracy of 96.21% and a loss of 0.2013. In comparison to earlier research works on Geez handwritten digit recognition, the study was able to attain higher recognition accuracy using the developed CNN model.

Threshold center-symmetric local binary convolutional neural networks for bilingual handwritten digit recognition

The writing style of the same writer varies from instance to instance in Arabic and English handwritten digit recognition, making handwritten digit recognition challenging. Currently, deep learning approaches are applied in many applications, including convolutional neural networks (CNNs) modified to produce other models, such as local binary convolutional neural networks (LBCNNs). An LBCNN is created by fusing a local binary pattern (LBP) with a CNN by reformulating the LBP as a convolution layer called a local binary convolution (LBC). However, LBCNNs suffer from the random assignment of 1, 0, or -1 to LBC weights, making LBCNNs less robust. Nevertheless, using another LBP-based technique, such as center-symmetric local binary patterns (CS-LBPs), can address such issues. In this paper, a new model based on CS-LBPs is proposed called center-symmetric local binary convolutional neural networks (CS-LBCNN), which addresses the issues of LBCNNs. Furthermore, an enhanced version of CS-LBCNNs called threshold center-symmetric local binary convolutional neural networks (TCS-LBCNNs) is proposed, which addresses another issue related to the zero-thresholding function. Finally, the proposed models are compared to state-of-the-art models, proving their ability by producing a more accurate and significant classification rate than the existing LBCNN models. For the bilingual dataset, the TCS-LBCNN enhances the accuracy of the LBCNN and CS-LBCNN by 0.15% and 0.03%, respectively. In addition, the comparison shows that the accuracy acquired by the TCS-LBCNN is the second-highest using the MNIST and MADBase datasets.

Optimize performance of a diffractive neural network by controlling the Fresnel number

To achieve better performance of a diffractive deep neural network, increasing its spatial complexity (neurons and layers) is commonly used. Subject to physical laws of optical diffraction, a deeper diffractive neural network (DNN) would be more difficult to implement, and the development of DNN is limited. In this work, we found controlling the Fresnel number can increase DNN’s capability of expression and its spatial complexity is even less. DNN with only one phase modulation layer was proposed and experimentally realized at 515 nm. With the optimal Fresnel number, the single-layer DNN reached a maximum accuracy of 97.08% in the handwritten digits recognition task.

Bio-synapse behavior controlled by interface engineering in ferroelectric tunnel memristors

The interface engineering is always an important way to modulate the behavior of the ferroelectric tunnel memristor (FTM), which directly affects its biological synaptic properties. Here, to investigate the effect of interface on bio-synapse performance, FTMs with the structure of Pt/ BaTiO3/La0.67Sr0.33MnO3 were studied, of which the interfaces of the device can be controlled through tailoring the termination of the SrTiO3 substrate and the unit-cell of BaTiO3 thin film by its growth mode. FTMs with different interfaces present opposite resistive switching behaviors due to the contrary polarization directions and the different band alignments in the BaTiO3 film. More importantly, the synaptic learning properties of FTMs can also be tuned by tailoring the interface. FTMs with different interface terminations could modulate different properties of long-term potentiation (LTP), long-term depression (LTD), spike-timing-dependent plasticity (STDP), and paired-pulse facilitation (PPF). Based on the synaptic behaviors of these two interface-engineered FTMs, the artificial neural network (ANN) system could be constructed to complete a handwritten digital image recognition process, and the accuracies of both are close to 90%. Our results provide a useful reference for tuning the functionalities of memristors through nanoscale interface engineering.

Voltage-dependent synaptic plasticity: Unsupervised probabilistic Hebbian plasticity rule based on neurons membrane potential

This study proposes voltage-dependent-synaptic plasticity (VDSP), a novel brain-inspired unsupervised local learning rule for the online implementation of Hebb's plasticity mechanism on neuromorphic hardware. The proposed VDSP learning rule updates the synaptic conductance on the spike of the postsynaptic neuron only, which reduces by a factor of two the number of updates with respect to standard spike timing dependent plasticity (STDP). This update is dependent on the membrane potential of the presynaptic neuron, which is readily available as part of neuron implementation and hence does not require additional memory for storage. Moreover, the update is also regularized on synaptic weight and prevents explosion or vanishing of weights on repeated stimulation. Rigorous mathematical analysis is performed to draw an equivalence between VDSP and STDP. To validate the system-level performance of VDSP, we train a single-layer spiking neural network (SNN) for the recognition of handwritten digits. We report 85.01 ± 0.76% (Mean ± SD) accuracy for a network of 100 output neurons on the MNIST dataset. The performance improves when scaling the network size (89.93 ± 0.41% for 400 output neurons, 90.56 ± 0.27 for 500 neurons), which validates the applicability of the proposed learning rule for spatial pattern recognition tasks. Future work will consider more complicated tasks. Interestingly, the learning rule better adapts than STDP to the frequency of input signal and does not require hand-tuning of hyperparameters.

Reconfigurable nonlinear photonic activation function for photonic neural network based on non-volatile opto-resistive RAM switch

Photonic neural network has been sought as an alternative solution to surpass the efficiency and speed bottlenecks of electronic neural network. Despite that the integrated Mach–Zehnder Interferometer (MZI) mesh can perform vector-matrix multiplication in photonic neural network, a programmable in-situ nonlinear activation function has not been proposed to date, suppressing further advancement of photonic neural network. Here, we demonstrate an efficient in-situ nonlinear accelerator comprising a unique solution-processed two-dimensional (2D) MoS2 Opto-Resistive RAM Switch (ORS), which exhibits tunable nonlinear resistance switching that allow us to introduce nonlinearity to the photonic neuron which overcomes the linear voltage-power relationship of typical photonic components. Our reconfigurable scheme enables implementation of a wide variety of nonlinear responses. Furthermore, we confirm its feasibility and capability for MNIST handwritten digit recognition, achieving a high accuracy of 91.6%. Our accelerator constitutes a major step towards the realization of in-situ photonic neural network and pave the way for the integration of photonic integrated circuits (PIC).

Pattern recognition with neuromorphic computing using magnetic field-induced dynamics of skyrmions.

Nonlinear phenomena in physical systems can be used for brain-inspired computing with low energy consumption. Response from the dynamics of a topological spin structure called skyrmion is one of the candidates for such a neuromorphic computing. However, its ability has not been well explored experimentally. Here, we experimentally demonstrate neuromorphic computing using nonlinear response originating from magnetic field–induced dynamics of skyrmions. We designed a simple-structured skyrmion-based neuromorphic device and succeeded in handwritten digit recognition with the accuracy as large as 94.7% and waveform recognition. Notably, there exists a positive correlation between the recognition accuracy and the number of skyrmions in the devices. The large degrees of freedom of skyrmion systems, such as the position and the size, originate from the more complex nonlinear mapping, the larger output dimension, and, thus, high accuracy. Our results provide a guideline for developing energy-saving and high-performance skyrmion neuromorphic computing devices.

Digit Recognition Using MNIST Dataset

Abstract: In this paper, we have performed handwritten digit recognition using MNIST datasets using Support Vector Machines (SVM), Multi-Layer Perceptron (MLP) and Convolution Neural Network (CNN) models. Our main goal is to compare the accuracy of the above models along with their execution time to obtain the best possible model for digit recognition. Reliability of humans over machines has never been so high that from classifying objects in photographs to adding sound to silent movies can all be done using deep learning and machine learning algorithms. Similarly, handwriting recognition is one of the important areas research and development with a range of possibilities that could be achieved. Handwriting recognition (HWR), also known as handwritten text recognition (HTR), is a capability computers to receive and interpret comprehensible handwritten input from sources such as paper documents, photos, touch screen

Automatic Handwritten Digit Recognition Model

The Human Visual System is a marvel of the world. People can readily recognise digits. But it is not as simple as it seems. The human brain has a million neurons and billions of connections between them, which makes this exceptionally complex task of image processing easier. People can effortlessly recognize digits. However, it turns into a challenging task for computers to recognize digits. Simple hunches about how to recognize digits become difficult to express algorithmically. Moreover, there is a significant variation in writing from person to person, which makes it immensely complex. The MNIST digit recognition system is the working of a machine to train itself so that it can recognize digits from different sources like emails, bank cheques, papers, images, etc. Convolutional Neural Networks (CNNs) are very effective in perceiving the structure of handwritten characters/words in ways that help in automatic extraction of distinct features and make CNN the most suitable approach for solving handwriting recognition problems. Our aim in the proposed work is to explore the various design options like number of layers, stride size, kernel size for CNN-based handwritten digit recognition. Our objective is to achieve comparable accuracy by using a pure CNN architecture without ensemble architecture, as ensemble architectures introduce increased computational cost and high testing complexity. Thus, a CNN architecture is proposed in order to achieve accuracy, along with reduced operational complexity and cost.

Compact Modeling of Nanocluster Functionality as a Higher-Order Neuron

Disordered nanoclusters with multielectrode input–output functionality had recently been experimentally realized with energy-efficient and emergent computational capacity, and thus an interconnected network of several such nanoclusters had been proposed to realize artificial neural networks. To aid that end, here we show that nanocluster functionality can be fit to the simplest dendritic neuron model (DNM), where the only form of nonlinearity is due to multiplicative interactions. This work brings into the spotlight higher-order neural networks (known for their efficient encoding of geometric invariances) to serve as an explainable baseline model of nano-networks against which experimentalists can compare more sophisticated models (deep neural networks or physics-based models such as the lin-min network introduced here) and provides ground for designing novel approximate hardware and a statistical mechanics analysis of the learning performance of interconnected nanoclusters versus perceptrons (where neurons output a nonlinear function of the weighted sum of their inputs). A network with just ten higher-order neurons is shown to achieve a classification accuracy of more than 96% on the MNIST benchmark for handwritten digit recognition (which required 100 times more neurons in three-layer perceptrons).

Improved artificial synapse performance of Pt/HfO2/BiFeO3/HfO2/TiN memristor through N2 annealing

A high-performance synaptic plasticity memristor based on Pt/HfO2/BiFeO3/HfO2/TiN structure was developed by magnetron sputtering technology, and the effects of N2 annealing treatment on its resistance and synaptic bionic properties were studied. The results show that with the N2 post annealing process, the memristor can exhibits excellent switching windows, outstanding uniformity and high durability. An improvement of ON/OFF resistance ratio of at least two orders of magnitude is achieved, and the pulse endurance can remain stable up to 106 s. Moreover, the basic synaptic biomimetic function is realized due to its excellent multi-level storage properties, and the conductivity of the memristor is modulated by multilayer perception (MLP) algorithm to simulate handwritten digital image recognition. Overall, its linearity and stability of the artificial synapse in the regulation process still have the space to be improved for the application, so it can be efficiently regulated by rapid thermal processing. It is convinced the tri-layer post-annealing HfO2-based devices hold the application potential of neuromorphic computing in the next generation memory devices.

Learning the Precise Feature for Cluster Assignment.

Clustering is one of the fundamental tasks in computer vision and pattern recognition. Recently, deep clustering methods (algorithms based on deep learning) have attracted wide attention with their impressive performance. Most of these algorithms combine deep unsupervised representation learning and standard clustering together. However, the separation of representation learning and clustering will lead to suboptimal solutions because the two-stage strategy prevents representation learning from adapting to subsequent tasks (e.g., clustering according to specific cues). To overcome this issue, efforts have been made in the dynamic adaption of representation and cluster assignment, whereas current state-of-the-art methods suffer from heuristically constructed objectives with the representation and cluster assignment alternatively optimized. To further standardize the clustering problem, we audaciously formulate the objective of clustering as finding a precise feature as the cue for cluster assignment. Based on this, we propose a general-purpose deep clustering framework, which radically integrates representation learning and clustering into a single pipeline for the first time. The proposed framework exploits the powerful ability of recently developed generative models for learning intrinsic features, and imposes an entropy minimization on the distribution of the cluster assignment by a dedicated variational algorithm. The experimental results show that the performance of the proposed method is superior, or at least comparable to, the state-of-the-art methods on the handwritten digit recognition, fashion recognition, face recognition, and object recognition benchmark datasets.

Multi-Stimuli-Responsive Synapse Based on Vertical van der Waals Heterostructures.

Brain-inspired intelligent systems demand diverse neuromorphic devices beyond simple functionalities. Merging biomimetic sensing with weight-updating capabilities in artificial synaptic devices represents one of the key research focuses. Here, we report a multiresponsive synapse device that integrates synaptic and optical-sensing functions. The device adopts vertically stacked graphene/h-BN/WSe2 heterostructures, including an ultrahigh-mobility readout layer, a weight-control layer, and a dual-stimuli-responsive layer. The unique structure endows synapse devices with excellent synaptic plasticity, short response time (3 μs), and excellent optical responsivity (105 A/W). To demonstrate the application in neuromorphic computing, handwritten digit recognition was simulated based on an unsupervised spiking neural network (SNN) with a precision of 90.89%, well comparable with the state-of-the-art results. Furthermore, multiterminal neuromorphic devices are demonstrated to mimic dendritic integration and photoswitching logic. Different from other synaptic devices, the research work validates multifunctional integration in synaptic devices, supporting the potential fusion of sensing and self-learning in neuromorphic networks.

Improving handwritten digit recognition using hybrid feature selection algorithm

PurposeThe amount of features in handwritten digit data is often very large due to the different aspects in personal handwriting, leading to high-dimensional data. Therefore, the employment of a feature selection algorithm becomes crucial for successful classification modeling, because the inclusion of irrelevant or redundant features can mislead the modeling algorithms, resulting in overfitting and decrease in efficiency.Design/methodology/approachThe minimum redundancy and maximum relevance (mRMR) and the recursive feature elimination (RFE) are two frequently used feature selection algorithms. While mRMR is capable of identifying a subset of features that are highly relevant to the targeted classification variable, mRMR still carries the weakness of capturing redundant features along with the algorithm. On the other hand, RFE is flawed by the fact that those features selected by RFE are not ranked by importance, albeit RFE can effectively eliminate the less important features and exclude redundant features.FindingsThe hybrid method was exemplified in a binary classification between digits “4” and “9” and between digits “6” and “8” from a multiple features dataset. The result showed that the hybrid mRMR + support vector machine recursive feature elimination (SVMRFE) is better than both the sole support vector machine (SVM) and mRMR.Originality/valueIn view of the respective strength and deficiency mRMR and RFE, this study combined both these methods and used an SVM as the underlying classifier anticipating the mRMR to make an excellent complement to the SVMRFE.

Digital Recognition Methods Based on Deep Learning

In this paper, the K-nearest neighbor algorithm and the convolutional neural network will be used to train the handwritten digit recognition model, respectively. To establish a reasonable model structure, and through the training data, the model can learn to reflect ten different handwritten number features and finally give the probability of predicting number corresponding to the likelihood of each number. Taking the learning process of the handwritten numeral recognition algorithm based on deep learning as a clue, from deep learning to convolutional neural network, from simple to deep, the relevant basic concepts, model construction, and training process of deep learning are learned and understood. Finally, the deep learning framework uses MNIST as the training dataset to train a model with high recognition rate and then combines it with Open CV technology to realize the identification of handwritten numbers. A reasonable model structure is used to accurately identify the handwritten numbers in the test set. The neural network of deep learning is established with TensorFlow to realize the classification and recognition of handwritten numbers. Various deep learning methods such as CNN and KNN are learned and compared to complete the construction of deep learning architecture. The MNIST dataset was preprocessed, features extracted, and identified. The program is to complete the training of neural network and the recognition of numbers in the image, the recognition results of deep learning methods used are counted and analyzed, and the recognition rates of two different methods are compared to find ways to optimize these methods and improve the recognition rate.

All‐Dielectric Metasurface Empowered Optical‐Electronic Hybrid Neural Networks

AbstractOptical computing has a series of advantages over its electronic counterpart, e.g., low energy consumption, high speed, and intrinsic parallelism. Diffraction deep neural networks (D2NNs) are a prominent example capable of processing images directly without addressing the spatial locations of each element. Despite the great successes, the D2NNs typically utilize the multilayer framework and face the severe challenge of misalignment in the optical region. Herein, a single metasurface‐based optical‐electronic hybrid neural network (OENN) is proposed and experimentally demonstrated. The OENN is composed of a titanium dioxide (TiO2) metasurface and a fully‐connected electronic layer. The combination of nonlocal neural layer and nonlinear transformation has significantly expanded the neural network capacity. Consequently, the classification accuracy on handwritten digits recognition can still be as high as 98.05% without employing the architecture of cascaded metasurfaces. The OENN shall shed light on the practical applications of optical computing in the visible spectrum.