Handwritten Digit Research Articles

Handwritten Number Recognition Using CNN and Comparison of Performances of Models Varying by Layers

Now-a-days, the emerging technologies like Deep Learning (DL) leads a drastic turn in the area of Machine Learning (ML) by translating into Artificially Intelligent (AI). DL has several fields of applications viz. medicine, health, surveillance, drones, sports, robotics. In DL, Convolutional Neural Network (CNN) is at the centre of amazing advances that accelerates the field of visual imagery classifications. CNN has utilized broadly in face-recognition, pattern-recognition, handwritten digit-recognition, text categorization, document analysis. The objective of this work is to detect the accuracy distinction of CNN for the classification of handwritten digits via several numbers of layers and epochs. In addition to this, evaluation of the accurateness is incorporated. CNN performance was validated by Modified National Institute of Standards and Technology (MNIST) dataset. By comparing all models varying by layers, a highest validation accuracy of 99.64 % has been reported and the lowest loss gained is 0.0239.

A Novel Approach for Hand-written Digit Classification Using Deep Learning

A Novel Approach for Hand-written Digit Classification Using Deep Learning

Online dynamical learning and sequence memory with neuromorphic nanowire networks

Nanowire Networks (NWNs) belong to an emerging class of neuromorphic systems that exploit the unique physical properties of nanostructured materials. In addition to their neural network-like physical structure, NWNs also exhibit resistive memory switching in response to electrical inputs due to synapse-like changes in conductance at nanowire-nanowire cross-point junctions. Previous studies have demonstrated how the neuromorphic dynamics generated by NWNs can be harnessed for temporal learning tasks. This study extends these findings further by demonstrating online learning from spatiotemporal dynamical features using image classification and sequence memory recall tasks implemented on an NWN device. Applied to the MNIST handwritten digit classification task, online dynamical learning with the NWN device achieves an overall accuracy of 93.4%. Additionally, we find a correlation between the classification accuracy of individual digit classes and mutual information. The sequence memory task reveals how memory patterns embedded in the dynamical features enable online learning and recall of a spatiotemporal sequence pattern. Overall, these results provide proof-of-concept of online learning from spatiotemporal dynamics using NWNs and further elucidate how memory can enhance learning.

Multifunctional, Self-Adhesive MXene-Based Hydrogel Flexible Strain Sensors for Hand-Written Digit Recognition with Assistance of Deep Learning.

The conductive hydrogel as a flexible sensor not only has certain mechanical flexibility but also can be used in the field of human health detection and human-computer interaction. Herein, by introduction of tannic acid (TA) with MXene into the polyacrylamide (PAM)/carboxymethyl chitosan (CMC) double-network hydrogel, a hydrogel with high stretchability, self-adhesion, and high sensitivity was prepared. CMC and PAM form a semi-interpenetrating double-network of high toughness and durability through electrostatic interactions and multiple hydrogen bonding networks. The abundant hydrophilic functional groups on TA and MXene form multiple hydrogen bonds simultaneously with the polymer network, ensuring high stretchability and sensitivity of the hydrogel. The hydrogel can display an accurate response to a variety of stimulus signals and can monitor both human joint movements and small physiological signal changes. It can also be combined with deep learning algorithms to classify handwritten digits with an accuracy rate of 98%. This work can promote the application of hydrogel sensors with durability and high sensitivity. The combination of algorithms and flexible sensors provides important ideas for the further development of flexible devices.

Design and Development of Handwritten Digit Recognition System Using Machine Learning

In any system ,the wide range of writing styles, handwritten digit recognition has proven to be a challenging task.The wide variety of writing styles has made it challenging to read handwritten numerals.. Numerous research have demonstrated the superior performance of neural networks in the classification of data. The major goal of this work is to compare various existing classification models in order to give effective and trustworthy methods for handwritten numeral recognition. As training sets, a total of 45,000 images with a pixel size of 2828 were employed. The original image was compared to the training sets and images. We also found that the classifier's accuracy increases with the amount of training data. The study's conclusion shows that K-NN maintains an accuracy of 96.7% as opposed to 96.8% while processing data at a rate nearly ten times faster than a neural network.

Classification of Metal Handwritten Digits Based on Microwave Diffractive Deep Neural Network

AbstractRecently, optical diffractive deep neural networks (D2NNs) have shown unprecedented superiority in terms of processing speed and power consumption. However, in the microwave band, complicated classification based on D2NN needs further investigation, which may accelerate the artificial intelligent tasks and simplify systems. Here, a three‐layer D2NN is constructed for handwritten digit classification in the microwave frequency. The excited electromagnetic wave, which passes through the metal plane engraved with different digit patterns as the input, will focus on the target plane at designated focal points through the D2NN platform. A detector array is deployed to collect the target plane energy for direct digit classification. Each layer of the proposed D2NN is composed of 1024 phase modulating meta‐units, and the phase distribution is generated through the stochastic gradient descent algorithm applied on the dataset. The network realizes an accuracy rate of 90% in numeric simulations, together with a 100% accuracy rate on the eighteen fabricated samples on the built‐up platform. The average focal efficiency reaches 18.7% and 11.7% in the simulation and experiment, respectively. The system can be seen as an alternative method for seamless in situ monitoring of security checks and near‐field sensing.

High-Performance Neuromorphic Computing and Logic Operation Based on a Self-Assembled Vertically Aligned Nanocomposite SrTiO3:MgO Film Memristor.

Neuromorphic computing based on memristors capable of in-memory computing is promising to break the energy and efficiency bottleneck of well-known von Neumann architectures. However, unstable and nonlinear conductance updates compromise the recognition accuracy and block the integration of neural network hardware. To this end, we present a highly stable memristor with self-assembled vertically aligned nanocomposite (VAN) SrTiO3:MgO films that achieve excellent resistive switching with low set/reset voltage variability (4.7%/-5.6%) and highly linear conductivity variation (nonlinearity = 0.34) by spatially limiting the conductive channels at the vertical interfaces. Various synaptic behaviors are simulated by continuously modulating the conductance. Especially, convolutional image processing using diverse crossbar kernels is demonstrated, and the artificial neural network achieves an overwhelming recognition accuracy of up to 97.50% for handwritten digits. Even under the perturbation of Poisson noise (λ = 10), 6% Salt and Pepper noise, and 5% Gaussian noise, the high recognition accuracies are retained at 95.43%, 94.56%, and 95.97%, respectively. Importantly, the logic memory function is proven experimentally based on the nonvolatile properties. This work provides a material system and design idea to achieve high-performance neuromorphic computing and logic operation.

Applying multi-layer perceptron and ResNet for handwritten digits recognition

The task of handwritten digit recognition is to recognizing the handwritten digits from pictures. Applying machine learning based models to automatically perform handwritten digit recognition task can significantly improve efficiency. This paper applies two machine learning based models, including multi-layer perceptron and residual neural network, for such a task. Firstly, this paper introduces the basic concept of the simple multi-layer perceptron model and then presents the structure of the residual neural network model. Subsequently, such two models are trained on the MNIST corpus, one of the classical dataset for the handwritten digit recognition task. The data pre-processing, like the splitting of training and test set, is described. Also, the processes of testing and training of the two models are presented. According to the experiments on the test set of MNIST, it is observed that the residual neural network can achieve better performance where the accuracy score is 99.240%, while the accuracy score of the multi-layer perceptron model is 97.260%.

Photonic convolutional neural network with robustness against wavelength deviations.

We experimentally explore the practicality of integrated multiwavelength laser arrays (MLAs) for photonic convolutional neural network (PCNN). MLAs represent excellent performance for PCNN, except for imperfect wavelength spacings due to fabrication variation. Therefore, the performance of PCNN with non-ideal wavelength spacing is investigated experimentally and numerically for the first time. The results show that there exists a certain tolerance for wavelength deviation on the degradation of the structural information of the extracted feature map, leading to the robustness of photonic recognition accuracy under non-ideal wavelength spacing. The results suggest that scalable MLAs could serve as an alternative source for the PCNN, to support low-cost optical computing scenarios. For a benchmark classification task of MNIST handwritten digits, the photonic prediction accuracy of 91.2% for stride 1 × 1 scheme using the testing dataset are experimentally obtained at speeds on the order of tera operations per second, compared to 94.14% on computer. The robust performance, flexible spectral control, low cost, large bandwidth and parallel processing capability of the PCNN driven by scalable MLAs may broaden the application possibilities of photonic neural networks in next generation data computing applications.

High optical contrast and multi-level storage of the ultracompact plasmonic device based on phase change materials

The combination of photonic devices and chalcogenide phase change materials (PCMs) provides a potential solution for photonic storage and computing. However, there are still some issues that need to be addressed for currently integrated phase-change photonic memory such as low speed, high energy consumption, and large footprint. To enable stronger interaction between light and PCMs, we utilize the plasmonic enhancement effect to amplify the local electric field, resulting in high energy efficiency in photonic storage and computation. This work proposes a nanoscale elliptical plasmonic nanoantenna that dramatically improves switching speed and energy by placing an Ag elliptical ring wrapped with Ge2Sb2Te5 (GST) on a silicon waveguide. The device exhibits crystalline and amorphous states with insertion losses of 0.3 dB and 1.7 dB, respectively, at a wavelength of 1550 nm. Write and erase operations can be completed by optical pulses of 1.5 ns–1 mW and 3 ns–3 mW, respectively. An ultra-small device volume of only 4.465 × 10−4 (µm3) is capable of achieving a switching contrast ratio of 28.5 %. Moreover, the non-volatile storage of multi-level intermediate states facilitates a wide weight-variation range and allows for precise weight updates, enhancing the overall performance and accuracy of the system. In the handwritten digit recognition task of the three-layer perceptron, the device demonstrates a recognition accuracy rate of 95 %, indicating that our approach can enable a more efficient neuromorphic photonic computing network.

Two-Stage Feature Generator for Handwritten Digit Classification.

In this paper, a novel feature generator framework is proposed for handwritten digit classification. The proposed framework includes a two-stage cascaded feature generator. The first stage is based on principal component analysis (PCA), which generates projected data on principal components as features. The second one is constructed by a partially trained neural network (PTNN), which uses projected data as inputs and generates hidden layer outputs as features. The features obtained from the PCA and PTNN-based feature generator are tested on the MNIST and USPS datasets designed for handwritten digit sets. Minimum distance classifier (MDC) and support vector machine (SVM) methods are exploited as classifiers for the obtained features in association with this framework. The performance evaluation results show that the proposed framework outperforms the state-of-the-art techniques and achieves accuracies of 99.9815% and 99.9863% on the MNIST and USPS datasets, respectively. The results also show that the proposed framework achieves almost perfect accuracies, even with significantly small training data sizes.

Universal adversarial perturbations for multiple classification tasks with quantum classifiers

Abstract Quantum adversarial machine learning is an emerging field that studies the vulnerability of quantum learning systems against adversarial perturbations and develops possible defense strategies. Quantum universal adversarial perturbations are small perturbations, which can make different input samples into adversarial examples that may deceive a given quantum classifier. This is a field that was rarely looked into but worthwhile investigating because universal perturbations might simplify malicious attacks to a large extent, causing unexpected devastation to quantum machine learning models. In this paper, we take a step forward and explore the quantum universal perturbations in the context of heterogeneous classification tasks. In particular, we find that quantum classifiers that achieve almost state-of-the-art accuracy on two different classification tasks can be both conclusively deceived by one carefully-crafted universal perturbation. This result is explicitly demonstrated with well-designed quantum continual learning models with elastic weight consolidation method to avoid catastrophic forgetting, as well as real-life heterogeneous datasets from hand-written digits and medical MRI images. Our results provide a simple and efficient way to generate universal perturbations on heterogeneous classification tasks and thus would provide valuable guidance for future quantum learning technologies.

Quantum generative adversarial networks based on a readout error mitigation method with fault tolerant mechanism

Readout errors caused by measurement noise are a significant source of errors in quantum circuits, which severely affect the output results and are an urgent problem to be solved in noisy-intermediate scale quantum (NISQ) computing. In this paper, we use the bit-flip averaging (BFA) method to mitigate frequent readout errors in quantum generative adversarial networks (QGAN) for image generation, which simplifies the response matrix structure by averaging the qubits for each random bit-flip in advance, successfully solving problems with high cost of measurement for traditional error mitigation methods. Our experiments were simulated in Qiskit using the handwritten digit image recognition dataset under the BFA-based method, the Kullback–Leibler (KL) divergence of the generated images converges to 0.04, 0.05, and 0.1 for readout error probabilities of p = 0.01, p = 0.05, and p = 0.1, respectively. Additionally, by evaluating the fidelity of the quantum states representing the images, we observe average fidelity values of 0.97, 0.96, and 0.95 for the three readout error probabilities, respectively. These results demonstrate the robustness of the model in mitigating readout errors and provide a highly fault tolerant mechanism for image generation models.

A novel deep belief network architecture with interval type-2 fuzzy set based uncertain parameters towards enhanced learning

This paper proposes a novel Deep Belief Network (DBN) architecture, the ‘Interval Type-2 Fuzzy DBN (IT2FDBN)’, which models the weights and biases with IT2 FSs. Thus, it introduces a novel algorithm for augmented deep leaning, which has the capability to address all the limitations of the classical DBN (CDBN) and T1 fuzzy DBN (T1FDBN). We comparatively evaluate the performance of the IT2FDBN by conducting experiments using the popular MNIST handwritten digit recognition datasets. Additionally, to demonstrate its robustness and generalization capabilities, we also conduct experiments taking two noisy variants of MNIST dataset, viz. the MNIST with AWGN (additive white Gaussian noise) and the MNIST with motion blur. We conduct extensive simulations by considering different combinations of nodes in the hidden layers of the DBN for better model selection. We thoroughly compare the results using well-known performance measures such as root mean square error (RMSE) and Error rate. We show that, in terms of RMSE values and error rates, the proposed IT2FDBN outperforms both T1FDBN and CDBN across all the three datasets. Further, we also provide the results of convergence, runtime-based comparison, and statistical analysis in support of our proposal.

Pulse-efficient quantum machine learning

Quantum machine learning algorithms based on parameterized quantum circuits are promising candidates for near-term quantum advantage. Although these algorithms are compatible with the current generation of quantum processors, device noise limits their performance, for example by inducing an exponential flattening of loss landscapes. Error suppression schemes such as dynamical decoupling and Pauli twirling alleviate this issue by reducing noise at the hardware level. A recent addition to this toolbox of techniques is pulse-efficient transpilation, which reduces circuit schedule duration by exploiting hardware-native cross-resonance interaction. In this work, we investigate the impact of pulse-efficient circuits on near-term algorithms for quantum machine learning. We report results for two standard experiments: binary classification on a synthetic dataset with quantum neural networks and handwritten digit recognition with quantum kernel estimation. In both cases, we find that pulse-efficient transpilation vastly reduces average circuit durations and, as a result, significantly improves classification accuracy. We conclude by applying pulse-efficient transpilation to the Hamiltonian Variational Ansatz and show that it delays the onset of noise-induced barren plateaus.

Voltage-Mode Ferroelectric Synapse for Neuromorphic Computing.

Ferroelectric materials with a modulable polarization extent hold promise for exploring voltage-driven neuromorphic hardware, in which direct current flow can be minimized. Utilizing a single active layer of an insulating ferroelectric polymer, we developed a voltage-mode ferroelectric synapse that can continuously and reversibly update its states. The device states are straightforwardly manifested in the form of variable output voltage, enabling large-scale direct cascading of multiple ferroelectric synapses to build a deep physical neural network. Such a neural network based on potential superposition rather than current flow is analogous to the biological counterpart driven by action potentials in the brain. A high accuracy of over 97% for the simulation of handwritten digit recognition is achieved using the voltage-mode neural network. The controlled ferroelectric polarization, revealed by piezoresponse force microscopy, turns out to be responsible for the synaptic weight updates in the ferroelectric synapses. The present work demonstrates an alternative strategy for the design and construction of emerging artificial neural networks.

Recognising Image Shapes from Image Parts, not Neural Parts

This paper describes an image processing method that makes use of image parts instead of neural parts. Neural networks excel at image or pattern recognition and they do this by constructing complex networks of weighted values that can cover the complexity of the pattern data. These features however are integrated holistically into the network, which means that they can be difficult to use in an individual sense. A different method might scan individual images and use a more local method to try to recognise the features in it. This paper suggests such a method and it is conjectured that this method is more ‘intelligent’ than a traditional neural network. The image parts that it creates not only have more meaning, but they can also be put into a positional context and allow for an explainable result. Tests show that it can be quite accurate, on some handwritten digit datasets, but not as accurate as a neural network. The fact that it offers an explainable interface however, could make it interesting.

Nonnegative/Binary matrix factorization for image classification using quantum annealing

Classical computing has borne witness to the development of machine learning. The integration of quantum technology into this mix will lead to unimaginable benefits and be regarded as a giant leap forward in mankind’s ability to compute. Demonstrating the benefits of this integration now becomes essential. With the advance of quantum computing, several machine-learning techniques have been proposed that use quantum annealing. In this study, we implement a matrix factorization method using quantum annealing for image classification and compare the performance with traditional machine-learning methods. Nonnegative/binary matrix factorization (NBMF) was originally introduced as a generative model, and we propose a multiclass classification model as an application. We extract the features of handwritten digit images using NBMF and apply them to solve the classification problem. Our findings show that when the amount of data, features, and epochs is small, the accuracy of models trained by NBMF is superior to classical machine-learning methods, such as neural networks. Moreover, we found that training models using a quantum annealing solver significantly reduces computation time. Under certain conditions, there is a benefit to using quantum annealing technology with machine learning.

Handwritten Digit Recognition Using CNN

This project explores the application of Convolutional Neural Networks in Handwritten digit recognition. Exploiting the widely used MNIST dataset, we proposed a deep learning model that encompasses the Ensemble model that is capable of predicting and recognizing the handwritten digits and achieves remarkable precision. The dataset contains 70000 images in total, where 55000 images are for training, 5000 images for validating, and 10000 images for testing. By using CNN, we surpassed the traditional way of training the algorithm with machine learning. By incorporating robust pre-processing techniques and innovative training ideas and strategies, our model showcases resilience to real-world problems and challenges which includes reading the postal codes in the written mail, reading the digits in the handwritten checks, digits drawn on the mobile touch panel, etc. To better extract the features of the complex handwritten digits, bagging is used from the Ensemble techniques. The objective of this project is to find a better optimizer to work with ensemble models, which encompasses the merging of optimizers with the deep learning models. Key Words: Handwritten digits, CNN, Ensemble Model, Optimizers, MNIST.

Optimization Strategy for Generative Adversarial Networks Design

Generative Adversarial Networks (GANs) are a powerful class of deep learning models that can generate realistic synthetic data. However, designing and optimizing GANs can be a difficult task due to various technical challenges. The article provides a comprehensive analysis of solution methods for GAN performance optimization. The research covers a range of GAN design components, including loss functions, activation functions, batch normalization, weight clipping, gradient penalty, stability problems, performance evaluation, mini-batch discrimination, and other aspects. The article reviews various techniques used to address these challenges and highlights the advancements in the field. The article offers an up-to-date overview of the state-of-the-art methods for structuring, designing, and optimizing GANs, which will be valuable for researchers and practitioners. The implementation of the optimization strategy for the design of standard and deep convolutional GANs (handwritten digits and fingerprints) developed by the authors is discussed in detail, the obtained results confirm the effectiveness of the proposed optimization approach.