Handwritten Digit Recognition Research Articles

Field-free spin–orbit torque magnetization switching in Pt/CoTb devices grown on flexible substrates for neuromorphic computing

Flexible spintronic devices based on spin–orbit torque (SOT)-induced perpendicular magnetization switching (PMS) have attracted increasing attention due to their high storage intensity and good programming capability. However, to achieve deterministic PMS, an in-plane auxiliary magnetic field is required, which greatly limits its application. Here, we show that “robust” magnetic field-free SOT-driven PMS is realized in the oblique sputtered Pt/CoTb multilayers grown on a flexible polyimide substrate. “Robust” means the magnetic field-free SOT switching is highly repeatable and stable after 100 bending cycles under various bending conditions. Additionally, the fabricated flexible multilayers exhibit nearly linear and nonvolatile multistate plasticity as synapses and a nonlinear sigmoid activation function when acting as neurons. We construct a fully connected neural network for handwritten digit recognition, achieving an over 96.27% recognition rate. Our findings may spur further investigations on the SOT-based flexible spintronic devices for wearable artificial intelligence applications.

Ultra-broadband all-optical nonlinear activation function enabled by MoTe2/optical waveguide integrated devices

All-optical nonlinear activation functions (NAFs) are crucial for enabling rapid optical neural networks (ONNs). As linear matrix computation advances in integrated ONNs, on-chip all-optical NAFs face challenges such as limited integration, high latency, substantial power consumption, and a high activation threshold. In this work, we develop an integrated nonlinear optical activator based on the butt-coupling integration of two-dimensional (2D) MoTe2 and optical waveguides (OWGs). The activator exhibits an ultra-broadband response from visible to near-infrared wavelength, a low activation threshold of 0.94 μW, a small device size (~50 µm2), an ultra-fast response rate (2.08 THz), and high-density integration. The excellent nonlinear effects and broadband response of 2D materials have been utilized to create all-optical NAFs. These activators were applied to simulate MNIST handwritten digit recognition, achieving an accuracy of 97.6%. The results underscore the potential application of this approach in ONNs. Moreover, the classification of more intricate CIFAR-10 images demonstrated a generalizable accuracy of 94.6%. The present nonlinear activator promises a general platform for three-dimensional (3D) ultra-broadband ONNs with dense integration and low activation thresholds by integrating a variety of strong nonlinear optical (NLO) materials (e.g., 2D materials) and OWGs in glass.

Handwritten Digit Recognition by Deep Learning

This paper evaluates recent advances in handwritten digit recognition models, focusing on strategies developed and deployed in practical applications. The project utilizes both traditional and deep learning approaches, employing architectures such as Convolutional Neural Networks (CNNs). This paper explores the comparative performance of various models, discusses their deployment in real-world scenarios, and highlights future prospects for enhancing handwritten digit recognition technology. Keywords: Handwritten Digit Recognition, Multi-Layer Perceptron (MLP), Convolutional Neural Networks (CNN), AlexNet, Deep Learning for Image Classification, Neural Networks, Image Preprocessing, Data Augmentation, Feature Extraction, Image to Grayscale Conversion, Image Normalization, Training and Validation, Accuracy Metrics, Model Evaluation, Transfer Learning, Classification Algorithms, Real- Time Processing, Computer Vision, Image Recognition.

Memristor-based circuit design of BiLSTM network

The bidirectional long short-term memory (BiLSTM) network involves significant amount of parameter computations. This paper proposes the memristor-based bidirectional long short-term memory (MBiLSTM) network, with its capability of in-memory computing and parallel computing, can accelerates the parameter computations speed. The MBiLSTM network circuit is composed of normalization circuit, two memristor-based long short-term memory (LSTM) circuits, memristor-based resnet circuit, memristor-based dense circuitand winner-take-all (WTA) circuit. The voltage signals are scaled to the setting range by normalization circuit, memristor-based LSTM circuit is responsible for extracting features from the dataset, memristor-based resnet circuit can enhance the overall performance of the network, memristor-based dense circuit ensures that the final outputs dimension matches the dimension of the target signals, WTA circuit outputs the maximum voltage of memristor-based dense circuit. The effectiveness of the MBiLSTM network is validated through gait recognition experiment and handwritten digit recognition experiment. The stability, robustness and potential errors in the manufacturing process of memristance are analyzed.

Development of a conditional variational autoencoder for handwritten digit recognition.

This study examines the performance of Conditional Variational Autoencoder (CVAE) in handwritten digit recognition. Using the MNIST dataset, two variants of the CVAE models — convolutional and multilevel architecture — were developed and compared. The research methodology includes comprehensive data preprocessing, architecture design, training, and thorough evaluation processes. The obtained data highlight the better performance of the convolutional model-based CVAE in achieving recognition accuracy compared to its multilayer counterpart. Evaluation metrics include analysis of original and reconstructed images, comparison of hidden layer vector distribution patterns, and visualization of loss function dynamics. In addition, the study highlights the practical implications of CVAEs in various fields, highlighting their performance in digit recognition tasks due to their inherent robustness and extraordinary generalizability.

Comparative analysis of CNN models for handwritten digit recognition

The paper discusses the subject of convolutional neural networks used for handwritten digit classification. The purpose of the research is to evaluate the accuracy, performance, training, and classification time of three OCR networks (VGG-16, VGG-19 and AlexNet) and compare them with each other while selecting the most optimal one. The popular MNIST dataset of 70,000 images was used for the study. For each model, a preliminary study was conducted to determine the optimal parameters in the form of the number of input data and number of training epochs. The result of the work indicates that, despite the longer training and classification time, the AlexNet model achieved the highest precision, recall, and F1-score, indicating its ability to effectively classify images.

Reconfigurable Multilevel Storage and Neuromorphic Computing Based on Multilayer Phase-Change Memory.

In the era of big data, the amount of global data is increasing exponentially, and the storage and processing of massive data put forward higher requirements for memory. To deal with this challenge, high-density memory and neuromorphic computing have been widely investigated. Here, a gradient-doped multilayer phase-change memory with two-level states, four-level states, and linear conductance evolution using different pulse operations is proposed. The mechanism of multilevel states is revealed through high-resolution transmission electron microscopy (HRTEM) and finite-element analysis (FEA), which show that the sequential phase change among different sublayers is realized due to the different physical properties of the sublayers with different doping concentrations. Taking advantage of the devices' linear conductance evolution characteristic, a handwritten digit (28 × 28 pixel) recognition task is implemented with a high learning accuracy of 93.46% by building a simulated artificial neural network made up of this gradient-doped multilayer phase-change memory. It is proved that this gradient-doped multilayer phase-change memory is capable of both binary multilevel digital storage and brain-inspired analog in-memory computing in the same device, enabling reconfigurable applications in the future.

Evaporated Copper‐Based Perovskite Dynamic Memristors for Reservoir Computing Systems

AbstractDynamic memristors are considered as the optimal hardware devices for reservoir computing (RC) enabled by their nonlinear conductance variations. This significantly reduces the extensive training workload typically required by traditional neural networks. Lead halide perovskites, with their tunable band structure and active ion migration properties, have emerged as highly promising materials for developing dynamic memristors. However, large‐scale and consistently stable production remains a challenge for perovskite functional films, while lead elements' toxicity and environmental impact also partly restrict their practical device utilization. In this work, lead‐free copper‐based perovskite (i.e., CsCu2I3) films are prepared by thermal evaporation for constructing dynamic memristors. The effective conductivity modulation of CsCu2I3‐based memristor can be utilized in artificial neural networks, achieving a high handwritten digit recognition accuracy of 91.2%. In addition, the RC system is also constructed based on the dynamic behavior of the devices, by which a letter recognition accuracy of 98.2% with simple training is achieved. This technology provides a feasible pathway to construct copper‐based perovskite dynamic memristors for future neural network information processing.

BiFeO3/SrTiO3 superlattice-like based ferroelectric memristors with pronounced artificial synaptic plasticity

Memristors are attracting widespread attention for their multilevel storage capacity and synaptic learning ability. Ferroelectric memristors, with resistance switching based on polarization reversal, offer uniform threshold voltage and analog resistance modulation. By preparing superlattice-like ferroelectric films, the ferroelectric properties can be significantly enhanced, thereby improving the memristive characteristics. In this study, the constructed BiFeO3/SrTiO3 (BFO/STO) superlattice-like structure increases the remanent polarization by approximately 200 % compared to a single-layer BFO structure, as well as the dramatically enhanced resistance switching performance. Notably, the Pt/(BFO/STO)2/NSTO device shows superior storage performance, with extended endurance (1000 cycles), reliable retention (10000 seconds) and fundamental synaptic behaviors being demonstrated. Finally, simulations for handwritten digit recognition classification, involving modulation of conductance to vary synaptic weights, achieve an impressive accuracy of 87.46 %, showing great promising for applications in information storage and neuromorphic computing.

Recognizing Handwritten Digits on MNIST Dataset using KNN Algorithm

Handwritten digit recognition is a critical job in computer vision and is used as a frequent benchmark for testing machine learning algorithms. This work describes the creation of a recognition system utilizing the K-Nearest Neighbors (KNN) method, which was chosen for its simplicity and ease of understanding. The system is built on the MNIST dataset, which contains a vast number of photographs of handwritten digits. The process begins with data collection and investigation, which involves analyzing the content and properties of the MNIST dataset to better comprehend the range of handwriting styles. This is followed by data preparation, which consists of standardizing pixel values to increase algorithm performance and dividing the dataset into training and testing sets. This split aids in assessing the model's generalization capabilities. KNN was selected as the model due to its non-parametric nature, meaning it does not make assumptions about the data's underlying distribution. This characteristic makes it versatile and effective for classification tasks like digit recognition. The model is trained in a unique way compared to other machine learning methods: instead of learning parameters, KNN stores all the training data and uses it during the prediction phase to classify new data points based on the nearest neighbors. The performance of the model was evaluated using accuracy as a metric, which measures the percentage of correctly classified digits. Different values of kkk, the number of neighbors, were tested to find the optimal setting. The results showed that KNN could effectively classify handwritten digits with significant accuracy. Finally, the paper discusses future work and deployment considerations. It highlights the potential for deploying KNN-based digit recognition systems in real-world applications and suggests further research to refine preprocessing techniques, optimize model parameters, and explore more advanced models, such as neural networks, to enhance accuracy and efficiency. This study demonstrates the practical applicability of KNN in digit recognition and outlines a path for future improvements.

Real-time execution of SNN models with synaptic plasticity for handwritten digit recognition on SIMD hardware.

Recent advancements in neuromorphic computing have led to the development of hardware architectures inspired by Spiking Neural Networks (SNNs) to emulate the efficiency and parallel processing capabilities of the human brain. This work focuses on testing the HEENS architecture, specifically designed for high parallel processing and biological realism in SNN emulation, implemented on a ZYNQ family FPGA. The study applies this architecture to the classification of digits using the well-known MNIST database. The image resolutions were adjusted to match HEENS' processing capacity. Results were compared with existing work, demonstrating HEENS' performance comparable to other solutions. This study highlights the importance of balancing accuracy and efficiency in the execution of applications. HEENS offers a flexible solution for SNN emulation, allowing for the implementation of programmable neural and synaptic models. It encourages the exploration of novel algorithms and network architectures, providing an alternative for real-time processing with efficient energy consumption.

Symmetric silicon microring resonator optical crossbar array for accelerated inference and training in deep learning

Photonic integrated circuits are emerging as a promising platform for accelerating matrix multiplications in deep learning, leveraging the inherent parallel nature of light. Although various schemes have been proposed and demonstrated to realize such photonic matrix accelerators, the in situ training of artificial neural networks using photonic accelerators remains challenging due to the difficulty of direct on-chip backpropagation on a photonic chip. In this work, we propose a silicon microring resonator (MRR) optical crossbar array with a symmetric structure that allows for simple on-chip backpropagation, potentially enabling the acceleration of both the inference and training phases of deep learning. We demonstrate a 4×4 circuit on a Si-on-insulator platform and use it to perform inference tasks of a simple neural network for classifying iris flowers, achieving a classification accuracy of 93.3%. Subsequently, we train the neural network using simulated on-chip backpropagation and achieve an accuracy of 91.1% in the same inference task after training. Furthermore, we simulate a convolutional neural network for handwritten digit recognition, using a 9×9 MRR crossbar array to perform the convolution operations. This work contributes to the realization of compact and energy-efficient photonic accelerators for deep learning.

Integrated sources model: A new space-learning model for heterogeneous multi-view data reduction, visualization, and clustering

In machine learning, multi-view data involve multiple distinct sets of attributes (“views”) for a common set of observations; when each view has the same attributes considered in different contexts, the data are said to contain multiple views of homogeneous format, which can be conceptualized as a tensor. In this article, we describe a novel approach for integrating multiple views of heterogeneous format into a common latent space using a workflow that involves non-negative matrix and tensor factorization (NMF/NTF). This approach, which we refer to as the integrated sources model (ISM), consists of two main steps: Embedding and analysis. In the embedding step, the views are transformed into matrices with common non-negative components. In the analysis step, the transformed views are combined into a tensor and decomposed using NTF. We also present a variant of ISM; the integrated latent sources model (ILSM), which offers significant advantages over ISM in terms of computational power and in cases where the views are highly unbalanced with regard to the number of attributes per view. Noteworthy, ISM can be extended to process multi-omic and multi-view datasets even in the presence of missing views. We provide a proof-of-concept analysis using five examples, including the UCI Digits (the University of California Irvine Pen-Based Recognition of Handwritten Digits) dataset, a public cell-type gene signatures dataset, and a multi-omic single-cell dataset. These examples demonstrate that, in most cases, multi-view clustering is better achieved with ISM or its variant ILSM than with other latent space approaches. We also show how the non-negativity and sparsity of the ISM model components enable straightforward interpretations, in contrast to other approaches that involve latent factors of mixed signs. Finally, we present potential applications to single-cell multi-omics and spatial mapping, including spatial imaging, spatial transcriptomics, and computational biology, which are currently under evaluation. ISM relies on state-of-the-art algorithms invoked through a simple workflow implemented in Python.

Implementation of Logic Function and Memristive Behavior in Synthetic Antiferromagnet with Strong Immunity to Perpendicular Magnetic Field Interference

Spintronic devices with heavy metal/ferromagnet structures have shown potential applications for in‐memory computing. However, the sensitivity of ferromagnet to external magnetic field raises a challenge for the practical applications of these devices. The usage of synthetic antiferromagnet (SAF) can effectively address this issue due to its good magnetic field immunity. This work demonstrates the implementation of all 16 types of Boolean logic functions via controlling the magnetization states of the SAF layer by using current pulses, and also the stable multistate storage under the interference of perpendicular magnetic field in these devices. The SAF devices also exhibit synaptic plasticity under the stimulation of current pulses. The artificial neural network constructed based on the SAF device can perform handwritten digit recognition tasks with an accuracy rate of 90%. These results showcase the viability and superiority of the SAF device in building logic‐in‐memory architecture for its strong immunity to magnetic field interference.

Black phosphorus quantum dots functionalized with photochromic poly(vinylspiropyran)-grafted polydopamine for transient digital-type memristors

Abstract The covalent functionalization of black phosphorus quantum dots (BPQDs) with organic species or polymers will inevitably change or damage their electronic structure and intrinsic structure. To address this problem and explore the application of BPQDs in transient digital-type memristors, a polydopamine (PDA) thin film is first synthesized in situ onto the surface of BPQDs to produce a donor–acceptor-type BPQDs@PDA composite that is directly used to react with 2-bromoisobutyryl bromide to give BPQDs@PDA-Br. By using BPQDs@PDA-Br as an atom transfer radical polymerization agent, a large number of polyvinylspiropyran (PSP) chains are in situ grown from the PDA surface to yield BPQDs@PDA-PSP. Upon ultraviolet (UV)–visible light illumination, the 2 isomers of the spiropyran (ring-closed spiropyran form and ring-opened merocyanine) in the PSP moieties will interconvert into each other rapidly. As expected, the as-fabricated indium tin oxide (ITO)/BPQDs@PDA-PSP/ITO device exhibits typical nonvolatile digital-type memristive performance under visible irradiation, with a small turn-on voltage of −1.52 V, a turn-off voltage of +1.16 V, and an ON/OFF ratio current ratio of 1.02 × 104. Upon UV illumination, the information stored in the device is quickly and completely erased within 6 s. By utilizing a simple memristor-based convolutional neural network, one can easily realize handwritten digit recognition. After 10 epochs of training, numeral recognition accuracy can reach up to 96.21%.

Synergistic insights: Exploring continuous learning and explainable AI in handwritten digit recognition

Deep Neural Networks achieve outstanding results; however, their reliance on a static environment with fixed data poses challenges in dynamic scenarios where data continuously evolves. Being capable of learning, adapting, and generalizing continually in a scalable, successful, and efficient manner is crucial for the sustainable development of AI systems. The classical solution of retraining the model using both old and new data is time-consuming and expensive. Continual Learning tackles the problem of learning new data distributions without the need for retraining from scratch. Furthermore, the task of recognizing unlabeled images using previously acquired knowledge becomes challenging, particularly when the new data needs to be incrementally annotated without starting the training process from scratch.To gain a deeper understanding of how “Black Box” neural networks make decisions, it is important to visualize components inside the model that affect the error rate throughout the decision-making process. The Continual Self-Learning model on label-less historical digits yields increasingly perceptive interpretations. This paper aims to establish a literature review of the latest advances in continual learning for computer vision tasks, to articulate catastrophic forgetting using Explainable Artificial Intelligence on both split MNIST and the historical digit dataset DIDA, and to shed light on important but still understudied topics.

High-Performance 2D Ambipolar MoTe2 Lateral Memristors by Mild Oxidation.

2D transition metal dichalcogenides (TMDCs) have been intensively explored in memristors for brain-inspired computing. Oxidation, which is usually unavoidable and harmful in 2D TMDCs, could also be used to enhance their memristive performances. However, it is still unclear how oxidation affects the resistive switching behaviors of 2D ambipolar TMDCs. In this work, a mild oxidation strategy is developed to greatly enhance the resistive switching ratio of ambipolar 2H-MoTe2 lateral memristors by more than 10 times. Such an enhancement results from the amplified doping due to O2 and H2O adsorption and the optimization of effective gate voltage distribution by mild oxidation. Moreover, the ambipolarity of 2H-MoTe2 also enables a change of resistive switching direction, which is uncommon in 2D memristors. Consequently, as an artificial synapse, the MoTe2 device exhibits a large dynamic range (≈200) and a good linearity (1.01) in long-term potentiation and depression, as well as a high-accuracy handwritten digit recognition (>96%). This work not only provides a feasible and effective way to enhance the memristive performance of 2D ambipolar materials, but also deepens the understanding of hidden mechanisms for RS behaviors in oxidized 2D materials.

Halide perovskite photovoltaics for in-sensor reservoir computing

As intelligent electronics get laden with multimodal sensors, the data transfer and computation increase their energy expenditure. Consequently, researchers aim to develop efficient computing paradigms or integrate energy harvesting from ambient sources. Halide perovskites possess unique photophysics and coupled ionic-electronic dynamics that actualise memory devices for brain-inspired computing. Synergising the computing capability with their conventional light harvesting efficacy could aptly address the aforementioned problem. In a novel approach, the transient open circuit voltage (VOC) of a methylammonium lead bromide-based solar cell was exploited to serve as a self-powered volatile short-term memory for optoelectronic in-sensor reservoir computing. The origin of the memory is attributed to the influence of mobile ions on the carrier generation and recombination in the device. The system's versatility and task specificity were shown by engineering the volatility of the memory. The benchmarking task of MNIST handwritten digit recognition was performed with the highly reproducible and robust transformation of optical inputs into unique reservoir states. To demonstrate the high nonlinearity, second-order time-series prediction (NARMA2) was performed. Finally, an exemplary cardiac health-monitoring application was showcased by monolithic reading and processing of a physiological time series known as photoplethysmography (PPG) to identify atrial fibrillation with increased computational efficiency.

Handwritten digit and Roman string recognition using gated CNN with federated learning

Handwritten digit and Roman string recognition using gated CNN with federated learning

Application of Deep Learning Algorithm in Visual Optimization of Industrial Design

In order to understand the application of degree learning algorithms in industrial design visual optimization, the author proposes an application research based on deep learning algorithms in industrial design visual optimization. The author first starts with the basic principles of deep learning algorithms and provides a detailed explanation of the basic structure of the single-layer network of deep learning algorithms, as well as the restricted Boltzmann machine and its training process. Finally, an example of the performance improvement brought by the application of deep learning technology in handwritten digit recognition was given through an automatic encoder, and a simple summary of deep learning technology was made. Secondly, the NCI matching algorithm is used to match industrial design products and reconstruct the point cloud of industrial products to accurately detect their features. On this basis, deep learning algorithms are applied to construct a visual optimization model for industrial design, determine the output format of the model scene and the output situation of industrial design, and process the model according to the changing characteristics of the industrial design model to comprehensively edit the data of the model. Finally, targeted optimization of the industrial design visual optimization model is carried out based on the technical characteristics of deep learning algorithms, in order to complete the industrial design visual optimization. The experimental comparison results show that the industrial design visual optimization method optimized in this design has higher visual clarity than the traditional industrial design method, reaching 98%. Greatly improves the visual clarity of industrial design.