Handwritten Digit Recognition Research Articles

Handwritten Digit Recognition System using Machine Learning

Abstract: Handwritten digit recognition remains a crucial area of research in pattern recognition and machine learning. In this paper, we present a novel approach to enhance handwritten digit recognition systems by incorporating deep learning techniques and an interactive graphical user interface (GUI). Our system employs convolutional neural networks (CNNs) for feature extraction and classification, allowing for improved accuracy and robustness in digit recognition tasks.

ScriptCalc: Handwriting Recognition with Mathematical Equation Solving

Abstract: The present paper proposes a novel algorithm for recognition of handwritten digits. For this, the present paper classified the digits into two groups: one group consists of blobs with/without stems and the other digits with stems only. The blobs are identified based on a new concept called morphological region filling methods. This eliminates the problem of finding the size of blobs and their structuring elements. The digits with blobs and stems are identified by a new concept called ‘connected component’. This method completely eliminates the complex process of recognition of horizontal or vertical lines and the property called ‘concavities’. The digits with only stems are recognized, by extending stems into blobs by using connected component approach of morphology. The present method has been applied and tested with various handwritten digits from modified NIST (National Institute of Standards and Technology) handwritten digit database (MNIST), and the success rate has been given. The present method is also compared with various existing methods. Once the digits are recognized, they are assembled back into their respective positions within the equation. Mathematical equation solving techniques are then applied to evaluate the expression and obtain the result

Hand Written Digit Recognition System

Abstract: This research paper provides a comprehensive analysis of the application of Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs) for the task of handwritten digit recognition, focusing on the extensively studied MNIST dataset. The study delves into the strengths and weaknesses of both approaches, considering various aspects such as accuracy, computational efficiency, and robustness. Through an in-depth exploration of the literature and empirical evidence, this paper aims to offer valuable insights into the advancements, challenges, and future directions in the field of handwritten digit recognition.

Neuromorphic Computing in Synthetic Antiferromagnets by Spin‐Orbit Torque Induced Magnetic‐Field‐Free Magnetization Switching

AbstractSynthetic antiferromagnet (SAF) with high thermal stability, ultra‐fast spin dynamics, and highly efficient spin‐orbit torque switching has great application potential in neuromorphic computing hardware. However, two challenges, the weakening of Hall signal in the remanent state and the need for a large auxiliary magnetic field for perpendicular magnetization switching, greatly limit the advantages of SAF in neuromorphic computing. In this work, both the enhanced anomalous Hall resistance and magnetic‐field‐free perpendicular magnetization switching are achieved by using oblique sputtering to fabricate the Pt/CoPt/Ru/CoTb SAF with strong interlayer exchange coupling and magnetic moment compensation. The fabricated SAF as synapse shows nearly linear, nonvolatile multistate plasticity, and as neuron exhibits a nonlinear sigmoid activation function, which are used to construct a fully connected neural network with a remarkable 97.0–98.1% recognition rate for the handwritten digits. Additionally, SAF serving as spike‐timing‐dependent plasticity synapse is used to construct an adaptive, unsupervised learning spiking neural network, and achieve an 87.0% accuracy in handwritten digit recognition. The findings exhibit the promise of SAFs as specialized hardware for high‐performance neuromorphic computing, offering high recognition rates and low power consumption.

Flexible Low‐Voltage MXene Floating‐Gate Synaptic Transistor for Neuromorphic Computing and Cognitive Learning

AbstractNeuromorphic computing, inspired by the functionality of biological neural networks, has emerged as a promising paradigm for artificial intelligence applications, especially in the field of flexible electronics. Among the various artificial synaptic devices, floating‐gate synaptic transistors exhibit long‐term synaptic plasticity, but they face the challenge of achieving flexible compatibility. In this work, the first demonstration of a flexible MXene floating‐gate synaptic transistor is reported, which uses multiple layers of MXene as floating gates and MXene nanosheets as charge state modulators. The device shows excellent mechanical flexibility and can operate at low voltages, which improves its suitability for wearable electronic devices. It can also emulate Pavlovian conditioned reflexes under external stress, suggesting its potential for cognitive learning. Moreover, the device is utilized for handwritten digit recognition by simulating a fully connected neural network, achieving a high recognition accuracy of 92.0%. This demonstrates its practical applicability in neuromorphic computing. Besides, this research achieves the patterning of MXene and its application in flexible floating‐gate transistors. It provides a new solution for the integrated fabrication of flexible artificial synaptic devices.

A pure pyrochlore phase ferroelectric thin film diode for optoelectric artificial synapse

In this work, Au/Bi2Ti2O7/ITO ferroelectric diodes based on Schottky barrier were fabricated on ITO substrates by the sol-gel method. The device has a simple structure and good diode performance. Under UV light irradiation at 23.5 mW/cm2, the device exhibits an open-circuit voltage of 0.31 V and a short-circuit current of −0.0425 μA. It shows that the device has good optoelectronic properties. Au/Bi2Ti2O7/ITO ferroelectric diodes also exhibited behaviors similar to biological synapses, including long-term plasticity, short-term plasticity, paired-pulse facilitation, and spike-timing-dependent plasticity when stimulated by pulses. We also build convolutional neural networks and use the MNIST dataset for handwritten digit recognition.

Handwritten Digit Recognition

Handwritten digit recognition is the intelligence of computers to recognize digits written by humans. But it becomes one of the most challenging tasks for machines as handwritten digits are not perfect and can be made with many different: flavors, size, thickness. Thus, as a solution to this problem, Handwriting digit recognition model comes into picture. Many machine learning techniques have been employed to solve the handwritten digit recognition problem. This paper focuses on Neural Network (NN) approaches. Among the three famous NN approaches: deep neural network (DNN), deep belief network (DBN) and convolutional neural network (CNN), the specialization of CNN as compared to other NN of being able to detect pattern is what makes it so useful for recognizing handwritten digits. Humans can very easily see, read & write any handwritten digits, when written in proper format. Even if the digits are not written in proper format we can use our logic and predict what digit it could be. But It is a hard task for the machine to recognize handwritten digits as these are not perfect and can be made with many different flavors. Thus handwritten digit recognition is the solution to this problem which uses the image of a digit and recognizes the digit present in the image. The handwritten digit recognition is the ability of computers to recognize human handwritten digits. The goal is to build a model that can efficiently and reliably recognize the digits and output the proper result. Amongst all the other neural networks, working and implementing a model using Convolution Neural Network gives out the most precise results. It is most popularly used for analyzing images as well as for other data analysis or classification problems. CNN has hidden layers called convolutional layers. These layers work the same way as other layers do but here we need to specify the no of filters each layer should have These filters are actually what detects the pattern. Patterns could be edges, corners, circles or any complex other objects like eyes, ears or even deeper full dogs, cats, etc. Thus, the specialization of CNN as compared to other NN of being able to detect patterns is what makes it so useful for recognizing handwritten digits Key Word: Neural Networks; Convolutional Neural Networks (CNN); Image Processing; Optical Character Recognition (OCR); MNIST Dataset

Optimization research of handwritten digit recognition algorithm based on hardware neural networks

In the modern era of digitization, the widespread application of handwritten numerical data has led to an exponential increase in data volume, necessitating efficient real-time recognition and processing systems. This project is grounded in the foundational structure of neural networks, with a specific focus on integrating these networks into hardware through Very-Large-Scale Integration (VLSI). The primary objective is to achieve faster processing speeds, lower power consumption, and real-time operations. The central emphasis of this study is the design and implementation of a hardware neural network system tailored specifically for handwritten digit recognition. Drawing from neural network theory, we explore the construction of a hardware-based handwritten digit classifier. The fundamental model employed is the single-layer perceptron neuron model. Upon receiving 28x28 pixel grayscale images, the image classifier utilizes pre-trained weights, activation functions, and a maximum selector to compare and output recognition results for numerical digits. The concrete implementation is realized using the Verilog hardware description language, coupled with algorithm optimization strategies to enhance performance and efficiency. This research endeavor aims to provide an effective hardware solution for real-time handwritten digit recognition in the digital age.

A hardware-implemented neural network for the analysis of handwritten numerals

This study presents a specialized hardware-accelerated neural network tailored for the recognition of handwritten digits in 28x28 pixel grayscale images. Employing the perceptron model, our single-layer neural network is composed of 10 neurons, each handling inputs from all pixels to generate an output. The digit recognition is determined by the neuron with the highest output value. Implemented in synthesizable Verilog, the design complies with a constraint of 350 multipliers. To achieve this, this paper employs a combination of parallel processing and pipelining, breaking down the 785 multiplications needed for each digit into 8 stages, simultaneously processing 98 data points per clock cycle. In testbench evaluations, the final design exhibits impressive performance, successfully recognizing the majority of the provided images, and attaining a remarkable 99% accuracy rate, all with a minimal delay of just 115 clocks. This accomplishment is achieved using only 99 multipliers and 107 adders, showcasing the efficiency and effectiveness of our hardware-accelerated neural network for handwritten digit recognition.

Federated Learning Implementation with Privacy Leakage Prevention for Hand-Written Digit Recognition

Federated learning (FL) has brought significant advantages to applications where collaborative learning should occur at multiple participating devices to enhance user experience in specific tasks. However, FL results in privacy leakage when n-1 clients collude to infer the model of another client. In this paper, we not only implement an FL framework but propose a methodology for preventing privacy leakage while realizing machine learning-based automatic hand-written digit recognition. Our framework supports the FL of deep networks where models trained locally are averaged. Two machine learning models Convolutional Neural Network (CNN) and Multilayer Perceptron (MLP) are implemented with FL. We proposed an algorithm, Federated Averaging with Privacy Leakage Prevention (FA-PLP), for model averaging to be done by the server. Our algorithm exploits differential privacy (DP) for realizing model averaging while getting rid of chances of privacy leakage. We evaluated our framework with two distributions of the MNIST dataset. Our empirical results revealed that FA-PLP with the CNN model could achieve the highest accuracy of 95.38%.

CFL：Data distribution edge clustering algorithm based on deep Federated learning

Federated learning (FL) allows end-devices to train local models on their respective local datasets and collaborate with the server to train a global predictive model, thus achieving the goal of machine learning while protecting the privacy and sensitive data of end-devices. However, simultaneous access to the server by a large number of end devices may result in increased transmission latency and some local models may have malicious behavior, converging in the opposite direction to the global model. As a result, additional communication costs can occur during the federation learning process. Existing research has mainly focused on reducing the number of communication rounds or cleaning dirty local data. In order to decrease the overall amount of local updates, we provide an edge-based model cleaning and device clustering strategy in this study. By computing the cosine similarity between local update parameters and global model parameters over a multi-dimensional space, the approach assesses if local updates are necessary while preventing pointless communications. In addition, end devices are grouped together based on where they are connected to the network and connect to the cloud via mobile edge nodes in clusters, therefore lowering the latency associated with high concurrent server access. Convolutional neural networks and Soft max regression are used, for instance, to perform MNIST handwritten digit recognition, and the effectiveness of the suggested approach to enhancing communication efficiency is confirmed. According to experimental findings, the edge-based model cleaning and device clustering technique decreases the quantity of local updates by 60% and speeds up the model's convergence by 10.3% when compared to the conventional FL approach.

A bioinspired MXene-based flexible sensory neuron for tactile near-sensor computing

Multifunctional near-sensor computing sensory neurons based on the integration of flexible sensors and memristors hold promise to achieve unprecedented development in large-scale neuromorphic perception circuits and human-machine interaction systems. However, the traditional sensory-memory integrated systems are based on the integration of different device modules, which are subjected to redundant analog-to-digital conversion circuits and poor flexibility. Here, an all MXene-based flexible sensory neuron fabricated by a room temperature energy-efficient, and controllable oxidation preparation technique is reported. The near-sensor computing system is seamlessly integrated a MXene-based pressure sensor array for acquisition of tactile signals with a flexible MXene-based memristor array for simulating the implementation of perceived behavior. The tactile sensor array shows a wide range (0.1–100 kPa), high linear sensitivity (23.9 kPa−1), and the memristor array exhibits repeatable and stable bipolar resistive switching behavior (retention >100 cycles, endurance >103 s) and excellent flexibility (R=1.0 mm). As a conceptual demonstration, our devices realize high-precision recognition of handwritten digit recognition (93.21%) and human motion (97.96%), which demonstrate the feasibility for real-time health monitoring based on signals collected by sensors and artificial neural networks constructed by the memristors. The MXene-based sensor-memristor system offers a prominent contribution towards advanced application of intelligence in electronic skin and wearable electronic equipment.

Compact eternal diffractive neural network chip for extreme environments

Artificial intelligence applications in extreme environments place high demands on hardware robustness, power consumption, and speed. Recently, diffractive neural networks have demonstrated superb advantages in high-throughput light-speed reasoning. However, the robustness and lifetime of existing diffractive neural networks cannot be guaranteed, severely limiting their compactness and long-term inference accuracy. Here, we have developed a millimeter-scale and robust bilayer-integrated diffractive neural network chip with virtually unlimited lifetime for optical inference. The two diffractive layers with binary phase modulation were engraved on both sides of a quartz wafer. Optical inference of handwritten digital recognition was demonstrated. The results showed that the chip achieved 82% recognition accuracy for ten types of digits. Moreover, the chip demonstrated high-performance stability at high temperatures. The room-temperature lifetime was estimated to be 1.84×1023 trillion years. Our chip satisfies the requirements for diffractive neural network hardware with high robustness, making it suitable for use in extreme environments.

Photoelectric synapses based on all-two-dimensional ferroelectric semiconductor heterojunction

Photoelectric synapses are attracting intensive attention due to its low power consumption and adaptive learning. However, traditional ferroelectric field effect transistors are not conducive to the integrated application in artificial intelligence systems. Here, we design the all two-dimensional photoelectric synapse device based on WSe2/MoS2/α-In2Se3 ferroelectric van der Waals heterojunction, which has high memory capacity (memory on/off = 105) and synaptic function. In addition, we simulate an artificial neural network to modify the handwritten digit recognition of the National Institute of Standards and Technology. In particular, the recognition rates are 92.4% and 93.6% for electrical synapse and photoelectric synapse, respectively. This work provides an effective strategy for achieving stable integration of neuromorphic computing.

Photonic implementation of the input and reservoir layers for a reservoir computing system based on a single VCSEL with two Mach-Zehnder modulators

Hardware implementation of reservoir computing (RC), which could reduce the power consumption of machine learning and significantly enhance data processing speed, holds the potential to develop the next generation of machine learning hardware devices and chips. Due to the existing solution only implementing reservoir layers, the information processing speed of photonics RC system are limited. In this paper, a photonic implementation of a VMM-RC system based on single Vertical Cavity Surface Emitting Laser (VCSEL) with two Mach Zehnder modulators (MZMs) has been proposed. Unlike previous work, both the input and reservoir layers are realized in the optical domain. Additionally, the impact of various mask signals, such as Two-level mask, Six-level mask, and chaos mask signal, employed in system, has been investigated. The system's performance improves with the use of more complex mask(t). The minimum Normalized mean square error (NMSE) can reach 0.0020 (0.0456) for Santa-Fe chaotic time series prediction in simulation (experiment), while the minimum Word Error Rate (WER) can 0.0677 for handwritten digits recognition numerically. The VMM-RC proposed is instrumental in advancing the development of photonic RC by overcoming the long-standing limitations of photonic RC systems in reservoir implementation. Linear matrix computing units (the input layer) and nonlinear computing units (the reservoir layer) are simultaneously implemented in the optical domain, significantly enhancing the information processing speed of photonic RC systems.

Pen Stroke Digit Recognition Using CNN

Hand-written character and digit recognition have been one of the most exigent and engrossing field of pattern recognition and image processing. The main aim of this paper is to demonstrate and represent the work which is related to hand-written digit recognition. The hand-written digit recognition is a very exigent task. In this recognition task, the numbers are not accurately written or scripted as they differ in shape or size; due to which the feature extraction and segmentation of hand-written numerical script is arduous. The vertical and horizontal projections methods are used for the purpose of segmentation in the proposed work. KNN is applied for recognition and classification. The digit recognition is a very exigent task. In this recognition task, the numbers are not accurately written or scripted as they differ in shape or size; due to which the feature extraction and segmentation of numerical script is arduous. The vertical and horizontal projections methods are used for the purpose of segmentation in the proposed work. CNN is applied for recognition, classification and we are also getting nice accuracy while doing prediction.

Lateral ionic-gated graphene synaptic transistor with transition from paired-pulse facilitation to depression for filtering and image recognition

The transition from paired-pulse facilitation to paired-pulse depression is of significant importance for regulating biologic neural network. Here, we fabricate and investigate a lateral ionic-gated graphene synaptic transistor (GST) with short gate length. We control the channel current via e-field-dependent movement and diffusion of ions for realizing the transition from paired-pulse facilitation to depression. This phenomenon comes from the over-diffusion and trapping of a finite number of ions and the short ion-diffusion length under large negative gate pulse stimuli. This device successfully emulates facilitation and depression synaptic functions, for leveraging the accumulation, migration and diffusion of ions. Moreover, enhanced high-pass filtering and conversion from long-term depression to long-term potentiation are achieved by changing pulses time interval. Our results indicate that our device possesses excellent frequency-dependent synaptic plasticity, making it highly suitable for information filtering up to 33 Hz in neuromorphic systems, and image edge enhancement for neural image processing. In additional, a simulated artificial neural network built on lateral ionic-gated GSTs for handwritten digit recognition can achieve a learning accuracy of 82.4 %.

Artificial synaptic simulating pain-perceptual nociceptor and brain-inspired computing based on Au/Bi3.2La0.8Ti3O12/ITO memristor

Recently, memristors have garnered widespread attention as neuromorphic devices that can simulate synaptic behavior, holding promise for future commercial applications in neuromorphic computing. In this paper, we present a memristor with an Au/Bi3.2La0.8Ti3O12 (BLTO)/ITO structure, demonstrating a switching ratio of nearly 103 over a duration of 104 s. It successfully simulates a range of synaptic behaviors, including long-term potentiation and depression, paired-pulse facilitation, spike-timing-dependent plasticity, spike-rate-dependent plasticity etc. Interestingly, we also employ it to simulate pain threshold, sensitization, and desensitization behaviors of pain-perceptual nociceptor (PPN). Lastly, by introducing memristor differential pairs (1T1R-1T1R), we train a neural network, effectively simplifying the learning process, reducing training time, and achieving a handwriting digit recognition accuracy of up to 97.19 %. Overall, the proposed device holds immense potential in the field of neuromorphic computing, offering possibilities for the next generation of high-performance neuromorphic computing chips.

Direct Optical Convolution Computing Based on Arrayed Waveguide Grating Router

AbstractOptical convolution computing is gaining traction owing to its inherent parallelism, multi‐dimensional processing, and energy efficiency. To handle input dimensions of N, conventional implementations necessitate N2 optical elements, such as Mach–Zehnder interferometers or micro‐ring resonators, to process multiply‐accumulate (MAC) operations, limiting scalability and resulting in elevated power consumption. Here, a direct convolution computing method based on wavelength routing, utilizing the unique sliding property of an arrayed waveguide grating router (AWGR) to perform the sliding window operation of the convolution in the wavelength–space domains is proposed. With two input vectors directly loaded onto two modulator arrays, the convolution result is instantaneously produced at a photodetector array. The entire convolution computation is executed within a single clock cycle without the need for preprocessing or decomposition into elementary MAC operations. The number of active elements is minimal, only needed for input/output. The proposed optical convolution unit has striking advantages of high scalability, high speed, and processing simplicity compared to those based on optical matrix‐vector multipliers. In the first experimental demonstration, a remarkable classification accuracy of up to 98.2% in handwritten digit recognition tasks using a LeNet‐5 neural network is achieved.

Silicon photonic neuromorphic accelerator using integrated coherent transmit-receive optical sub-assemblies

Neural networks, having achieved breakthroughs in many applications, require extensive convolutions and matrix-vector multiplication operations. To accelerate these operations, benefiting from power efficiency, low latency, large bandwidth, massive parallelism, and CMOS compatibility, silicon photonic neural networks have been proposed as a promising solution. In this study, we propose a scalable architecture based on a silicon photonic integrated circuit and optical frequency combs to offer high computing speed and power efficiency. A proof-of-concept silicon photonics neuromorphic accelerator based on integrated coherent transmit–receive optical sub-assemblies, operating over 1TOPS with only one computing cell, is experimentally demonstrated. We apply it to process fully connected and convolutional neural networks, achieving a competitive inference accuracy of up to 96.67% in handwritten digit recognition compared to its electronic counterpart. By leveraging optical frequency combs, the approach’s computing speed is possibly scalable with the square of the cell number to realize over 1 Peta-Op/s. This scalability opens possibilities for applications such as autonomous vehicles, real-time video processing, and other high-performance computing tasks.