MNIST Research Articles

Application of DMSFNN-COA technique for brand image design

Color plays a pivotal role in product design, as it can evoke emotional responses from users. Understanding these emotional needs is crucial for effective brand image design. This paper introduces a novel approach, the Brand Image Design using Deep Multi-Scale Fusion Neural Network optimized with Cheetah Optimization Algorithm (BID-DMSFNN-COA), for classifying product color brand images as “Stylish” and “Natural”. By leveraging deep learning techniques and optimization algorithms, the proposed method aims to enhance brand image accuracy and address existing challenges in product color trend forecasting research. Initially, data are collected from the Mnist Data Set. The data are then supplied into the pre-processing section. In the pre-processing segment, it removes the noise and enhances the input image utilizing master slave adaptive notch filter. The Deep Multi-Scale Fusion Neural Network optimized with cheetah optimization algorithm effectively classifies the product colour brand image as “Stylish” and “Natural”. Implemented on the MATLAB platform, the BID-DMSFNN-COA technique achieves remarkable accuracy rates of 99 % for both “Natural” and “Stylish” classifications. In comparison, existing methods such as BID-GNN, BID-ANN, and BID-CNN yield lower accuracy rates ranging from 65 % to 85 % for “Stylish” and 65 %–70 % for “Natural” product color brand image design. The simulation outcomes reveal the superior performance of the BID-DMSFNN-COA technique across various metrics including accuracy, F-score, precision, recall, sensitivity, specificity, and ROC analysis. Notably, the proposed method consistently outperforms existing approaches, providing higher values across all evaluation criteria. These findings underscore the effectiveness of the BID-DMSFNN-COA technique in enhancing brand image design through accurate product color classification.

Enhancing Data Security through Machine Learning-based Key Generation and Encryption

In an era marked by growing concerns about data security and privacy, the need for robust encryption techniques has become a matter of paramount importance. The primary goal of this study is to protect sensitive information during transmission while ensuring efficient and reliable decryption at the receiver's side. To generate robust and unique cryptographic keys, the proposed approach trains an autoencoder neural network based on hashing and optionally generated prime numbers in the MNIST dataset. The key serves as the foundation for secure communication. An additional security layer to the cryptographic algorithm passing through the first ciphertext, was employed utilizing the XORed and Blum-Blum-Shub (BBS) generators to make the system resistant to various types of attacks. This approach offers a robust and innovative solution for secure data transmission, combining the strengths of autoencoder-based key generation and cryptographic encryption. Its effectiveness is demonstrated through testing and simulations.

Metaheuristics Approach for Hyperparameter Tuning of Convolutional Neural Network

Deep learning is an artificial intelligence technique that has been used for various tasks. Deep learning performance is determined by its hyperparameter, architecture, and training (connection weight and bias). Finding the right combination of these aspects is very challenging. Convolution neural networks (CNN) is a deep learning method that is commonly used for image classification. It has many hyperparameters; therefore, tuning its hyperparameter is difficult. In this research, a metaheuristic approach is proposed to optimize the hyperparameter of convolution neural networks. Three metaheuristic methods are used in this research: ant colony optimization (ACO), genetic algorithm (GA), and Harmony Search (HS). The metaheuristics methods are used to find the best combination of 8 hyperparameters with 8 options each which creates 1.6. 107 of solution space. The solution space is too large to explore using manual tuning. The Metaheuristics method will bring benefits in terms of finding solutions in the search space more effectively and efficiently. The performance of the metaheuristic methods is evaluated using MNIST datasets. The experiment results show that the accuracy of ACO, GA and HS are 99,7%, 97.7% and 89,9% respectively. The computational times for the ACO, GA and HS algorithms are 27.9 s, 22.3 s, and 56.4 s, respectively. It shows that ACO performs the best among the three algorithms in terms of accuracy, however, its computational time is slightly longer than GA. The results of the experiment reveal that the metaheuristic approach is promising for the hyperparameter tuning of CNN. Future research can be directed toward solving larger problems or improving the metaheuristics operator to improve its performance.

MRD‐GAN: Multi‐representation discrimination GAN for enhancing the diversity of the generated data

AbstractThe generative adversarial network (GAN) is a highly effective member of the generative models category and is extensively employed for generating realistic samples across various domains. The fundamental concept behind GAN involves two networks, a generator and a discriminator, competing against each other. During the training process, generator and discriminator networks encounter several issues that can potentially affect the quality and diversity of the generated samples. One such critical issue is mode collapse, where the generator fails to create varied samples. To tackle this issue, this article introduces a GAN approach called the multi‐representation discrimination GAN (MRD‐GAN). In this approach, the discriminator supports concurrent network discrimination flows to manage different representations of the data through various transformation functions, such as dimension rescaling, brightness adjustment, and gamma correction applied to the input data of the discriminator. We use a fusion function to aggregate the output of all flows and return a consolidated loss value to update the generator's weights. Hence, the discriminator conveys diverse feedback to the generator. The proposed approach has been evaluated on four distinct benchmarks, namely CelebA, Cifar‐10, Fashion‐Mnist, and Mnist. The experimental results demonstrate that the proposed approach surpasses the existing state‐of‐the‐art GAN models in terms of FID metric that measures the diversity of the generated samples. Significantly, the proposed approach demonstrates remarkable FID scores of 14.02, 30.19, 9.42, and 3.14 on the CelebA, Cifar‐10, Fashion‐Mnist, and Mnist datasets, respectively.

Handwritten Digit Recognition

Abstract: This paper presents a powerful Handwritten Digit Recognition System that combines a Graphical User Interface (GUI) based on Tkinter with Convolutional Neural Networks (CNNs). Our approach, which makes use of the MNIST dataset, includes careful data pretreatment to facilitate efficient CNN model training. Convolutional and pooling layers are included in the design of the model, and they are optimized with the Adam optimizer for higher learning rates. The evaluation's findings demonstrate excellent memory, accuracy, and precision. Furthermore, real-time digit drawing is made possible via an intuitive Tkinter GUI, which confirms the model's applicability. By showing the effectiveness of CNNs and offering an interactive platform for natural user interaction, the research provides a holistic solution to handwritten digit recognition. This method is promising for various uses in digit recognition scenarios, highlighting its flexibility and usefulness in real-world situations.

Hand Written Digit Recognition System

Abstract: This research paper provides a comprehensive analysis of the application of Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs) for the task of handwritten digit recognition, focusing on the extensively studied MNIST dataset. The study delves into the strengths and weaknesses of both approaches, considering various aspects such as accuracy, computational efficiency, and robustness. Through an in-depth exploration of the literature and empirical evidence, this paper aims to offer valuable insights into the advancements, challenges, and future directions in the field of handwritten digit recognition.

DRD-GAN: A Novel Distributed Conditional Wasserstein Deep Convolutional Relativistic Discriminator GAN with Improved Convergence

Generative Adversarial Network (GAN) exhibited significant capabilities in many applications including image enhancement and manipulation, language translation, generating images/videos from text, creating art and music, and so on. However, train ing GANs using large datasets remains highly computationally intensive for most of the standalone systems. Additionally, standalone GANs often exhibit poor synchronization between their generator and discriminator with unstable training , poor convergence along with a large number of mode collapse s and vanishing / exploding gradients. Standalone GANs also failed to learn in a decentralized environment, where the data is distributed among several client machines. Some researchers have lately used the most prevalent decentralized setting available today, called Federated Learning (FL) to develop distributed -GAN strategies as the possible solutions, although their implementations mostly failed to address the above issues mainly because of: the training instability within the distributed component s, which eventually leads to the poor synchronization among the generator s and discriminator s scattered over several machines. In this work, we developed a computationally inexpensive Wasserstein conditional Distributed Relativistic Discriminator-GAN or DRD-GAN to alleviate the above issues. DRD-GAN stabilizes its train ing (with non-convex losses) by keeping a global generator in the central server and relativistic discriminator s in the local client s (one discriminator per client ), and uses Wasserstein-1 for computing local and global losses. It eventually avoids mode collapse s, vanishing/exploding gradient s (both in the presence of iid and non-iid samples) and helps DRD-GAN to produce high-quality fake images. Apart from that, the sheer unavailability of a capable conditional distributed -GAN model has become another motivation behind the current work. Essentially, we revisited the FL paradigms, locating one discriminator per client , and a generator in the central server that aggregates the updates coming from multiple discriminator s. Relativistic discriminator s in the client s are train ed on both iid and non-iid private data. We presented a detailed mathematical formulation of DRD-GAN and empirically evaluated our claims using CIFAR-10, MNIST, EuroSAT, and CelebA datasets.

Evaluation of Classification Performance of New Layered Convolutional Neural Network Architecture on Offline Handwritten Signature Images

While there are many verification studies on signature images using deep learning algorithms in the literature, there is a lack of studies on the classification of signature images. Signatures are used as a means of identification for banking, security controls, symmetry, certificates, and contracts. In this study, the aim was to design network architectures that work very fast in areas that require only signature images. For this purpose, a new Si-CNN network architecture with existing layers was designed. Afterwards, a new loss function and layer (Si-CL), a novel architecture using Si-CL as classification layer in Si-CNN to increase the performance of this architecture, was designed. This architecture was called Si-CNN+NC (New Classification). Si-CNN and Si-CNN+NC were trained with two datasets. The first dataset which was used for training is the “C-Signatures” (Classification Signatures) dataset, which was created to test these networks. The second dataset is the “Cedar” dataset, which is a benchmark dataset. The number of classes and sample numbers in the two datasets are symmetrical with each other. To compare the performance of the trained networks, four of the most well-known pre-trained networks, GoogleNet, DenseNet201, Inceptionv3, and ResNet50, were also trained with the two datasets with transfer learning. The findings of the study showed that the proposed network models can learn features from two different handwritten signature images and achieve higher accuracy than other benchmark models. The test success of the trained networks showed that the Si-CNN+NC network outperforms the others, in terms of both accuracy and speed. Finally, Si-CNN and Si-CNN+NC networks were trained with the gold standard dataset MNIST and showed superior performance. Due to its superior performance, Si-CNN and Si-CNN+NC can be used by signature experts as an aid in a variety of applications, including criminal detection and forgery.

A pure pyrochlore phase ferroelectric thin film diode for optoelectric artificial synapse

In this work, Au/Bi2Ti2O7/ITO ferroelectric diodes based on Schottky barrier were fabricated on ITO substrates by the sol-gel method. The device has a simple structure and good diode performance. Under UV light irradiation at 23.5 mW/cm2, the device exhibits an open-circuit voltage of 0.31 V and a short-circuit current of −0.0425 μA. It shows that the device has good optoelectronic properties. Au/Bi2Ti2O7/ITO ferroelectric diodes also exhibited behaviors similar to biological synapses, including long-term plasticity, short-term plasticity, paired-pulse facilitation, and spike-timing-dependent plasticity when stimulated by pulses. We also build convolutional neural networks and use the MNIST dataset for handwritten digit recognition.

Handwritten Digit Recognition

Handwritten digit recognition is the intelligence of computers to recognize digits written by humans. But it becomes one of the most challenging tasks for machines as handwritten digits are not perfect and can be made with many different: flavors, size, thickness. Thus, as a solution to this problem, Handwriting digit recognition model comes into picture. Many machine learning techniques have been employed to solve the handwritten digit recognition problem. This paper focuses on Neural Network (NN) approaches. Among the three famous NN approaches: deep neural network (DNN), deep belief network (DBN) and convolutional neural network (CNN), the specialization of CNN as compared to other NN of being able to detect pattern is what makes it so useful for recognizing handwritten digits. Humans can very easily see, read & write any handwritten digits, when written in proper format. Even if the digits are not written in proper format we can use our logic and predict what digit it could be. But It is a hard task for the machine to recognize handwritten digits as these are not perfect and can be made with many different flavors. Thus handwritten digit recognition is the solution to this problem which uses the image of a digit and recognizes the digit present in the image. The handwritten digit recognition is the ability of computers to recognize human handwritten digits. The goal is to build a model that can efficiently and reliably recognize the digits and output the proper result. Amongst all the other neural networks, working and implementing a model using Convolution Neural Network gives out the most precise results. It is most popularly used for analyzing images as well as for other data analysis or classification problems. CNN has hidden layers called convolutional layers. These layers work the same way as other layers do but here we need to specify the no of filters each layer should have These filters are actually what detects the pattern. Patterns could be edges, corners, circles or any complex other objects like eyes, ears or even deeper full dogs, cats, etc. Thus, the specialization of CNN as compared to other NN of being able to detect patterns is what makes it so useful for recognizing handwritten digits Key Word: Neural Networks; Convolutional Neural Networks (CNN); Image Processing; Optical Character Recognition (OCR); MNIST Dataset

Accelerating handwritten number classifier with pipelined Multiply-Accumulate units in neural networks

This research project explores the development and assessment of a Handwritten Number Classifier based on Neural Networks, employing the widely used MNIST dataset. The primary objective of this study is to enhance computational efficiency, and to achieve this, it concentrates on integrating and contrasting non-pipelined and pipelined structures within the Multiply-Accumulate (MAC) unit. The initial phase involved conducting MATLAB simulations, which yielded promising results regarding the accuracy of weight calculations. Subsequently, Hardware Description Language (HDL) testing was carried out to further validate the classifiers performance. In the HDL testing phase, the classifier incorporating the pipelined MAC unit demonstrated a substantial 42.9% enhancement in processing speed when compared to its non-pipelined counterpart. These results highlight the potential advantages of employing pipeline processing in neural network architectures, emphasizing its effectiveness in achieving faster and more efficient image classification, particularly when dealing with extensive datasets. In conclusion, this research project not only presents valuable insights into improving the efficiency of neural network-based image classifiers but also lays the groundwork for potential future endeavors. These future directions may include adapting the classifier to handle more complex datasets and addressing emerging challenges.

An Improved BGE-Adam Optimization Algorithm Based on Entropy Weighting and Adaptive Gradient Strategy

This paper introduces an enhanced variant of the Adam optimizer—the BGE-Adam optimization algorithm—that integrates three innovative technologies to augment the adaptability, convergence, and robustness of the original algorithm under various training conditions. Firstly, the BGE-Adam algorithm incorporates a dynamic β parameter adjustment mechanism that utilizes the rate of gradient variations to dynamically adjust the exponential decay rates of the first and second moment estimates (β1 and β2), the adjustment of β1 and β2 is symmetrical, which means that the rules that the algorithm considers when adjusting β1 and β2 are the same. This design helps to maintain the consistency and balance of the algorithm, allowing the optimization algorithm to adaptively capture the trending movements of gradients. Secondly, it estimates the direction of future gradients by a simple gradient prediction model, combining historic gradient information with the current gradient. Lastly, entropy weighting is integrated into the gradient update step. This strategy enhances the model’s exploratory nature by introducing a certain amount of noise, thereby improving its adaptability to complex loss surfaces. Experimental results on classical datasets, MNIST and CIFAR10, and gastrointestinal disease medical datasets demonstrate that the BGE-Adam algorithm has improved convergence and generalization capabilities. In particular, on the specific medical image gastrointestinal disease test dataset, the BGE-Adam optimization algorithm achieved an accuracy of 69.36%, a significant improvement over the 67.66% accuracy attained using the standard Adam algorithm; on the CIFAR10 test dataset, the accuracy of the BGE-Adam algorithm reached 71.4%, which is higher than the 70.65% accuracy of the Adam optimization algorithm; and on the MNIST dataset, the BGE-Adam algorithm’s accuracy was 99.34%, surpassing the Adam optimization algorithm’s accuracy of 99.23%. The BGE-Adam optimization algorithm exhibits better convergence and robustness. This research not only demonstrates the effectiveness of the combination of these three technologies but also provides new perspectives for the future development of deep learning optimization algorithms.

Handwritten Digit Recognition of MNIST dataset using Deep Learning Convolutional Neural Network (CNN)

With applications ranging from postal services to digitized document processing, handwritten digit identification is a key problem in the fields of machine learning and computer vision. By utilizing their capacity to automatically extract hierarchical characteristics from unprocessed pixel data, Convolutional Neural Networks (CNNs) have become effective instruments for addressing this job. This study presents a thorough investigation of a CNN-based method that uses the MNIST dataset to recognize handwritten numbers. We explore the design, implementation, and performance assessment of the CNN model, demonstrating its ability to achieve high accuracy on tasks involving the recognition of numbers. We also go over the significance of our results for the larger picture of image categorization and suggest directions for further investigation and advancement. Key Words: CNN, MNIST, Convolutional Layer, Pooling layer, Max Pooling, Neural Networks, Preprocessing, Dropout Layer, Activation layer, Rectified Linear Unit (ReLU), Epochs, MNIST dataset

Federated learning in smart home: A dynamic contract-based incentive approach with task preferences

Federated Learning (FL) is a new type of distributed machine learning paradigm aimed at addressing data privacy and security issues in the Internet of Things (IoT) context. In the training process of federated learning model of smart home, data owners need to constantly update cached data, which lead to resources consumption and service latency. Meanwhile, the update cost of the data owners will change with the passage of time when the model owner requests data update. Therefore, this paper first puts forward an incentive scheme based on two-period dynamic contract theory under the information asymmetry. The scheme can balance the weighted preference of the model owner for age of information (AoI) and service latency, as well as encourage more data owners to participate in model training to increase the utility of model owner. Then we formally prove the feasibility of the proposed dynamic contract, which satisfies the constraints of individual rationality and intertemporal incentive compatibility. The experimental results on MNIST dataset show that the accuracy of the proposed dynamic contract is improved by at least 4% compared with the existing contracts. Additionally, compared with traditional contract and consistent pricing strategy, the model owner obtains more profit from the proposed contract.

Federated variational generative learning for heterogeneous data in distributed environments

Distributedly training models across diverse clients with heterogeneous data samples can significantly impact the convergence of federated learning. Various novel federated learning methods address these challenges but often require significant communication resources and local computational capacity, leading to reduced global inference accuracy in scenarios with imbalanced label data distribution and quantity skew. To tackle these challenges, we propose FedVGL, a Federated Variational Generative Learning method that directly trains a local generative model to learn the distribution of local features and improve global target model inference accuracy during aggregation, particularly under conditions of severe data heterogeneity. FedVGL facilitates distributed learning by sharing generators and latent vectors with the global server, aiding in global target model training from mapping local data distribution to the variational latent space for feature reconstruction. Additionally, FedVGL implements anonymization and encryption techniques to bolster privacy during generative model transmission and aggregation. In comparison to vanilla federated learning, FedVGL minimizes communication overhead, demonstrating superior accuracy even with minimal communication rounds. It effectively mitigates model drift in scenarios with heterogeneous data, delivering improved target model training outcomes. Empirical results establish FedVGL's superiority over baseline federated learning methods under severe label imbalance and data skew condition. In a Label-based Dirichlet Distribution setting with α=0.01 and 10 clients using the MNIST dataset, FedVGL achieved an exceptional accuracy over 97% with the VGG-9 target model.

Federated Learning Implementation with Privacy Leakage Prevention for Hand-Written Digit Recognition

Federated learning (FL) has brought significant advantages to applications where collaborative learning should occur at multiple participating devices to enhance user experience in specific tasks. However, FL results in privacy leakage when n-1 clients collude to infer the model of another client. In this paper, we not only implement an FL framework but propose a methodology for preventing privacy leakage while realizing machine learning-based automatic hand-written digit recognition. Our framework supports the FL of deep networks where models trained locally are averaged. Two machine learning models Convolutional Neural Network (CNN) and Multilayer Perceptron (MLP) are implemented with FL. We proposed an algorithm, Federated Averaging with Privacy Leakage Prevention (FA-PLP), for model averaging to be done by the server. Our algorithm exploits differential privacy (DP) for realizing model averaging while getting rid of chances of privacy leakage. We evaluated our framework with two distributions of the MNIST dataset. Our empirical results revealed that FA-PLP with the CNN model could achieve the highest accuracy of 95.38%.

Spintronic Artificial Synapses Using Voltage‐Controlled Multilevel Magnetic States

AbstractNeuromorphic computing offers energy‐efficient computations for large‐scale data processing compared to conventional von Neumann computing. The artificial synapse, a key element for learning and memory operations in neuromorphic computing, requires multi‐state characteristics and the capability to change and store its states. The implementation of hardware‐based artificial synapses using nonvolatile memory provides significant advantages in terms of energy consumption and circuit area compared to their currently employed CMOS‐based counterparts. In this regard, spintronic devices have emerged as a promising candidate due to their desirable properties for artificial synapses, including multilevel formability, non‐volatility, and outstanding writing performance. In this study, spintronic artificial synapses utilizing voltage‐controlled multilevel magnetic states and energy‐ and area‐efficient artificial neural network architectures associated with them are demonstrated. The multilevel states are created by gradually modulating the magnetic easy‐axis orientation from perpendicular to in‐plane and vice versa, which is achieved either by sequentially applying gate voltage pulses or by adjusting the pulse width of the gate voltage. Based on these spintronic artificial synapses, convolutional neural network (CNN) and spiking neural network (SNN) architectures are constructed, demonstrating high recognition accuracy for the MNIST dataset with improved energy efficiency and a reduced circuit area.

MCAD: Multi-classification anomaly detection with relational knowledge distillation

With the wide application of deep learning in anomaly detection (AD), industrial vision AD has achieved remarkable success. However, current AD usually focuses on anomaly localization and rarely investigates anomaly classification. Furthermore, anomaly classification is currently requested for quality management and anomaly reason analysis. Therefore, it is essential to classify anomalies while improving the accuracy of AD. This paper designs a novel multi-classification AD (MCAD) framework to achieve high-accuracy AD with an anomaly classification function. In detail, the proposal model based on relational knowledge distillation consists of two components. The first one employs a teacher–student AD model, utilizing a relational knowledge distillation approach to transfer the interrelationships of images. The teacher–student critical layer feature activation values are used in the knowledge transfer process to achieve anomaly detection. The second component realizes anomaly multi-classification using the lightweight convolutional neural network. Our proposal has achieved 98.95, 96.04, and 92.94% AUROC AD results on MNIST, FashionMNIST, and CIFAR10 datasets. Meanwhile, we earn 97.58 and 98.10% AUROC for AD and localization in the MVTecAD dataset. The average classification accuracy of anomaly classification has reached 76.37% in fifteen categories of the MVTec-AD dataset. In particular, the classification accuracy of the leather category has gained 95.24%. The results on the MVTec-AD dataset show that MCAD achieves excellent detection, localization, and classification results.

Single-pixel imaging based on self-supervised conditional mask classifier-free guidance

Reconstructing high-quality images at a low measurement rate is a pivotal objective of Single-Pixel Imaging (SPI). Currently, deep learning methods achieve this by optimizing the loss between the target image and the original image, thereby constraining the potential of low measurement values. We employ conditional probability to ameliorate this, introducing the classifier-free guidance model (CFG) for enhanced reconstruction. We propose a self-supervised conditional masked classifier-free guidance (SCM-CFG) for single-pixel reconstruction. At a 10% measurement rate, SCM-CFG efficiently completed the training task, achieving an average peak signal-to-noise ratio (PSNR) of 26.17 dB on the MNIST dataset. This surpasses other methods of photon imaging and computational ghost imaging. It demonstrates remarkable generalization performance. Moreover, thanks to the outstanding design of the conditional mask in this paper, it can significantly enhance the accuracy of reconstructed images through overlay. SCM-CFG achieved a notable improvement of an average of 7.3 dB in overlay processing, in contrast to only a 1 dB improvement in computational ghost imaging. Subsequent physical experiments validated the effectiveness of SCM-CFG.

Differentially private federated learning with Laplacian smoothing

Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. However, an adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models. In this paper, we investigate a utility enhancement scheme based on Laplacian smoothing for differentially private federated learning (DP-Fed-LS), to improve the statistical precision of parameter aggregation with injected Gaussian noise without losing privacy budget. Our key observation is that the aggregated gradients in federated learning often enjoy a type of smoothness, i.e. sparsity in a graph Fourier basis with polynomial decays of Fourier coefficients as frequency grows, which can be exploited by the Laplacian smoothing efficiently. Under a prescribed differential privacy budget, convergence error bounds with tight rates are provided for DP-Fed-LS with uniform subsampling of heterogeneous non-iid data, revealing possible utility improvement of Laplacian smoothing in effective dimensionality and variance reduction, among others. Experiments over MNIST, SVHN, and Shakespeare datasets show that the proposed method can improve model accuracy with DP-guarantee and membership privacy under both uniform and Poisson subsampling mechanisms.