MNIST Research Articles

Local differential privacy federated learning based on heterogeneous data multi-privacy mechanism

Federated learning enables the development of robust models without accessing users data directly. However, recent studies indicate that federated learning remains vulnerable to privacy leakage. To address this issue, local differential privacy mechanisms have been incorporated into federated learning. Nevertheless, local differential privacy will reduce the availability of data. To explore the balance between privacy budgets and data availability in federated learning, we propose federated learning for clustering hierarchical aggregation with adaptive piecewise mechanisms under multiple privacy-FedAPCA as a way to balance the relationship between privacy preservation and model accuracy. First, we introduce an adaptive piecewise mechanism that dynamically adjusts perturbation intervals based on the data ranges across different layers of the model, ensuring minimized perturbation variance while maintaining the same level of privacy. Second, we propose two dynamic privacy budget allocation methods, which are allocating the privacy budget based on global accuracy and global loss, and allocating the privacy budget based on local accuracy and loss, to ensure that better model accuracy can be achieved under the same privacy budget. Finally, we propose a clustering hierarchical aggregation method in the model aggregation stage, and the model is updated and aggregated after the unbiased estimation of the disturbance in each cluster according to the variance of each layer. FedAPCA improves the balance between privacy preservation and model accuracy. Our experimental results, comparing FedAPCA with the SOTA multi-privacy local differential privacy federated learning frameworks on the MNIST and CIFAR-10 datasets, demonstrate that FedAPCA improves model accuracy by 1%–2%.

Oxygen-Plasma-Treated Al/TaOX/Al Resistive Memory for Enhanced Synaptic Characteristics.

In this study, we investigate the impact of O2 plasma treatment on the performance of Al/TaOX/Al-based resistive random-access memory (RRAM) devices, focusing on applications in neuromorphic systems. Comparative analysis using scanning electron microscopy and X-ray photoelectron spectroscopy confirmed the differences in chemical composition between O2-plasma-treated and untreated RRAM cells. Direct-current measurements showed that O2-plasma-treated RRAM cells exhibited significant improvements over untreated RRAM cells, including higher on/off ratios, improved uniformity and distribution, longer retention times, and enhanced durability. The conduction mechanism is investigated by current-voltage (I-V) curve fitting. In addition, paired-pulse facilitation (PPF) is observed using partial short-term memory. Furthermore, 3- and 4-bit weight tuning with auto-pulse-tuning algorithms was achieved to improve the controllability of the synapse weight for the neuromorphic system, maintaining retention times exceeding 103 s in the multiple states. Neuromorphic simulation with an MNIST dataset is conducted to evaluate the synaptic device.

Text-Driven Video Prediction

Current video generation models usually convert signals indicating appearance and motion received from inputs (e.g., image and text) or latent spaces (e.g., noise vectors) into consecutive frames, fulfilling a stochastic generation process for the uncertainty introduced by latent code sampling. However, this generation pattern lacks deterministic constraints for both appearance and motion, leading to uncontrollable and undesirable outcomes. To this end, we propose a new task called Text-driven Video Prediction (TVP). Taking the first frame and text caption as inputs, this task aims to synthesize the following frames. Specifically, appearance and motion components are provided by the image and caption separately. The key to addressing the TVP task depends on fully exploring the underlying motion information in text descriptions, thus facilitating plausible video generation. In fact, this task is intrinsically a cause-and-effect problem, as the text content directly influences the motion changes of frames. To investigate the capability of text in causal inference for progressive motion information, our TVP framework contains a Text Inference Module (TIM), producing step-wise embeddings to regulate motion inference for subsequent frames. In particular, a refinement mechanism incorporating global motion semantics guarantees coherent generation. Extensive experiments are conducted on Something-Something V2 and Single Moving MNIST datasets. Experimental results demonstrate that our model achieves better results over other baselines, verifying the effectiveness of the proposed framework.

Label-Only Membership Inference Attack Based on Model Explanation

It is well known that machine learning models (e.g., image recognition) can unintentionally leak information about the training set. Conventional membership inference relies on posterior vectors, and this task becomes extremely difficult when the posterior is masked. However, current label-only membership inference attacks require a large number of queries during the generation of adversarial samples, and thus incorrect inference generates a large number of invalid queries. Therefore, we introduce a label-only membership inference attack based on model explanations. It can transform a label-only attack into a traditional membership inference attack by observing neighborhood consistency and perform fine-grained membership inference for vulnerable samples. We use feature attribution to simplify the high-dimensional neighborhood sampling process, quickly identify decision boundaries and recover a posteriori vectors. It also compares different privacy risks faced by different samples through finding vulnerable samples. The method is validated on CIFAR-10, CIFAR-100 and MNIST datasets. The results show that membership attributes can be identified even using a simple sampling method. Furthermore, vulnerable samples expose the model to greater privacy risks.

Trainable Reference Spikes Improve Temporal Information Processing of SNNs With Supervised Learning.

Spiking neural networks (SNNs) are the next-generation neural networks composed of biologically plausible neurons that communicate through trains of spikes. By modifying the plastic parameters of SNNs, including weights and time delays, SNNs can be trained to perform various AI tasks, although in general not at the same level of performance as typical artificial neural networks (ANNs). One possible solution to improve the performance of SNNs is to consider plastic parameters other than just weights and time delays drawn from the inherent complexity of the neural system of the brain, which may help SNNs improve their information processing ability and achieve brainlike functions. Here, we propose reference spikes as a new type of plastic parameters in a supervised learning scheme in SNNs. A neuron receives reference spikes through synapses providing reference information independent of input to help during learning, whose number of spikes and timings are trainable by error backpropagation. Theoretically, reference spikes improve the temporal information processing of SNNs by modulating the integration of incoming spikes at a detailed level. Through comparative computational experiments, we demonstrate using supervised learning that reference spikes improve the memory capacity of SNNs to map input spike patterns to target output spike patterns and increase classification accuracy on the MNIST, Fashion-MNIST, and SHD data sets, where both input and target output are temporally encoded. Our results demonstrate that applying reference spikes improves the performance of SNNs by enhancing their temporal information processing ability.

Proton-gated organic thin-film transistors for leaky integrate-and-fire convolutional spiking neural networks

Artificial spiking neurons, integral to the functionality of spiking neural networks, are designed to mimic the information transmission via discrete spikes in biological nervous systems. Traditional approaches that necessitate the charging of capacitors and the inclusion of discharge circuits for neuron membrane potential integration and leakage, present challenges in terms of cost and space efficiency. To overcome the challenges, this work proposes a hardware leaky integrate-and-fire neuron based on organic thin-film transistors. Under the electric field, the ion dynamics in the gate electrolyte can mimic the processes of membrane potential integration, leakage, and reset in spiking neurons. The convolutional spiking neural networks composed of such organic spiking neurons achieves excellent recognition rates (∼97.26 %) on the MNIST dataset. This indicates that the organic spiking neuron has enormous potential in next-generation non-von Neumann neuromorphic computing.

Uncertainty-Aware Federated Reinforcement Learning for Optimizing Accuracy and Energy in Heterogeneous Industrial IoT

The Internet of Things (IoT) technology has revolutionized various industries by allowing data collection, analysis, and decision-making in real time through interconnected devices. However, challenges arise in implementing Federated Learning (FL) in heterogeneous industrial IoT environments, such as maintaining model accuracy with non-Independent and Identically Distributed (non-IID) datasets and straggler IoT devices, ensuring computation and communication efficiency, and addressing weight aggregation issues. In this study, we propose an Uncertainty-Aware Federated Reinforcement Learning (UA-FedRL) method that dynamically selects epochs of individual clients to effectively manage heterogeneous industrial IoT devices and improve accuracy, computation, and communication efficiency. Additionally, we introduce the Predictive Weighted Average Aggregation (PWA) method to tackle weight aggregation issues in heterogeneous industrial IoT scenarios by adjusting the weights of individual models based on their quality. The UA-FedRL addresses the inherent complexities and challenges of implementing FL in heterogeneous industrial IoT environments. Extensive simulations in complex IoT environments demonstrate the superior performance of UA-FedRL on both MNIST and CIFAR-10 datasets compared to other existing approaches in terms of accuracy, communication efficiency, and computation efficiency. The UA-FedRL algorithm attain an accuracy of 96.83% on the MNIST dataset and 62.75% on the CIFAR-10 dataset, despite the presence of 90% straggler IoT devices, attesting to its robust performance and adaptability in different datasets.

Defense against Adversarial Attacks in Image Recognition Based on Multilayer Filters

The security and privacy of a system are urgent issues in achieving secure and efficient learning-based systems. Recent studies have shown that these systems are susceptible to subtle adversarial perturbations applied to inputs. Although these perturbations are difficult for humans to detect, they can easily mislead deep learning classifiers. Noise injection, as a defense mechanism, can offer a provable defense against adversarial attacks by reducing sensitivity to subtle input changes. However, these methods face issues of computational complexity and limited adaptability. We propose a multilayer filter defense model, drawing inspiration from filter-based image denoising techniques. This model inserts a filtering layer after the input layer and before the convolutional layer, and incorporates noise injection techniques during the training process. This model substantially enhances the resilience of image classification systems to adversarial attacks. We also investigated the impact of various filter combinations, filter area sizes, standard deviations, and filter layers on the effectiveness of defense. The experimental results indicate that, across the MNIST, CIFAR10, and CIFAR100 datasets, the multilayer filter defense model achieves the highest average accuracy when employing a double-layer Gaussian filter (filter area size of 3×3, standard deviation of 1). We compared our method with two filter-based defense models, and the experimental results demonstrated that our method attained an average accuracy of 71.9%, effectively enhancing the robustness of the image recognition classifier against adversarial attacks. This method not only performs well on small-scale datasets but also exhibits robustness on large-scale datasets (miniImageNet) and modern models (EfficientNet and WideResNet).

Bidirectional Decoupled Distillation for Heterogeneous Federated Learning.

Federated learning enables multiple devices to collaboratively train a high-performance model on the central server while keeping their data on the devices themselves. However, due to the significant variability in data distribution across devices, the aggregated global model's optimization direction may differ from that of the local models, making the clients lose their personality. To address this challenge, we propose a Bidirectional Decoupled Distillation For Heterogeneous Federated Learning (BDD-HFL) approach, which incorporates an additional private model within each local client. This design enables mutual knowledge exchange between the private and local models in a bidirectional manner. Specifically, previous one-way federated distillation methods mainly focused on learning features from the target class, which limits their ability to distill features from non-target classes and hinders the convergence of local models. To solve this limitation, we decompose the network output into target and non-target class logits and distill them separately using a joint optimization of cross-entropy and decoupled relative-entropy loss. We evaluate the effectiveness of BDD-HFL through extensive experiments on three benchmarks under IID, Non-IID, and unbalanced data distribution scenarios. Our results show that BDD-HFL outperforms state-of-the-art federated distillation methods across five baselines, achieving at most 3% improvement in average classification accuracy on the CIFAR-10, CIFAR-100, and MNIST datasets. The experiments demonstrate the superiority and generalization capability of BDD-HFL in addressing personalization challenges in federated learning.

Tackling Non-IID Data and Data Poisoning in Federated Learning using Adversarial Synthetic Data

Federated learning (FL) involves joint model training by various devices while preserving the privacy of their data. However, it presents a challenge of dealing with heterogeneous data located on participating devices. This issue can further be complicated by the appearance of malicious clients, aiming to sabotage the training process by poisoning local data. In this context, a problem of differentiating between poisoned and non-IID data appears. To address it, a technique utilizing data-free synthetic data generation is proposed, using a reverse concept of adversarial attack. Adversarial inputs allow for improving the training process by measuring clients' coherence, favoring trustworthy participants. Experimental results, obtained from the image classification tasks for MNIST, EMNIST, and CIFAR-10 datasets are reported and analyzed.

Sparse Personalized Federated Learning.

Federated learning (FL) is a collaborative machine learning technique to train a global model (GM) without obtaining clients' private data. The main challenges in FL are statistical diversity among clients, limited computing capability among clients' equipment, and the excessive communication overhead between the server and clients. To address these challenges, we propose a novel sparse personalized FL scheme via maximizing correlation (FedMac). By incorporating an approximated l1 -norm and the correlation between client models and GM into standard FL loss function, the performance on statistical diversity data is improved and the communicational and computational loads required in the network are reduced compared with nonsparse FL. Convergence analysis shows that the sparse constraints in FedMac do not affect the convergence rate of the GM, and theoretical results show that FedMac can achieve good sparse personalization, which is better than the personalized methods based on the l2 -norm. Experimentally, we demonstrate the benefits of this sparse personalization architecture compared with the state-of-the-art personalization methods (e.g., FedMac, respectively, achieves 98.95%, 99.37%, 90.90%, 89.06%, and 73.52% accuracy on the MNIST, FMNIST, CIFAR-100, Synthetic, and CINIC-10 datasets under non-independent and identically distributed (i.i.d.) variants).

Identifying alternately poisoning attacks in federated learning online using trajectory anomaly detection method

Implicit poisoning in federated learning is a significant threat, with malicious nodes subtly altering gradient parameters each round, making detection difficult. This study investigates this problem, revealing that temporal analysis alone struggles to identify such covert attacks, which can bypass online methods like cosine similarity and clustering. Common detection methods rely on offline analysis, resulting in delayed responses. However, recalculating gradient updates reveals distinct characteristics of malicious clients. Based on this finding, we designed a privacy-preserving detection algorithm using trajectory anomaly detection. Singular values of matrices are used as features, and an improved Isolation Forest algorithm processes these to detect malicious behavior. Experiments on MNIST, FashionMNIST, and CIFAR-10 datasets show our method achieves 94.3% detection accuracy and a false positive rate below 1.2%, indicating its high accuracy and effectiveness in detecting implicit model poisoning attacks.

Implementing handwritten text recognition using deep learning with TensorFlow: An MNIST dataset approach

Handwritten text recognition (HTR) is a pivotal technology with extensive applications in document digitization, postal automation, and educational tools. This paper delves into the implementing a deep learning-based system for recognizing handwritten digits using TensorFlow and the MNIST dataset. The MNIST dataset, a widely-used benchmark, comprises 60,000 training images and 10,000 testing images of handwritten digits, each standardized to a 28x28 pixel grayscale format. Leveraging the power of Convolutional Neural Networks (CNNs), our model effectively extracts features and classifies digits with high accuracy. The model architecture consists of multiple convolutional layers for feature extraction, pooling layers for dimensionality reduction, and fully connected layers for classification. Preprocessing steps include normalizing pixel values and one-hot encoding the labels, ensuring the data is optimally formatted for training. The TensorFlow framework, known for its robustness and scalability, facilitates the development and deployment of the model. Through a series of experiments, the model demonstrates impressive performance, achieving high accuracy on the MNIST test set. This paper underscores the potential of deep learning in handwritten text recognition. It sets the stage for future enhancements, such as recognizing more complex handwritten texts and optimizing the system for practical applications. The results highlight the effectiveness of deep learning techniques in overcoming the challenges associated with handwritten text recognition, paving the way for advanced, real-world implementations.

Image reconstruction through a nonlinear scattering medium via deep learning

Image reconstruction through the opaque medium has great significance in fields of biophotonics, optical imaging, mesoscopic physics, and optical communications. Previous researches are limited in the simple linear scattering process. Here, we develop a nonlinear speckle decoder network, which can reconstruct the phase information of the fundamental frequency wave via the nonlinear scattering signal. Further, we validate the ability of our model to recover simple and complex structures by using MNIST and CIFAR data sets, respectively. We then show that the model is able to restore the image information through different sets of nonlinear diffusers and reconstruct the image of a kind of completely unseen object category. The proposed method paves the way to nonlinear scattering imaging and information encryption.

Unrecognizable yet identifiable: Image distortion with preserved embeddings

Biometric authentication systems play a crucial role in modern security systems. However, maintaining the balance of privacy and integrity of stored biometrics derivative data while achieving high recognition accuracy is often challenging. Addressing this issue, we introduce an innovative image transformation technique that effectively renders facial images unrecognizable to the eye while maintaining their identifiability by neural network models, which allows the distorted photo version to be stored for further verification. While initially intended for biometrics systems, the proposed methodology can be used in various artificial intelligence applications to distort the visual data and keep the derived features close. By experimenting with widely used datasets LFW and MNIST, we show that it is possible to build the distortion that changes the image content by more than 70% while maintaining the same recognition accuracy. We compare our method with previously state-of-the-art approaches. We publically release the source code.11https://github.com/ZamDimon/distortion-generator/tree/v1.0.0.

Forecasting the Right Crop Nutrients for Specific Crops Based on Collected Data Using an Artificial Neural Network (ANN)

In farming technologies, it is difficult to properly provide the accurate crop nutrients for respective crops. For this reason, farmers are experiencing enormous problems. Although various types of machine learning (deep learning and convolutional neural networks) have been used to identify crop diseases, as has crop classification-based image processing, they have failed to forecast accurate crop nutrients for various crops, as crop nutrients are numerical instead of visual. Neural networks represent an opportunity for the precision agriculture sector to more accurately forecast crop nutrition. Recent technological advancements in neural networks have begun to provide greater precision, with an array of opportunities in pattern recognition. Neural networks represent an opportunity to effectively solve numerical data problems. The aim of the current study is to estimate the right crop nutrients for the right crops based on the data collected using an artificial neural network. The crop data were collected from the MNIST dataset. To forecast the precise nutrients for the crops, ANN models were developed. The entire system was simulated in a MATLAB environment. The obtained results for forecasting accurate nutrients were 99.997%, 99.996%, and 99.997% for validation, training, and testing, respectively. Therefore, the proposed algorithm is suitable for forecasting accurate crop nutrients for the crops.

Single-pixel object classification using ordered illumination patterns

The fixed-ordering illumination patterns lack an effective information interchange with objects. This leads to the light intensity values carrying random feature information, affecting the model's building and recognition capability. Furthermore, this method needs numerous illumination pattern projection counts, which results in low sampling efficiency. Therefore, we propose a new optimized-ordering illumination patterns method. In this paper, we first integrate fixed-ordering illumination patterns with category-representative images to obtain light intensity values that carry category feature information. Subsequently, the mechanisms of intra-category feature importance and inter-category feature differences are introduced to reorder the light intensity values and eliminate index numbers to generate an optimized-ordering index array. Finally, based on this index array, illumination patterns are extracted from the original fixed ordering method to form the new optimized-ordering illumination patterns. Experimental results show that in simulation experiments using the MNIST, Fashion MNIST, TRI, and Multi-category datasets, the optimized ordering method achieved recognition accuracies of 90.00%, 80.00%, 80.00%, and 80.00% with only 30 sampling times. Actual tests with physical objects such as mini numbers and clothing also confirmed these findings. The method offers a new research approach by enhancing recognition accuracy and sampling efficiency in single-pixel object classification.

Design of Diffractive Neural Networks for Solving Different Classification Problems at Different Wavelengths

We consider the problem of designing a diffractive neural network (DNN) consisting of a set of sequentially placed phase diffractive optical elements (DOEs) and intended for the optical solution of several given classification problems at different operating wavelengths, so that each classification problem is solved at the corresponding wavelength. The problem of calculating the DNN is formulated as the problem of minimizing a functional that depends on the functions of the diffractive microrelief height of the DOEs constituting the DNN and represents the error in solving the given classification problems at the operating wavelengths. We obtain explicit and compact expressions for the derivatives of this functional, and using them, we formulate a gradient method for the DNN calculation. Using this method, we design DNNs for solving the following three classification problems at three different wavelengths: the problem of classifying handwritten digits from the MNIST database, the problem of classifying fashion products from the Fashion MNIST database, and the problem of classifying ten handwritten letters from the EMNIST database. The presented simulation results of the designed DNNs demonstrate the high performance of the proposed method.

Semantic memory-based dynamic neural network using memristive ternary CIM and CAM for 2D and 3D vision.

The brain is dynamic, associative, and efficient. It reconfigures by associating the inputs with past experiences, with fused memory and processing. In contrast, AI models are static, unable to associate inputs with past experiences, and run on digital computers with physically separated memory and processing. We propose a hardware-software co-design, a semantic memory-based dynamic neural network using a memristor. The network associates incoming data with the past experience stored as semantic vectors. The network and the semantic memory are physically implemented on noise-robust ternary memristor-based computing-in-memory (CIM) and content-addressable memory (CAM) circuits, respectively. We validate our co-designs, using a 40-nm memristor macro, on ResNet and PointNet++ for classifying images and three-dimensional points from the MNIST and ModelNet datasets, which achieves not only accuracy on par with software but also a 48.1 and 15.9% reduction in computational budget. Moreover, it delivers a 77.6 and 93.3% reduction in energy consumption.

Ultralow Power In-Sensor Neuronal Computing with Oscillatory Retinal Neurons for Frequency-Multiplexed, Parallel Machine Vision.

In-sensor and near-sensor computing architectures enable multiply accumulate operations to be carried out directly at the point of sensing. In-sensor architectures offer dramatic power and speed improvements over traditional von Neumann architectures by eliminating multiple analog-to-digital conversions, data storage, and data movement operations. Current in-sensor processing approaches rely on tunable sensors or additional weighting elements to perform linear functions such as multiply accumulate operations as the sensor acquires data. This work implements in-sensor computing with an oscillatory retinal neuron device that converts incident optical signals into voltage oscillations. A computing scheme is introduced based on the frequency shift of coupled oscillators that enables parallel, frequency multiplexed, nonlinear operations on the inputs. An experimentally implemented 3 × 3 focal plane array of coupled neurons shows that functions approximating edge detection, thresholding, and segmentation occur in parallel. An example of inference on handwritten digits from the MNIST database is also experimentally demonstrated with a 3 × 3 array of coupled neurons feeding into a single hidden layer neural network, approximating a liquid-state machine. Finally, the equivalent energy consumption to carry out image processing operations, including peripherals such as the Fourier transform circuits, is projected to be <20 fJ/OP, possibly reaching as low as 15 aJ/OP.