Neural Network Architecture Search Research Articles

SANAS-Net: spatial attention neural architecture search for breast cancer detection

<p>The utilization of mammography images plays a vital role in the prompt detection and treatment of breast cancer. Breast imaging techniques aid medical professionals in assessing the dimensions, morphology, and spatial orientation of breast lesions, facilitating the differentiation between benign and malignant conditions. Breast tissue can vary widely in terms of density, composition, and structure, leading to complexities in distinguishing between benign and malignant conditions. The primary contribution of this paper is the proposal of a spatial attention-based neural architecture search network (SANAS-Net) technique that incorporates a spatial attention mechanism, enabling the model to learn and prioritize key regions within mammograms (MMs). Multi-head attention is employed within the transformer blocks to effectively capture a wide range of spatial relations and feature interactions. Global contextual information was integrated into the transformer blocks by means of introducing positional embeddings. Several practical studies have been undertaken to verify the effectiveness of our methodology in identifying fully attentive networks that exhibit good performance in distinguishing between malignant and benign breast cancer cases. The experimental study reached a test accuracy of 89.95%, which is way higher than previously proposed algorithms for mammography imagebased breast cancer detection.</p>

Efficient and lightweight convolutional neural network architecture search methods for object classification

Determining the architecture of deep learning models is a complex task. Several automated search techniques have been proposed, but these methods typically require high-performance graphics processing units (GPUs), manual parameter adjustments, and specific training approaches. This study introduces an efficient, lightweight convolutional neural network architecture search approach tailored for object classification. It features an optimized search space design and a novel controller design.This study introduces a refined search space design incorporating optimizations in both spatial and operational aspects. The focus is on the synergistic integration of convolutional units, dimension reduction units, and the stacking of Convolutional Neural Network (CNN) architectures. To enhance the search space, ShuffleNet modules are integrated, reducing the number of parameters and training time. Additionally, BlurPool is implemented in the dimension reduction unit operation to achieve translational invariance, alleviate the gradient vanishing problem, and optimize unit compositions. Moreover, an innovative controller model, Stage LSTM, is proposed based on Long Short-Term Memory (LSTM) to generate lightweight architectural sequences. In conclusion, the refined search space design and the Stage LSTM controller model are synergistically combined to establish an efficient and lightweight architecture search technique termed Stage and Lightweight Network Architecture Search (SLNAS).The experimental results highlight the superior performance of the optimized search space design, primarily when implemented with the Stage LSTM controller model. This approach shows significantly improved accuracy and stability compared to random, traditional LSTM, and Genetic Algorithm (GA) controller models, with statistically significant differences. Notably, the Stage LSTM controller excels in accuracy while producing models with fewer parameters within the expanded architecture search space.The study adopts the Stage LSTM controller model due to its ability to approximate optimal sequence structures, particularly when combined with the optimized search space design, referred to as SLNAS. SLNAS's performance is evaluated through experiments and comparisons with other Neural Architecture Search (NAS) and object classification methods from different researchers. These experiments consider model parameters, hardware resources, model stability, and multiple datasets. The results show that SLNAS achieves a low error rate of 2.86 % on the CIFAR-10 dataset after just 0.2 days of architecture search, matching the performance of manually designed models but using only 2 % of the parameters. SLNAS consistently demonstrates robust performance across various image classification domains, with an approximate parameter count 700,000.To summarize, SLNAS emerges as a highly effective automated network architecture search method tailored for image classification. It streamlines the model design process, making it accessible to researchers without specialized knowledge in deep learning. Optimizing this method unlocks the full potential of deep learning across diverse research areas. Interested parties can publicly access the source code and pre-trained models through the following link: https://github.com/huanyu-chen/LNASG-and-SLNAS-model.

Neural network architecture search model for thermal radiation in dense particulate systems

In the CFD-DEM simulation of the many dense particulate systems, particle-scale thermal radiation is an important heat transfer mode under high temperatures. In this work, neural network architecture search model with different layer connection matrix is applied to the view factor regression of the thermal radiation in dense particulate systems. Deep neural networks with 3346 feasible architectures are evaluated by the HyperBand algorithm to find the local optimal solution. Neural architecture search model trained by the big data of the view factor gives a good prediction of the macroscopic radiative properties and it is in general agreement with the empirical correlations and experimental data. The particle–wall radiation decreases strongly with the distance and the maximum interaction depth is about 2.0 times the sphere diameter. The trained deep neural network model provides an efficient data-driven closure to discuss the thermal radiation of the particle–particle and particle–wall interactions in particle bed.

Computer vision for eye diseases detection using pre‐trained deep learning techniques and raspberry Pi

AbstractEarly diagnosis of eye diseases is very important to prevent visual impairment and guide appropriate treatment methods. This paper presents a unique approach that can detect numerous eye diseases automatically. Initially, this approach used the pre‐trained ImageNet models that provides various pre‐trained models for training the acquired data. The existing data sets are composed of 645 data images acquired clinically, represented by two groups of subjects as healthy and others holding the proposed eye defect like cataracts, foreign bodies, glaucoma, subconjunctival haemorrhage, and viral conjunctivitis. Followed by comparisons of the pre‐trained model's coefficients and prediction performance. Later, the first‐class execution model is integrated within the Raspberry Pi staging and the real‐time digital camera detection. The evaluation process used the confusion matrix, model accuracy, precision factor, recall coefficient, F1 score, and the Matthews Correlation Coefficient (MCC). Resulting in the performance of these pre‐trained ImageNet models used in this study represented by 93% (InceptionResNetV2), 90% (MobileNet), 86% (Residual Network ResNet50), 85% (InceptionV3), 78% (Visual Geometry Group VGG19), and 72% (Neural Architecture Search Network NASNetMobile). The results show that the InceptionResNetV2 achieved the highest performance. This proposed approach shows its efficiency and strength by early detection of the subject's unhealthy eyes through real‐time monitoring in the field of ophthalmology.

Enhanced Gradient for Differentiable Architecture Search.

In recent years, neural architecture search (NAS) methods have been proposed for the automatic generation of task-oriented network architecture in image classification. However, the architectures obtained by existing NAS approaches are optimized only for classification performance and do not adapt to devices with limited computational resources. To address this challenge, we propose a neural network architecture search algorithm aiming to simultaneously improve the network performance and reduce the network complexity. The proposed framework automatically builds the network architecture at two stages: block-level search and network-level search. At the stage of block-level search, a gradient-based relaxation method is proposed, using an enhanced gradient to design high-performance and low-complexity blocks. At the stage of network-level search, an evolutionary multiobjective algorithm is utilized to complete the automatic design from blocks to the target network. The experimental results demonstrate that our method outperforms all evaluated hand-crafted networks in image classification, with an error rate of 3.18% on Canadian Institute for Advanced Research (CIFAR10) and an error rate of 19.16% on CIFAR100, both at network parameter size less than 1 M. Obviously, compared with other NAS methods, our method offers a tremendous reduction in designed network architecture parameters.

Neural architecture search via similarity adaptive guidance

Evolutionary neural network architecture search (ENAS) has attracted the attention of many experts due to its global optimization capabilities to automatically search for convolutional neural network architectures based on the target task. The current search space for ENAS is not to design a fully structured network, but to search for smaller cell architectures to reduce search costs. However, blind search strategies do not effectively utilize the potential experience of the population. In order to utilize the potential experience learned by the current population to guide the evolutionary search of the population, we propose a similarity guided neural network architecture search algorithm based on cell architecture, which utilizes the similarity between pairwise architectures in the population as empirical knowledge learned by the population. Our proposed algorithm provides a novel method for calculating architecture similarity, which calculates architecture similarity separately from the cell and macro-structure. Then we decouple the connections and operations in the cell and calculate connection and operation similarity separately. In addition, we propose adaptive similarity selection and binary tournament selection strategies to enhance the algorithm’s global and local search capabilities and effectively explore the search space. Finally, we design an improved single-point crossover operator to enhance the local search ability of the evolutionary operator. The experimental results show that SAGNAS is a competitive algorithm that achieves 97.44% and 81.60% in CIFAR10 and CIFAR100 with only 1.9 GPU-days spent.

Defect analysis of 3D printed object using transfer learning approaches

Additive manufacturing (AM) is rapidly gaining traction across diverse industries, including healthcare, aerospace, and automotive, mainly due to its ability to enhance production efficiency. However, a critical challenge in AM is the early detection of defects, which can significantly reduce production costs and improve productivity. Addressing this need, our study explored the effectiveness of machine learning (ML) approaches, mainly focusing on transfer learning (TL) models, for defect detection in 3D-printed objects. We utilized a range of TL models such as visualgeometrygroup (VGG)-16, inception-residualnetwork (InceptionResNet)-V2, residual network (ResNet)-50, mobilenetwork (MobileNet)-V2, VGG19, ResNet101, extreme inception (Xception), efficientnetwork (EfficientNet)-B0, EfficientNetB7, efficientnetworkV2medium (EfficientNetV2M), and neural architecture search network large (NASNetLarge), analyzing images of 3D-printed objects across three datasets using various statistical measures. An ablation study was also conducted using adaptive moment estimation (Adam), stochastic gradient descent (Sgd), and root mean square propagation (Rmsprop) optimizers. We identified that TL models such as VGG16, MobileNetV2, InceptionResNetV2, and NASNetLarge significantly outperform others, especially when optimized with Adam and Rmsprop. Conversely, we found that models like EfficientNetB0, EfficientNetB7, and EfficientNetV2M exhibited lower performance when paired with the Sgd optimizer. Additionally, Local Interpretable Model-agnostic (LIME)-based approaches are used to provide explanations of the models’ predictions. These findings offer substantial insights into model optimization and integration, aiming to enhance AM product quality through advanced image-based monitoring and inspection.

Improved tomato leaf disease classification through adaptive ensemble models with exponential moving average fusion and enhanced weighted gradient optimization.

Tomato is one of the most popular and most important food crops consumed globally. The quality and quantity of yield by tomato plants are affected by the impact made by various kinds of diseases. Therefore, it is essential to identify these diseases early so that it is possible to reduce the occurrences and effect of the diseases on tomato plants to improve the overall crop yield and to support the farmers. In the past, many research works have been carried out by applying the machine learning techniques to segment and classify the tomato leaf images. However, the existing machine learning-based classifiers are not able to detect the new types of diseases more accurately. On the other hand, deep learning-based classifiers with the support of swarm intelligence-based optimization techniques are able to enhance the classification accuracy, leading to the more effective and accurate detection of leaf diseases. This research paper proposes a new method for the accurate classification of tomato leaf diseases by harnessing the power of an ensemble model in a sample dataset of tomato plants, containing images pertaining to nine different types of leaf diseases. This research introduces an ensemble model with an exponential moving average function with temporal constraints and an enhanced weighted gradient optimizer that is integrated into fine-tuned Visual Geometry Group-16 (VGG-16) and Neural Architecture Search Network (NASNet) mobile training methods for providing improved learning and classification accuracy. The dataset used for the research consists of 10,000 tomato leaf images categorized into nine classes for training and validating the model and an additional 1,000 images reserved for testing the model. The results have been analyzed thoroughly and benchmarked with existing performance metrics, thus proving that the proposed approach gives better performance in terms of accuracy, loss, precision, recall, receiver operating characteristic curve, and F1-score with values of 98.7%, 4%, 97.9%, 98.6%, 99.97%, and 98.7%, respectively.

Ancient mural dynasty recognition algorithm based on a neural network architecture search

A neural network model needs to be manually designed for ancient mural dynasty recognition, and this paper proposes an ancient mural dynasty recognition algorithm that is based on a neural architecture search (NAS). First, the structural edge information of mural images is extracted for use by the neural network model in recognizing mural missions. Second, an NAS algorithm that is based on contrast selection (CS) simplifies the architecture search to an incremental CS and then searches for the optimal network architecture on the mural dataset. Finally, the identified optimal network architecture is used for training and testing to complete the mural dynasty recognition task. The results show that the top accuracy of the proposed method on the mural dataset is 88.10%, the recall rate is 87.52%, and the precision rate is 87.69%. Each evaluation index used by the neural network model is superior to that of classical network models such as AlexNet and ResNet-50. Compared with NAS methods such as ASNG and MIGO, the accuracy of mural dynasty recognition is higher by an average of 4.27% when using the proposed method. The proposed method is verified on CIFAR-10, CIFAR-100, ImageNet16-120 and other datasets and achieves a good recognition accuracy in the NAS-bench-201 search space, which averages 93.26%, 70.73% and 45.34%, respectively, on the abovementioned datasets.

Empowering EEG motor imagery classification with deep transfer learning approach

AbstractThe brain‐computer interface (BCI) enables individuals with impairments to interact with the real world without relying on the neuromuscular pathway. BCI leverages artificial intelligence (AI) models for control. It can capture brain activity patterns associated with mental processes and convert them into commands for actuators. One promising application of BCI is in rehabilitation centres. When compared to traditional methods integrated with machine learning (ML) approaches that were initially explored in the past decade, there is a relatively limited number of studies utilizing deep learning (DL) approaches in BCI applications. A novel approach has been developed for the automatic recognition of motor imagery (MI) tasks. This article introduces the improved sparrow search algorithm with deep transfer learning‐based EEG motor imagery classification (ISSADTL‐EEGMIC) algorithm for BCIs. The presented ISSADTL‐EEGMIC algorithm primarily focuses on the automated classification of motor imagery tasks. To achieve this, the ISSADTL‐EEGMIC method first preprocesses EEG signals using the wavelet packet decomposition (WPD) technique. Additionally, it employs the neural architecture search network (NASNet) technique for feature extraction. Moreover, the ISSADTL‐EEGMIC model performs the classification process using the stacked sparse autoencoder (SSAE) model. Furthermore, hyperparameter optimization of the SSAE model is performed using the ISSA technique, which helps achieve maximum classification performance. The results from the simulation of the ISSADTL‐EEGMIC algorithm have been examined using two benchmark datasets. The experimental findings suggest that it outperforms recent approaches in terms of performance.

A Novel IoT-based Arrhythmia Detection System with ECG Signals Using a Hybrid Convolutional Neural Network and Neural Architecture Search Network

A Novel IoT-based Arrhythmia Detection System with ECG Signals Using a Hybrid Convolutional Neural Network and Neural Architecture Search Network

Enhancing fish freshness prediction using NasNet-LSTM

Fish freshness is an essential feature of the fishing industry because it affects the safety and quality of the product. Precisely estimating seafood freshness is critical for consumer pleasure and waste reduction. This study presented a fish freshness prediction framework using two datasets from Kaggle, which were combined but highly imbalanced. Both upscaling (SMOTEENN) and downscaling (Random Under Sampler) methods were used to address the dataset imbalance. Neural Architecture Search Network (NasNet) and Long Short-Term Memory Networks (LSTM) models were employed to extract features from images. A feature selection technique was also applied to identify the most relevant features from the extracted features. The proposed NasNet-LSTM approach achieved impressive Matthew's correlation coefficient (MCC) and Cohen's kappa coefficient (KC) scores of 99.1%. The models were also cross-validated using a 5-fold method, resulting in MCC and KC values of 97%. Moreover, the p-value and confidence intervals of the proposed method were analyzed.

A novel transfer method with neural network architecture searching to predict asymptomatic of “akizuki” pear cork spot disorder on near-infrared spectroscopy

“Akizuki” pear cork spot disorder is a physiological disease, the ability to effectively diagnose the diseased fruit can directly affect the fruit quality. To improve the diagnostic performance of asymptomatic samples, this paper proposes a novel spectral transfer modelling method based on neural network architecture searching, named TranNAS_NIR. The experiments show that the model by the proposed TranNAS_NIR method is more effective. The accuracy of the target domain test set reaches 82.61%, which is 5.3% better than the model with transfer component analysis (TCA) and 10.5% better than the model constructed only relying on the target domain data and without the transfer learning method. The novel transfer method with neural network architecture search proposed in this paper for the asymptomatic prediction model of “Akizuki” pear cork spot disorder also further provides a theoretical basis for the NIR model improvement of other asymptomatic diseases.

ETNAS: An energy consumption task-driven neural architecture search

Neural Architecture Search (NAS) is crucial in the field of sustainable computing as it facilitates the development of highly efficient and effective neural networks. However, it cannot automate the deployment of neural networks to accommodate specific hardware resources and task requirements. This paper introduces ETNAS, which is a hardware-aware multi-objective optimal neural network architecture search algorithm based on the differentiable neural network architecture search method (DARTS). The algorithm searches for a lower-power neural network architecture with guaranteed inference accuracy by modifying the loss function of the differentiable neural network architecture search. We modify the dense network in DARTS to simultaneously search for networks with a lower memory footprint, enabling them to run on memory-constrained edge-end devices. We collected data on the power consumption and time consumption of numerous common operators on FPGA and Domain-Specific Architectures (DSA). The experimental results demonstrate that ETNAS achieves comparable accuracy performance and time efficiency while consuming less power compared to state-of-the-art algorithms, thereby validating its effectiveness in practical applications and contributing to the reduction of carbon emissions in intelligent cyber–physical systems (ICPS) edge computing inference.

AutoKeras for Fake News Identification in Arabic: Leveraging Deep Learning with an Extensive Dataset

Social media and the World Wide Web have led to a worrying rise in spreading false information, which presents a significant worldwide issue. Identifying and preventing false information is crucial in promoting an informed and knowledgeable society. The identification of false information, specifically in the Arabic dialect, presents inherent difficulties due to its diverse characteristics and linguistic intricacies. This study implements AutoKeras, a deep learning-based machine learning framework. Using advanced optimization techniques, the neural network architecture search, hyperparameter adjustments, and model selection can all be automated in AutoKeras. Therefore, it is suitable for our fake news detection task. The methodology employs proficient deep learning algorithms and natural language processing methods to acquire distinct characteristics that enable accurate differentiation between genuine and fake news. The present study uses various sources, including news websites, social media platforms, and blogs, to construct the dataset. The AutoKeras-based approach is superior to multiple state-of-the-art approaches to detecting fabricated news in Arabic, as evidenced by the experimental results. The suggested method outperforms 93.2% accuracy in identifying fake news, demonstrating its superior efficacy. This demonstrates the great promise of the deep learning-based Auto model for detecting false information.

Graph neural network architecture search for rotating machinery fault diagnosis based on reinforcement learning

In order to improve the accuracy of fault diagnosis, researchers are constantly trying to develop new diagnostic models. However, limited by the inherent thinking of human beings, it has always been difficult to build a pioneering architecture for rotating machinery fault diagnosis. In order to solve this problem, this paper uses reinforcement learning algorithm based on adjacency matrix to carry out network architecture search (NAS) of rotating machinery fault diagnosis model. A reinforcement learning agent for deep deterministic policy gradient (DDPG) is developed based on actor–critic neural networks. The observation state of reinforcement learning is used to develop the graph neural network (GNN) diagnosis model, and the diagnosis accuracy is fed back to the agent as a reward for updating the reinforcement learning parameters. The MFPT bearing fault datasets and the developed gear pitting fault experimental data are used to validate the proposed network architecture search method based on reinforcement learning (RL-NAS). The proposed method is proved to be practical and effective in various aspects such as fault diagnosis ability, search space, search efficiency and multi-working condition performance.

CAM-NAS: An Efficient and Interpretable Neural Architecture Search Model Based on Class Activation Mapping

Artificial intelligence (AI) has made rapid progress in recent years, but as the complexity of AI models and the need to deploy them on multiple platforms gradually increases, the design of network model structures for specific platforms becomes more difficult. A neural network architecture search (NAS) serves as a solution to help experts discover new network structures that are suitable for different tasks and platforms. However, traditional NAS algorithms often consume time and many computational resources, especially when dealing with complex tasks and large-scale models, and the search process can become exceptionally time-consuming and difficult to interpret. In this paper, we propose a class activation graph-based neural structure search method (CAM-NAS) to address these problems. Compared with traditional NAS algorithms, CAM-NAS does not require full training of submodels, which greatly improves the search efficiency. Meanwhile, CAM-NAS uses the class activation graph technique, which makes the searched models have better interpretability. In our experiments, we tested CAM-NAS on an NVIDIA RTX 3090 graphics card and showed that it can evaluate a submodel in only 0.08 seconds, which is much faster than traditional NAS methods. In this study, we experimentally evaluated CAM-NAS using the CIFAR-10 and CIFAR-100 datasets as benchmarks. The experimental results show that CAM-NAS achieves very good results. This not only proves the efficiency of CAM-NAS, but also demonstrates its powerful performance in image classification tasks.

Research on an ultrasonic detection method for weld defects based on neural network architecture search

In order to further reduce the subjectivity of network design and improve the ability of model feature extraction, an ultrasonic detection method for weld defects based on neural network architecture search is proposed. Through the designed multi-level and multi-branch search space and an untrained architecture search and evaluation method, an efficient defect classification network was automatically constructed to complete the task of weld defect classification. Experiments were carried out on a self-constructed data set, and compared with the manually designed model, the classification accuracy of defect types reached 95.26% when the number of parameters was only 7.3 M. Compared with the model constructed using neural network architecture search, the proposed method can reduce the searching time to 8.29% of the baseline model while weighing multiple conflicting objectives, which proved the efficiency and effectiveness of the proposed method.

ANAS: Asymptotic NAS for large-scale proxyless search and multi-task transfer learning

Neural Architecture Search (NAS) is an emerging solution to design a lightweight network for researchers to obtain a trade-off between accuracy and speed, releasing researchers from tedious repeated trials. However, the main shortcoming of NAS is its high and unstable memory consumption of the search work, especially for large-scale tasks. In this study, the proposed Asymptotic Neural Architecture Search network (ANAS) achieved a proxyless search for large-scale tasks with economic and stable memory consumption. Instead of proxy search like other NAS algorithms, ANAS achieved the large-scale proxyless search that directly learns deep neural network architecture for target task. ANAS reduced the peak value of memory consumption by an asymptotic method, and kept the memory consumption stable by the linkage change of a series of key indexes. A pruning operation and efficient candidate operations decreased the total memory consumption. Finally, ANAS achieved a good trade-off between accuracy and speed for classification tasks on CIFAR-10, CIFAR-100, and ImageNet datasets. Besides, except for the classification task, it achieved excellent multi-task transfer learning ability for implementing the segmentation task on CamVid and Cityscapes. ANAS reached 22.8% test errs with 5 M parameter on ImageNet, and 72.9 mIoU (mean Intersection over Union) with 119.9 FPS (Frames Per Second) on Cityscapes dataset.

EPC-DARTS: Efficient partial channel connection for differentiable architecture search

With weight-sharing and continuous relaxation strategies, the differentiable architecture search (DARTS) proposes a fast and effective solution to perform neural network architecture search in various deep learning tasks. However, unresolved issues, such as the inefficient memory utilization, and the poor stability of the search architecture due to channels randomly selected, which has even caused performance collapses, are still perplexing researchers and practitioners. In this paper, a novel efficient channel attention mechanism based on partial channel connection for differentiable neural architecture search, termed EPC-DARTS, is proposed to address these two issues. Specifically, we design an efficient channel attention module, which is applied to capture cross-channel interactions and assign weight based on channel importance, to dramatically improve search efficiency and reduce memory occupation. Moreover, only partial channels with higher weights in the mixed calculation of operation are used through the efficient channel attention mechanism, and thus unstable network architectures obtained by the random selection operation can also be avoided in the proposed EPC-DARTS. Experimental results show that the proposed EPC-DARTS achieves remarkably competitive performance (CIFAR-10/CIFAR-100: a test accuracy rate of 97.60%/84.02%), compared to other state-of-the-art NAS methods using only 0.2 GPU-Days.