Network Architecture Search Research Articles

Network architecture search methods for constructing efficient fault diagnosis models in rotating machinery

Abstract The development of high-performance fault diagnosis models for specific tasks requires substantial expertise. Neural Architecture Search (NAS) offers a promising solution, but most NAS methodologies are hampered by lengthy search durations and low efficiency, and few researchers have applied these methods within the fault diagnosis domain. This paper introduces a novel differentiable architecture search (DARTS) method tailored for constructing efficient fault diagnosis models for rotating machinery, designed to rapidly and effectively search for network models suitable for specific datasets. Specifically, this study constructs a completely new and advanced search space, incorporating various efficient, lightweight convolutional operations to reduce computational complexity. To enhance the stability of the differentiable network architecture search process and reduce fluctuations in model accuracy, this study proposes a novel Multi-scale Pyramid Squeeze Attention (MPSA) module. This module aids in the learning of richer multi-scale feature representations and adaptively recalibrates the weights of multi-dimensional channel attention. The proposed method was validated on two rotating machinery fault datasets, demonstrating superior performance compared to manually designed networks and general network search methods, with notably improved diagnostic effectiveness.

Method for Rail Surface Defect Detection Based on Neural Network Architecture Search

Abstract This study addresses the inherent limitations of implementing neural network architecture search algorithms for rail surface defect detection, including low search efficiency and the oversight of edge features on the rail surface. A sophisticated multi-level neural network architecture search framework is proposed that integrates and emphasizes rail surface edge features. The framework utilizes the Z-Score normalization method to quantify the edge concern of rail surface defect samples, combined with an Edge-Loss function to enhance edge feature recognition capabilities. Furthermore, acknowledging the sensitivity of defect features to spatial resolution changes, a multi-level neural network architecture search space is meticulously designed. In the cell-level search space, a method combining partial channel sampling with operation pruning is employed to enhance model search efficiency and regularization. In the network-level search space, optimal paths for resolution change are established, allowing for the screening and aggregation of defect features at various levels to facilitate the adaptive extraction of multi-scale edge defect features. Experimental outcomes indicate that this method significantly reduces computational resource usage by approximately 75% and increases mIOU by 2.6% relative to traditional architecture search methods. Moreover, it demonstrates robust capability in accurately recognizing defective edges on rail surfaces, thereby substantiating the method's effectiveness.

A novel automated neural network architecture search method of air target intent recognition

Modern air battlefield operations are characterized by flexibility and change, and the battlefield evolves rapidly and intricately. However, traditional air target intent recognition methods, which mainly rely on manually designed neural network models, find it difficult to maintain sustained and excellent performance in such a complex and changing environment. To address the problem of the adaptability of neural network models in complex environments, we propose a lightweight Transformer model (TransATIR) with a strong adaptive adjustment capability, based on the characteristics of air target intent recognition and the neural network architecture search technique. After conducting extensive experiments, it has been proved that TransATIR can efficiently extract the deep feature information from battlefield situation data by utilizing the neural architecture search algorithm, in order to quickly and accurately identify the real intention of the target. The experimental results indicate that TransATIR significantly improves recognition accuracy compared to the existing state-of-the-art methods, and also effectively reduces the computational complexity of the model.

Automated fault diagnosis of rotating machinery using sub domain greedy Network Architecture search

Network Architecture Search (NAS) automates hyperparameter adjustments in deep learning models, offering a potent solution for building intelligent fault diagnosis models. Despite its potential, NAS faces challenges in this application. The primary issues include the excessive time required when searching across the full data domain and the inefficacy of the common random search strategy in yielding high-performance sub networks. This study addresses these challenges through several key initiatives: It defines a specific library of hyperparameters tailored for intelligent fault diagnosis tasks; constructs sub data domain model clusters to reduce the computational costs associated with NAS; and introduces a hyperparameter search strategy leveraging a greedy algorithm, which enhances the search efficiency for optimal diagnostic sub network and minimizes its prediction errors. When applied to datasets involving gear and bearing faults, the proposed method demonstrates a reduction in both the time cost of searches and the prediction errors of sub networks, compared to traditional Network Architecture Search. Moreover, it outperforms manually tuned deep learning models by simultaneously optimizing multiple sets of hyperparameter and automatically generating higher accuracy diagnostic sub networks.

Anti-islanding image detection and optimization of distributed power supply using neural network architecture search

Anti-islanding detection (AILD) for distributed power sources plays an important role on the stable operation of power grid systems and the safety of electrical systems. In order to improve the detection accuracy, we propose neural network architecture search (NAS) based approach for anti-islanding image detection and optimization of distributed power sources. It combines the fast region-based convolutional neural network (Faster-R-CNN) with the differentiable architecture search (Darts), utilizing electrical signal image data acquired from cameras or photovoltaic (PV) inverters, thereby enhancing the accuracy and efficiency of detection. We conduct a comparative analysis of two methods for obtaining distributed power supply (DPS) anti-islanding image data: camera-based acquisition and PV inverter-based acquisition. Our observation reveals that the image data acquired through cameras is more conducive for the learning process of DCNN. The proposed algorithm was compared with other convolutional neural network (CNN) models, validating its performance superiority. This provides a novel perspective and methodology for the advancement of AILD for distributed power sources, and enhances the security and stability of DPS.

Constrained multi-scale dense connections for biomedical image segmentation

Multi-scale dense connection has been widely used in the biomedical image community to enhance the segmentation performance. In this way, features from all or most scales are aggregated or iteratively fused. However, by analyzing the details, we discover that some connections involving distant scales may not contribute to, or even harm, the performance, while they always introduce a noticeable increase in computational cost. In this paper, we propose constrained multi-scale dense connections (CMDC) for biomedical image segmentation. In contrast to current general lightweight approaches, we first introduce two methods, a naive method and a network architecture search (NAS)-based method, to remove redundant connections and verify the optimal connection configuration, thereby improving overall efficiency and accuracy. The results demonstrate that the two approaches obtain a similar optimal configuration in which most features at the adjacent scales are connected. Then, we applied the optimal configuration to various backbone networks to build constrained multi-scale dense networks (CMD-Net). Experimental results evaluated on eight image segmentation datasets covering biomedical images and natural images demonstrate the effectiveness of CMD-Net across a variety of backbone networks (FCN, U-Net, DeepLabV3, SegNet, FCNsa, ConvUNeXt) with a much lower increase in computational cost. Furthermore, CMD-Net achieves state-of-the-art performance on four publicly available datasets. We believe that the CMDC method can offer valuable insight for ways to engage in dense connectivity at multiple scales within communities. The source code has been made available at https://github.com/JerRuy/CMD-Net.

Research on Self-Learning Control Method of Reusable Launch Vehicle Based on Neural Network Architecture Search

Research on Self-Learning Control Method of Reusable Launch Vehicle Based on Neural Network Architecture Search

SN-CNN: A Lightweight and Accurate Line Extraction Algorithm for Seedling Navigation in Ridge-Planted Vegetables

In precision agriculture, after vegetable transplanters plant the seedlings, field management during the seedling stage is necessary to optimize the vegetable yield. Accurately identifying and extracting the centerlines of crop rows during the seedling stage is crucial for achieving the autonomous navigation of robots. However, the transplanted ridges often experience missing seedling rows. Additionally, due to the limited computational resources of field agricultural robots, a more lightweight navigation line fitting algorithm is required. To address these issues, this study focuses on mid-to-high ridges planted with double-row vegetables and develops a seedling band-based navigation line extraction model, a Seedling Navigation Convolutional Neural Network (SN-CNN). Firstly, we proposed the C2f_UIB module, which effectively reduces redundant computations by integrating Network Architecture Search (NAS) technologies, thus improving the model’s efficiency. Additionally, the model incorporates the Simplified Attention Mechanism (SimAM) in the neck section, enhancing the focus on hard-to-recognize samples. The experimental results demonstrate that the proposed SN-CNN model outperforms YOLOv5s, YOLOv7-tiny, YOLOv8n, and YOLOv8s in terms of the model parameters and accuracy. The SN-CNN model has a parameter count of only 2.37 M and achieves an mAP@0.5 of 94.6%. Compared to the baseline model, the parameter count is reduced by 28.4%, and the accuracy is improved by 2%. Finally, for practical deployment, the SN-CNN algorithm was implemented on the NVIDIA Jetson AGX Xavier, an embedded computing platform, to evaluate its real-time performance in navigation line fitting. We compared two fitting methods: Random Sample Consensus (RANSAC) and least squares (LS), using 100 images (50 test images and 50 field-collected images) to assess the accuracy and processing speed. The RANSAC method achieved a root mean square error (RMSE) of 5.7 pixels and a processing time of 25 milliseconds per image, demonstrating a superior fitting accuracy, while meeting the real-time requirements for navigation line detection. This performance highlights the potential of the SN-CNN model as an effective solution for autonomous navigation in field cross-ridge walking robots.

Augmenting machine learning of Grad–Shafranov equilibrium reconstruction with Green's functions

This work presents a method for predicting plasma equilibria in tokamak fusion experiments and reactors. The approach involves representing the plasma current as a linear combination of basis functions using principal component analysis of plasma toroidal current densities (Jt) from the EFIT-AI equilibrium database. Then utilizing EFIT's Green's function tables, basis functions are created for the poloidal flux (ψ) and diagnostics generated from the toroidal current (Jt). Similar to the idea of a physics-informed neural network (NN), this physically enforces consistency between ψ, Jt, and the synthetic diagnostics. First, the predictive capability of a least squares technique to minimize the error on the synthetic diagnostics is employed. The results show that the method achieves high accuracy in predicting ψ and moderate accuracy in predicting Jt with median R2 = 0.9993 and R2 = 0.978, respectively. A comprehensive NN using a network architecture search is also employed to predict the coefficients of the basis functions. The NN demonstrates significantly better performance compared to the least squares method with median R2 = 0.9997 and 0.9916 for Jt and ψ, respectively. The robustness of the method is evaluated by handling missing or incorrect data through the least squares filling of missing data, which shows that the NN prediction remains strong even with a reduced number of diagnostics. Additionally, the method is tested on plasmas outside of the training range showing reasonable results.

Efficient and lightweight convolutional neural network architecture search methods for object classification

Determining the architecture of deep learning models is a complex task. Several automated search techniques have been proposed, but these methods typically require high-performance graphics processing units (GPUs), manual parameter adjustments, and specific training approaches. This study introduces an efficient, lightweight convolutional neural network architecture search approach tailored for object classification. It features an optimized search space design and a novel controller design.This study introduces a refined search space design incorporating optimizations in both spatial and operational aspects. The focus is on the synergistic integration of convolutional units, dimension reduction units, and the stacking of Convolutional Neural Network (CNN) architectures. To enhance the search space, ShuffleNet modules are integrated, reducing the number of parameters and training time. Additionally, BlurPool is implemented in the dimension reduction unit operation to achieve translational invariance, alleviate the gradient vanishing problem, and optimize unit compositions. Moreover, an innovative controller model, Stage LSTM, is proposed based on Long Short-Term Memory (LSTM) to generate lightweight architectural sequences. In conclusion, the refined search space design and the Stage LSTM controller model are synergistically combined to establish an efficient and lightweight architecture search technique termed Stage and Lightweight Network Architecture Search (SLNAS).The experimental results highlight the superior performance of the optimized search space design, primarily when implemented with the Stage LSTM controller model. This approach shows significantly improved accuracy and stability compared to random, traditional LSTM, and Genetic Algorithm (GA) controller models, with statistically significant differences. Notably, the Stage LSTM controller excels in accuracy while producing models with fewer parameters within the expanded architecture search space.The study adopts the Stage LSTM controller model due to its ability to approximate optimal sequence structures, particularly when combined with the optimized search space design, referred to as SLNAS. SLNAS's performance is evaluated through experiments and comparisons with other Neural Architecture Search (NAS) and object classification methods from different researchers. These experiments consider model parameters, hardware resources, model stability, and multiple datasets. The results show that SLNAS achieves a low error rate of 2.86 % on the CIFAR-10 dataset after just 0.2 days of architecture search, matching the performance of manually designed models but using only 2 % of the parameters. SLNAS consistently demonstrates robust performance across various image classification domains, with an approximate parameter count 700,000.To summarize, SLNAS emerges as a highly effective automated network architecture search method tailored for image classification. It streamlines the model design process, making it accessible to researchers without specialized knowledge in deep learning. Optimizing this method unlocks the full potential of deep learning across diverse research areas. Interested parties can publicly access the source code and pre-trained models through the following link: https://github.com/huanyu-chen/LNASG-and-SLNAS-model.

Neural network architecture search model for thermal radiation in dense particulate systems

In the CFD-DEM simulation of the many dense particulate systems, particle-scale thermal radiation is an important heat transfer mode under high temperatures. In this work, neural network architecture search model with different layer connection matrix is applied to the view factor regression of the thermal radiation in dense particulate systems. Deep neural networks with 3346 feasible architectures are evaluated by the HyperBand algorithm to find the local optimal solution. Neural architecture search model trained by the big data of the view factor gives a good prediction of the macroscopic radiative properties and it is in general agreement with the empirical correlations and experimental data. The particle–wall radiation decreases strongly with the distance and the maximum interaction depth is about 2.0 times the sphere diameter. The trained deep neural network model provides an efficient data-driven closure to discuss the thermal radiation of the particle–particle and particle–wall interactions in particle bed.

Enhanced Gradient for Differentiable Architecture Search.

In recent years, neural architecture search (NAS) methods have been proposed for the automatic generation of task-oriented network architecture in image classification. However, the architectures obtained by existing NAS approaches are optimized only for classification performance and do not adapt to devices with limited computational resources. To address this challenge, we propose a neural network architecture search algorithm aiming to simultaneously improve the network performance and reduce the network complexity. The proposed framework automatically builds the network architecture at two stages: block-level search and network-level search. At the stage of block-level search, a gradient-based relaxation method is proposed, using an enhanced gradient to design high-performance and low-complexity blocks. At the stage of network-level search, an evolutionary multiobjective algorithm is utilized to complete the automatic design from blocks to the target network. The experimental results demonstrate that our method outperforms all evaluated hand-crafted networks in image classification, with an error rate of 3.18% on Canadian Institute for Advanced Research (CIFAR10) and an error rate of 19.16% on CIFAR100, both at network parameter size less than 1 M. Obviously, compared with other NAS methods, our method offers a tremendous reduction in designed network architecture parameters.

GLNAS: Greedy Layer-wise Network Architecture Search for low cost and fast network generation

The process of applying machine learning algorithms to practical problems can be a challenging and tedious task for non-experts. Previous research has sought to alleviate this burden by introducing automated machine learning techniques, including Network Architecture Search (NAS) and Differentiable Architecture Search (DARTS). However, these methods use a fixed number of layers and predefined skip connections which impose limitations on the generation of an optimal network architecture. In this paper, we propose a novel approach called Greedy Layer-wise Network Architecture Search (GLNAS), which trains network layers one after another and evaluates the network’s performance after each layer is added. GLNAS also assesses the effectiveness of skip connections between layers by testing various outputs of previous layers as an input to the current layer. Our experiment results demonstrate that the network generated by GLNAS requires fewer parameters (i.e., 3.5 millions in both CIFAR-10 and CIFAR-100 datasets) and GPU resources during the searching phase (i.e., 0.17 and 0.24 GPU days in CIFAR-10 and CIFAR-100 datasets respectively) than many existing methods.

Neural architecture search via similarity adaptive guidance

Evolutionary neural network architecture search (ENAS) has attracted the attention of many experts due to its global optimization capabilities to automatically search for convolutional neural network architectures based on the target task. The current search space for ENAS is not to design a fully structured network, but to search for smaller cell architectures to reduce search costs. However, blind search strategies do not effectively utilize the potential experience of the population. In order to utilize the potential experience learned by the current population to guide the evolutionary search of the population, we propose a similarity guided neural network architecture search algorithm based on cell architecture, which utilizes the similarity between pairwise architectures in the population as empirical knowledge learned by the population. Our proposed algorithm provides a novel method for calculating architecture similarity, which calculates architecture similarity separately from the cell and macro-structure. Then we decouple the connections and operations in the cell and calculate connection and operation similarity separately. In addition, we propose adaptive similarity selection and binary tournament selection strategies to enhance the algorithm’s global and local search capabilities and effectively explore the search space. Finally, we design an improved single-point crossover operator to enhance the local search ability of the evolutionary operator. The experimental results show that SAGNAS is a competitive algorithm that achieves 97.44% and 81.60% in CIFAR10 and CIFAR100 with only 1.9 GPU-days spent.

Ancient mural dynasty recognition algorithm based on a neural network architecture search

A neural network model needs to be manually designed for ancient mural dynasty recognition, and this paper proposes an ancient mural dynasty recognition algorithm that is based on a neural architecture search (NAS). First, the structural edge information of mural images is extracted for use by the neural network model in recognizing mural missions. Second, an NAS algorithm that is based on contrast selection (CS) simplifies the architecture search to an incremental CS and then searches for the optimal network architecture on the mural dataset. Finally, the identified optimal network architecture is used for training and testing to complete the mural dynasty recognition task. The results show that the top accuracy of the proposed method on the mural dataset is 88.10%, the recall rate is 87.52%, and the precision rate is 87.69%. Each evaluation index used by the neural network model is superior to that of classical network models such as AlexNet and ResNet-50. Compared with NAS methods such as ASNG and MIGO, the accuracy of mural dynasty recognition is higher by an average of 4.27% when using the proposed method. The proposed method is verified on CIFAR-10, CIFAR-100, ImageNet16-120 and other datasets and achieves a good recognition accuracy in the NAS-bench-201 search space, which averages 93.26%, 70.73% and 45.34%, respectively, on the abovementioned datasets.

Implementation of hand‐object pose estimation using SSD and YOLOV5 model for object grasping by SCARA robot

AbstractEnforcement of advanced deep learning methods in hand‐object pose estimation is an imperative method for grasping the objects safely during the human–robot collaborative tasks. The position and orientation of a hand‐object from a two‐dimensional image is still a crucial problem under various circumstances like occlusion, critical lighting, and salient region detection and blur images. In this paper, the proposed method uses an enhanced MobileNetV3 with single shot detection (SSD) and YOLOv5 to ensure the improvement in accuracy and without compromising the latency in the detection of hand‐object pose and its orientation. To overcome the limitations of higher computation cost, latency and accuracy, the Network Architecture Search and NetAdapt Algorithm is used in MobileNetV3 that perform the network search for parameter tuning and adaptive learning for multiscale feature extraction and anchor box offset adjustment due to auto‐variance of weight in the level of each layers. The squeeze‐and‐excitation block reduces the computation and latency of the model. Hard‐swish activation function and feature pyramid networks are used to prevent over fitting the data and stabilizing the training. Based on the comparative analysis of MobileNetV3 with its predecessor and YOLOV5 are carried out, the obtained results are 92.8% and 89.7% of precision value, recall value of 93.1% and 90.2%, mAP value of 93.3% and 89.2%, respectively. The proposed methods ensure better grasping for robots by providing the pose estimation and orientation of hand‐objects with tolerance of −1.9 to 2.15 mm along x, −1.55 to 2.21 mm along y, −0.833 to 1.51 mm along z axis and −0.233° to 0.273° along z‐axis.

Target aware network architecture search and compression for efficient knowledge transfer

Target aware network architecture search and compression for efficient knowledge transfer

An efficient evolutionary architecture search for variational autoencoder with alternating optimization and adaptive crossover

Variational autoencoder is a commonly unsupervised learning model. However, its complex structure hinders the utilization of the network architecture search algorithm to release researchers from tedious manual design. To design excellent architectures automatically, this paper proposes an efficient evolutionary architecture search for variational autoencoder with alternating optimization and adaptive crossover(AOC-VAE). Firstly, to alleviate the problem of large search space when automatically designing variational autoencoders, AOC-VAE designs an alternating optimized search mechanism based on the specific coupling of encoder and decoder in variational autoencoders, which reduces the original huge search space almost to half. Then, AOC-VAE can find quickly the optimal individual in the solution space by designing an adaptive crossover mechanism. In early evolutionary period, the structural differences between individuals are relatively significant, making crossover operations more inclined to exchange structural information between individuals. As evolution progresses, the individual structures in the population tend to be similar, and the exchange of information concentrates on the parameter. Finally, in the optimization process, a fitness evaluation mechanism based on dynamic weights is designed to accurately find out the outstanding individuals under the current optimization goal. Individual fitness in the population is more inclined to be affected by the current optimization goal, thus guiding the population to evolve according to the optimization goal at different stages. AOC-VAE is verified on MNIST, SVHN, CIFAR-10, and CIFAR-100 benchmark datasets and compared with 14 algorithms. The experimental results show that the VAE network structure designed by the AOC-VAE performs well in the image classification task.

Stability Analysis and Controller Synthesis Using Single-Hidden-Layer ReLU Neural Networks

This paper presents algorithms to solve analysis and controller synthesis problems for dynamical systems modeled as a recurrent single-hidden-layer rectified linear unit neural network (ReLU NN), or equivalently, a piecewise affine dynamical system. Such models are interesting since they may arise through the use of modern machine learning methods for system identification, or as closed-loop solutions in certain classes of model predictive control (MPC) problems. A key idea in the proposed approach is to use piecewise affine Lyapunov functions parametrized as ReLU neural networks, and similarly parametrized controllers. This compatible representation between the Lyapunov function and the dynamics simplifies the automation of analysis of and controller synthesis for learned models. Our method of verifying a candidate Lyapunov function is faster than methods based on mixed integer programming. We ‘learn’ controllers and Lyapunov functions using both weight updates and network architecture search, without gradients. We demonstrate the proposed algorithm on examples involving learned models, explicit MPC controllers, and constrained controller synthesis.

A Novel IoT-based Arrhythmia Detection System with ECG Signals Using a Hybrid Convolutional Neural Network and Neural Architecture Search Network

A Novel IoT-based Arrhythmia Detection System with ECG Signals Using a Hybrid Convolutional Neural Network and Neural Architecture Search Network