Candidate Architectures Research Articles

Gradient-Based Neural Architecture Search: A Comprehensive Evaluation

One of the challenges in deep learning involves discovering the optimal architecture for a specific task. This is effectively tackled through Neural Architecture Search (NAS). Neural Architecture Search encompasses three prominent approaches—reinforcement learning, evolutionary algorithms, and gradient descent—that have demonstrated noteworthy potential in identifying good candidate architectures. However, approaches based on reinforcement learning and evolutionary algorithms often necessitate extensive computational resources, requiring hundreds of GPU days or more. Therefore, we confine this work to a gradient-based approach due to its lower computational resource demands. Our objective encompasses identifying the optimal gradient-based NAS method and pinpointing opportunities for future enhancements. To achieve this, a comprehensive evaluation of the use of four major Gradient descent-based architecture search methods for discovering the best neural architecture for image classification tasks is provided. An overview of these gradient-based methods, i.e., DARTS, PDARTS, Fair DARTS and Att-DARTS, is presented. A theoretical comparison, based on search spaces, continuous relaxation strategy and bi-level optimization, for deriving the best neural architecture is then provided. The strong and weak features of these methods are also listed. Experimental results for comparing the error rate and computational cost of these gradient-based methods are analyzed. These experiments involved using bench marking datasets CIFAR-10, CIFAR-100 and ImageNet. The results show that PDARTS is better and faster among the examined methods, making it a potent candidate for automating Neural Architecture Search. By effectively conducting a comparative analysis, our research provides valuable insights and future research directions to address the criticism and gaps in the literature.

HGNAS++: Efficient Architecture Search for Heterogeneous Graph Neural Networks

Heterogeneous graphs are commonly used to describe networked data with multiple types of nodes and edges. Heterogeneous Graph Neural Networks (HGNNs) are powerful tools for analyzing heterogeneous graphs. However, designing neural architectures of HGNNs requires extensive domain knowledge and time-consuming manual work. Recently, neural architecture search algorithms have become popular in automatically designing neural architectures for homogeneous graph neural networks. In this paper, we present a Heterogeneous Graph Neural Architecture Search algorithm (HGNAS for short) which allows the automatic design of heterogeneous graph neural architectures. Specifically, HGNAS first designs a new search space based on existing popular HGNNs. Then, HGNAS uses a policy network as the controller to sample and find the best neural architecture from the designed search space by maximizing the expected accuracy of the selected architectures on a given validation dataset. Moreover, we design a new method HGNAS++ to improve the efficiency of HGNAS by training the RNN controller within a generative adversarial learning framework. The basic idea of HGNAS++ is to embed a pairwise ranker into the reinforcement learning based architecture search algorithm. The pairwise ranker can be taken as a discriminator which selects more accurate architectures between pairs of candidate architectures. Then, the RNN controller can be updated more efficiently by only using a relatively small number of candidate architectures selected by the pairwise ranker. Experiments on real-world heterogeneous graph datasets show that HGNAS is capable of designing novel HGNNs that beat the best human-invented HGNNs. On the benchmark datasets, HGNAS++ improves HGNAS in terms of evaluation cost, with a reduction of 50% of the evaluated candidate architectures and a decrease of 24% in search time on average. As a byproduct, HGNAS++ can find sparse yet powerful neural architectures for HGNNs.

ASMEvoNAS: Adaptive segmented multi-objective evolutionary network architecture search

Network architecture search (NAS) has attracted much attention as an automatic design technique of network architecture. In particular, multi-objective evolutionary algorithms (MOEAs) have been a popular kind of optimizer in NAS due to their global optimization capability. However, as a population-based iterative search method, MOEAs are subject to the unbearable computational cost of individual evaluation on multiple objectives at each generation, which affects their generalization ability and transferability of MOEA-based NAS. Therefore, an adaptive segmented multi-objective evolutionary network architecture search (ASMEvoNAS) method is proposed in this paper. Firstly, an adaptive segmented evaluation strategy is designed to adaptively select different but more suitable objectives to efficiently assess the candidate architectures at different evolutionary stages, instead of evaluating them by all the considered objectives simultaneously. Thus, the computational cost and complexity of the search process can be controlled and reduced to some extent. Secondly, a preference-based pre-selection strategy is designed to filter out the initialized architectures with high parameter quantities to reduce the total parameter scale of the whole population and memory consumption. Last, a novel desirable gene reservation-based crossover and a directed connection-based mutation are proposed to produce offspring. Experimental results show that ASMEvoNAS shows promising performance on CIFAR-10, CIFAR-100, and ImageNet with error rates of 2.21%, 15.57%, and 24.43% top-1, respectively. The proposed method reduces the search cost to 0.36 GPU-Days on CIFAR-10 while maintaining competitive classification performance compared to state-of-the-art networks. In addition, ASMEvoNAS presents superior performance when dealing with the considered transfer tasks, as well as the benchmark dataset of NAS.

Direct Vibrational Stark Shift Probe of Quasi-Fermi Level Alignment in Metal Nanoparticle Catalyst-Based Metal-Insulator-Semiconductor Junction Photoelectrodes.

Photoelectrodes consisting of metal-insulator-semiconductor (MIS) junctions are a promising candidate architecture for water splitting and for the CO2 reduction reaction (CO2RR). The photovoltage is an essential indicator of the driving force that a photoelectrode can provide for surface catalytic reactions. However, for MIS photoelectrodes that contain metal nanoparticles, direct photovoltage measurements at the metal sites under operational conditions remain challenging. Herein, we report a new in situ spectroscopic approach to probe the quasi-Fermi level of metal catalyst sites in heterogeneous MIS photoelectrodes via surface-enhanced Raman spectroscopy. Using a CO2RR photocathode, nanoporous p-type Si modified with Ag nanoparticles, as a prototype, we demonstrate a selective probe of the photovoltage of ∼0.59 V generated at the Si/SiOx/Ag junctions. Because it can directly probe the photovoltage of MIS heterogeneous junctions, this vibrational Stark probing approach paves the way for the thermodynamic evaluation of MIS photoelectrodes with varied architectural designs.

AutoCTS+: Joint Neural Architecture and Hyperparameter Search for Correlated Time Series Forecasting

Sensors in cyber-physical systems often capture interconnected processes and thus emit correlated time series (CTS), the forecasting of which enables important applications. The key to successful CTS forecasting is to uncover the temporal dynamics of time series and the spatial correlations among time series. Deep learning-based solutions exhibit impressive performance at discerning these aspects. In particular, automated CTS forecasting, where the design of an optimal deep learning architecture is automated, enables forecasting accuracy that surpasses what has been achieved by manual approaches. However, automated CTS solutions remain in their infancy and are only able to find optimal architectures for predefined hyperparameters and scale poorly to large-scale CTS. To overcome these limitations, we propose AutoCTS+, a joint, scalable framework, to automatically devise effective CTS forecasting models. Specifically, we encode each candidate architecture and accompanying hyperparameters into a joint graph representation. We introduce an efficient Architecture-Hyperparameter Comparator (AHC) to rank all architecture-hyperparameter pairs, and we then further evaluate the top-ranked pairs to select an architecture-hyperparameter pair as the final model. Extensive experiments on six benchmark datasets demonstrate that AutoCTS+ not only eliminates manual efforts but also is capable of better performance than manually designed and existing automatically designed CTS models. In addition, it shows excellent scalability to large CTS.

Steaming in Plain Sight

May 01 2023 Steaming in Plain Sight Angela Loescher-Montal, Angela Loescher-Montal Angela Loescher-Montal is a fourth-year dual degree student at MIT, currently completing her M. Arch and M. S. in Real Estate Development. She is focusing her research on the mediated nature of our urban realm and the increasing privatization of public space. She is also in the process of founding a start-up called Harness the Heat with three friends, which looks at the possibilities of re-channeling sources of wasted heat to heat outdoor spaces in the winter. Search for other works by this author on: This Site Google Scholar Latifa Alkhayat, Latifa Alkhayat Latifa Alkhayat is a Master of Architecture candidate at MIT. Her work delves into the themes of material processes and systems, ecology, resource extraction, and the national narratives that tie them. As an architectural researcher and practitioner, she has experience in Manama, London, and Cambridge. Search for other works by this author on: This Site Google Scholar Ous Abou Ras, Ous Abou Ras Ous Abou Ras is a graduate student in the M. Arch program at MIT. He is interested in how math and physical phenomenon can shape our built environment, and practices his intrigue by employing computational workflows. He holds a Bachelor's degree from the University of Toronto where he pursued a double major in Architecture and Physics and studied the ways in which the laws of physics shape our cities and architecture across cultures. Search for other works by this author on: This Site Google Scholar Jules Kleitman Jules Kleitman Search for other works by this author on: This Site Google Scholar Author and Article Information Angela Loescher-Montal Angela Loescher-Montal is a fourth-year dual degree student at MIT, currently completing her M. Arch and M. S. in Real Estate Development. She is focusing her research on the mediated nature of our urban realm and the increasing privatization of public space. She is also in the process of founding a start-up called Harness the Heat with three friends, which looks at the possibilities of re-channeling sources of wasted heat to heat outdoor spaces in the winter. Latifa Alkhayat Latifa Alkhayat is a Master of Architecture candidate at MIT. Her work delves into the themes of material processes and systems, ecology, resource extraction, and the national narratives that tie them. As an architectural researcher and practitioner, she has experience in Manama, London, and Cambridge. Ous Abou Ras Ous Abou Ras is a graduate student in the M. Arch program at MIT. He is interested in how math and physical phenomenon can shape our built environment, and practices his intrigue by employing computational workflows. He holds a Bachelor's degree from the University of Toronto where he pursued a double major in Architecture and Physics and studied the ways in which the laws of physics shape our cities and architecture across cultures. Jules Kleitman Online ISSN: 2572-7338 Print ISSN: 1091-711X © 2023 Angela Loescher-Montal, Latifa Alkhayat, Ous Abou Ras, & Jules Kleitman2023Angela Loescher-Montal, Latifa Alkhayat, Ous Abou Ras, & Jules Kleitman Thresholds (2023) (51): 226–229. https://doi.org/10.1162/thld_a_00803 Cite Icon Cite Permissions Share Icon Share Facebook Twitter LinkedIn MailTo Views Icon Views Article contents Figures & tables Video Audio Supplementary Data Peer Review Search Site Citation Angela Loescher-Montal, Latifa Alkhayat, Ous Abou Ras, Jules Kleitman; Steaming in Plain Sight. Thresholds 2023; (51): 226–229. doi: https://doi.org/10.1162/thld_a_00803 Download citation file: Ris (Zotero) Reference Manager EasyBib Bookends Mendeley Papers EndNote RefWorks BibTex toolbar search Search Dropdown Menu toolbar search search input Search input auto suggest filter your search All ContentAll JournalsThresholds Search Advanced Search This content is only available as a PDF. © 2023 Angela Loescher-Montal, Latifa Alkhayat, Ous Abou Ras, & Jules Kleitman2023Angela Loescher-Montal, Latifa Alkhayat, Ous Abou Ras, & Jules Kleitman Article PDF first page preview Close Modal You do not currently have access to this content.

Addressable and adaptable intercellular communication via DNA messaging

Engineered consortia are a major research focus for synthetic biologists because they can implement sophisticated behaviors inaccessible to single-strain systems. However, this functional capacity is constrained by their constituent strains’ ability to engage in complex communication. DNA messaging, by enabling information-rich channel-decoupled communication, is a promising candidate architecture for implementing complex communication. But its major advantage, its messages’ dynamic mutability, is still unexplored. We develop a framework for addressable and adaptable DNA messaging that leverages all three of these advantages and implement it using plasmid conjugation in E. coli. Our system can bias the transfer of messages to targeted receiver strains by 100- to 1000-fold, and their recipient lists can be dynamically updated in situ to control the flow of information through the population. This work lays the foundation for future developments that further utilize the unique advantages of DNA messaging to engineer previously-inaccessible levels of complexity into biological systems.

Machine-learning-driven Architectural Selection of Adders and Multipliers in Logic Synthesis

Designing high-performance adders and multiplier components for diverse specifications and constraints is of practical concern. However, selecting the best architecture for adder or multiplier, which largely affects the performance of synthesized circuits, is difficult. To tackle this difficulty, a machine-learning-driven approach is proposed for automatic architectural selection of adders and multipliers. It trains a machine learning model for classification through learning a number of existing design schemes and their performance data. Experimental results show that the proposed approach based on a multi-perception neural network achieves as high as 94% prediction accuracy with negligible inference time. On a CPU server, the proposed approach runs about 4× faster than a brute-force approach trying four candidate architectures and consumes 10%~20% less runtime than the DesignWare datapath generator for obtaining the optimal adder/multiplier circuit. The adder (multiplier) generated with the proposed approach achieves performance metrics close to the optimal and has 1.6% (5.2%) less area and 2.2% (7.1%) more worst negative slack averagely than that generated with the DesignWare datapath generator. Our experiment also shows that the proposed approach is not sensitive to the size of training subset.

SFA: Searching faster architectures for end-to-end automatic speech recognition models

Recently End-to-end (E2E) Automatic Speech Recognition (ASR) has been widely used due to its advantages over the hybrid method. Even though existing E2E ASR models have achieved impressive performance, they usually take a large model size and suffer from a slow inference speed in real-world applications. To obtain faster models for E2E ASR, we propose searching faster architectures with the help of neural architecture search (NAS) in this paper, named SFA. SFA consists of one search space that contains a set of candidate architectures and one search algorithm responsible for searching the optimal architecture from the search space. On one hand, SFA designs a topology-fused search space to integrate different topologies of existing architectures (e.g. Transformer, Conformer) and explore more brand-new ones. On the other hand, combined with the training criterion of E2E ASR, SFA develops a speed-aware differentiable search algorithm to search faster architectures according to target hardware devices. Additionally, a connectionist temporal classification based progressive search algorithm is proposed to reduce the difficulty of the architecture search and obtain better performance. On two commonly-used Mandarin datasets, SFA can effectively improve the inference speed of existing E2E ASR models with comparable performance and achieve at most 2.46×/ 1.98× CPU/GPU speedup than the best human-designed baselines.

Facilitating hardware-aware neural architecture search with learning-based predictive models

Neural architecture search (NAS), which automatically explores the efficient model design, has achieved ground-breaking advances in recent years. To achieve the optimal model latency on deployment platforms, a performance tuning process is usually needed to select reasonable parameters and implementations for each neural network operator. As the tuning process is time-consuming, it is impractical for tuning each candidate architecture generated in the search procedure. Recent NAS systems usually utilize theoretical metrics or rule-based heuristics on-device latency to approximately estimate the model performance. Nevertheless, we discovered that there is still a gap between the estimated latency and the optimal latency, potentially causing a sub-optimal solution for neural architecture search. This paper presents an accurate and efficient approach for estimating the practical model latency on target platforms, which employs lightweight learning-based predictive models (LBPMs) to assist to obtain the realistic deployment-time model latency with acceptable run-time overhead, thereby facilitating hardware-aware neural architecture search. We propose an LBPM-based NAS framework, LBPM-NAS, and evaluate it by searching model architectures for ImageNet classification and facial landmark localization tasks on various hardware platforms. Experimental results show that the LBPM-NAS achieves up to 2.4× performance boost compared with the baselines under the same-level accuracy.

Multiobjective Reinforcement Learning-Based Neural Architecture Search for Efficient Portrait Parsing.

This article dedicates to automatically explore efficient portrait parsing models that are easily deployed in edge computing or terminal devices. In the interest of the tradeoff between the resource cost and performance, we design the multiobjective reinforcement learning (RL)-based neural architecture search (NAS) scheme, which comprehensively balances the accuracy, parameters, FLOPs, and inference latency. Finally, under varying hyperparameter configurations, the search procedure emits a bunch of excellent objective-oriented architectures. The combination of two-stage training with precomputing and memory-resident feature maps effectively reduces the time consumption of the RL-based NAS method, so that we complete approximately 1000 search iterations in two GPU days. To accelerate the convergence of the lightweight candidate architecture, we incorporate knowledge distillation into the training of the search process. This also provides a reasonable evaluation signal to the RL controller that enables it to converge well. In the end, we conduct full training with outstanding Pareto-optimal architectures, so that a series of excellent portrait parsing models (with only approximately 0.3M parameters) is received. Furthermore, we directly transfer the architectures searched on CelebAMask-HQ (Portrait Parsing) to other portrait and face segmentation tasks. Finally, we achieve the state-of-the-art performance of 96.5% MIOU on EG1800 (portrait segmentation) and 91.6% overall F1 -score on HELEN (face labeling). That is, our models significantly surpass the artificial network on the accuracy, but with lower resource consumption and higher real-time performance.

RelativeNAS: Relative Neural Architecture Search via Slow-Fast Learning.

Despite the remarkable successes of convolutional neural networks (CNNs) in computer vision, it is time-consuming and error-prone to manually design a CNN. Among various neural architecture search (NAS) methods that are motivated to automate designs of high-performance CNNs, the differentiable NAS and population-based NAS are attracting increasing interests due to their unique characters. To benefit from the merits while overcoming the deficiencies of both, this work proposes a novel NAS method, RelativeNAS. As the key to efficient search, RelativeNAS performs joint learning between fast learners (i.e., decoded networks with relatively lower loss value) and slow learners in a pairwise manner. Moreover, since RelativeNAS only requires low-fidelity performance estimation to distinguish each pair of fast learner and slow learner, it saves certain computation costs for training the candidate architectures. The proposed RelativeNAS brings several unique advantages: 1) it achieves state-of-the-art performances on ImageNet with top-1 error rate of 24.88%, that is, outperforming DARTS and AmoebaNet-B by 1.82% and 1.12%, respectively; 2) it spends only 9 h with a single 1080Ti GPU to obtain the discovered cells, that is, 3.75× and 7875× faster than DARTS and AmoebaNet, respectively; and 3) it provides that the discovered cells obtained on CIFAR-10 can be directly transferred to object detection, semantic segmentation, and keypoint detection, yielding competitive results of 73.1% mAP on PASCAL VOC, 78.7% mIoU on Cityscapes, and 68.5% AP on MSCOCO, respectively. The implementation of RelativeNAS is available at https://github.com/EMI-Group/RelativeNAS.

Designing an Observing System to Study the Surface Biology and Geology (SBG) of the Earth in the 2020s.

Observations of planet Earth from space are a critical resource for science and society. Satellite measurements represent very large investments and United States (US) agencies organize their effort to maximize the return on that investment. The US National Research Council conducts a survey of Earth science and applications to prioritize observations for the coming decade. The most recent survey prioritized a visible to shortwave infrared imaging spectrometer and a multispectral thermal infrared imager to meet a range of needs for studying Surface Biology and Geology (SBG). SBG will be the premier integrated observatory for observing the emerging impacts of climate change by characterizing the diversity of plant life and resolving chemical and physiological signatures. It will address wildfire risk, behavior, and recovery as well as responses to hazards such as oil spills, toxic minerals in minelands, harmful algal blooms, landslides, and other geological hazards. The SBG team analyzed needed instrument characteristics (spatial, temporal, and spectral resolutions, measurement uncertainty) and assessed the cost, mass, power, volume, and risk of different architectures. We present an overview of the Research and Applications trade-study analysis of algorithms, calibration and validation needs, and societal applications with specifics of substudies detailed in other articles in this special collection. We provide a value framework to converge from hundreds down to three candidate architectures recommended for development. The analysis identified valuable opportunities for international collaboration to increase the revisit frequency, adding value for all partners, leading to a clear measurement strategy for an observing system architecture.

Experiences With Deep Learning Enhanced Steering Mechanisms for Debugging of Fundamental Cloud Services

Cloud architecture blueprints or reference architectures allow the reuse of existing knowledge and best practices when creating new cloud native solutions. Therefore, debugging reference architecture candidates (or their new versions) is an extremely crucial but tedious and time-consuming task due to the deployment of complex services in typical multi-tenant and non-deterministic environments. During the debugging/testing/maintenance scenarios, we might be able to achieve greater levels of test coverage (and eventually improved reliability) by modelling and verifying at least their most fundamental building blocks. The main objective of our work is to integrate the stochastic modelling and verification techniques based on deep learning methods into the debugging cycle in order to handle large state spaces more efficiently, i.e. by steering the process of traversing state space towards suspicious situations that may result in potential bugs in the actual system with smart steering during the traversal. For this purpose, our presented and illustrated approach combines (among others) Continuous Time Markov Chain modelling (CTMC) techniques with deep learning methods including autoencoder, Long Short-Term Memory (LSTM) and Graph Neural Network (GNN) models. Our experiences are summarized with widespread cloud design patterns including load balancing and service mesh topologies.

Convolutional neural network architecture search based on fractal decomposition optimization algorithm

This paper presents a new approach to design the architecture and optimize the hyperparameters of a deep convolutional neural network (CNN) via of the Fractal Decomposition Algorithm (FDA).This optimization algorithm was recently proposed to solve continuous optimization problems. It is based on a geometric fractal decomposition that divides the search space while searching for the best solution possible. As FDA is effective in single-objective optimization, in this work we aim to prove that it can also be successfully applied to fine-tuning deep neural network architectures.Moreover, a new formulation based on bi-level optimization is proposed to separate the architecture search composed of discrete parameters from hyperparameters’ optimization. This is motivated by the fact that automating the construction of deep neural architecture has been an important focus over recent years as manual construction is considerably time-consuming, error-prone, and requires in-depth knowledge. To solve the bi-level problem thus formulated, a random search is performed aiming to create a set of candidate architectures. Then, the best ones are finetuned using FDA. CIFAR-10 and CIFAR-100 benchmarks were used to evaluate the performance of the proposed approach. The results obtained are among the state of the art in the corresponding class of networks (low number of parameters and chained-structured CNN architectures). The results are emphasized by the fact that the whole process was performed using low computing power with only 3 NVIDIA V100 GPUs. The source code is available at https://github.com/alc1218/Convolutional-Neural-Network-Architecture-Search-Based-on-Fractal-Decomposition-Optimization.

Evolutionary Search for Complete Neural Network Architectures With Partial Weight Sharing

Neural architecture search (NAS) provides an automatic solution in designing network architectures. Unfortunately, the direct search for complete task-dependent network architectures is laborious since training and evaluating complete neural architectures over a large search space are computationally prohibitive. Recently, one-shot NAS (OSNAS) has attracted great attention in the NAS community because it significantly speeds up the candidate architecture evaluation procedure through weight sharing. However, the full weight sharing training paradigm in OSNAS may result in strong interference across candidate architectures and mislead the architecture search. To alleviate the problem, we propose a partial weight sharing OSNAS framework that directly evolves complete neural network architectures. In particular, we suggest a novel node representation scheme that randomly activates a subset of nodes of the one-shot model in each generation to reduce the weight coupling in the one-shot model. During the evolutionary search, a tailored crossover operator randomly samples the nodes from two parent individuals or a single parent to construct new candidate architectures, thus effectively constraining the degree of weight sharing. Furthermore, we introduce a new mutation operator that replaces the chosen nodes of the one-shot model with randomly generated nodes to enhance the exploratory capability. Finally, we encode a set of pyramidal convolution operations in the search space, enabling the evolved neural networks to capture different levels of details in the images. The proposed method is examined and compared with 26 state-of-the-art algorithms on ten image classification tasks, including CIFAR series, CINIC10, ImageNet, and MedMNIST series. The experimental results demonstrate that the proposed method can computationally much more efficiently find neural architectures that achieve comparable classification accuracy to the state-of-the-art designs.

Evolutionary Architectural Search for Generative Adversarial Networks

The Generative Adversarial Network (GAN) has shown powerfulness in various real-world artificial intelligence applications. However, its network architecture is generally designed through a manual trial-and-error process, which is relatively tedious, slow, and sub-optimal. This paper hence develops an evolutionary architectural search (EAS) technique to automate the entire design process of the GAN. In particular, different objective functions are used in the generator of a GAN as variation operators. This helps train the generator with various candidate architectures and their associated weights to play adversarial training against the discriminator of the GAN. Following evaluations by the discriminator, superior candidate generators survive to the next generation and evolve for potentially better architectures and their weights simultaneously, leading to more stabilized GANs with improved performance. The GAN designed through EAS is termed an EAS-GAN in this paper and is tested against existing evolutionary and other state-of-the-art GANs. The test results show that the EAS-GAN offers better generative performance overall, with the Fréchet inception distance scoring 22.1, 38.8, and 8.3 on the CIFAR-10, STL-10, and LSUN bedroom data sets, respectively.

How AI-enabled SDN technologies improve the security and functionality of industrial IoT network: Architectures, enabling technologies, and opportunities

The ongoing expansion of the Industrial Internet of Things (IIoT) is enabling the possibility of effective Industry 4.0, where massive sensing devices in heterogeneous environments are connected through dedicated communication protocols. This brings forth new methods and models to fuse the information yielded by the various industrial plant elements and generates emerging security challenges that we have to face, providing ad-hoc functions for scheduling and guaranteeing the network operations. Recently, the large development of Software-Defined Networking (SDN) and Artificial Intelligence (AI) technologies have made feasible the design and control of scalable and secure IIoT networks. This paper studies how AI and SDN technologies combined can be leveraged towards improving the security and functionality of these IIoT networks. After surveying the state-of-the-art research efforts in the subject, the paper introduces a candidate architecture for AI-enabled Software-Defined IIoT Network (AI-SDIN) that divides the traditional industrial networks into three functional layers. And with this aim in mind, key technologies (Blockchain-based Data Sharing, Intelligent Wireless Data Sensing, Edge Intelligence, Time-Sensitive Networks, Integrating SDN&TSN, Distributed AI) and improve applications based on AI-SDIN are also discussed. Further, the paper also highlights new opportunities and potential research challenges in control and automation of IIoT networks.

Unsupervised brain imaging 3D anomaly detection and segmentation with transformers.

Pathological brain appearances may be so heterogeneous as to be intelligible only as anomalies, defined by their deviation from normality rather than any specific set of pathological features. Amongst the hardest tasks in medical imaging, detecting such anomalies requires models of the normal brain that combine compactness with the expressivity of the complex, long-range interactions that characterise its structural organisation. These are requirements transformers have arguably greater potential to satisfy than other current candidate architectures, but their application has been inhibited by their demands on data and computational resources. Here we combine the latent representation of vector quantised variational autoencoders with an ensemble of autoregressive transformers to enable unsupervised anomaly detection and segmentation defined by deviation from healthy brain imaging data, achievable at low computational cost, within relative modest data regimes. We compare our method to current state-of-the-art approaches across a series of experiments with 2D and 3D data involving synthetic and real pathological lesions. On real lesions, we train our models on 15,000 radiologically normal participants from UK Biobank and evaluate performance on four different brain MR datasets with small vessel disease, demyelinating lesions, and tumours. We demonstrate superior anomaly detection performance both image-wise and pixel/voxel-wise, achievable without post-processing. These results draw attention to the potential of transformers in this most challenging of imaging tasks.

Derivation Method of Architectural Asset Value Enhancement Zones in South Korea

In June 2014, South Korea enacted a law to systematically preserve and utilize architectural assets to strengthen national competitiveness through the enhancement of architectural culture. An architectural asset value enhancement zone (AAVE) can be designated for an area in which a unique spatial environment has been created, centered on excellent architectural assets or in which architectural assets are densely concentrated. However, five years after the law was promulgated, while 14 local governments had completed basic investigations of architectural assets, only three had been designated as AAVE zones (as of March 2020). This is because the criteria for non-Hanok architectural assets are unclear, making it difficult to specify the scope of designation. This study aims to present and verify a methodology for deriving more effective AAVE zones. After establishing the criteria for architectural asset candidates, densely populated areas across the country were identified using GIS. Subsequently, a methodology was derived to classify candidate areas for the enhancement zone, based on the locations of these densely populated areas and designated/registered cultural heritage sites. The effectiveness of the methodology was reviewed through an actual area analysis, which indicated that the methodology is highly applicable to AAVE zones.