Architecture Search Research Articles

Exploring Topography Downscaling Methods for Hyper‐Resolution Land Surface Modeling

AbstractHyper‐resolution land surface modeling provides an unprecedented opportunity to simulate locally relevant water and energy cycles. However, the available meteorological forcing data is often insufficient to fulfill the requirements of hyper‐resolution modeling. Here, we developed a comprehensive downscaling framework based on topography‐adjusted methods and automated machine learning (AutoML). With this framework, a 90 m and hourly atmospheric forcing data set was developed from ERA5 data at a 0.25° resolution, and the Common Land Model (CoLM) was then forced with the developed forcing data over two complex terrain regions (the Heihe River Basin and Upper Colorado River Basin). We systematically evaluated the downscaled forcing and the CoLM outputs against both in situ observations and gridded data. The ground‐based validation results suggested consistent improvements for all downscaled forcing variables with mean RMSE improved by 6.362%–95.86%. The downscaled forcings, which incorporated detailed topographic features, offered improved magnitude estimates, achieving a comparable level of performance to that of regional reanalysis forcing data. The downscaled forcing driving the CoLM model showed comparable or better skills in simulating water and energy fluxes, as verified by in situ validations. The hyper‐resolution simulations provided a detailed and more reasonable description of land surface processes and attained similar spatial patterns and magnitudes with high‐resolution land surface data, especially over highly elevated areas. Additionally, this study highlighted the benefits of using mountain radiation theory‐based shortwave radiation downscaling models and AutoML‐assisted precipitation downscaling models. These findings emphasized the significance of integrating topography‐based downscaling methods for hillslope‐scale simulations.

Scale-Aware Neural Architecture Search for Multivariate Time Series Forecasting

Multivariate time series (MTS) forecasting has attracted much attention in many intelligent applications. It is not a trivial task, as we need to consider both intra-variable dependencies and inter-variable dependencies. However, existing works are designed for specific scenarios, and require much domain knowledge and expert efforts, which is difficult to transfer between different scenarios. In this paper, we propose a scale-aware neural architecture search framework for MTS forecasting (SNAS4MTF). A multi-scale decomposition module transforms raw time series into multi-scale sub-series, which can preserve multi-scale temporal patterns. An adaptive graph learning module infers the different inter-variable dependencies under different time scales without any prior knowledge. For MTS forecasting, a search space is designed to capture both intra-variable dependencies and inter-variable dependencies at each time scale. The multi-scale decomposition, adaptive graph learning, and neural architecture search modules are jointly learned in an end-to-end framework. Extensive experiments on two real-world datasets demonstrate that SNAS4MTF achieves a promising performance compared with the state-of-the-art methods.

Trainability maximization using estimation of distribution algorithms assisted by surrogate modelling for quantum architecture search

Quantum architecture search (QAS) involves optimizing both the quantum parametric circuit configuration but also its parameters for a variational quantum algorithm. Thus, the problem is known to be multi-level as the performance of a given architecture is unknown until its parameters are tuned using classical routines. Moreover, the task becomes even more complicated since well-known trainability issues, e.g., barren plateaus (BPs), can occur. In this paper, we aim to achieve two improvements in QAS: (1) to reduce the number of measurements by an online surrogate model of the evaluation process that aggressively discards architectures of poor performance; (2) to avoid training the circuits when BPs are present. To detect the presence of the BPs, we employed a recently developed metric, information content, which only requires measuring the energy values of a small set of parameters to estimate the magnitude of cost function’s gradient. The main idea of this proposal is to leverage a recently developed metric which can be used to detect the onset of vanishing gradients to ensure the overall search avoids such unfavorable regions. We experimentally validate our proposal for the variational quantum eigensolver and showcase that our algorithm is able to find solutions that have been previously proposed in the literature for the Hamiltonians; but also to outperform the state of the art when initializing the method from the set of architectures proposed in the literature. The results suggest that the proposed methodology could be used in environments where it is desired to improve the trainability of known architectures while maintaining good performance.

Spotting Leaders in Organizations with Graph Convolutional Networks, Explainable Artificial Intelligence, and Automated Machine Learning

Over the past few decades, the study of leadership theory has expanded across various disciplines, delving into the intricacies of human behavior and defining the roles of individuals within organizations. Its primary objective is to identify leaders who play significant roles in the communication flow. In addition, behavioral theory posits that leaders can be distinguished based on their daily conduct, while social network analysis provides valuable insights into behavioral patterns. Our study investigates five and six types of social networks frequently observed in different organizations. This study is conducted using datasets we collected from an IT company and public datasets collected from a manufacturing company for the thorough evaluation of prediction performance. We leverage PageRank and effective word embedding techniques to obtain novel features. State-of-the-art performance is obtained using various statistical machine learning methods, graph convolutional networks (GCN), automated machine learning (AutoML), and explainable artificial intelligence (XAI). More specifically, our approach can achieve state-of-the-art performance with an accuracy close to 90% for leaders identification with data from projects of different types. This investigation contributes to the establishment of sustainable leadership practices by aiding organizations in retaining their leadership talent.

Interpretable Machine Learning-Aided Optical Deciphering of Serum Exosomes for Early Detection, Staging, and Subtyping of Lung Cancer.

Lung cancer (LC) is the leading cause of cancer-related mortality worldwide, underscoring an urgent need for strategies that enable early detection and phenotypic classification. Here, we conducted a label-free surface-enhanced Raman spectroscopic (SERS) analysis of serum exosomes from 643 participants to elucidate the biochemical deregulation associated with LC progression and the unique phenotypes of different LC subtypes. Iodide-modified silver nanofilms were prepared to rapidly acquire SERS spectra with a high signal-to-noise ratio using 0.5 μL of patient exosomes. We performed interpretable and automated machine learning (ML) analysis of differential SERS features of serum exosomes to build LC diagnostic models, which achieved accuracies of 100% and 81% for stage I lung adenocarcinoma and its preneoplasia, respectively. In addition, the ML-derived exosomal SERS models effectively recognized different LC subtypes and disease stages to guide precision treatment. Our findings demonstrate that spectral fingerprinting of circulating exosomes holds promise for decoding the clinical status of LC, thus aiding in improving the clinical management of patients.

A digital twin of intelligent robotic grasping based on single-loop-optimized differentiable architecture search and sim-real collaborative learning

A digital twin of intelligent robotic grasping based on single-loop-optimized differentiable architecture search and sim-real collaborative learning

A Neural Architecture Search CNN for Alzheimer’s Disease Classification

The evolution of automated machine learning (AutoML) is gradually reengineering the design of deep learning architectures for various imaging tasks. AutoML effectively develops model architectures and tunes hyperparameters through neural architecture search (NAS). Deep learning model architecture design is generally considered a tedious and time-consuming task that requires mastery skills to develop robust and better-performing models for imaging tasks. Again, the model's hyperparameters must be well-tuned to ensure optimal performances, which can be tedious and time-consuming if the hyperparameters are manually selected; using existing hyperparameter optimization algorithms can be expensive regarding resources. This study addresses these challenges in developing an optimal convolutional neural network (CNN) for classifying Alzheimer's (AD). The study, therefore, adopted a NAS approach to generate a CNN model architecture using a customized search space comprising only CNN patterns implemented with a NAS framework. The search was done for ten (10) trials, yielding a CNN architecture with an accuracy of 95.85% and a loss of 0.22. Training the model with a 10-fold cross-validation approach using a 0.0009 learning rate for 150 epochs improved the model's performance. The model recorded 97.17% accuracy, 97.21% precision, 97.14% recall, and a 0.99 area under the curve (AUC) in classifying AD as one of AD, mild cognitive impairment (MCI), and normal control (NC). The model obtained 98.06%, 98.66%, and 98.62% accuracy on binary classes of AD/NC, AD/MCI, and NC/MCI, respectively. The model generally showed robustness and better performance than existing CNN architectures.

A Comparative Analysis of Automated Machine Learning Tools: A Use Case for Autism Spectrum Disorder Detection

Automated Machine Learning (AutoML) enhances productivity and efficiency by automating the entire process of machine learning model development, from data preprocessing to model deployment. These tools are accessible to users with varying levels of expertise and enable efficient, scalable, and accurate classification across different applications. This paper evaluates two popular AutoML tools, the Tree-Based Pipeline Optimization Tool (TPOT) version 0.10.2 and Konstanz Information Miner (KNIME) version 5.2.5, comparing their performance in a classification task. Specifically, this work analyzes autism spectrum disorder (ASD) detection in toddlers as a use case. The dataset for ASD detection was collected from various rehabilitation centers in Pakistan. TPOT and KNIME were applied to the ASD dataset, with TPOT achieving an accuracy of 85.23% and KNIME achieving 83.89%. Evaluation metrics such as precision, recall, and F1-score validated the reliability of the models. After selecting the best models with optimal accuracy, the most important features for ASD detection were identified using these AutoML tools. The tools optimized the feature selection process and significantly reduced diagnosis time. This study demonstrates the potential of AutoML tools and feature selection techniques to improve early ASD detection and outcomes for affected children and their families.

Comparison of Pre-Defined Automatic Machine Learning (AutoML) for MBTI Personality Prediction of Twitter Users using Binary Classification Approach

The Myers-Briggs Type Indicator (MBTI) is a personality test that is globally accepted and used as a method for identifying personality. MBTI uses a four-factor linear model to characterize a person's behaviour patterns. This feature is often used to pursue career opportunities, make decisions, manage leadership, and deal with stress. In particular, MBTI personality prediction has been widely conducted and well performed using Recurrent Neural Network (RNN) based on Twitter data because it indirectly reveals most of a person's personality through their tweets. However, deep understanding is needed in building RNN-based solutions. Hence it will take a lot of time and resources to produce an excellent model architecture and the parameters used. Therefore, this study proposed the Auto Machine Learning (AutoML) method with a pre-defined search space to determine the correct model architecture and hyperparameters based on the results of data analysis. Thus, the search algorithm can exploit environments with suitable configurations in general. There are two pre-defined search spaces employed in this study, i.e. (i) two RNN algorithms, including Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), and (ii) pre-trained Word2Ves as word embedding. In addition, this study compares the model's performance that employs preprocessing and raw data (without preprocessing). The first result shows that the preprocessing increases the F1-Score values for LSTM and GRU by 2.35% and 2.02%, respectively. Subsequently, the LSTM outperformed GRU by the values of F1-Score at 0.35% and accuracy at 0.76%. The implementation of LSTM with pre-processed data in pre-defined AutoML with Word2Vec as a word embedding technique can provide good performance on long and complex data sequences such as Twitter data for predicting its user personality.

Non-invasive detection of hydraulic cylinder leakage using computer vision and time-frequency analysis

ABSTRACT This study presents a meticulous investigation facilitated by a bespoke high-pressure test rig. The rig stands as a cornerstone of the research endeavour, providing the capability to simulate diverse leakage scenarios under controlled conditions. With its capacity to replicate conditions akin to real-world hydraulic systems, including various levels of severity such as healthy operation, moderate leakage and severe leakage. The study delves into the intricate analysis of time-series discharge flow rate signals within this experimental framework. The study utilizes sophisticated signal processing techniques, particularly the Continuous Wavelet Transform (CWT), to observe frequency components and temporal occurrences. Despite prevalent challenges distinguishing between leakage severity levels, the CWT-based analysis provides crucial insights into the dominant low frequencies (2.3–3.3 hz) characterising all leakage conditions. An advanced computer vision-based methodology is devised to address the complexities inherent in leakage differentiation, integrating cutting-edge models such as You Only Look Once (YOLO-V7) and You Only Look Once-Neural Architecture Search (YOLO-NAS). These models are meticulously trained using annotated CWT images of flow rates corresponding to different leakage conditions. Notably, the superiority of YOLO-NAS in terms of both speed and accuracy underscores its efficacy in automated leakage detection. In summary, this comprehensive approach, underpinned by the innovative hydraulic test rig and advanced signal processing coupled with state-of-the-art computer vision techniques, presents a significant advancement in the realm of internal leakage detection in hydraulic systems.

Survival prediction in diffuse large B-cell lymphoma patients: multimodal PET/CT deep features radiomic model utilizing automated machine learning

PurposeWe sought to develop an effective combined model for predicting the survival of patients with diffuse large B-cell lymphoma (DLBCL) based on the multimodal PET-CT deep features radiomics signature (DFR-signature).Methods369 DLBCL patients from two medical centers were included in this study. Their PET and CT images were fused to construct the multimodal PET-CT images using a deep learning fusion network. Then the deep features were extracted from those fused PET-CT images, and the DFR-signature was constructed through an Automated machine learning (AutoML) model. Combined with clinical indexes from the Cox regression analysis, we constructed a combined model to predict the progression-free survival (PFS) and the overall survival (OS) of patients. In addition, the combined model was evaluated in the concordance index (C-index) and the time-dependent area under the ROC curve (tdAUC).ResultsA total of 1000 deep features were extracted to build a DFR-signature. Besides the DFR-signature, the combined model integrating metabolic and clinical factors performed best in terms of PFS and OS. For PFS, the C-indices are 0.784 and 0.739 in the training cohort and internal validation cohort, respectively. For OS, the C-indices are 0.831 and 0.782 in the training cohort and internal validation cohort.ConclusionsDFR-signature constructed from multimodal images improved the classification accuracy of prognosis for DLBCL patients. Moreover, the constructed DFR-signature combined with NCCN-IPI exhibited excellent potential for risk stratification of DLBCL patients.

Forecasting severe respiratory disease hospitalizations using machine learning algorithms

BackgroundForecasting models predicting trends in hospitalization rates have the potential to inform hospital management during seasonal epidemics of respiratory diseases and the associated surges caused by acute hospital admissions. Hospital bed requirements for elective surgery could be better planned if it were possible to foresee upcoming peaks in severe respiratory illness admissions. Forecasting models can also guide the use of intervention strategies to decrease the spread of respiratory pathogens and thus prevent local health system overload. In this study, we explore the capability of forecasting models to predict the number of hospital admissions in Auckland, New Zealand, within a three-week time horizon. Furthermore, we evaluate probabilistic forecasts and the impact on model performance when integrating laboratory data describing the circulation of respiratory viruses.MethodsThe dataset used for this exploration results from active hospital surveillance, in which the World Health Organization Severe Acute Respiratory Infection (SARI) case definition was consistently used. This research nurse-led surveillance has been implemented in two public hospitals in Auckland and provides a systematic laboratory testing of SARI patients for nine respiratory viruses, including influenza, respiratory syncytial virus, and rhinovirus. The forecasting strategies used comprise automatic machine learning, one of the most recent generative pre-trained transformers, and established artificial neural network algorithms capable of univariate and multivariate forecasting.ResultsWe found that machine learning models compute more accurate forecasts in comparison to naïve seasonal models. Furthermore, we analyzed the impact of reducing the temporal resolution of forecasts, which decreased the model error of point forecasts and made probabilistic forecasting more reliable. An additional analysis that used the laboratory data revealed strong season-to-season variations in the incidence of respiratory viruses and how this correlates with total hospitalization cases. These variations could explain why it was not possible to improve forecasts by integrating this data.ConclusionsActive SARI surveillance and consistent data collection over time enable these data to be used to predict hospital bed utilization. These findings show the potential of machine learning as support for informing systems for proactive hospital management.

Daily NO2 Simulation Research Based on Automatic Machine Learning Ensemble Models

To understand the spatial distribution of NO2 near the surface, we utilized measured data from NO2 monitoring stations and combined it with column concentration data from the Tropospheric Monitoring Instrument （TROPOMI）, taking the Yangtze River Delta region as the study area. We considered the impact of factors such as population, elevation, and meteorological conditions on NO2 levels. We used automated machine learning to select five machine-learning algorithms with high simulation accuracy, namely ET, RF, XGBoost, LightGBM, and Catboost, and then integrated these five algorithms using the Stacking model to simulate the daily NO2 concentration in the Yangtze River Delta region from March 2020 to February 2021. The results indicated that the RMAE and MAE values of the Stacking ensemble model were 7.078 and 5.270, respectively, which outperformed the single algorithms of ET, RF, XGBoost, LightGBM, and Catboost. The spatial distribution of high NO2 concentrations in the Yangtze River Delta region, consisting of three provinces and one municipality, exhibited a U-shaped pattern with the convergence point located at the intersection of the three provinces, extending towards the southwest. Notably, urban pollution was particularly significant in the urban agglomerations centered around Shanghai, Hangzhou, Nanjing, and Hefei. There were 27 cities that exceeded the national standard daily limit. Changzhou was the city with the most serious NO2 pollution, with the NO2 concentration exceeding the standard for 14 d, followed by Shanghai, with 13 d. In terms of seasonal variation, the order of severity was as follows： winter, autumn, spring, and summer, with the least NO2 pollution occurring on July 9th during the summer, and the most severe NO2 pollution was observed on December 23rd during the winter.

Efficient machine learning models across multiple datasets for autism spectrum disorder diagnoses

Autism Spectrum Disorder (ASD) presents a complex neurodevelopmental challenge with diverse behavioral symptoms that overlap with other mental disorders, complicating diagnosis. Challenges in existing systems include the complexity and diversity of ASD symptoms, as well as the need for innovative and efficient detection strategies across different age groups. The application of machine learning (ML) offers a promising avenue for revolutionizing ASD prediction and diagnosis. The proposed work aims to overcome these challenges by leveraging ML and Automated Machine Learning (AutoML) techniques, combined with comprehensive datasets and advanced analytics, to enhance ASD detection and prediction capabilities. This study introduces the Autism Spectrum Disorder Detection (ASDD) Framework, which systematically applies ML and AutoML across text-based datasets for various age groups. A combined dataset is generated through the concatenation of various data sources, enriching the diagnostic scope. The framework incorporates AutoML tools such as Lazy Predict, TPOT, and AutoKeras, alongside feature selection methods like Principal Component Analysis (PCA) and Chi-Square. It also employs robust hyperparameter optimization techniques, including Bayesian optimization, Random Search, and genetic algorithms, and uses advanced ensemble methods like voting and bagging to boost prediction accuracy. An empirical evaluation of 27 distinct ML models across seven datasets, including a synthesized dataset named DB8, demonstrates exceptional performance, with many models achieving 100% accuracy and perfect F1 scores in various contexts. These outcomes highlight the effectiveness of the proposed framework in advancing ASD detection and prediction methodologies.

Clinical Impact of Radiologist’s Alert System on Patient Care for High-risk Incidental CT Findings: A Machine Learning-Based Risk Factor Analysis

Rationale and ObjectivesEfficient communication between radiologists and clinicians ordering computed tomography (CT) examinations is crucial for managing high-risk incidental CT findings (ICTFs). Herein, we introduced a Radiologist’s Alert and Patient Care Follow-up System (APCFS) for high-risk ICTFs. This study aimed to analyze the ICTFs detected by this system and the factors associated with them. Materials and MethodsThis retrospective study was approved by the institutional review board. We analyzed 52,331 CT examinations conducted between 2019 and 2021. In cases where high-risk ICTFs were identified, radiologists utilized APCFS to prompt ordering clinicians for further patient care. We assessed the frequency, affected body organs, presence or absence of therapeutic interventions, and diagnoses of high-risk ICTFs. An automated machine learning platform was employed to analyze the factors associated with high-risk ICTFs. ResultsAmong the 52,331 CT examinations, 507 (0.96%) revealed high-risk ICTFs, primarily affecting the lung (18.0%). Of these 507 high-risk ICTFs, 117 (23.1%) underwent therapeutic interventions, while 362 (71.4%) required only follow-up. Of the 117 cases undergoing interventions, 61 (52.1%) required surgery. Of the 219 high-risk ICTFs leading to a confirmed diagnosis, 146 (66.7%) were neoplastic lesions, including 88 (60.3%) malignancies, and 73 (33.3%) were non-neoplastic lesions. The top three risk factors associated with high-risk ICTFs in the regularized logistic regression model were the imaging protocol (especially aortic valve implantation planning protocol), imaging area (especially whole-body imaging), and clinical department (especially cardiology). ConclusionUtilizing APCFS, high-risk ICTFs were detected in approximately 1% of all CT examinations, likely associated with specific imaging protocols, areas, and clinical departments.

NAS FD Lung: A novel lung assist diagnostic system based on neural architecture search

In the detection and recognition of lung nodules, pulmonary nodules vary in size and shape and contain many similar tissues and organs around them, leading to the problems of both missed detection and false detection in existing detection algorithms. Designing proprietary detection and recognition networks manually requires substantial professional expertise. This process is time-consuming and labour-intensive and leads to issues like parameter redundancy and improper feature selection. Therefore, this paper proposes a new pulmonary CAD (computer-aided diagnosis) system for pulmonary nodules, NAS FD Lung (Using the NAS approach to search deep FPN and DPN networks), that can automatically learn and generate a deep learning network tailored to pulmonary nodule detection and recognition task requirements. NAS FD Lung aims to use automatic search to generate deep learning networks in the auxiliary diagnosis of pulmonary nodules to replace the manual design of deep learning networks. NAS FD Lung comprises two automatic search networks: BM NAS-FPN (Using NAS methods to search for deep FPN structures with Binary operation and Matrix multiplication fusion methods) network for nodule detection and NAS-A-DPN (Using the NAS approach to search deep DPN networks with attention mechanism) for nodule identification. The proposed technique is tested on the LUNA16 dataset, and the experimental results show that the model is superior to many existing state-of-the-art approaches. The detection accuracy of lung nodules is 98.23%. Regarding the lung nodules classification, the accuracy, specificity, sensitivity, and AUC values achieved 96.32%,97.14%,95.82%, and 98.33%, respectively.

Automated Neural Architecture Search for Cardiac Amyloidosis Classification from [18F]-Florbetaben PET Images.

Medical image classification using convolutional neural networks (CNNs) is promising but often requires extensive manual tuning for optimal model definition. Neural architecture search (NAS) automates this process, reducing human intervention significantly. This study applies NAS to [18F]-Florbetaben PET cardiac images for classifying cardiac amyloidosis (CA) sub-types (amyloid light chain (AL) and transthyretin amyloid (ATTR)) and controls. Following data preprocessing and augmentation, an evolutionary cell-based NAS approach with a fixed network macro-structure is employed, automatically deriving cells' micro-structure. The algorithm is executed five times, evaluating 100 mutating architectures per run on an augmented dataset of 4048 images (originally 597), totaling 5000 architectures evaluated. The best network (NAS-Net) achieves 76.95% overall accuracy. K-fold analysis yields mean ± SD percentages of sensitivity, specificity, and accuracy on the test dataset: AL subjects (98.7 ± 2.9, 99.3 ± 1.1, 99.7 ± 0.7), ATTR-CA subjects (93.3 ± 7.8, 78.0 ± 2.9, 70.9 ± 3.7), and controls (35.8 ± 14.6, 77.1 ± 2.0, 96.7 ± 4.4). NAS-derived network performance rivals manually determined networks in the literature while using fewer parameters, validating its automatic approach's efficacy.

Spatio-temporal change and driving mechanisms of land use/cover in Qarhan Salt Lake area during from 2000 to 2020, based on machine learning

The significance of land use classification has garnered attention due to its implications for climate and ecosystems. This paper establishes a connection by introducing and applying automatic machine learning (Auto ML) techniques to salt lake landscape, with a specific focus on the Qarhan Salt Lake area. Utilizing Landsat-5 Thematic Mappe (TM) and Landsat-8 Operational Land Imager (OLI) imagery, six machine learning algorithms were employed to classify eight land use types from 2000 to 2020. Results show that XGBLD performed optimally with 77% accuracy. Over two decades, salt fields, construction land, and water areas increased due to transformations in saline land and salt flats. The exposed lakes area exhibited a rise followed by a decline, mainly transforming into salt flats. Agricultural land areas slightly increased, influenced by both human activities and climate. Our analysis reveals a strong correlation between salt fields and precipitation, while exposed lakes demonstrate a significant negative correlation with evaporation and temperature, highlighting their vulnerability to climate change. Additionally, human water usage was identified as a significant factor impacting land use change, emphasizing the dual influence of anthropogenic activities and natural factors. This paper addresses the void in the application of Auto ML in salt lake environments and provides valuable insights into the dynamic evolution of land use types in the Qarhan Salt Lake region.

Is Automated Machine Learning useful for ocular toxoplasmosis identification and classification of the inflammatory activity?

PurposeTo evaluate the performance of automated machine learning (AutoML) models in diagnosing ocular toxoplasmosis (OT) and classifying its inflammatory activity from fundus photographs. DesignCross-sectional study. MethodsFundus photographs from OT patients in two Colombian referral centers and an open-source OT database were classified into active OT, inactive OT, and no OT. Image quality assessment excluded images with artifacts but included blurry images due to vitritis. Photos were uploaded to Amazon Web Services S3 and Google Cloud Bucket. Two models were developed on each platform: a binary model (active/inactive OT vs. no OT) and a multiclass model (active OT, inactive OT, and no OT). Datasets were split into 70% for training, 20% for testing, and 10% for validation. Sensitivity, specificity, precision, accuracy, F1-score, the area under the precision-recall curve (AUPRC), and Cohen's Kappa were calculated for each platform and model. An external validation using an open-source Image bank was performed. ResultsThe binary model on AWS showed a sensitivity of 0.97, specificity of 0.98, and AUPRC of 1.00, while the Google Cloud binary model had a sensitivity of 0.82, specificity of 0.91, and AUPRC of 0.91. The multiclass model on AWS achieved an F1 score of 0.88, with Cohen's Kappa of 0.81, while the Google Cloud model reached an F1 score of 0.88, with Cohen's Kappa of 0.82. External validation for Google Cloud achieved an accuracy of 87.5% and 80.3% in the binary and multiclass models, respectively ConclusionsAutoML is a powerful tool for diagnosing OT and classifying inflammatory activity, potentially guiding diagnosis and treatment decisions.

Training-Free Transformer Architecture Search With Zero-Cost Proxy Guided Evolution.

Transformers have shown remarkable performance, however, their architecture design is a time-consuming process that demands expertise and trial-and-error. Thus, it is worthwhile to investigate efficient methods for automatically searching high-performance Transformers via Transformer Architecture Search (TAS). In order to improve the search efficiency, training-free proxy based methods have been widely adopted in Neural Architecture Search (NAS). Whereas, these proxies have been found to be inadequate in generalizing well to Transformer search spaces, as confirmed by several studies and our own experiments. This paper presents an effective scheme for TAS called TRansformer Architecture search with ZerO-cost pRoxy guided evolution (T-Razor) that achieves exceptional efficiency. First, through theoretical analysis, we discover that the synaptic diversity of multi-head self-attention (MSA) and the saliency of multi-layer perceptron (MLP) are correlated with the performance of corresponding Transformers. The properties of synaptic diversity and synaptic saliency motivate us to introduce the ranks of synaptic diversity and saliency that denoted as DSS++ for evaluating and ranking Transformers. DSS++ incorporates correlation information among sampled Transformers to provide unified scores for both synaptic diversity and synaptic saliency. We then propose a block-wise evolution search guided by DSS++ to find optimal Transformers. DSS++ determines the positions for mutation and crossover, enhancing the exploration ability. Experimental results demonstrate that our T-Razor performs competitively against the state-of-the-art manually or automatically designed Transformer architectures across four popular Transformer search spaces. Significantly, T-Razor improves the searching efficiency across different Transformer search spaces, e.g., reducing required GPU days from more than 24 to less than 0.4 and outperforming existing zero-cost approaches. We also apply T-Razor to the BERT search space and find that the searched Transformers achieve competitive GLUE results on several Neural Language Processing (NLP) datasets. This work provides insights into training-free TAS, revealing the usefulness of evaluating Transformers based on the properties of their different blocks.