Two-step Training Research Articles

A Feature Fusion Model Based on Temporal Convolutional Network for Automatic Sleep Staging Using Single-Channel EEG.

Sleep staging is a crucial task in sleep monitoring and diagnosis, but clinical sleep staging is both time-consuming and subjective. In this study, we proposed a novel deep learning algorithm named feature fusion temporal convolutional network (FFTCN) for automatic sleep staging using single-channel EEG data. This algorithm employed a one-dimensional convolutional neural network (1D-CNN) to extract temporal features from raw EEG, and a two-dimensional CNN (2D-CNN) to extract time-frequency features from spectrograms generated through continuous wavelet transform (CWT) at the epoch level. These features were subsequently fused and further fed into a temporal convolutional network (TCN) to classify sleep stages at the sequence level. Moreover, a two-step training strategy was used to enhance the model's performance on an imbalanced dataset. Our proposed method exhibits superior performance in the 5-class classification task for healthy subjects, as evaluated on the SHHS-1, Sleep-EDF-153, and ISRUC-S1 datasets. This work provided a straightforward and promising method for improving the accuracy of automatic sleep staging using only single-channel EEG, and the proposed method exhibited great potential for future applications in professional sleep monitoring, which could effectively alleviate the workload of sleep technicians.

Combining clinical and molecular data for personalized treatment in acute myeloid leukemia: A machine learning approach

Background and ObjectiveThe standard of care in Acute Myeloid Leukemia patients has remained essentially unchanged for nearly 40 years. Due to the complicated mutational patterns within and between individual patients and a lack of targeted agents for most mutational events, implementing individualized treatment for AML has proven difficult. We reanalysed the BeatAML dataset employing Machine Learning algorithms. The BeatAML project entails patients extensively characterized at the molecular and clinical levels and linked to drug sensitivity outputs. Our approach capitalizes on the molecular and clinical data provided by the BeatAML dataset to predict the ex vivo drug sensitivity for the 122 drugs evaluated by the project. MethodsWe utilized ElasticNet, which produces fully interpretable models, in combination with a two-step training protocol that allowed us to narrow down computations. We automated the genes’ filtering step by employing two metrics, and we evaluated all possible data combinations to identify the best training configuration settings per drug. ResultsWe report a Pearson correlation across all drugs of 0.36 when clinical and RNA sequencing data were combined, with the best-performing models reaching a Pearson correlation of 0.67. When we trained using the datasets in isolation, we noted that RNA Sequencing data (Pearson: 0.36) attained three times the predictive power of whole exome sequencing data (Pearson: 0.11), with clinical data falling somewhere in between (Pearson 0.26). Lastly, we present a paradigm of clinical significance. We used our models’ prediction as a drug sensitivity score to rank an individual's expected response to treatment. We identified 78 patients out of 89 (88 %) that the proposed drug was more potent than the administered one based on their ex vivo drug sensitivity data. ConclusionsIn conclusion, our reanalysis of the BeatAML dataset using Machine Learning algorithms demonstrates the potential for individualized treatment prediction in Acute Myeloid Leukemia patients, addressing the longstanding challenge of treatment personalization in this disease. By leveraging molecular and clinical data, our approach yields promising correlations between predicted drug sensitivity and actual responses, highlighting a significant step forward in improving therapeutic outcomes for AML patients.

Hybrid CNN-Mamba network for single-pixel imaging

Recent progress in single-pixel imaging (SPI) has exhibited remarkable performance using deep neural networks, e.g., convolutional neural networks (CNNs) and vision Transformers (ViTs). Nonetheless, it is challenging for existing methods to well model object image from single-pixel detections that have a long-range dependency, where CNNs are constrained by their local receptive fields, and ViTs suffer from high quadratic complexity of attention mechanism. Inspired by the Mamba architecture, known for its proficiency in handling long sequences and global contextual information with enhanced computational efficiency as state space models (SSMs), we propose a hybrid network of CNN and Mamba for SPI, named CMSPI. The proposed CMSPI integrates the local feature extraction capability of convolutional layers with the abilities of SSMs for efficiently capturing the long-range dependency, and the design of complementary split-concat structure, depthwise separable convolution, and residual connection enhance learning power of network model. Besides, CMSPI adopts a two-step training strategy, which makes reconstruction performance better and hardware-friendly. Simulations and real experiments demonstrate that CMSPI has higher imaging quality, lower memory consumption, and less computational burden than the state-of-the-art SPI methods.

Automated segmentation and classification of supraspinatus fatty infiltration in shoulder magnetic resonance image using a convolutional neural network.

Goutallier's fatty infiltration of the supraspinatus muscle is a critical condition in degenerative shoulder disorders. Deep learning research primarily uses manual segmentation and labeling to detect this condition. Employing unsupervised training with a hybrid framework of segmentation and classification could offer an efficient solution. To develop and assess a two-step deep learning model for detecting the region of interest and categorizing the magnetic resonance image (MRI) supraspinatus muscle fatty infiltration according to Goutallier's scale. A retrospective study was performed from January 1, 2019 to September 20, 2020, using 900 MRI T2-weighted images with supraspinatus muscle fatty infiltration diagnoses. A model with two sequential neural networks was implemented and trained. The first sub-model automatically detects the region of interest using a U-Net model. The second sub-model performs a binary classification using the VGG-19 architecture. The model's performance was computed as the average of five-fold cross-validation processes. Loss, accuracy, Dice coefficient (CI. 95%), AU-ROC, sensitivity, and specificity (CI. 95%) were reported. Six hundred and six shoulders MRIs were analyzed. The Goutallier distribution was presented as follows: 0 (66.50%); 1 (18.81%); 2 (8.42%); 3 (3.96%); 4 (2.31%). Segmentation results demonstrate high levels of accuracy (0.9977 ± 0.0002) and Dice score (0.9441 ± 0.0031), while the classification model also results in high levels of accuracy (0.9731 ± 0.0230); sensitivity (0.9000 ± 0.0980); specificity (0.9788 ± 0.0257); and AUROC (0.9903 ± 0.0092). The two-step training method proposed using a deep learning model demonstrated strong performance in segmentation and classification tasks.

A non-local based microcrack segmentation model optimized for effective high resolution and low-power devices

Concrete is notable for its strength, durability, and versatility, making it a fundamental material in construction. Nevertheless, microcracks occur in concrete structures due to various factors, and the size of these cracks gradually widens over time, which can lead to spalling. Furthermore, the advent of climate change accelerates concrete degradation, posing significant challenges to structural maintenance. Recently, deep learning-based computer vision techniques, especially convolutional neural networks, have shown promising results in detecting and segmenting cracks in concrete walls. In this study, we propose a two-step training strategy for a microcrack segmentation model optimized for high resolution and low-power devices. The first step is to develop a segmentation model that can detect microcracks through high-resolution images. This strategy includes generating a high-quality CrackHQ dataset using super-resolution and a Transformer-based model training process named CegFormer. The second step involves converting the trained model to a lightweight version, increasing inference speed and making it operable on low-power devices. While performing lightweight, the model performance inevitably is degraded. We restored the degraded crack recognition performance by applying knowledge distillation. The experimental results of comparing baseline and CegFormer showed a 12.82% improvement in microcrack recognition performance. The detector optimized for use in actual fields improved recognition performance by up to 5.16% and reduced inference speed by 51.52%. The utilization of the two-step training strategy enabled the optical sensors to accurately detect microcracks even from far fields. Future work will focus on developing crack detectors that take into account both indoor and outdoor environmental conditions, as well as different texture of surface materials, for application in building inspection.

Hypnos: A Contactless Sleep Stage Monitoring System Using UWB Signals

As a fundamental physiological process, sleep plays a vital role in human health. High-quality sleep requires a reasonable distribution of sleep duration over different sleep stages. Recently, contactless solutions have been used for in-home sleep stage monitoring via wireless signals as it enables monitoring daily sleep in a non-intrusive manner. However, various factors, such as the subject's physiological characteristics during sleep, the subject's health status, and even the sleep environment, pose challenges to wireless signal analysis. In this paper, we propose Hypnos, a contactless sleep monitoring system that identifies different sleep stages using an ultra-wideband (UWB) device. Hypnos enables automated bed localization and extracts signals containing coarse-grained body movements and fine-grained chest movements due to breathing and heartbeat from the subject, which acts as the preparation step for sleep staging. The key to our system is a seq2seq deep learning model, which adopts an attention-based sequence encoder to learn the patterns and transitions within and between sleep epochs and combines with contrastive learning to improve the generalizability of the encoder. Particularly, we incorporate sleep apnea detection as an auxiliary task into the model to reduce the interference of sleep apnea with sleep staging. Moreover, we design a two-step training for better adaptation of subjects with different severities of sleep disorders. We conduct extensive experiments on 100 subjects, including healthy individuals and patients with sleep disorders, and the experimental results show that Hypnos achieves excellent performance in multi-stage sleep classification (including 5-stage sleep classification), and outperforms other baseline methods.

Reinforcement learning-based satellite formation attitude control under multi-constraint

As the complexity of space missions increases, the constraints on satellite attitude control become more stringent, particularly for satellites working in orbit formation. This paper introduces a novel method, based on the categorization and modeling of different constraints, for attitude control of satellite formations under multiple constraints. The method employs the Phased Priority Reinforcement Learning (PPRL) approach, which utilizes Deep Deterministic Policy Gradient (DDPG) technology. Considering the complexity of constraints and the challenge posed by the high control dimensionality due to multi-satellite coordination, the method addresses these challenges through a two-step training strategy. The first step addresses the multi-constraint issue for individual satellites and increases the priority of single-satellite training experience data in the experience replay buffer of the second step to enhance data utilization efficiency. To address the issue of reward sparsity in complex high-dimensional constraint models, a detailed reward mechanism is proposed, incorporating both local and global constraints into the reward function, thereby achieving both efficient and effective attitude control. This approach not only meets dynamic, state, and performance constraints but also demonstrates adaptability and robustness through numerical simulations. Compared to traditional methods, this approach achieves significant improvements in control performance and constraint satisfaction, offering a novel solution pathway for high-dimensional control problems in multi-constraint satellite formations.

Reconfigurable inverse design of phononic crystal sensor based on a deep learning accelerated evolution strategy

Phononic crystals offer valuable sensing capabilities due to their high sensitivity to changes in sound velocity of analytes. In this work, a reconfigurable inverse design approach of phononic crystal sensors is achieved using a deep learning accelerated evolution strategy. The training data is acquired through finite element method (FEM). Two multilayer perceptrons (MLP) are constructed and trained to predict the center frequency and bandwidth of a passband in the dispersion relation. Utilizing a two-step training approach enables rapid accuracy enhancements and swift reconstruction of network targets from NaCl to KCl solutions. The trained networks accelerate the optimization process, yielding a phononic crystal with good detection ability. Compared to FEM, invoking the trained networks can reduce optimization time by a factor of 105. The optimized and initial structures are both fabricated and experimentally tested. The robust linear relation between the resonant peak and the solution concentration indicates significant sensing value. The experimental results are in good agreement with the FEM simulations. In the detection of NaCl solution, the optimized phononic crystal sensor has a sensitivity increase of 375% and a Q-factor of 999%. Our research demonstrates that the data-driven deep learning network is a very powerful tool for the design and optimization of phononic crystal devices.

Pre-trained multimodal large language model enhances dermatological diagnosis using SkinGPT-4

Large language models (LLMs) are seen to have tremendous potential in advancing medical diagnosis recently, particularly in dermatological diagnosis, which is a very important task as skin and subcutaneous diseases rank high among the leading contributors to the global burden of nonfatal diseases. Here we present SkinGPT-4, which is an interactive dermatology diagnostic system based on multimodal large language models. We have aligned a pre-trained vision transformer with an LLM named Llama-2-13b-chat by collecting an extensive collection of skin disease images (comprising 52,929 publicly available and proprietary images) along with clinical concepts and doctors’ notes, and designing a two-step training strategy. We have quantitatively evaluated SkinGPT-4 on 150 real-life cases with board-certified dermatologists. With SkinGPT-4, users could upload their own skin photos for diagnosis, and the system could autonomously evaluate the images, identify the characteristics and categories of the skin conditions, perform in-depth analysis, and provide interactive treatment recommendations.

Promoting critical reading instruction in higher education: a three-step training scheme facilitated by using corpus technology

Abstract This study examines the development of students’ cognitive skills through hands-on concordancing, also known as the inductive Data-driven learning (DDL) method (Cobb, T. 1997. Is there any measurable learning from hands-on concordancing? System 25(3). 301–315). This method involves learners directly using concordance software to identify language rules by observing and working with concordance results. While previous research has explored DDL’s role in language acquisition, its impact on cognitive growth has received less attention. Building on a corpus-based language pedagogy (CBLP) (Ma, Q., J. Tang & S. Lin. 2022a. The development of corpus-based language pedagogy for TESOL teachers: A two-step training approach facilitated by online collaboration. Computer Assisted Language Learning 35(9). 2731–2760, Ma, Q., R. Yuan, L. M. E. Cheung & J. Yang. 2022b. Teacher paths for developing corpus-based language pedagogy: A case study. Computer Assisted Language Learning. 1–32), we designed a three-step training scheme that included an online task, a computer lab lesson, and a student independent corpus exploration task. The study involved 45 second-year English majors. The data were collected through surveys, interviews, and written reports, focusing on student learning of cognitive skills such as speculating, noticing, inferring, and verifying, as synthesized by Chun (2011. Learners as corpus researchers: Developing cognitive skills through hands-on concordancing. Foreign Languages Education 18(3). 73–103). The results indicate a positive response to the CBLP, with students’ independent exploration task contributing to improved critical reading and thinking skills. This highlights the effectiveness of our three-step CBLP scheme in developing cognitive skills and enhancing independent research. Overall, the study emphasizes the potential of hands-on concordancing and CBLP in fostering critical reading skills.

A CNN Model for Physical Activity Recognition and Energy Expenditure Estimation from an Eyeglass-Mounted Wearable Sensor.

Metabolic syndrome poses a significant health challenge worldwide, prompting the need for comprehensive strategies integrating physical activity monitoring and energy expenditure. Wearable sensor devices have been used both for energy intake and energy expenditure (EE) estimation. Traditionally, sensors are attached to the hip or wrist. The primary aim of this research is to investigate the use of an eyeglass-mounted wearable energy intake sensor (Automatic Ingestion Monitor v2, AIM-2) for simultaneous recognition of physical activity (PAR) and estimation of steady-state EE as compared to a traditional hip-worn device. Study data were collected from six participants performing six structured activities, with the reference EE measured using indirect calorimetry (COSMED K5) and reported as metabolic equivalents of tasks (METs). Next, a novel deep convolutional neural network-based multitasking model (Multitasking-CNN) was developed for PAR and EE estimation. The Multitasking-CNN was trained with a two-step progressive training approach for higher accuracy, where in the first step the model for PAR was trained, and in the second step the model was fine-tuned for EE estimation. Finally, the performance of Multitasking-CNN on AIM-2 attached to eyeglasses was compared to the ActiGraph GT9X (AG) attached to the right hip. On the AIM-2 data, Multitasking-CNN achieved a maximum of 95% testing accuracy of PAR, a minimum of 0.59 METs mean square error (MSE), and 11% mean absolute percentage error (MAPE) in EE estimation. Conversely, on AG data, the Multitasking-CNN model achieved a maximum of 82% testing accuracy in PAR, a minimum of 0.73 METs MSE, and 13% MAPE in EE estimation. These results suggest the feasibility of using an eyeglass-mounted sensor for both PAR and EE estimation.

Con-CDVAE: A method for the conditional generation of crystal structures

In recent years, progress has been made in generating new crystalline materials using generative machine learning models, though gaps remain in efficiently generating crystals based on target properties. This paper proposes the Con-CDVAE model, an extension of the Crystal Diffusion Variational Autoencoder (CDVAE), for conditional crystal generation. We introduce innovative components, design a two-step training method, and develop three unique generation strategies to enhance model performance. The effectiveness of Con-CDVAE is demonstrated through extensive testing under various conditions, including both single and combined property targets. Ablation studies further underscore the critical role of the new components in achieving our model’s performance. Additionally, we validate the physical credibility of the generated crystals through Density Functional Theory (DFT) calculations, confirming Con-CDVAE’s potential in material science research.

BrepMFR: Enhancing machining feature recognition in B-rep models through deep learning and domain adaptation

Feature Recognition (FR) plays a crucial role in modern digital manufacturing, serving as a key technology for integrating Computer-Aided Design (CAD), Computer-Aided Process Planning (CAPP) and Computer-Aided Manufacturing (CAM) systems. The emergence of deep learning methods in recent years offers a new approach to address challenges in recognizing highly intersecting features with complex geometric shapes. However, due to the high cost of labeling real CAD models, neural networks are usually trained on computer-synthesized datasets, resulting in noticeable performance degradation when applied to real-world CAD models. Therefore, we propose a novel deep learning network, BrepMFR, designed for Machining Feature Recognition (MFR) from Boundary Representation (B-rep) models. We transform the original B-rep model into a graph representation as network-friendly input, incorporating local geometric shape and global topological relationships. Leveraging a graph neural network based on Transformer architecture and graph attention mechanism, we extract the feature representation of high-level semantic information to achieve machining feature recognition. Additionally, employing a two-step training strategy under a transfer learning framework, we enhance BrepMFR's generalization ability by adapting synthetic training data to real CAD data. Furthermore, we establish a large-scale synthetic CAD model dataset inclusive of 24 typical machining features, showcasing diversity in geometry that closely mirrors real-world mechanical engineering scenarios. Extensive experiments across various datasets demonstrate that BrepMFR achieves state-of-the-art machining feature recognition accuracy and performs effectively on CAD models of real-world mechanical parts.

Physics-based neural network as constitutive law for finite element analysis of sintering

In the sintering of advanced ceramics, a digital twin consisted of a finite element model can predict the final shape and microstructure of a sintered ceramic. However, to minimise uncertainties, it is essential to continually update the mechanical properties in the digital twin using data collected from the manufacturing process. One promising approach for achieving this is through artificial neural networks (ANN). This study introduces a machine learning strategy to update the constitutive behaviour of advanced ceramics in finite element analysis of sintering deformation. A major challenge in implementing machine learning in material processing is the huge amount of data required by the training and validation of an ANN, which are often unavailable or incomplete in a real manufacturing process. This study demonstrates that the data requirement can be reduced by employing a two-step training technique. Firstly, the ANN is trained using a nonlinear constitutive law, which describes a general relationship between the strain rates and stresses. Subsequently, the weights and biases of the ANN are transferred for the retraining using limited experimental data for an actual ceramic. It is shown that such approach can successfully capture the nonlinear constitutive behaviour of fine-grained alumina without demanding for large amount of experimental data. A case study is provided, highlighting the feasibility of implementing the ANN in a commercial finite element package, to replace the constitutive law and predict the shrinkage and distortion of a sintering part. In particular, the sintered dumb-bell shape part, simulated using the retrained ANN, showed grain size and relative density that markedly different from those using the nonlinear constitutive law. It is important to note that the proposed methodology is generic and can be used to create ANNs to replace constitutive laws in finite element analysis in digital twins for a wide range of other engineering processes.

One model, two brains: Automatic fetal brain extraction from MR images of twins

Fetal brain extraction from magnetic resonance (MR) images is of great importance for both clinical applications and neuroscience studies. However, it is a challenging task, especially when dealing with twins, which are commonly existing in pregnancy. Currently, there is no brain extraction method dedicated to twins, raising significant demand to develop an effective twin fetal brain extraction method. To this end, we propose the first twin fetal brain extraction framework, which possesses three novel features. First, to narrow down the region of interest and preserve structural information between the two brains in twin fetal MR images, we take advantage of an advanced object detector to locate all the brains in twin fetal MR images at once. Second, we propose a Twin Fetal Brain Extraction Network (TFBE-Net) to further suppress insignificant features for segmenting brain regions. Finally, we propose a Two-step Training Strategy (TTS) to learn correlation features of the single fetal brain for further improving the performance of TFBE-Net. We validate the proposed framework on a twin fetal brain dataset. The experiments show that our framework achieves promising performance on both quantitative and qualitative evaluations, and outperforms state-of-the-art methods for fetal brain extraction.

Phase smoothing for diffractive deep neural networks

Optical neural networks have shown promising advantages over its electronic counterpart due to fast computational speed, high throughput, and low energy consumption. For the deep neural networks mimicked by the cascaded phase masks, the conventional training algorithm results in abruptly varying phase profiles. In this work, we show that even two kinds of diffraction formulas give completely different results for the identical rough phase masks, which denotes that the training process harvests the inaccuracy of the used diffraction modeling, which prevents the results from being replicated when the diffraction is formularized by other numerical methods or implemented in the real world. The modeling inaccuracy of each phase mask accumulates to prevent the successful optical implementation. In this work, we propose a two-step training framework to enforce smooth phase masks, and thus reduce the mismatch between numerical modeling and practical deployment.

TTDAT: Two-Step Training Dual Attention Transformer for Malware Classification Based on API Call Sequences

The surge in malware threats propelled by the rapid evolution of the internet and smart device technology necessitates effective automatic malware classification for robust system security. While existing research has primarily relied on some feature extraction techniques, issues such as information loss and computational overhead persist, especially in instruction-level tracking. To address these issues, this paper focuses on the nuanced analysis of API (Application Programming Interface) call sequences between the malware and system and introduces TTDAT (Two-step Training Dual Attention Transformer) for malware classification. TTDAT utilizes Transformer architecture with original multi-head attention and an integrated local attention module, streamlining the encoding of API sequences and extracting both global and local patterns. To expedite detection, we introduce a two-step training strategy: ensemble Transformer models to generate class representation vectors, thereby bolstering efficiency and adaptability. Our extensive experiments demonstrate TTDAT’s effectiveness, showcasing state-of-the-art results with an average F1 score of 0.90 and an accuracy of 0.96.

Adaptive Multioutput Gradient RBF Tracker for Nonlinear and Nonstationary Regression.

Multioutput regression of nonlinear and nonstationary data is largely understudied in both machine learning and control communities. This article develops an adaptive multioutput gradient radial basis function (MGRBF) tracker for online modeling of multioutput nonlinear and nonstationary processes. Specifically, a compact MGRBF network is first constructed with a new two-step training procedure to produce excellent predictive capacity. To improve its tracking ability in fast time-varying scenarios, an adaptive MGRBF (AMGRBF) tracker is proposed, which updates the MGRBF network structure online by replacing the worst performing node with a new node that automatically encodes the newly emerging system state and acts as a perfect local multioutput predictor for the current system state. Extensive experimental results confirm that the proposed AMGRBF tracker significantly outperforms existing state-of-the-art online multioutput regression methods as well as deep-learning-based models, in terms of adaptive modeling accuracy and online computational complexity.

Self-Supervised Adversarial Variational Learning

A natural approach for representation learning is to combine the inference mechanisms of VAEs and the generative abilities of GANs, within a new model, namely VAEGAN. Most existing VAEGAN models would jointly train the generator and inference modules, which has limitations when learning representations generated by a pre-trained GAN model without data. In this paper, we develop a novel hybrid model, called the Self-Supervised Adversarial Variational Learning (SS-AVL) which introduces a two-step optimization procedure training separately the generator and the inference model. The primary advantage of SS-AVL over existing VAEGAN models is that SS-AVL optimizes the inference models in a self-supervised learning manner where the samples used for training the inference models are drawn from the generator distribution instead of using real samples. This can allow SS-AVL to learn representations from arbitrary GAN models without using real data. Additionally, we employ information maximization into the context of increasing the maximum likelihood, which encourages SS-AVL to learn meaningful latent representations. We perform extensive experiments to demonstrate the effectiveness of the proposed SS-AVL model.

LHC hadronic jet generation using convolutional variational autoencoders with normalizing flows

In high energy physics, one of the most important processes for collider data analysis is the comparison of collected and simulated data. Nowadays the state-of-the-art for data generation is in the form of Monte Carlo (MC) generators. However, because of the upcoming high-luminosity upgrade of the Large Hadron Collider (LHC), there will not be enough computational power or time to match the amount of needed simulated data using MC methods. An alternative approach under study is the usage of machine learning generative methods to fulfill that task. Since the most common final-state objects of high-energy proton collisions are hadronic jets, which are collections of particles collimated in a given region of space, this work aims to develop a convolutional variational autoencoder (ConVAE) for the generation of particle-based LHC hadronic jets. Given the ConVAE’s limitations, a normalizing flow (NF) network is coupled to it in a two-step training process, which shows improvements on the results for the generated jets. The ConVAE+NF network is capable of generating a jet in , making it one of the fastest methods for this task up to now.