Training Deep Neural Networks Research Articles

IM-LIF: Improved Neuronal Dynamics With Attention Mechanism for Direct Training Deep Spiking Neural Network

IM-LIF: Improved Neuronal Dynamics With Attention Mechanism for Direct Training Deep Spiking Neural Network

FTPipeHD: A Fault-Tolerant Pipeline-Parallel Distributed Training Approach for Heterogeneous Edge Devices

With the increasing proliferation of Internet-of-Things (IoT) devices, there is a growing trend towards distributing the power of deep learning (DL) among edge devices rather than centralizing it at the cloud. To deploy deep and complex models at edge devices with limited resources, model partitioning of deep neural network (DNN) models has been widely studied. However, most of the existing literature only considers distributing the inference model while still training the model at the cloud. In this paper, we propose FTPipeHD, a novel DNN training approach that trains DNN models across distributed heterogeneous devices with the fault-tolerance mechanism. To accelerate the training with the time-varying computing power of each device, we optimize the partition points dynamically according to real-time computing capacities. We also propose a novel weight redistribution approach that replicates the weights to both the neighboring nodes and the central node periodically, which combats the failure of multiple devices during training while incurring limited communication costs. Our numerical results demonstrate that FTPipeHD is 6.8 times faster in training than the state-of-the-art method when the computing capacity of the best device is 10 times greater than the worst one. It is also shown that the proposed method is able to accelerate the training even with the existence of device failures.

Robustness evaluation of deep neural networks for endoscopic image analysis: Insights and strategies

Computer-aided detection and diagnosis systems (CADe/CADx) in endoscopy are commonly trained using high-quality imagery, which is not representative for the heterogeneous input typically encountered in clinical practice. In endoscopy, the image quality heavily relies on both the skills and experience of the endoscopist and the specifications of the system used for screening. Factors such as poor illumination, motion blur, and specific post-processing settings can significantly alter the quality and general appearance of these images. This so-called domain gap between the data used for developing the system and the data it encounters after deployment, and the impact it has on the performance of deep neural networks (DNNs) supportive endoscopic CAD systems remains largely unexplored. As many of such systems, for e.g. polyp detection, are already being rolled out in clinical practice, this poses severe patient risks in particularly community hospitals, where both the imaging equipment and experience are subject to considerable variation. Therefore, this study aims to evaluate the impact of this domain gap on the clinical performance of CADe/CADx for various endoscopic applications. For this, we leverage two publicly available data sets (KVASIR-SEG and GIANA) and two in-house data sets. We investigate the performance of commonly-used DNN architectures under synthetic, clinically calibrated image degradations and on a prospectively collected dataset including 342 endoscopic images of lower subjective quality. Additionally, we assess the influence of DNN architecture and complexity, data augmentation, and pretraining techniques for improved robustness. The results reveal a considerable decline in performance of 11.6% (±1.5) as compared to the reference, within the clinically calibrated boundaries of image degradations. Nevertheless, employing more advanced DNN architectures and self-supervised in-domain pre-training effectively mitigate this drop to 7.7% (±2.03). Additionally, these enhancements yield the highest performance on the manually collected test set including images with lower subjective quality. By comprehensively assessing the robustness of popular DNN architectures and training strategies across multiple datasets, this study provides valuable insights into their performance and limitations for endoscopic applications. The findings highlight the importance of including robustness evaluation when developing DNNs for endoscopy applications and propose strategies to mitigate performance loss.

Hybrid Precision Floating-Point (HPFP) Selection to Optimize Hardware-Constrained Accelerator for CNN Training.

The rapid advancement in AI requires efficient accelerators for training on edge devices, which often face challenges related to the high hardware costs of floating-point arithmetic operations. To tackle these problems, efficient floating-point formats inspired by block floating-point (BFP), such as Microsoft Floating Point (MSFP) and FlexBlock (FB), are emerging. However, they have limited dynamic range and precision for the smaller magnitude values within a block due to the shared exponent. This limits the BFP's ability to train deep neural networks (DNNs) with diverse datasets. This paper introduces the hybrid precision (HPFP) selection algorithms, designed to systematically reduce precision and implement hybrid precision strategies, thereby balancing layer-wise arithmetic operations and data path precision to address the shortcomings of traditional floating-point formats. Reducing the data bit width with HPFP allows more read/write operations from memory per cycle, thereby decreasing off-chip data access and the size of on-chip memories. Unlike traditional reduced precision formats that use BFP for calculating partial sums and accumulating those partial sums in 32-bit Floating Point (FP32), HPFP leads to significant hardware savings by performing all multiply and accumulate operations in reduced floating-point format. For evaluation, two training accelerators for the YOLOv2-Tiny model were developed, employing distinct mixed precision strategies, and their performance was benchmarked against an accelerator utilizing a conventional brain floating point of 16 bits (Bfloat16). The HPFP selection, employing 10 bits for the data path of all layers and for the arithmetic of layers requiring low precision, along with 12 bits for layers requiring higher precision, results in a 49.4% reduction in energy consumption and a 37.5% decrease in memory access. This is achieved with only a marginal mean Average Precision (mAP) degradation of 0.8% when compared to an accelerator based on Bfloat16. This comparison demonstrates that the proposed accelerator based on HPFP can be an efficient approach to designing compact and low-power accelerators without sacrificing accuracy.

An efficient bandwidth-adaptive gradient compression algorithm for distributed training of deep neural networks

In distributed deep learning with data parallelism, communication bottleneck throttles the efficiency of model training. Recent studies adopt versatile gradient compression techniques, with communication sparsification standing out as an effective approach for reducing the number of gradients to be transmitted. However, the deployment of gradient sparsification is adversely influenced by the change of network environment in real systems, and existing methods either neglect bandwidth dynamics during training or experience drastic fluctuation of compression ratios. In this paper, we propose ACE, a novel adaptive gradient compression mechanism with high communication efficiency under bandwidth variation. ACE adapts the sparsification ratio to the average bandwidth in a time window, other than following its dynamics exactly. To accurately compute the compression ratio, we first profile the compression time and model a single iteration time consisting of communication, computation and compression operations. We then present a practical model to fit the needed training rounds till convergence, and formulate an optimization problem to compute the optimal sparsification ratio. We conduct experiments on different DNN models in different network environments and compare various methods in terms of convergence and model quality. The experimental results show that ACE achieves up to 9.39× and 1.28× training speedups over fixed and state-of-the-art adaptive compression methods.

When to Grow? A Fitting Risk-Aware Policy for Layer Growing in Deep Neural Networks

Neural growth is the process of growing a small neural network to a large network and has been utilized to accelerate the training of deep neural networks. One crucial aspect of neural growth is determining the optimal growth timing. However, few studies investigate this systematically. Our study reveals that neural growth inherently exhibits a regularization effect, whose intensity is influenced by the chosen policy for growth timing. While this regularization effect may mitigate the overfitting risk of the model, it may lead to a notable accuracy drop when the model underfits. Yet, current approaches have not addressed this issue due to their lack of consideration of the regularization effect from neural growth. Motivated by these findings, we propose an under/over fitting risk-aware growth timing policy, which automatically adjusts the growth timing informed by the level of potential under/overfitting risks to address both risks. Comprehensive experiments conducted using CIFAR-10/100 and ImageNet datasets show that the proposed policy achieves accuracy improvements of up to 1.3% in models prone to underfitting while achieving similar accuracies in models suffering from overfitting compared to the existing methods.

IPRemover: A Generative Model Inversion Attack against Deep Neural Network Fingerprinting and Watermarking

Training Deep Neural Networks (DNNs) can be expensive when data is difficult to obtain or labeling them requires significant domain expertise. Hence, it is crucial that the Intellectual Property (IP) of DNNs trained on valuable data be protected against IP infringement. DNN fingerprinting and watermarking are two lines of work in DNN IP protection. Recently proposed DNN fingerprinting techniques are able to detect IP infringement while preserving model performance by relying on the key assumption that the decision boundaries of independently trained models are intrinsically different from one another. In contrast, DNN watermarking embeds a watermark in a model and verifies IP infringement if an identical or similar watermark is extracted from a suspect model. The techniques deployed in fingerprinting and watermarking vary significantly because their underlying mechanisms are different. From an adversary's perspective, a successful IP removal attack should defeat both fingerprinting and watermarking. However, to the best of our knowledge, there is no work on such attacks in the literature yet. In this paper, we fill this gap by presenting an IP removal attack that can defeat both fingerprinting and watermarking. We consider the challenging data-free scenario whereby all data is inverted from the victim model. Under this setting, a stolen model only depends on the victim model. Experimental results demonstrate the success of our attack in defeating state-of-the-art DNN fingerprinting and watermarking techniques. This work reveals a novel attack surface that exploits generative model inversion attacks to bypass DNN IP defenses. This threat must be addressed by future defenses for reliable IP protection.

Towards Trustworthy Deep Learning

Deep neural networks (DNNs) have achieved unprecedented success across many scientific and engineering fields in the last decades. Despite its empirical success, unfortunately, recent studies have shown that there are various failure modes and blindspots in DNN models which may result in unexpected serious failures and potential harms, e.g. the existence of adversarial examples and small perturbations. This is not acceptable especially for safety critical and high stakes applications in the real-world, including healthcare, self-driving cars, aircraft control systems, hiring and malware detection protocols. Moreover, it has been challenging to understand why and when DNNs will fail due to their complicated structures and black-box behaviors. Lacking interpretability is one critical issue that may seriously hinder the deployment of DNNs in high-stake applications, which need interpretability to trust the prediction, to understand potential failures, and to be able to mitigate harms and eliminate biases in the model. To make DNNs trustworthy and reliable for deployment, it is necessary and urgent to develop methods and tools that can (i) quantify and improve their robustness against adversarial and natural perturbations, and (ii) understand their underlying behaviors and further correct errors to prevent injuries and damages. These are the important first steps to enable Trustworthy AI and Trustworthy Machine Learning. In this talk, I will survey a series of research efforts in my lab contributed to tackling the grand challenges in (i) and (ii). In the first part of my talk, I will overview our research effort in Robust Machine Learning since 2017, where we have proposed the first attack-agnostic robustness evaluation metric, the first efficient robustness certification algorithms for various types of perturbations, and efficient robust learning algorithms across supervised learning to deep reinforcement learning. In the second part of my talk, I will survey a series of exciting results in my lab on accelerating interpretable machine learning and explainable AI. Specifically, I will show how we could bring interpretability into deep learning by leveraging recent advances in multi-modal models. I'll present recent works in our group on automatically dissecting neural networks with open vocabulary concepts, designing interpretable neural networks without concept labels, and briefly overview our recent efforts on demystifying black-box DNN training process, automated neuron explanations for Large Language Models and the first robustness evaluation of a family of neuron-level interpretation techniques.

Algorithmic Foundation of Federated Learning with Sequential Data

The current analysis of federated optimization algorithms for training deep neural networks assumes that the data is non-sequential (e.g., images), which incurs a smooth loss objective. In contrast, edge devices generate lots of sequential data every day, where these sequences exhibit significant sequential correlation at different time stamps (e.g., text messages). In order to learn from such sequential data, people typically use a class of neural networks that is inherently nonsmooth, with a potentially unbounded smoothness parameter. Examples include recurrent neural networks, long-short-term memory networks, and transformers. It remains unclear how to design provably efficient algorithms for training these neural networks to learn from sequential data. My goal is to lay the algorithmic foundation of federated learning with sequential data, which contributes novel algorithms for learning from a range of real-world sequential data (e.g., natural language, electronic health record, transportation, time series, etc.) using state-of-the-art deep neural networks. In this talk, I will first motivate the problem by showing that the transformer, which is widely used for sequential data learning, has an unbounded smooth landscape. Then, I will introduce provably efficient federated deep learning algorithms in the presence of unbounded smoothness. In particular, I will introduce a few efficient algorithms for various settings of federated learning, including homogeneous data, heterogeneous data, and partial client participation. The main result is twofold. First, we show that the designed algorithms provably small computational and communication complexities. Second, we establish fundamental hardness results in the unbounded smoothness setting. Ultimately, I will discuss the future challenges of extending our research framework from small-scale neural networks to large language models.

Adversarial Attacks on the Interpretation of Neuron Activation Maximization

Feature visualization is one of the most popular techniques used to interpret the internal behavior of individual units of trained deep neural networks. Based on activation maximization, they consist of finding synthetic or natural inputs that maximize neuron activations. This paper introduces an optimization framework that aims to deceive feature visualization through adversarial model manipulation. It consists of finetuning a pre-trained model with a specifically introduced loss that aims to maintain model performance, while also significantly changing feature visualization. We provide evidence of the success of this manipulation on several pre-trained models for the classification task with ImageNet.

Inconsistency-Based Data-Centric Active Open-Set Annotation

Active learning, a method to reduce labeling effort for training deep neural networks, is often limited by the assumption that all unlabeled data belong to known classes. This closed-world assumption fails in practical scenarios with unknown classes in the data, leading to active open-set annotation challenges. Existing methods struggle with this uncertainty. We introduce NEAT, a novel, computationally efficient, data-centric active learning approach for open-set data. NEAT differentiates and labels known classes from a mix of known and unknown classes, using a clusterability criterion and a consistency mea- sure that detects inconsistencies between model predictions and feature distribution. In contrast to recent learning-centric solutions, NEAT shows superior performance in active open- set annotation, as our experiments confirm. Additional details on the further evaluation metrics, implementation, and archi- tecture of our method can be found in the public document at https://arxiv.org/pdf/2401.04923.pdf.

Convolutional Channel-Wise Competitive Learning for the Forward-Forward Algorithm

The Forward-Forward (FF) Algorithm has been recently proposed to alleviate the issues of backpropagation (BP) commonly used to train deep neural networks. However, its current formulation exhibits limitations such as the generation of negative data, slower convergence, and inadequate performance on complex tasks. In this paper we take the main ideas of FF and improve them by leveraging channel-wise competitive learning in the context of convolutional neural networks for image classification tasks. A layer-wise loss function is introduced that promotes competitive learning and eliminates the need for negative data construction. To enhance both the learning of compositional features and feature space partitioning, a channel-wise feature separator and extractor block is proposed that complements the competitive learning process. Our method outperforms recent FF-based models on image classification tasks, achieving testing errors of 0.58%, 7.69%, 21.89%, and 48.77% on MNIST, Fashion-MNIST, CIFAR-10 and CIFAR-100 respectively. Our approach bridges the performance gap between FF learning and BP methods, indicating the potential of our proposed approach to learn useful representations in a layer-wise modular fashion, enabling more efficient and flexible learning. Our source code and supplementary material are available at https://github.com/andreaspapac/CwComp.

Toward More Generalized Malicious URL Detection Models

This paper reveals a data bias issue that can profoundly hinder the performance of machine learning models in malicious URL detection. We describe how such bias can be diagnosed using interpretable machine learning techniques and further argue that such biases naturally exist in the real world security data for training a classification model. To counteract these challenges, we propose a debiased training strategy that can be applied to most deep-learning based models to alleviate the negative effects of the biased features. The solution is based on the technique of adversarial training to train deep neural networks learning invariant embedding from biased data. Through extensive experimentation, we substantiate that our innovative strategy fosters superior generalization capabilities across both CNN-based and RNN-based detection models. The findings presented in this work not only expose a latent issue in the field but also provide an actionable remedy, marking a significant step forward in the pursuit of more reliable and robust malicious URL detection.

Abstract 2300: AI enabled whole exome& transcriptome liquid biopsy addressing MCED, MRD, and therapy selection on a single platform

Abstract Effective clinical management of cancer patients requires an accurate and early diagnosis, highly sensitive monitoring of minimal residual disease (MRD), and precise therapy selection. There are multiple tests available that attempt to address each of these needs independently with varying degrees of clinical utility. Caris Assure is a proprietary circulating nucleic acid sequencing platform that couples whole exome and transcriptome (WES/WTS) sequencing on white blood cells and plasma with advanced machine learning techniques to satisfy all three testing needs on one platform. This test detects SNVs, INDELs, structural variants, copy number, gene expression, tumor mutational burden (TMB), microsatellite instability (MSI), fragment length, and aneuploidy of both somatic (tumor and clonal hematopoiesis) and germline origin. Caris’ extensive database of over 350,000 tissue WES/WTS from solid malignancies was used to train deep learning neural networks to identify the molecular underpinnings of cancer. These networks were then deployed on WES/WTS data from plasma and buffy coat in pursuit of signals that can inform early detection, MRD and therapy selection. Validation studies were performed to characterize the analytic and clinical performance of Assure on over 3000 patient blood samples. These samples include ~1000 non-cancer patients (controls), ~1700 newly diagnosed patients where blood was collected at surgery (early detection), ~500 early-stage patients during adjuvant therapy at multiple time points (MRD), and ~200 locally advanced/metastatic patients where matched tissue testing was also performed (therapy selection). For early detection, stratification of blood samples from patients with stage I-IV cancer versus those with no reported cancer resulted in an AUC > 0.99 and included over 30 types of solid tumors. Notably, at 99.5% specificity, the sensitivities for stages I-IV (n= 119, 54, 50, 27) were 73%, 80%, 76%, and 89%. In the MRD setting for high-risk patients, the disease-free survival of patients whose cancers were predicted to recur was significantly shorter (39.6 mo) than those predicted not to recur (93.4 mo) (HR: 5.18, 95%CI: 2.94-9.09, p<.00001). This performance was observed across multiple lineages of cancer including but not limited to breast, colon, lung, and bladder. Lastly, for therapy selection, detection of driver mutations where blood was collected within 30 days of matched tissue demonstrated high concordance with a PPA of 93.8% and PPV of 96.8%. CHIP correction proved to be essential as ~35% percent of patients had CHIP mutations, including KRAS, BRAF, ATM, BRCA1/2, findings that could lead to improper therapy selection. Herein, we demonstrate for the first time a single liquid biopsy assay that addresses the entire continuum of care in clinical oncology with optimal diagnostic, prognostic, and predictive utility for patients and physicians. Citation Format: Jim Abraham, Valeriy Domenyuk, Maria Perdigones Borderias, Takayuki Yoshino, Elisabeth I. Heath, Emil Lou, Stephen Liu, John Marshall, Wafik S. El-Deiry, Anthony Shields, Martin Dietrich, Yoshiaki Nakamura, Takao Fujisawa, David D. Halbert, Dominic Sacchetti, Seth Stahl, Adam Stark, Sergey Klimov, Sourabh Antani, Chadi Nabhan, Jeffrey Swensen, George Poste, Matt Oberley, Milan Radovich, George W. Sledge, David Spetzler. AI enabled whole exome& transcriptome liquid biopsy addressing MCED, MRD, and therapy selection on a single platform [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2300.

A comprehensive study of a long-term creep thermo-mechanical fatigue behavior monitoring of BFRP composite pipeline using electrical capacitance sensors and deep learning algorithm

The composite pipeline is a relatively new and viable alternative pipeline to the more commonly used traditional one due to its good mechanical and fatigue properties and lower production cost. For this purpose, it is critical to assess the mechanical and fatigue performance of composite pipeline material under various working conditions, particularly for monitoring long-term creep thermo-mechanical fatigue behavior. In this paper, a long-term creep thermo-mechanical fatigue behavior in a basalt fiber reinforced polymer laminated composite pipeline is detected through an integrated expert system consisting of the electrical capacitance sensors and a deep learning algorithm. First, a multi-physics finite element model is established for the simulation of a long-term creep thermo-mechanical fatigue behavior in basalt fiber reinforced polymer composite pipelines subjected to long-term fatigue loading of internal pressure and thermal effect. Second, theoretical model results of long-term creep thermo-mechanical fatigue compliance (Sf(t)) over the time of creep are analyzed in pipeline material using the modulus degradation approach. Finally, an electrical potential change between electrical capacitance sensors electrodes corresponding to Sf(t) over the time of creep for some levels of long-term creep thermo-mechanical fatigue (Rf%) is recorded and then used in these datasets for training of the novel deep neural network based on one of the most widely used of the deep neural network families is the convolutional neural network, to predict Sf(t) in pipeline for various Rf% not included in the previous finite element model evaluation (i.e. electrical capacitance sensors technique). In this paper first is detected the long-term creep thermo-mechanical fatigue behavior for Rf%=25%,50%, and 75% from a finite element model and modulus degradation approach, and then is predicted the long-term creep thermo-mechanical fatigue behavior for Rf%=15%,35%,60%, and 85% respectively via deep neural network. The proposed method results are in good agreement with the experimental results available in the literature, thus verifying the accuracy and reliability of the proposed technique and its applicability to other different composite structures.

Adaptive Force Field Parameter Optimization for Expanding Reaction Simulations within Wide-Ranged Temperature.

Large-scale and long-term simulation of chemical reactions are key research topics in computational chemistry. However, there are still difficulties in simulating high-temperature reactions, such as polymer thermal decomposition. Herein, we introduce an adaptive potential parameter optimization framework designed to automatically fine-tune parameters, and the application of it to optimize ReaxFF parameters enhances the accuracy of chemical reaction simulations conducted at experimental temperatures. To achieve this, we leverage the power of Random Forests and interpretable machine learning techniques that enable the identification and selection of parameters that exert a substantial influence on the target attribute. By training deep neural network (NN) models, we established optimized parameter associations with reference properties. We train deep neural network (NN) models to establish the relationship between the optimized parameters and reference properties. We employ a Genetic Algorithm (GA) to utilize the surrogate NN model and the quantum mechanical targets to speed up the search for the optimal parameters. Our simulation results of resin pyrolysis show that the adaptive optimized ReaxFF can predict the peak temperature more accurately and obtain reasonable product composition under conditions that more closely resemble experimental scenarios. This work facilitates advances in force field parameter optimization for more accurate and universal reaction simulations.

An optimized neural network with AdaHessian for cryptojacking attack prediction for Securing Crypto Exchange Operations of MEC applications

Bitcoin exchange security is crucial because of MEC's widespread use. Cryptojacking has compromised MEC app security and bitcoin exchange ecosystem functionality. This paper propose a cutting-edge neural network and AdaHessian optimization technique for cryptojacking prediction and defense. We provide a cutting-edge deep neural network (DNN) cryptojacking attack prediction approach employing pruning, post-training quantization, and AdaHessian optimization. To solve these problems, this paper apply pruning, post-training quantization, and AdaHessian optimization. A new framework for quick DNN training utilizing AdaHessian optimization can detect cryptojacking attempts with reduced computational cost. Pruning and post-training quantization improve the model for low-CPU on-edge devices. The proposed approach drastically decreases model parameters without affecting Cryptojacking attack prediction. The model has Recall 98.72%, Precision 98.91%, F1-Score 99.09%, MSE 0.0140, RMSE 0.0137, and MAE 0.0139. Our solution beats state-of-the-art approaches in precision, computational efficiency, and resource consumption, allowing more realistic, trustworthy, and cost-effective machine learning models. We address increasing cybersecurity issues holistically by completing the DNN optimization-security loop. Securing Crypto Exchange Operations delivers scalable and efficient Cryptojacking protection, improving machine learning, cybersecurity, and network management.

Reusing Convolutional Neural Network Models through Modularization and Composition

With the widespread success of deep learning technologies, many trained deep neural network (DNN) models are now publicly available. However, directly reusing the public DNN models for new tasks often fails due to mismatching functionality or performance. Inspired by the notion of modularization and composition in software reuse, we investigate the possibility of improving the reusability of DNN models in a more fine-grained manner. Specifically, we propose two modularization approaches named CNNSplitter and GradSplitter, which can decompose a trained convolutional neural network (CNN) model for N -class classification into N small reusable modules. Each module recognizes one of the N classes and contains a part of the convolution kernels of the trained CNN model. Then, the resulting modules can be reused to patch existing CNN models or build new CNN models through composition. The main difference between CNNSplitter and GradSplitter lies in their search methods: the former relies on a genetic algorithm to explore search space, while the latter utilizes a gradient-based search method. Our experiments with three representative CNNs on three widely used public datasets demonstrate the effectiveness of the proposed approaches. Compared with CNNSplitter, GradSplitter incurs less accuracy loss, produces much smaller modules (19.88% fewer kernels), and achieves better results on patching weak models. In particular, experiments on GradSplitter show that (1) by patching weak models, the average improvement in terms of precision, recall, and F1-score is 17.13%, 4.95%, and 11.47%, respectively, and (2) for a new task, compared with the models trained from scratch, reusing modules achieves similar accuracy (the average loss of accuracy is only 2.46%) without a costly training process. Our approaches provide a viable solution to the rapid development and improvement of CNN models.

Temporal teacher with masked transformers for semi-supervised action proposal generation

By conditioning on unit-level predictions, anchor-free models for action proposal generation have displayed impressive capabilities, such as having a lightweight architecture. However, task performance depends significantly on the quality of data used in training, and most effective models have relied on human-annotated data. Semi-supervised learning, i.e., jointly training deep neural networks with a labeled dataset as well as an unlabeled dataset, has made significant progress recently. Existing works have either primarily focused on classification tasks, which may require less annotation effort, or considered anchor-based detection models. Inspired by recent advances in semi-supervised methods on anchor-free object detectors, we propose a teacher-student framework for a two-stage action detection pipeline, named Temporal Teacher with Masked Transformers (TTMT), to generate high-quality action proposals based on an anchor-free transformer model. Leveraging consistency learning as one self-training technique, the model jointly trains an anchor-free student model and a gradually progressing teacher counterpart in a mutually beneficial manner. As the core model, we design a Transformer-based anchor-free model to improve effectiveness for temporal evaluation. We integrate bi-directional masks and devise encoder-only Masked Transformers for sequences. Jointly training on boundary locations and various local snippet-based features, our model predicts via the proposed scoring function for generating proposal candidates. Experiments on the THUMOS14 and ActivityNet-1.3 benchmarks demonstrate the effectiveness of our model for temporal proposal generation task.

Combination of DCE-MRI and NME-DWI via Deep Neural Network for Predicting Breast Cancer Molecular Subtypes

BackgroundTo explore whether the combination of dynamic contrast-enhanced (DCE) magnetic resonance imaging (MRI) and nonmono-exponential (NME) model-based diffusion-weighted imaging (DWI) via deep neural network (DNN) can improve the prediction of breast cancer molecular subtypes compared to either imaging technique used alone. Patients and MethodsThis prospective study examined 480 breast cancers in 475 patients undergoing DCE-MRI and NME-DWI at 3.0 T. Breast cancers were classified as follows: human epidermal growth factor receptor 2 enriched (HER2-enriched), luminal A, luminal B (HER2–), luminal B (HER2+), and triple-negative subtypes. A total of 20% cases were withheld as an independent test dataset, and the remaining cases were used to train DNN with an 80% to 20% training-validation split and 5-fold cross-validation. The diagnostic accuracies of DNN in 5-way subtype classification between the DCE-MRI, NME-DWI, and their combined multiparametric-MRI datasets were compared using analysis of variance with least significant difference posthoc test. Areas under the receiver-operating characteristic curves were calculated to assess the performances of DNN in binary subtype classification between the 3 datasets. ResultsThe 5-way classification accuracies of DNN on both DCE-MRI (0.71) and NME-DWI (0.64) were significantly lower (P < .05) than on multiparametric-MRI (0.76), while on DCE-MRI was significantly higher (P < .05) than on NME-DWI. The comparative results of binary classification between the 3 datasets were consistent with the 5-way classification. ConclusionThe combination of DCE-MRI and NME-DWI via DNN achieved a significant improvement in breast cancer molecular subtype prediction compared to either imaging technique used alone. Additionally, DCE-MRI outperformed NME-DWI in differentiating subtypes.