Cascade Architecture Research Articles

The Combinatorial Fusion Cascade as a Neural Network

The combinatorial fusion cascade provides a surprisingly simple and complete explanation for the origin of the genetic code based on competing protocodes. Although its molecular basis is only beginning to be uncovered, it represents a natural pattern of information generation from initial signals and has potential applications in designing more-efficient neural networks. By utilizing the properties of the combinatorial fusion cascade, we demonstrate its embedding into deep neural networks with sequential fully connected layers using the dynamic matrix method and compare the resulting modifications. We observe that the Fiedler Laplacian eigenvector of a combinatorial cascade neural network does not reflect the cascade architecture. Instead, eigenvectors associated with the cascade structure exhibit higher Laplacian eigenvalues and are distributed widely across the network. We analyze a text classification model consisting of two sequential transformer layers with an embedded cascade architecture. The cascade shows a significant influence on the classifier’s performance, particularly when trained on a reduced dataset (approximately 3% of the original). The properties of the combinatorial fusion cascade are further examined for their application in training neural networks without relying on traditional error backpropagation.

Toward Closing the Loop in Image-to-Image Conversion in Radiotherapy: A Quality Control Tool to Predict Synthetic Computed Tomography Hounsfield Unit Accuracy.

In recent years, synthetic Computed Tomography (CT) images generated from Magnetic Resonance (MR) or Cone Beam Computed Tomography (CBCT) acquisitions have been shown to be comparable to real CT images in terms of dose computation for radiotherapy simulation. However, until now, there has been no independent strategy to assess the quality of each synthetic image in the absence of ground truth. In this work, we propose a Deep Learning (DL)-based framework to predict the accuracy of synthetic CT in terms of Mean Absolute Error (MAE) without the need for a ground truth (GT). The proposed algorithm generates a volumetric map as an output, informing clinicians of the predicted MAE slice-by-slice. A cascading multi-model architecture was used to deal with the complexity of the MAE prediction task. The workflow was trained and tested on two cohorts of head and neck cancer patients with different imaging modalities: 27 MR scans and 33 CBCT. The algorithm evaluation revealed an accurate HU prediction (a median absolute prediction deviation equal to 4 HU for CBCT-based synthetic CTs and 6 HU for MR-based synthetic CTs), with discrepancies that do not affect the clinical decisions made on the basis of the proposed estimation. The workflow exhibited no systematic error in MAE prediction. This work represents a proof of concept about the feasibility of synthetic CT evaluation in daily clinical practice, and it paves the way for future patient-specific quality assessment strategies.

Probabilistic regression for autonomous terrain relative navigation via multi-modal feature learning

The extension of human spaceflight across an ever-expanding domain, in conjunction with intricate mission architectures demands a paradigm shift in autonomous navigation algorithms, especially for the powered descent phase of planetary landing. Deep learning architectures have previously been explored to perform low-dimensional localization with limited success. Due to the expectations regarding novel algorithms in the context of real missions, the proposed approaches must be rigorously evaluated in extraneous scenarios and demonstrate sufficient robustness. In the current work, a novel formulation is proposed to train CNN-based Deep Learning (DL) models in a multi-layer cascading architecture and utilize the resulting classification probabilities as regression weights to estimate the position of the lander spacecraft. The approach leverages image intensity and depth data provided by multiple sensors on board to accurately determine the spacecraft’s location relative to the observed terrain at a specific altitude. Navigation performance is validated through Monte Carlo analysis, demonstrating the efficacy of the proposed DL architecture and the subsequent state-estimation framework across several simulated scenarios. It shows tremendous promise in extending the multi-modal feature learning approach to realistic missions.

Sparse Cascade: Multi-stage cascade fusion and shape dictionary guided network for segmentation of microscopic cytology images

Microscopic cytology image instance segmentation is critical for clinical diagnosis, treatment planning, and biomedical research. However, challenges such as cell overlapping, external interferences, and irregular shapes hinder accurate segmentation. In this paper, a microscopic cytology image instance segmentation algorithm based on a multi-stage cascaded fusion and shape-guided strategy is proposed to improve instance segmentation accuracy. To mitigate the issue of external interference and cell overlap, a feedforward fusion mechanism is introduced in the multi-stage cascade architecture to effectively integrate the mask features from multiple stages. To make the shape of the segmentation result more consistent with the shape of the cell, and to alleviate the issues of unclear boundaries and incomplete shape segmentation caused by cellular overlap and irregular shapes, the sparse code guidance branch based on the explicit shape guidance strategy is introduced. Experiments on the private embryonic cell dataset and the ISBI2014 dataset indicate that the network has achieved improvements over the Cascade Mask R-CNN, with enhancements of 2.51% and 3.92% in bounding boxes mean Average Precision (mAP), and 2.82% and 4.70% in segmentation mAP, respectively. The experimental results indicate that the network proposed in this paper can output more accurate bounding boxes and instance masks.

Cascadable optical nonlinear activation function based on Ge-Si.

To augment the capabilities of optical computing, specialized nonlinear devices as optical activation functions are crucial for enhancing the complexity of optical neural networks. However, existing optical nonlinear activation function devices often encounter challenges in preparation, compatibility, and multi-layer cascading. Here, we propose a cascadable optical nonlinear activation function architecture based on Ge-Si structured devices. Leveraging dual-source modulation, this architecture achieves cascading and wavelength switching by compensating for loss. Experimental comparisons with traditional Ge-Si devices validate the cascading capability of the new architecture. We first verified the versatility of this activation function in a MNIST task, and then in a multi-layer optical dense neural network designed for complex gesture recognition classification, the proposed architecture improves accuracy by an average of 23% compared to a linear network and 15% compared to a network with a traditional activation function architecture. With its advantages of cascadability and high compatibility, this work underscores the potential of all-optical activation functions for large-scale optical neural network scaling and complex task handling.

Mutual learning with memory for semi-supervised pest detection.

Effectively monitoring pest-infested areas by computer vision is essential in precision agriculture in order to minimize yield losses and create early scientific preventative solutions. However, the scale variation, complex background, and dense distribution of pests bring challenges to accurate detection when utilizing vision technology. Simultaneously, supervised learning-based object detection heavily depends on abundant labeled data, which poses practical difficulties. To overcome these obstacles, in this paper, we put forward innovative semi-supervised pest detection, PestTeacher. The framework effectively mitigates the issues of confirmation bias and instability among detection results across different iterations. To address the issue of leakage caused by the weak features of pests, we propose the Spatial-aware Multi-Resolution Feature Extraction (SMFE) module. Furthermore, we introduce a Region Proposal Network (RPN) module with a cascading architecture. This module is specifically designed to generate higher-quality anchors, which are crucial for accurate object detection. We evaluated the performance of our method on two datasets: the corn borer dataset and the Pest24 dataset. The corn borer dataset encompasses data from various corn growth cycles, while the Pest24 dataset is a large-scale, multi-pest image dataset consisting of 24 classes and 25k images. Experimental results demonstrate that the enhanced model achieves approximately 80% effectiveness with only 20% of the training set supervised in both the corn borer dataset and Pest24 dataset. Compared to the baseline model SoftTeacher, our model improves mAP @0.5 (mean Average Precision) at 7.3 compared to that of SoftTeacher at 4.6. This method offers theoretical research and technical references for automated pest identification and management.

SDFuse: Semantic-injected dual-flow learning for infrared and visible image fusion

Infrared and visible image fusion (IVIF) strives to render fused results that preserve the strengths of the source images (e.g., texture details and thermal highlights) while boosting downstream tasks. However, existing semantic-driven methods typically combine segmentation and fusion tasks via cascade architecture, which fails to sufficiently exploit semantic knowledge. To address this limitation, an innovative semantic-injected network with dual flows is proposed for IVIF, called SDFuse. Concretely, we first apply two pseudo-Siamese feature extraction modules (FEM) to obtain shallow structure features, which involve skip connections to prevent the disappearance of information. Subsequently, dual flows are constructed to support the joint learning paradigm of two tasks. The segmentation flow excavates abundant semantic knowledge from shallow features through fulfilling high-level segmentation task. The fusion flow is introduced to achieve the aggregation of highlights and textures. Note that SGM in fusion flow effectuates the injection of deep semantic knowledge into shallow structural features. Eventually, the region consistency module (RCM) with group normalization leads the network to focus on foreground objects and generate context-consistent fusion images. Our fusion results not only effectively fuse the information of the source image but also have advantages in downstream tasks. Extensive experiments over multiple representative benchmarks prove that SDFuse outperforms current state-of-the-art approaches.

Cascaded framework for earthquake building damage detection combining spatial and frequency domain feature integration

Abstract. Building collapse is a major cause of casualties after an earthquake, so accurately extracting building damage information is critical for post-earthquake assessment and rescue. Currently, most deep learning methods focus on the end-to-end detection of building collapse. However, in real-world earthquake scenarios, the end-to-end computational process often lacks flexibility and struggles to meet the requirements of rapid emergency response. To address this issue, this paper proposes a cascaded framework that combines pre-earthquake building extraction and post-earthquake building damage classification. The proposed framework includes two sections: (1) Progressive building semantic segmentation model in the joint frequency domain. This model is designed to accurately extract buildings prior to an earthquake, with the goal of minimizing error propagation throughout the cascading process. The model addresses the spatial similarity of buildings under complicated backgrounds, as well as the high internal heterogeneity of buildings, by utilizing frequency domain techniques. It compensates for the shortcomings of traditional models in terms of incomplete information extraction through the effective integration of global and local information. Finally, the model employs edge priors for edge regularization. (2) Rapid building damage classification process. Based on the accurate building extraction results, a fast and efficient classification process is developed. This process uses a simple and lightweight classification network to effectively extract building damage information caused by the earthquake. The superiority of the proposed framework is validated through comparison with traditional cascading architectures and end-to-end models. The results show that the cascading framework not only provides accurate pre-earthquake building extraction, but also enables efficient and accurate post-earthquake damage classification, which meets the requirements of rapid post-earthquake emergency response. This balance of accuracy and speed is essential for effective disaster management and recovery.

AMPActiPred: A three‐stage framework for predicting antibacterial peptides and activity levels with deep forest

AbstractThe emergence and spread of antibiotic‐resistant bacteria pose a significant public health threat, necessitating the exploration of alternative antibacterial strategies. Antibacterial peptide (ABP) is a kind of antimicrobial peptide (AMP) that has the potential ability to fight against bacteria infection, offering a promising avenue for developing novel therapeutic interventions. This study introduces AMPActiPred, a three‐stage computational framework designed to identify ABPs, characterize their activity against diverse bacterial species, and predict their activity levels. AMPActiPred employed multiple effective peptide descriptors to effectively capture the compositional features and physicochemical properties of peptides. AMPActiPred utilized deep forest architecture, a cascading architecture similar to deep neural networks, capable of effectively processing and exploring original features to enhance predictive performance. In the first stage, AMPActiPred focuses on ABP identification, achieving an Accuracy of 87.6% and an MCC of 0.742 on an elaborate dataset, demonstrating state‐of‐the‐art performance. In the second stage, AMPActiPred achieved an average GMean at 82.8% in identifying ABPs targeting 10 bacterial species, indicating AMPActiPred can achieve balanced predictions regarding the functional activity of ABP across this set of species. In the third stage, AMPActiPred demonstrates robust predictive capabilities for ABP activity levels with an average PCC of 0.722. Furthermore, AMPActiPred exhibits excellent interpretability, elucidating crucial features associated with antibacterial activity. AMPActiPred is the first computational framework capable of predicting targets and activity levels of ABPs. Finally, to facilitate the utilization of AMPActiPred, we have established a user‐friendly web interface deployed at https://awi.cuhk.edu.cn/∼AMPActiPred/.

On-chip ultra-high rejection and narrow bandwidth filter based on coherency-broken cascaded cladding-modulated gratings

Bragg filters are of essential importance for chip-scale photonic systems. However, the implementation of filters with sub-nanometer bandwidth and rejection beyond 70 dB is hindered by the high index contrast of the silicon-on-insulator platform, which makes filters prone to fabrication imperfections. In this paper, we propose to combine coherency-broken cascading architecture and cladding modulation to circumvent the intrinsic limitation. The cascading architecture effectively prevents the accumulation of phase errors, while the cladding modulation offers additional design freedom to reduce the coupling coefficient. A bimodal Bragg filter with a testing-equipment-limited rejection level of 74 dB and a 40 dB bandwidth of 0.44 nm is experimentally demonstrated. The minimum feature size is 90 nm, which significantly relieves the fabrication constraints.

Robust Adaptive Control Based on Incremental Nonlinear Dynamic Inversion for a Quadrotor in Presence of Partial Actuator Fault

This paper presents a novel nonlinear robust adaptive trajectory tracking control architecture for stabilizing and controlling a quadrotor in the presence of actuator partial faults. The proposed control strategy utilizes an Incremental Nonlinear Dynamic Inversion (INDI) algorithm as the baseline controller in the inner loop and augments a nonlinear model reference adaptive controller in the outer loop to ensure robustness against unmodeled faults. Additionally, a modified PID controller is introduced in the most outer-loop to track the desired path. The effects of actuator faults are modeled by considering sudden variations in motor thrust and torques. To enhance the control algorithm's robustness, a projection operator is employed in the robust adaptive structure. Comparative performance evaluations with a previous successful algorithm implemented on a quadrotor model demonstrate that the proposed controller achieves full controllability of the faulty quadrotor in pitch, roll, and yaw channels in the presence of actuator partial faults up to 50%.

VIRA: An exoplanet atmospheric retrieval framework for JWST transmission spectroscopy

Abstract JWST observations are leading to important new insights into exoplanetary atmospheres through transmission spectroscopy. In order to harness the full potential of the broad spectral range and high sensitivity of JWST, atmospheric retrievals of exoplanets require a high level of robustness and accuracy in the underlying models. We present the VIRA retrieval framework which implements a range of modelling and inference capabilities motivated by early JWST observations of exoplanet transmission spectra. This includes three complementary approaches to modelling atmospheric composition, three atmospheric aerosol models, including a physically-motivated Mie scattering approach, and consideration of correlated noise. VIRA enables a cascading retrieval architecture involving a sequence of retrievals with increasing sophistication. We demonstrate VIRA using a JWST transmission spectrum of the hot Saturn WASP-39 b in the ∼1–5 μm range. In addition to confirming prior chemical inferences, we retrieve molecular abundances for H2O, CO, CO2, SO2 and H2S, resulting in super-solar elemental abundances of log(O/H) = −2.0 ± 0.2, log(C/H) = −2.1 ± 0.2 and log(S/H) = −3.6 ± 0.2, along with C/O and S/O ratios of $0.83^{+0.05}_{-0.07}$ and $0.029^{+0.012}_{-0.009}$, respectively, in the free chemistry case. The abundances correspond to $20.1^{+10.5}_{-8.1}\times$, $28.2^{+16.3}_{-12.1}\times$ and $20.8^{+10.3}_{-7.5}\times$ solar values for O/H, C/H and S/H, respectively, compared to C/H =8.67 ± 0.35 × solar for Saturn. Our results demonstrate how JWST transmission spectroscopy combined with retrieval frameworks like VIRA can measure multi-elemental abundances for giant exoplanets and enable comparative characterisation with solar system planets.

On the Expressivity of Recurrent Neural Cascades

Recurrent Neural Cascades (RNCs) are the recurrent neural networks with no cyclic dependencies among recurrent neurons. This class of recurrent networks has received a lot of attention in practice. Besides training methods for a fixed architecture such as backpropagation, the cascade architecture naturally allows for constructive learning methods, where recurrent nodes are added incrementally one at a time, often yielding smaller networks. Furthermore, acyclicity amounts to a structural prior that even for the same number of neurons yields a more favourable sample complexity compared to a fully-connected architecture. A central question is whether the advantages of the cascade architecture come at the cost of a reduced expressivity. We provide new insights into this question. We show that the regular languages captured by RNCs with sign and tanh activation with positive recurrent weights are the star-free regular languages. In order to establish our results we developed a novel framework where capabilities of RNCs are assessed by analysing which semigroups and groups a single neuron is able to implement. A notable implication of our framework is that RNCs can achieve the expressivity of all regular languages by introducing neurons that can implement groups.

Decoding Global Preferences: Temporal and Cooperative Dependency Modeling in Multi-Agent Preference-Based Reinforcement Learning

Designing accurate reward functions for reinforcement learning (RL) has long been challenging. Preference-based RL (PbRL) offers a promising approach by using human preferences to train agents, eliminating the need for manual reward design. While successful in single-agent tasks, extending PbRL to complex multi-agent scenarios is nontrivial. Existing PbRL methods lack the capacity to comprehensively capture both temporal and cooperative aspects, leading to inadequate reward functions. This work introduces an advanced multi-agent preference learning framework that effectively addresses these limitations. Based on a cascading Transformer architecture, our approach captures both temporal and cooperative dependencies, alleviating issues related to reward uniformity and intricate interactions among agents. Experimental results demonstrate substantial performance improvements in multi-agent cooperative tasks, and the reconstructed reward function closely resembles expert-defined reward functions. The source code is available at https://github.com/catezi/MAPT.

CasOmniMVS: Cascade Omnidirectional Depth Estimation with Dynamic Spherical Sweeping

Estimating 360∘ depth from multiple cameras has been a challenging problem. However, existing methods often adopt a fixed-step spherical sweeping approach with densely sampled spheres and use numerous 3D convolutions in networks, which limits the speed of algorithms in practice. Additionally, obtaining high-precision depth maps of real scenes poses a challenge for the existing algorithms. In this paper, we design a cascade architecture using a dynamic spherical sweeping method that progressively refines the depth estimation from coarse to fine over multiple stages. The proposed method adaptively adjusts sweeping intervals and ranges based on the predicted depth and the uncertainty from the previous stage, resulting in a more efficient cost aggregation performance. The experimental results demonstrated that our method achieved state-of-the-art accuracy with reduced GPU memory usage and time consumption compared to the other methods. Furthermore, we illustrate that our method achieved satisfactory performance on real-world data, despite being trained on synthetic data, indicating its generalization potential and practical applicability.

Partial Power Converter for Electric Vehicle Hybrid Energy Storage System Using a Controlled Current Source Cascade Architecture

Partial Power Converter for Electric Vehicle Hybrid Energy Storage System Using a Controlled Current Source Cascade Architecture

Cascading Machine Learning to Monitor Volcanic Thermal Activity Using Orbital Infrared Data: From Detection to Quantitative Evaluation

Several satellite missions are currently available to provide thermal infrared data at different spatial resolutions and revisit time. Furthermore, new missions are planned thus enabling to keep a nearly continuous ‘eye’ on thermal volcanic activity around the world. This massive volume of data requires the development of artificial intelligence (AI) techniques for the automatic processing of satellite data in order to extract significant information about volcano conditions in a short time. Here, we propose a robust machine learning approach to accurately detect, recognize and quantify high-temperature volcanic features using Sentinel-2 MultiSpectral Instrument (S2-MSI) imagery. We use the entire archive of high spatial resolution satellite data containing more than 6000 S2-MSI scenes at ten different volcanoes around the world. Combining a ‘top-down’ cascading architecture, two different machine learning models, a scene classifier (SqueezeNet) and a pixel-based segmentation model (random forest), we achieved a very high accuracy, namely 95%. These results show that the cascading approach can be applied in near-real time to any available satellite image, providing a full description of the scene, with an important contribution to the monitoring, mapping and characterization of volcanic thermal features.

CTG-Net: an efficient cascaded framework driven by terminal guidance mechanism for dilated pancreatic duct segmentation

Pancreatic duct dilation indicates a high risk of various pancreatic diseases. Segmentation for dilated pancreatic duct (DPD) on computed tomography (CT) image shows the potential to assist the early diagnosis, surgical planning and prognosis. Because of the DPD’s tiny size, slender tubular structure and the surrounding distractions, most current researches on DPD segmentation achieve low accuracy and always have segmentation errors on the terminal DPD regions. To address these problems, we propose a cascaded terminal guidance network to efficiently improve the DPD segmentation performance. Firstly, a basic cascaded segmentation architecture is established to get the pancreas and coarse DPD segmentation, a DPD graph structure is build on the coarse DPD segmentation to locate the terminal DPD regions. Then, a terminal anatomy attention module is introduced for jointly learning the local intensity from the CT images, feature cues from the coarse DPD segmentation and global anatomy information from the designed pancreas anatomy-aware maps. Finally, a terminal distraction attention module which explicitly learns the distribution of the terminal distraction regions is proposed to reduce the false positive and false negative predictions. We also propose a new metric called tDice to measure the terminal segmentation accuracy for targets with tubular structures and two other metrics for segmentation error evaluation. We collect our dilated pancreatic duct segmentation dataset with 150 CT scans from patients with five types of pancreatic tumors. Experimental results on our dataset show that our proposed approach boosts DPD segmentation accuracy by nearly 20% compared with the existing results, and achieves more than 9% improvement for the terminal segmentation accuracy compared with the state-of-the-art methods.

Deep Learning for Automated Contouring of Primary Gross Tumor Volumes by MRI for Radiation Therapy of Brain Metastasis

Radiotherapy is one of the most effective methods for the treatment of brain metastases (BMs). Traditional manual delineation of primary gross tumor volumes (GTV) of multiple BMs (especially small metastases) in radiotherapy practice is extremely labor intensive and highly dependent on oncologists' experience, achieving the precise and efficient automatic delineation of BMs is of great significance for efficient and homogeneous one-stop adaptive radiotherapy. We retrospectively collected 62 MRI (non-enhanced T1-weighted sequences) sequences of 50 patients with BMs from January 2020 to July 2021. An automatic model (BUC-Net) for automatic delineation BMs was proposed in this work, which was based on deep learning by combining 3D bottler layer module and the cascade architecture to improve the accuracy and efficient of BMs' automatic delineation, especially for small metastases with tiny size and relatively low contrast. The prosed method was compared with the existing 3D U-Net (U-Net) and 3D U-Net Cascade (U-Net Cascade). The performance of our proposed method was evaluated by Dice similarity coefficient (DSC), 95% Hausdorff distance (HD95) and average surface distance (ASD) with human experts. The automatic segmentation results of BUC-Net evaluated with 310 BMs in 13 test patients was summarized in Table 1. These BMs in each test patient were automatically delineated by two types of contours: as a whole tumor contour (Whole-delineation) and the multiple tumor contours (Multiple-delineation). BUC-Net performed the best mean DSC and HD95, which is significantly outperformed U-Net (Whole-delineation: 0.911 & 0.894 of DSC, Multiple-delineation: 0.794 & 0.754 of DSC, P < 0.05 for both) and U-Net cascade (Whole-delineation: 0.947 & 7.141 of HD95, Multiple-delineation: 0.902 & 1.171 of HD95, P < 0.05 for both); Additionally, BUC-Net achieved the best mean ASD for Whole-delineation and comparable ASD (0.290 & 0.277, P > 0) for Multiple-delineation with U-Net Cascade. Our results showed that the proposed approach is promising for the automatic delineation of BMs in MRI, which can be integrated into a radiotherapy workflow to significantly shorten segmentation time.

MiPEPPred-FRL: A Novel Method for Predicting Plant MiRNA-Encoded Peptides Using Adaptive Feature Representation Learning.

MicroRNAs (miRNAs) are an essential type of small molecule RNAs that play significant regulatory roles in organisms. Recent studies have demonstrated that small open reading frames (sORFs) harbored in primary miRNAs (pri-miRNAs) can encode small peptides, known as miPEPs. Plant miPEPs can increase the abundance and activity of cognate miRNAs by promoting the transcription of their corresponding pri-miRNAs, thereby modulating plant traits. Biological experiments are the most effective way to accurately identify miPEPs; however, they are time-consuming and expensive. Hence, an efficient computational method for the identification of miPEPs on a large scale is highly desirable. Up to now, there have been no specialized computational tools for identifying miPEPs. In this work, a novel predictor named miPEPPred-FRL based on an adaptive feature representation learning framework that consists of the feature transformation module and the cascade architecture has been proposed. The feature transformation module integrating a newly designed feature selection method and classifier selection rule is developed to convert sequence-based features into primary class and probabilistic features, which are then fed into the improved cascade architecture to obtain more stable and discriminative augmented features. Finally, the augmented features are utilized to construct the final predictor. Cross-validation experiments illustrate that the novel feature selection method and classifier selection rule contribute to boosting the feature representation ability of the framework. Furthermore, the high accuracy of miPEPPred-FRL on independent testing data suggests that it is a trustworthy and valuable tool for the identification of miPEPs.