Multiple Deep Neural Networks Research Articles

DyPanVO: a robust monocular visual odometry in dynamic environments by utilizing multiple deep neural networks

Abstract Traditional monocular Visual Odometry (VO) is typically based on the assumption of a static environment. However, it performs poorly in dynamic scenes, suffering from error accumulation and scale drift. To address these issues, a novel framework, named DyPanVO, was proposed, which incorporates multiple deep neural networks for feature point extraction, panoptic segmentation, and depth estimation. Firstly, learning-based methods are employed, specifically SuperPoint for robust feature extraction and LightGlue for accurate feature matching. Then, outlier points are eliminated by combining dynamic prior results from panoptic segmentation with epipolar geometry constraints to determine the motion state of objects. Additionally, the introduction of depth information ensures that scale is recovered and fundamentally solves the issue of error accumulation. Two types of experiment (outdoor and indoor scenes) are carried out to show the effectiveness of DyPanVO. Moreover, the comparison results demonstrate that it performs comparably with multi-view geometry-based and outperforms learning-based methods.

The Impact Mechanism of Non-Economic Policies on Social and Investor Disagreement in China: A Dual Analysis Based on Empirical Evidence and DSGE Models

This study investigates the integration of non-economic policies into the framework for assessing macroeconomic coherence as applied by the Chinese government, with a particular focus on green policies. We examine the impact of non-economic factors on social disagreement and investor disagreement (expectations), and how these influences interact with macroeconomic regulation, employing both empirical evidence and dynamic stochastic general equilibrium (DSGE) theoretical models. In the basic analysis section, we merge statistical data on social divergence with policy implementation, utilizing multiple regression and deep neural network models. Our findings provide direct evidence that non-economic policies significantly regulate social sentiment. Additionally, theoretical analyses using contagion models, grounded in real textual data on social and investor divergence, demonstrate that expectations of social sentiment can ultimately affect economic variables. In the extended analysis, we enhance the classic DSGE model to delineate the pathways through which non-economic policies impact the macroeconomy. Drawing from our analyses, we propose specific optimization measures for non-economic policies. The results indicate that targeted policy optimization can effectively manage social disagreement, thereby shaping expectations and harmonizing non-economic with economic policy initiatives. This alignment enhances the coherence of macroeconomic policy interventions. The innovative contribution of this study lies in its provision of both theoretical and empirical evidence supporting the formulation of non-economic policies for the first time, alongside specific recommendations for improving the consistency of macroeconomic policies.

Classifying School Scope Using Deep Neural Networks Based on Students' Surrounding Living Environments

This research investigates school scope classification using Deep Neural Networks (DNN), focusing on students living environments and educational opportunities. By addressing the interplay of socioeconomic and educational factors, the study aims to develop an analytical framework for understanding how environmental contexts shape academic trajectories. The research provides a nuanced understanding of the importance of features in educational classification by developing DNN models based on Spearman's Rank Correlation Coefficient (SRCC). The methodology employs machine learning techniques, integrating data wrangling, exploratory analysis, and multiple DNN models with K-fold cross-validation. The study analyzes 677 student records from two schools. The research examined multiple model configurations. Results show that the 'All Data' model achieved 83.08% accuracy, the 'Top 5' model 81.54%, and the 'Non-Top 5' model 79.23%. The SRCC-based approach revealed that while top correlated features are important, additional variables significantly contribute to model performance. The study highlights the profound impact of family background, social environment, and educational contexts on school selection. Furthermore, it demonstrates DNN's capability to uncover intricate, non-linear relationships, offering actionable insights for policymakers to leverage machine learning's potential in developing targeted educational strategies.

Enhancing object recognition: The role of object knowledge decomposition and component-labeled datasets

Deep learning models’ decision-making processes can be elusive, often raising concerns about their reliability. To address this, we have introduced the Object Knowledge Decomposition and Components Label Dataset (OKD-CL), designed to improve the interpretability and accuracy of object recognition models. This dataset includes 99 categories from PartImageNet, each detailed with clear physical structures that align with human visual concepts. In a hierarchical structure, every category is described by Abstract Component Knowledge (ACK) descriptions and each image instance comes with Explicit Visual Knowledge (EVK) masks, highlighting the visual components’ appearance. By evaluating multiple deep neural networks guided with ACK and EVK (dual-knowledge-guidance approach), we saw better accuracy and a higher Foreground Reasoning Ratio (FRR), confirming our knowledge-guided method’s effectiveness. When used on the Hard-ImageNet dataset, this approach reduced the model’s reliance on incorrect feature assumptions without sacrificing classification accuracy. This hierarchical comprehension encouraged by OKD-CL is crucial in minimizing incorrect feature associations and strengthening model robustness. The entire code and dataset are available on: https://github.com/XiGuaBo/OKD-CL.

A 2 : Towards Accelerator Level Parallelism for Autonomous Micromobility Systems

Autonomous micromobility systems (AMS) such as low-speed minicabs and robots are thriving. In AMS, multiple Deep Neural Networks execute in parallel on heterogeneous AI accelerators. An emerging paradigm called Accelerator Level Parallelism (ALP) suggests managing accelerators holistically. However, there lacks a specialized and practical solution populating ALP for an AMS, where the varying real-time requirements under different working scenarios bring an opportunity to dynamically tradeoff between latency and efficiency. Furthermore, accelerator heterogeneity introduces enormous configuration space, and the shared-memory architecture results in dynamic bandwidth interference. In this paper, we propose A 2 , a novel AMS resource manager optimizing energy and memory space efficiency under variable latency constraints. We gain insight from prior Learn&Control scheme to design an Analyze&Adapt scheme specialized for heterogeneous AI accelerators under shared-memory architecture. It features analyzing the system thoroughly offline to support two-step adaptation online. We build a prototype of A 2 and evaluate it on a commercial edge platform. We show that A 2 achieves 32.8% improvements in power and 13.8% in memory compared with control-based methods. As for timeliness enhancement, A 2 reduces the deadline violation rate by 9.2 percentage points (12.8% → 3.6%) on average compared to directly porting Learn&Control methods.

Generating Daily High-Resolution Regional XCO2 by Deep Neural Network and Multi-Source Data

CO2 is one of the primary greenhouse gases impacting global climate change, making it crucial to understand the spatiotemporal variations of CO2. Currently, commonly used satellites serve as the primary means of CO2 observation, but they often suffer from striping issues and fail to achieve complete coverage. This paper proposes a method for constructing a comprehensive high-spatiotemporal-resolution XCO2 dataset based on multiple auxiliary data sources and satellite observations, utilizing multiple simple deep neural network (DNN) models. Global validation results against ground-based TCCON data demonstrate the excellent accuracy of the constructed XCO2 dataset (R is 0.94, RMSE is 0.98 ppm). Using this method, we analyze the spatiotemporal variations of CO2 in China and its surroundings (region: 0°–60° N, 70°–140° E) from 2019 to 2020. The gapless and fine-scale CO2 generation method enhances people’s understanding of CO2 spatiotemporal variations, supporting carbon-related research.

DeepIOD: Towards A Context-Aware Indoor-Outdoor Detection Framework Using Smartphone Sensors.

Accurate indoor-outdoor detection (IOD) is essential for location-based services, context-aware computing, and mobile applications, as it enhances service relevance and precision. However, traditional IOD methods, which rely only on GPS data, often fail in indoor environments due to signal obstructions, while IMU data are unreliable on unseen data in real-time applications due to reduced generalizability. This study addresses this research gap by introducing the DeepIOD framework, which leverages IMU sensor data, GPS, and light information to accurately classify environments as indoor or outdoor. The framework preprocesses input data and employs multiple deep neural network models, combining outputs using an adaptive majority voting mechanism to ensure robust and reliable predictions. Experimental results evaluated on six unseen environments using a smartphone demonstrate that DeepIOD achieves significantly higher accuracy than methods using only IMU sensors. Our DeepIOD system achieves a remarkable accuracy rate of 98-99% with a transition time of less than 10 ms. This research concludes that DeepIOD offers a robust and reliable solution for indoor-outdoor classification with high generalizability, highlighting the importance of integrating diverse data sources to improve location-based services and other applications requiring precise environmental context awareness.

Evaluation of multiple deep neural networks for detection of intracranial dural arteriovenous fistula on susceptibility weighted angiography imaging.

The natural history of intracranial dural arteriovenous fistula (DAVF) is variable and early diagnosis is crucial in order to positively impact the clinical course of aggressive DAVF. Artificial intelligence (AI) based techniques can be promising in this regard, and in this study, we used various deep neural network (DNN) architectures to determine whether DAVF could be reliably identified on susceptibility-weighted angiography images (SWAN). A total of 3965 SWAN image slices from 30 digital subtraction angiographically proven DAVF patients and 4380 SWAN image slices from 40 age-matched patients with normal MRI findings as control group were included. The images were categorized as either DAVF or normal and the data was trained using various DNN such as VGG-16, EfficientNet-B0, and ResNet-50. Various DNN architectures showed the accuracy of 95.96% (VGG-16), 91.75% (EfficientNet-B0), and 86.23% (ResNet-50) on the SWAN image dataset. ROC analysis yielded an area under the curve of 0.796 (p < .001), best for VGG-16 model. Criterion of seven consecutive positive slices for DAVF diagnosis yielded a sensitivity of 74.68% with a specificity of 69.15%, while setting eight slices improved the sensitivity to above 80.38%, with a decrease of specificity up to 56.38%. Based on seven consecutive positive slices criteria, EfficientNet-B0 yielded a sensitivity of 73.21% with a specificity of 45.92% and ResNet-50 yielded a sensitivity of 72.39% with a specificity of 67.42%. This study shows that DNN can extract discriminative features of SWAN for the classification of DAVF from normal with good accuracy, reasonably good sensitivity and specificity.

Collaborative intrusion detection using weighted ensemble averaging deep neural network for coordinated attack detection in heterogeneous network

Detecting coordinated attacks in cybersecurity is challenging due to their sophisticated and distributed nature, making traditional Intrusion Detection Systems often ineffective, especially in heterogeneous networks with diverse devices and systems. This research introduces a novel Collaborative Intrusion Detection System (CIDS) using a Weighted Ensemble Averaging Deep Neural Network (WEA-DNN) designed to detect such attacks. The WEA-DNN combines deep learning techniques and ensemble methods to enhance detection capabilities by integrating multiple Deep Neural Network (DNN) models, each trained on different data subsets with varying architectures. Differential Evolution optimizes the model’s contributions by calculating optimal weights, allowing the system to collaboratively analyze network traffic data from diverse sources. Extensive experiments on real-world datasets like CICIDS2017, CSE-CICIDS2018, CICToNIoT, and CICBotIoT show that the CIDS framework achieves an average accuracy of 93.8%, precision of 78.6%, recall of 60.4%, and an F1-score of 62.4%, surpassing traditional ensemble models and matching the performance of local DNN models. This demonstrates the practical benefits of WEA-DNN in improving detection capabilities in real-world heterogeneous network environments, offering superior adaptability and robustness in handling complex attack patterns.

Robust knowledge distillation based on feature variance against backdoored teacher model

Benefiting from large well-trained deep neural networks (DNNs), model compression has captured special attention for computing resource limited equipment, especially edge devices. Knowledge distillation (KD) is one of the widely used compression techniques for edge deployment, by obtaining a lightweight student model from a well-trained teacher model released on public platforms. However, it has been empirically noticed that the backdoor in the teacher model will be transferred to the student model during the process of KD. Although numerous KD methods have been proposed, most of them focus on the distillation of a high-performing student model without robustness consideration. Besides, some research adopts KD techniques as effective backdoor mitigation tools, but they fail to perform model compression at the same time. Consequently, it is still an open problem to well achieve two objectives of robust KD, i.e., student model’s performance and backdoor mitigation. To address these issues, we propose RobustKD, a robust knowledge distillation that compresses the model while mitigating backdoor based on feature variance. Specifically, RobustKD distinguishes the previous works in three key aspects: (1) effectiveness - by distilling the feature map of the teacher model after detoxification, the main task performance of the student model is comparable to that of the teacher model; (2) robustness - by reducing the characteristic variance between the teacher model and the student model, it mitigates the backdoor of the student model under backdoored teacher model scenario; (3) generic - RobustKD still has good performance in the face of multiple data models (e.g., WRN 28-4, Pyramid-200) and diverse DNNs (e.g., ResNet50, MobileNet). Comprehensive experiments are conducted on four datasets, six models, two distillation methods, and two backdoor attack methods, compared with four baselines, and the results verified that the proposed method achieves the state-of-the-art performance in both aspects of accuracy and robustness. In addition, RobustKD is still effective when adaptive attacks are considered. The code of RobustKD is open-sourced at https://github.com/Xming-Z/RobustKD.

Deciphering social traits and pathophysiological conditions from natural behaviors in common marmosets

Nonhuman primates (NHPs) are indispensable animal models by virtue of the continuity of behavioral repertoires across primates, including humans. However, behavioral assessment at the laboratory level has so far been limited. Employing the application of three-dimensional (3D) pose estimation and the optimal integration of subsequent analytic methodologies, we demonstrate that our artificial intelligence (AI)-based approach has successfully deciphered the ethological, cognitive, and pathological traits of common marmosets from their natural behaviors. By applying multiple deep neural networks trained with large-scale datasets, we established an evaluation system that could reconstruct and estimate the 3D poses of the marmosets, a small NHP that is suitable for analyzing complex natural behaviors in laboratory setups. We further developed downstream analytic methodologies to quantify a variety of behavioral parameters beyond motion kinematics. We revealed the distinct parental roles of male and female marmosets through automated detections of food-sharing behaviors using a spatial-temporal filter on 3D poses. Employing a recurrent neural network to analyze 3D pose time series data during social interactions, we additionally discovered that marmosets adjusted their behaviors based on others' internal state, which is not directly observable but can be inferred from the sequence of others' actions. Moreover, a fully unsupervised approach enabled us to detect progressively appearing symptomatic behaviors over a year in a Parkinson's disease model. The high-throughput and versatile nature of an AI-driven approach to analyze natural behaviors will open a new avenuefor neuroscience research dealing with big-data analyses of social and pathophysiological behaviors in NHPs.

Sex classification of 3D skull images using deep neural networks

Determining the fundamental characteristics that define a face as "feminine" or "masculine" has long fascinated anatomists and plastic surgeons, particularly those involved in aesthetic and gender-affirming surgery. Previous studies in this area have relied on manual measurements, comparative anatomy, and heuristic landmark-based feature extraction. In this study, we collected retrospectively at Cedars Sinai Medical Center (CSMC) a dataset of 98 skull samples, which is the first dataset of this kind of 3D medical imaging. We then evaluated the accuracy of multiple deep learning neural network architectures on sex classification with this dataset. Specifically, we evaluated methods representing three different 3D data modeling approaches: Resnet3D, PointNet++, and MeshNet. Despite the limited number of imaging samples, our testing results show that all three approaches achieve AUC scores above 0.9 after convergence. PointNet++ exhibits the highest accuracy, while MeshNet has the lowest. Our findings suggest that accuracy is not solely dependent on the sparsity of data representation but also on the architecture design, with MeshNet's lower accuracy likely due to the lack of a hierarchical structure for progressive data abstraction. Furthermore, we studied a problem related to sex determination, which is the analysis of the various morphological features that affect sex classification. We proposed and developed a new method based on morphological gradients to visualize features that influence model decision making. The method based on morphological gradients is an alternative to the standard saliency map, and the new method provides better visualization of feature importance. Our study is the first to develop and evaluate deep learning models for analyzing 3D facial skull images to identify imaging feature differences between individuals assigned male or female at birth. These findings may be useful for planning and evaluating craniofacial surgery, particularly gender-affirming procedures, such as facial feminization surgery.

Graph deep learning recognition of port ship behavior patterns from a network approach

The detection and recognition of ship mobile behavior patterns based on automatic identification system (AIS) data are crucial for ship navigation and maritime supervision. However, present methods, such as rule-based constraints and density clustering extraction, still suffer from restrictions imposed by artificial settings of parameters and thresholds. The lack of neighborhood context information in unstructured and serialized ship trajectory vector data makes it challenging to incorporate advanced methods such as deep learning for pattern learning. This study builds multiple semi-supervised deep learning graph neural network (GNN) models based on the message passing paradigm for recognizing ship mobile behavior patterns by constructing an AIS trajectory network and computing multi-order node features through Delaunay triangulation. It can get rid of the dependence on parameters of traditional rule-based and unsupervised learning methods. It also reduces the dependence on auxiliary information in trajectory qualitative by introducing multi-order node features. We conducted the experiments on cargo ship data from New York and Singapore to test our method. The model's classification accuracy is more than 99% for the New York dataset and more than 95% for the Singapore dataset. This method can be extended to other types of ships and traffic trajectory vector data with the excellent application potential of GNN in pattern recognition of vector mobile data. The code and experimental data are available at: https://github.com/destiny1103/DT-GNN.

Supplementation of deep neural networks with simplified physics-based features to increase accuracy of plate fundamental frequency predictions

To improve predictive machine learning-based models limited by sparse data, supplemental physics-related features are introduced into a deep neural network (DNN). While some approaches inject physics through differential equations or numerical simulation, improvements are possible using simplified relationships from engineering references. To evaluate this hypothesis, thin rectangular plates were simulated to generate training datasets. With plate dimensions and material properties as input features and fundamental natural frequency as the output, predictive performance of a data-driven DNN-based model is compared with models using supplemental inputs, such as modulus of rigidity. To evaluate model accuracy improvements, these additional features are injected into various DNN layers, and the network is trained with four different dataset sizes. When evaluated against independent data of similar features to the training sets, supplementation provides no statistically-significant prediction error reduction. However, notable accuracy gains occur when independent test data is of material and dimensions different from the original training set. Furthermore, when physics-enhanced data is injected into multiple DNN layers, reductions in mean error from 33.2% to 19.6%, 34.9% to 19.9%, 35.8% to 22.4%, and 43.0% to 28.4% are achieved for dataset sizes of 261, 117, 60, and 30, respectively, demonstrating potential for generalizability using a data supplementation approach. Additionally, when compared with other methods—such as linear regression and support vector machine (SVM) approaches—the physics-enhanced DNN demonstrates an order of magnitude reduction in percentage error for dataset sizes of 261, 117, and 60 and a 30% reduction for a size of 30 when compared with a cubic SVM model independently tested with data divergent from the training and validation set.

Data Representation for Deep Learning - Based Arabic Text Summarization Performance Using Python Results

A sequence-to-sequence model is used as the foundation for a suggested abstractive Arabic text summarizing system. Our goal is to create a sequence-to-sequence model by utilizing multiple deep artificial neural networks and determining which one performs the best. The encoder and decoder have been developed using several layers of recurrent neural networks, gated recurrent units, recursive neural networks, convolutional neural networks, long short-term memory, and bidirectional long short-term memory. We are re-implementing the fundamental summarization model in this study, which uses the sequence-to-sequence framework. Using a Google Colab Jupiter notebook that runs smoothly, we have constructed these models using the Keras library. The results further demonstrate that one of the key techniques that has led to breakthrough performance with deep neural networks is the use of Gensim for word embeddings over other text representations by abstractive summarization models, along with FastText, a library for efficient learning of word representations and sentence classification.

Ensemble detection of hand joint ankylosis and subluxation in radiographic images using deep neural networks

The modified total Sharp score (mTSS) is often used as an evaluation index for joint destruction caused by rheumatoid arthritis. In this study, special findings (ankylosis, subluxation, and dislocation) are detected to estimate the efficacy of mTSS by using deep neural networks (DNNs). The proposed method detects and classifies finger joint regions using an ensemble mechanism. This integrates multiple DNN detection models, specifically single shot multibox detectors, using different training data for each special finding. For the learning phase, we prepared a total of 260 hand X-ray images, in which proximal interphalangeal (PIP) and metacarpophalangeal (MP) joints were annotated with mTSS by skilled rheumatologists and radiologists. We evaluated our model using five-fold cross-validation. The proposed model produced a higher detection accuracy, recall, precision, specificity, F-value, and intersection over union than individual detection models for both ankylosis and subluxation detection, with a detection rate above 99.8% for the MP and PIP joint regions. Our future research will aim at the development of an automatic diagnosis system that uses the proposed mTSS model to estimate the erosion and joint space narrowing score.

Reliable ECG analysis using recognition scores from multiple deep neural networks

Reliable ECG analysis using recognition scores from multiple deep neural networks

Optimizing Job Offloading Schedule for Collaborative DNN Inference

Deep Neural Networks (DNNs) have been widely deployed in mobile applications. DNN inference latency is a critical metric to measure the service quality of those applications. Collaborative inference is a promising approach for latency optimization, where partial inference workloads are offloaded from mobile devices to cloud servers. Model partition problems for collaborative inference have been well studied. However, little attention has been paid to optimizing offloading pipeline for multiple DNN inference jobs. In practice, mobile devices usually need to process multiple DNN inference jobs simultaneously. We propose to jointly optimize the DNN partitioning and pipeline scheduling for multiple inference jobs. We theoretically analyze the optimal scheduling conditions for homogeneous chain-structure DNNs. Based on the analysis, we proposed near-optimal partitioning and scheduling methods for chain-structure DNNs. We also extend those methods for general-structure DNNs. In addition, we extend our problem scenario to handle heterogeneous DNN inference jobs. A layer-level scheduling algorithm is proposed. Theoretical analyses show that our proposed method is optimal when computation graphs are tree-structure. Our joint optimization methods are evaluated in a real-world testbed. Experiment results show that our methods can significantly reduce the overall inference latency of multiple inference jobs compared to partition-only or schedule-only approaches.

Progressive Poisoned Data Isolation for Training-Time Backdoor Defense

Deep Neural Networks (DNN) are susceptible to backdoor attacks where malicious attackers manipulate the model's predictions via data poisoning. It is hence imperative to develop a strategy for training a clean model using a potentially poisoned dataset. Previous training-time defense mechanisms typically employ an one-time isolation process, often leading to suboptimal isolation outcomes. In this study, we present a novel and efficacious defense method, termed Progressive Isolation of Poisoned Data (PIPD), that progressively isolates poisoned data to enhance the isolation accuracy and mitigate the risk of benign samples being misclassified as poisoned ones. Once the poisoned portion of the dataset has been identified, we introduce a selective training process to train a clean model. Through the implementation of these techniques, we ensure that the trained model manifests a significantly diminished attack success rate against the poisoned data. Extensive experiments on multiple benchmark datasets and DNN models, assessed against nine state-of-the-art backdoor attacks, demonstrate the superior performance of our PIPD method for backdoor defense. For instance, our PIPD achieves an average True Positive Rate (TPR) of 99.95% and an average False Positive Rate (FPR) of 0.06% for diverse attacks over CIFAR-10 dataset, markedly surpassing the performance of state-of-the-art methods. The code is available at https://github.com/RorschachChen/PIPD.git.

Critical Path-Based Backdoor Detection for Deep Neural Networks.

Backdoor attack to deep neural networks (DNNs) is among the predominant approaches to bring great threats into artificial intelligence. The existing methods to detect backdoor attacks focus on the perspective of distributions in DNNs, however, limited by its ability of generalization across DNN models. In this article, a critical-path-based backdoor detector (CPBD) is proposed, which approaches to detect backdoor attacks via DNN's interpretability. CPBD is designed to efficiently discover the characteristics of backdoors, which distinguish the critical paths in the attacked DNNs. To deal with the intractably large number of neurons, we propose to simplify the neurons, and the preserved key nodes are integrated into a set of critical paths. Thus, a DNN model can be formulated as a combination of several critical paths. Afterward, the detection of backdoors is performed based on the analysis of critical paths corresponding to different classes. Then, combining all the above steps, the CPBD algorithm is integrated to present the results in a standard and systematic manner. In addition, CPBD is able to locate neurons associated with malicious triggers, the combination of which is named as trigger propagation path. Extensive experiments are conducted, which testify the efficiency of the proposed method on multiple DNNs and different trigger sizes.