Deep Network Architecture Research Articles

Physics-Informed Geometry-Aware Neural Operator

Engineering design problems often involve solving parametric Partial Differential Equations (PDEs) under variable PDE parameters and domain geometry. Recently, neural operators have shown promise in learning PDE operators and quickly predicting the PDE solutions. However, training these neural operators typically requires large datasets, the acquisition of which can be prohibitively expensive. To overcome this, physics-informed training offers an alternative way of building neural operators, eliminating the high computational costs associated with Finite Element generation of training data. Nevertheless, current physics-informed neural operators struggle with limitations, either in handling varying domain geometries or varying PDE parameters. In this research, we introduce a novel method, the Physics-Informed Geometry-Aware Neural Operator (PI-GANO), designed to simultaneously generalize across both PDE parameters and domain geometries. We adopt a geometry encoder to capture the domain geometry features, and design a novel pipeline to integrate this component within the existing Deep Compositional Operator Network architecture. Numerical results demonstrate the accuracy and efficiency of the proposed method. All the codes and data related to this work are available on GitHub: https://github.com/uq-group/codes/tree/main/PI-GANO.

Adaptive Kernel Convolutional Stereo Matching Recurrent Network.

For binocular stereo matching techniques, the most advanced method currently is using an iterative structure based on GRUs. Methods in this class have shown high performance on both high-resolution images and standard benchmarks. However, simply replacing cost aggregation with a GRU iterative method leads to the original cost volume for disparity calculation lacking non-local geometric and contextual information. Based on this, this paper proposes a new GRU iteration-based adaptive kernel convolution deep recurrent network architecture for stereo matching. This paper proposes a kernel convolution-based adaptive multi-scale pyramid pooling (KAP) module that fully considers the spatial correlation between pixels and adds new matching attention (MAR) to refine the matching cost volume before inputting it into the iterative network for iterative updates, enhancing the pixel-level representation ability of the image and improving the overall generalization ability of the network. At present, the AKC-Stereo network proposed in this paper has a higher improvement than the basic network. On the Sceneflow dataset, the EPE of AKC-Stereo reaches 0.45, which is 0.02 higher than the basic network. On the KITTI 2015 dataset, the AKC-Stereo network outperforms the base network by 5.6% on the D1-all metric.

Deep Reinforcement Learning for Dynamic Task Scheduling in Edge-Cloud Environments

With The advent of the Internet of Things (IoT) and its use cases there is a necessity for improved latency which has led to edgecomputing technologies. IoT applications need a cloud environment and appropriate scheduling based on the underlying requirements of a given workload. Due to the mobility nature of IoT devices and resource constraints and resource heterogeneity, IoT application tasks need more efficient scheduling which is a challenging problem. The existing conventional and deep learning scheduling techniques have limitations such as lack of adaptability, issues with synchronous nature and inability to deal with temporal patterns in the workloads. To address these issues, we proposed a learning-based framework known as the Deep Reinforcement Learning Framework (DRLF). This is designed in such a way that it exploits Deep Reinforcement Learning (DRL) with underlying mechanisms and enhanced deep network architecture based on Recurrent Neural Network (RNN). We also proposed an algorithm named Reinforcement Learning Dynamic Scheduling (RLbDS) which exploits different hyperparameters and DRL-based decision-making for efficient scheduling. Real-time traces of edge-cloud infrastructure are used for empirical study. We implemented our framework by defining new classes for CloudSim and iFogSim simulation frameworks. Our empirical study has revealed that RLbDS out performs many existing scheduling methods.

Adaptive Multi-Function Radar Temporal Behavior Analysis

The performance of radar mode recognition has been significantly enhanced by the various architectures of deep learning networks. However, these approaches often rely on supervised learning and are susceptible to overfitting on the same dataset. As a transitional phase towards Cognitive Multi-Functional Radar (CMFR), Adaptive Multi-Function Radar (AMFR) possesses the capability to emit identical waveform signals across different working modes and states for task completion, with dynamically adjustable waveform parameters that adapt based on scene information. From a reconnaissance perspective, the valid signals received exhibit sparsity and localization in the time series. To address this challenge, we have redefined the reconnaissance-focused research priorities for radar systems to emphasize behavior analysis instead of pattern recognition. Based on our initial comprehensive digital system simulation model of a radar, we conducted reconnaissance and analysis from the perspective of the reconnaissance side, integrating both radar and reconnaissance aspects into environmental simulations to analyze radar behavior under realistic scenarios. Within the system, waveform parameters on the radar side vary according to unified rules, while resource management and task scheduling switch based on operational mechanisms. The target in the reconnaissance side maneuvers following authentic behavioral patterns while adjusting the electromagnetic space complexity in the environmental aspect as required. The simulation results indicate that temporal annotations in signal flow data play a crucial role in behavioral analysis from a reconnaissance perspective. This provides valuable insights for future radar behavior analysis incorporating temporal correlations and sequential dependencies.

CGLight: An effective indoor illumination estimation method based on improved convmixer and GauGAN

Illumination consistency is a key factor for seamlessly integrating virtual objects with real scenes in augmented reality (AR) systems. High dynamic range (HDR) panoramic images are widely used to estimate scene lighting accurately. However, generating environment maps requires complex deep network architectures, which cannot operate on devices with limited memory space. To address this issue, this paper proposes CGLight, an effective illumination estimation method that predicts HDR panoramic environment maps from a single limited field-of-view (LFOV) image. We first design a CMAtten encoder to extract features from input images, which learns the spherical harmonic (SH) lighting representation with fewer model parameters. Guided by the lighting parameters, we train a generative adversarial network (GAN) to generate HDR environment maps. In addition, to enrich lighting details and reduce training time, we specifically introduce the color consistency loss and independent discriminator, considering the impact of color properties on the lighting estimation task while improving computational efficiency. Furthermore, the effectiveness of CGLight is verified by relighting virtual objects using the predicted environment maps, and the root mean square error and angular error are 0.0494 and 4.0607 in the gray diffuse sphere, respectively. Extensive experiments and analyses demonstrate that CGLight achieves a balance between indoor illumination estimation accuracy and resource efficiency, attaining higher accuracy with nearly 4 times fewer model parameters than the ViT-B16 model.

Hierarchical Graph Convolutional Network Built by Multiscale Atlases for Brain Disorder Diagnosis Using Functional Connectivity.

Functional connectivity network (FCN) data from functional magnetic resonance imaging (fMRI) is increasingly used for the diagnosis of brain disorders. However, state-of-the-art studies used to build the FCN using a single brain parcellation atlas at a certain spatial scale, which largely neglected functional interactions across different spatial scales in hierarchical manners. In this study, we propose a novel framework to perform multiscale FCN analysis for brain disorder diagnosis. We first use a set of well-defined multiscale atlases to compute multiscale FCNs. Then, we utilize biologically meaningful brain hierarchical relationships among the regions in multiscale atlases to perform nodal pooling across multiple spatial scales, namely "Atlas-guided Pooling (AP)." Accordingly, we propose a multiscale-atlases-based hierarchical graph convolutional network (MAHGCN), built on the stacked layers of graph convolution and the AP, for a comprehensive extraction of diagnostic information from multiscale FCNs. Experiments on neuroimaging data from 1792 subjects demonstrate the effectiveness of our proposed method in the diagnoses of Alzheimer's disease (AD), the prodromal stage of AD [i.e., mild cognitive impairment (MCI)], as well as autism spectrum disorder (ASD), with the accuracy of 88.9%, 78.6%, and 72.7%, respectively. All results show significant advantages of our proposed method over other competing methods. This study not only demonstrates the feasibility of brain disorder diagnosis using resting-state fMRI empowered by deep learning but also highlights that the functional interactions in the multiscale brain hierarchy are worth being explored and integrated into deep learning network architectures for a better understanding of the neuropathology of brain disorders. The codes for MAHGCN are publicly available at "https://github.com/MianxinLiu/MAHGCN-code."

Hybrid Feature Extraction for Breast Cancer Classification Using the Ensemble Residual VGG16 Deep Learning Model

Introduction: Breast Cancer (BC) is a significant cause of high mortality amongst women globally and probably will remain a disease posing challenges about its detectability. Advancements in medical imaging technology have improved the accuracy and efficiency of breast cancer classification. However, tumor features' complexity and imaging data variability still pose challenges. Method: This study proposes the Ensemble Residual-VGG-16 model as a novel combination of the Deep Residual Network (DRN) and VGG-16 architecture. This model is purposely engineered with maximal precision for the task of breast cancer diagnosis based on mammography images. We assessed its performance by accuracy, recall, precision, and the F1-Score. All these metrics indicated the high performance of this Residual-VGG-16 model. The diagnostic residual-VGG16 performed exceptionally well with an accuracy of 99.6%, precision of 99.4%, recall of 99.7%, F1 score of 98.6%, and Mean Intersection over Union (MIoU) of 99.8% with MIAS datasets. Result: Similarly, the INBreast dataset achieved an accuracy of 93.8%, a precision of 94.2%, a recall of 94.5%, and an F1-score of 93.4% Conclusion: The proposed model is a significant advancement in breast cancer diagnosis, with high accuracy and potential as an automated grading.

Kilometer-Scale Precipitation Forecasting Utilizing Convolutional Neural Networks: A Case Study of Jiangsu’s Coastal Regions

High-resolution precipitation forecasts play a pivotal role in formulating comprehensive disaster prevention and mitigation plans. As spatial resolution enhances, striking a balance between computation, storage, and simulation accuracy becomes imperative to ensure optimal cost-effectiveness. Convolutional neural networks (CNNs), a cornerstone of deep learning, are examined in this study for their downscaling capabilities in precipitation simulation. During a precipitation event on 23 June 2022, in Jiangsu Province, China, distinct rain belts emerged in both southern and northern Jiangsu, precisely captured by a numerical model (the Weather Research and Forecasting, WRF) with a 3 km spatial resolution. Specifically, precipitation was prevalent in northern Jiangsu from 00:00 to 11:00 Beijing Time (BJT), transitioning to southern Jiangsu from 12:00 to 23:00 BJT on the same day. Upon dynamic downscaling, the model reproduced precipitation in these periods with an average error of 12.35 mm at 3 km and 12.48 mm at 1 km spatial resolutions. Employing CNN technology for statistical downscaling to a 1 km spatial resolution, samples from the initial period were utilized for training, while those from the subsequent period served for validation. Following dynamic downscaling, CNNs with four, five, six, and seven layers exhibited average errors of 8.86 mm, 8.93 mm, 9.71 mm, and 9.70 mm, respectively, accompanied by correlation coefficients of 0.550, 0.570, 0.574, and 0.578, respectively. This analysis indicates that for this precipitation event, a shallower CNN depth yields a lower average error and correlation coefficient, whereas a deeper architecture enhances the correlation coefficient. By employing deep network architectures, CNNs are capable of capturing nonlinear patterns and subtle local features from complex meteorological data, thereby providing more accurate predictions during the downscaling process. Leveraging faster computation and reduced storage requirements, machine learning has demonstrated immense potential in high-resolution forecasting research. There is significant scope for advancing technologies that integrate numerical models with machine learning to achieve higher-resolution numerical forecasts.

Rapid-Learning Collaborative Pushing and Grasping via Deep Reinforcement Learning and Image Masking

When multiple objects are positioned close together or stacked, pre-grasp operations such as pushing objects can be used to create space for the grasp, thereby improving the grasping success rate. This study develops a model based on a deep Q-learning network architecture and introduces a fully convolutional network to accurately identify pixels in the workspace image that correspond to target locations for exploration. In addition, this study incorporates image masking to limit the exploration area of the robotic arm, ensuring that the agent consistently explores regions containing objects. This approach effectively addresses the sparse reward problem and improves the convergence rate of the model. Experimental results from both simulated and real-world environments show that the proposed method accelerates the learning of effective grasping strategies. When image masking is applied, the success rate in the grasping task reaches 80% after 600 iterations. The time required to reach 80% success rate is 25% shorter when image masking is used compared to when it is not used. The main finding of this study is the direct integration of image masking technique with a deep reinforcement learning (DRL) algorithm, which offers significant advancement in robotic arm control. Furthermore, this study shows that image masking technique can substantially reduce training time and improve the object grasping success rate. This innovation enables the robotic arm to better adapt to scenarios that conventional DRL methods cannot handle, thereby improving training efficiency and performance in complex and dynamic industrial applications.

Emotion Recognition Using EEG Signals through the Design of a Dry Electrode Based on the Combination of Type 2 Fuzzy Sets and Deep Convolutional Graph Networks.

Emotion is an intricate cognitive state that, when identified, can serve as a crucial component of the brain-computer interface. This study examines the identification of two categories of positive and negative emotions through the development and implementation of a dry electrode electroencephalogram (EEG). To achieve this objective, a dry EEG electrode is created using the silver-copper sintering technique, which is assessed through Scanning Electron Microscope (SEM) and Energy Dispersive X-ray Analysis (EDXA) evaluations. Subsequently, a database is generated utilizing the designated electrode, which is based on the musical stimulus. The collected data are fed into an improved deep network for automatic feature selection/extraction and classification. The deep network architecture is structured by combining type 2 fuzzy sets (FT2) and deep convolutional graph networks. The fabricated electrode demonstrated superior performance, efficiency, and affordability compared to other electrodes (both wet and dry) in this study. Furthermore, the dry EEG electrode was examined in noisy environments and demonstrated robust resistance across a diverse range of Signal-To-Noise ratios (SNRs). Furthermore, the proposed model achieved a classification accuracy of 99% for distinguishing between positive and negative emotions, an improvement of approximately 2% over previous studies. The manufactured dry EEG electrode is very economical and cost-effective in terms of manufacturing costs when compared to recent studies. The proposed deep network, combined with the fabricated dry EEG electrode, can be used in real-time applications for long-term recordings that do not require gel.

Oscillations in an artificial neural network convert competing inputs into a temporal code.

The field of computer vision has long drawn inspiration from neuroscientific studies of the human and non-human primate visual system. The development of convolutional neural networks (CNNs), for example, was informed by the properties of simple and complex cells in early visual cortex. However, the computational relevance of oscillatory dynamics experimentally observed in the visual system are typically not considered in artificial neural networks (ANNs). Computational models of neocortical dynamics, on the other hand, rarely take inspiration from computer vision. Here, we combine methods from computational neuroscience and machine learning to implement multiplexing in a simple ANN using oscillatory dynamics. We first trained the network to classify individually presented letters. Post-training, we added temporal dynamics to the hidden layer, introducing refraction in the hidden units as well as pulsed inhibition mimicking neuronal alpha oscillations. Without these dynamics, the trained network correctly classified individual letters but produced a mixed output when presented with two letters simultaneously, indicating a bottleneck problem. When introducing refraction and oscillatory inhibition, the output nodes corresponding to the two stimuli activate sequentially, ordered along the phase of the inhibitory oscillations. Our model implements the idea that inhibitory oscillations segregate competing inputs in time. The results of our simulations pave the way for applications in deeper network architectures and more complicated machine learning problems.

Enhanced deep potential model for fast and accurate molecular dynamics: application to the hydrated electron.

In molecular simulations, neural network force fields aim at achieving ab initio accuracy with reduced computational cost. This work introduces enhancements to the Deep Potential network architecture, integrating a message-passing framework and a new lightweight implementation with various improvements. Our model achieves accuracy on par with leading machine learning force fields and offers significant speed advantages, making it well-suited for large-scale, accuracy-sensitive systems. We also introduce a new iterative model for Wannier center prediction, allowing us to keep track of electron positions in simulations of general insulating systems. We apply our model to study the solvated electron in bulk water, an ostensibly simple system that is actually quite challenging to represent with neural networks. Our trained model is not only accurate, but can also transfer to larger systems. Our simulation confirms the cavity model, where the electron's localized state is observed to be stable. Through an extensive run, we accurately determine various structural and dynamical properties of the solvated electron.

MixQuantBio: Towards extreme face and periocular recognition model compression with mixed-precision quantization

Current periocular and face recognition approaches utilize computationally costly deep neural networks, achieving notable recognition accuracies. Deploying such solutions in applications with limited computational resources requires minimizing their computational demand while maintaining similar recognition accuracies. Model compression techniques like model quantization can be used to reduce the computational costs of deep models. This approach is widely studied and applied to different machine-learning tasks, however it is understudied and investigated for biometrics. We propose in this work to reduce the computational cost of face and periocular recognition models using fixed- and mixed-precision model quantization. Specifically, we first quantize the full-precision models to fixed 8 and 6 bits, reducing the required memory footprint by 5x while maintaining, to a very large degree, the recognition accuracies. However, our achieved results demonstrated that by quantizing the models to extremely low b bits, e.g., below 6 bits, the accuracies significantly dropped, which motivated our investigation on mixed-precision quantization. Hence, we propose to utilize an iterative mixed-precision quantization scheme. In each iteration, the least important parameters are selected based on their weight magnitude and quantized to low b-bit precision and the model is fine-tuned. This approach is repeated until all parameters are quantized to low b-bit precision, achieving extreme reduction in memory footprint, e.g., 16x times, without significant loss in the model accuracies. The effectiveness of mixed- and fixed-precision quantization for biometric recognition models is studied and proved for two modalities, face and periocular, using three different deep network architectures and using different b bit precision.

Fiduciary-Free Frame Alignment for Robust Time-Lapse Drift Correction Estimation in Multi-Sample Cell Microscopy.

When analyzing microscopic time-lapse observations, frame alignment is an essential task to visually understand the morphological and translation dynamics of cells and tissue. While in traditional single-sample microscopy, the region of interest (RoI) is fixed, multi-sample microscopy often uses a single microscope that scans multiple samples over a long period of time by laterally relocating the sample stage. Hence, the relocation of the optics induces a statistical RoI offset and can introduce jitter as well as drift, which results in a misaligned RoI for each sample's time-lapse observation (stage drift). We introduce a robust approach to automatically align all frames within a time-lapse observation and compensate for frame drift. In this study, we present a sub-pixel precise alignment approach based on recurrent all-pairs field transforms (RAFT); a deep network architecture for optical flow. We show that the RAFT model pre-trained on the Sintel dataset performed with near perfect precision for registration tasks on a set of ten contextually unrelated time-lapse observations containing 250 frames each. Our approach is robust for elastically undistorted and translation displaced (x,y) microscopic time-lapse observations and was tested on multiple samples with varying cell density, obtained using different devices. The approach only performed well for registration and not for tracking of the individual image components like cells and contaminants. We provide an open-source command-line application that corrects for stage drift and jitter.

Semantic attention guided low-light image enhancement with multi-scale perception

Low-light environments often lead to complex degradation of captured images. However, most deep learning-based image enhancement methods for low-light conditions only learn a single-channel mapping relationship between the input image in low-light conditions and the desired image in normal light without considering semantic priors. This may cause the network to deviate from the original color of the region. In addition, deep network architectures are not suitable for low-light image recovery due to low pixel values. To address these issues, we propose a novel network called SAGNet. It consists of two branches:the main branch extracts global enhancement features at the level of the original image, and the other branch introduces semantic information through region-based feature learning and learns local enhancement features for semantic regions with multi-level perception to maintain color consistency. The extracted features are merged with the global enhancement features for semantic consistency and visualization. We also propose an unsupervised loss function to improve the network’s adaptability to general scenes and reduce the effect of sparse datasets. Extensive experiments and ablation studies show that SAGNet maintains color accuracy better in all cases and keeps natural luminance consistency across the semantic range.

Multi-scale dynamics by adjusting the leaking rate to enhance the performance of deep echo state networks.

The deep echo state network (Deep-ESN) architecture, which comprises a multi-layered reservoir layer, exhibits superior performance compared to conventional echo state networks (ESNs) owing to the divergent layer-specific time-scale responses in the Deep-ESN. Although researchers have attempted to use experimental trial-and-error grid searches and Bayesian optimization methods to adjust the hyperparameters, suitable guidelines for setting hyperparameters to adjust the time scale of the dynamics in each layer from the perspective of dynamical characteristics have not been established. In this context, we hypothesized that evaluating the dependence of the multi-time-scale dynamical response on the leaking rate as a typical hyperparameter of the time scale in each neuron would help to achieve a guideline for optimizing the hyperparameters of the Deep-ESN. First, we set several leaking rates for each layer of the Deep-ESN and performed multi-scale entropy (MSCE) analysis to analyze the impact of the leaking rate on the dynamics in each layer. Second, we performed layer-by-layer cross-correlation analysis between adjacent layers to elucidate the structural mechanisms to enhance the performance. As a result, an optimum task-specific leaking rate value for producing layer-specific multi-time-scale responses and a queue structure with layer-to-layer signal transmission delays for retaining past applied input enhance the Deep-ESN prediction performance. These findings can help to establish ideal design guidelines for setting the hyperparameters of Deep-ESNs.

Extra-abdominal trocar and instrument detection for enhanced surgical workflow understanding.

Video-based intra-abdominal instrument tracking for laparoscopic surgeries is a common research area. However, the tracking can only be done with instruments that are actually visible in the laparoscopic image. By using extra-abdominal cameras to detect trocars and classify their occupancy state, additional information about the instrument location, whether an instrument is still in the abdomen or not, can be obtained. This can enhance laparoscopic workflow understanding and enrich already existing intra-abdominal solutions. A data set of four laparoscopic surgeries recorded with two time-synchronized extra-abdominal 2D cameras was generated. The preprocessed and annotated data were used to train a deep learning-based network architecture consisting of a trocar detection, a centroid tracker and a temporal model to provide the occupancy state of all trocars during the surgery. The trocar detection model achieves an F1 score of . The prediction of the occupancy state yields an F1 score of , providing a first step towards enhanced surgical workflow understanding. The current method shows promising results for the extra-abdominal tracking of trocars and their occupancy state. Future advancements include the enlargement of the data set and incorporation of intra-abdominal imaging to facilitate accurate assignment of instruments to trocars.

A Multi-Level Interpretable Sleep Stage Scoring System by Infusing Experts' Knowledge Into a Deep Network Architecture.

In recent years, deep learning has shown potential and efficiency in a wide area including computer vision, image and signal processing. Yet, translational challenges remain for user applications due to a lack of interpretability of algorithmic decisions and results. This black box problem is particularly problematic for high-risk applications such as medical-related decision-making. The current study goal was to design an interpretable deep learning system for time series classification of electroencephalogram (EEG) for sleep stage scoring as a step toward designing a transparent system. We have developed an interpretable deep neural network that includes a kernel-based layer guided by a set of principles used for sleep scoring by human experts in the visual analysis of polysomnographic records. A kernel-based convolutional layer was defined and used as the first layer of the system and made available for user interpretation. The trained system and its results were interpreted in four levels from microstructure of EEG signals, such as trained kernels and effect of each kernel on the detected stages, to macrostructures, such as transitions between stages. The proposed system demonstrated greater performance than prior studies and the system learned information consistent with expert knowledge.

An efficient novel dual deep network architecture for video forgery detection

The technique of video copy-move forgery (CMF) is commonly employed in various industries; digital videography is regularly used as the foundation for vital graphic evidence that may be modified using the aforementioned method. Recently in the past few decades, forgery in digital images is detected via machine intellect. The second issue includes continuous allocation of parallel frames having relevant backgrounds erroneously results in false implications, detected as CMF regions third include as the CMF is divided into inter-frame or intra-frame forgeries to detect video copy is not possible by most of the existing methods. Thus, this research presents the dual deep network (DDN) for efficient and effective video copy-move forgery detection (VCMFD); DDN comprises two networks; the first detection network (DetNet1) extracts the general deep features and second detection network (DetNet2) extracts the custom deep features; both the network are interconnected as the output of DetNet1 is given to DetNet2. Furthermore, a novel algorithm is introduced for forged frame detection and optimization of the falsely detected frame. DDN is evaluated considering the two benchmark datasets REWIND and video tampering dataset (VTD) considering different metrics; furthermore, evaluation is carried through comparing the recent existing model. DDN outperforms the existing model in terms of various metrics.

Empowering deep neural quantum states through efficient optimization

Computing the ground state of interacting quantum matter is a long-standing challenge, especially for complex two-dimensional systems. Recent developments have highlighted the potential of neural quantum states to solve the quantum many-body problem by encoding the many-body wavefunction into artificial neural networks. However, this method has faced the critical limitation that existing optimization algorithms are not suitable for training modern large-scale deep network architectures. Here, we introduce a minimum-step stochastic-reconfiguration optimization algorithm, which allows us to train deep neural quantum states with up to 106 parameters. We demonstrate our method for paradigmatic frustrated spin-1/2 models on square and triangular lattices, for which our trained deep networks approach machine precision and yield improved variational energies compared to existing results. Equipped with our optimization algorithm, we find numerical evidence for gapless quantum-spin-liquid phases in the considered models, an open question to date. We present a method that captures the emergent complexity in quantum many-body problems through the expressive power of large-scale artificial neural networks.