Efficient Neural Network Architecture Research Articles

CoupNeRF: Property‐aware Neural Radiance Fields for Multi‐Material Coupled Scenario Reconstruction

AbstractNeural Radiance Fields (NeRFs) have achieved significant recognition for their proficiency in scene reconstruction and rendering by utilizing neural networks to depict intricate volumetric environments. Despite considerable research dedicated to reconstructing physical scenes, rare works succeed in challenging scenarios involving dynamic, multi‐material objects. To alleviate, we introduce CoupNeRF, an efficient neural network architecture that is aware of multiple material properties. This architecture combines physically grounded continuum mechanics with NeRF, facilitating the identification of motion systems across a wide range of physical coupling scenarios. We first reconstruct specific‐material of objects within 3D physical fields to learn material parameters. Then, we develop a method to model the neighbouring particles, enhancing the learning process specifically in regions where material transitions occur. The effectiveness of CoupNeRF is demonstrated through extensive experiments, showcasing its proficiency in accurately coupling and identifying the behavior of complex physical scenes that span multiple physics domains.

Enhancing capabilities of generative models through VAE-GAN integration: A review

Our review explores the integration of Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), which are pivotal in the realm of generative models. VAEs are renowned for their robust probabilistic foundations and capacity for complex data representation learning, while GANs are celebrated for generating high-fidelity images. Despite their strengths, both models have limitations: VAEs often produce less sharp outputs, and GANs face challenges with training stability. The hybrid VAE-GAN models harness the strengths of both architectures to overcome these limitations, enhancing output quality and diversity. We provide a comprehensive overview of VAEs and GANs technology developments, their integration strategies, and resultant performance improvements. Applications across various fields, such as artistic creation, medical imaging, e-commerce, and video gaming, highlight the transformative potential of these models. However, challenges in model robustness, ethical concerns, and computational demands persist, posing significant hurdles. Future research directions are poised to transform the VAE-GAN landscape significantly. Enhancing training stability remains a priority, with new approaches such as incorporating self-correcting mechanisms into GANs training being tested. Addressing ethical issues is also critical, as policymakers and technologists work together to develop standards that prevent misuse. Moreover, reducing computational costs is fundamental to democratizing access to these technologies. Projects such as the development of MobileNetV2 have made strides in creating more efficient neural network architectures that maintain performance while being less resource-intensive. Further, the exploration of VAE-GAN applications in fields like augmented reality and personalized medicine offers exciting opportunities for growth, as evidenced by recent pilot studies.

Automatic extraction of lightweight and efficient neural network architecture of heavy convolutional architectures to predict microsatellite instability from hematoxylin and eosin histology in gastric cancer

Automatic extraction of lightweight and efficient neural network architecture of heavy convolutional architectures to predict microsatellite instability from hematoxylin and eosin histology in gastric cancer

Hybrid synaptic structure for spiking neural network realization

Neural networks and neuromorphic computing represent fundamental paradigms as alternative approaches to Von-Neumann-based implementations, advancing in the applications of deep learning and machine vision. Nonetheless, conventional semiconductor circuits encounter challenges in achieving ultra-fast processing speed and low power consumption due to their dissipative properties. Conversely, single flux quantum circuits exhibit inherent spiking behavior, showcasing their characteristics as a promising candidate for spiking neural networks (SNNs). In this work, we present a compact hybrid synapse circuit to mimic the biological interconnect functionality, enabling the weighting operations for excitatory and inhibitory impulses. Additionally, the proposed structure facilitates input accumulation, which is performed before the activation function. In the experiments, our synaptic structure interfaces with a soma circuit fabricated using a commercial Nb process, underscoring its compatibility and supporting its potential for integration into efficient neural network architectures. The weight value on the synapse is configurable by utilizing cryo-CMOS circuits, providing adaptability to the inference networks. We’ve successfully designed, fabricated, and partially tested the JJ-Synapse within our cryocooler system, enabling high-speed inference implementation for SNNs.

Colon and lung cancer classification from multi-modal images using resilient and efficient neural network architectures

Automatic classification of colon and lung cancer images is crucial for early detection and accurate diagnostics. However, there is room for improvement to enhance accuracy, ensuring better diagnostic precision. This study introduces two novel dense architectures (D1 and D2) and emphasizes their effectiveness in classifying colon and lung cancer from diverse images. It also highlights their resilience, efficiency, and superior performance across multiple datasets. These architectures were tested on various types of datasets, including NCT-CRC-HE-100K (set of 100,000 non-overlapping image patches from hematoxylin and eosin (H&E) stained histological images of human colorectal cancer (CRC) and normal tissue), CRC-VAL-HE-7K (set of 7180 image patches from N = 50 patients with colorectal adenocarcinoma, no overlap with patients in NCT-CRC-HE-100K), LC25000 (Lung and Colon Cancer Histopathological Image), and IQ-OTHNCCD (Iraq-Oncology Teaching Hospital/National Center for Cancer Diseases), showcasing their effectiveness in classifying colon and lung cancers from histopathological and Computed Tomography (CT) scan images. This underscores the multi-modal image classification capability of the proposed models. Moreover, the study addresses imbalanced datasets, particularly in CRC-VAL-HE-7K and IQ-OTHNCCD, with a specific focus on model resilience and robustness. To assess overall performance, the study conducted experiments in different scenarios. The D1 model achieved an impressive 99.80 % accuracy on the NCT-CRC-HE-100K dataset, with a Jaccard Index (J) of 0.8371, a Matthew's Correlation Coefficient (MCC) of 0.9073, a Cohen's Kappa (Kp) of 0.9057, and a Critical Success Index (CSI) of 0.8213. When subjected to 10-fold cross-validation on LC25000, the D1 model averaged (avg) 99.96 % accuracy (avg J, MCC, Kp, and CSI of 0.9993, 0.9987, 0.9853, and 0.9990), surpassing recent reported performances. Furthermore, the ensemble of D1 and D2 reached 93 % accuracy (J, MCC, Kp, and CSI of 0.7556, 0.8839, 0.8796, and 0.7140) on the IQ-OTHNCCD dataset, exceeding recent benchmarks and aligning with other reported results. Efficiency evaluations were conducted in various scenarios. For instance, training on only 10 % of LC25000 resulted in high accuracy rates of 99.19 % (J, MCC, Kp, and CSI of 0.9840, 0.9898, 0.9898, and 0.9837) (D1) and 99.30 % (J, MCC, Kp, and CSI of 0.9863, 0.9913, 0.9913, and 0.9861) (D2). In NCT-CRC-HE-100K, D2 achieved an impressive 99.53 % accuracy (J, MCC, Kp, and CSI of 0.9906, 0.9946, 0.9946, and 0.9906) with training on only 30 % of the dataset and testing on the remaining 70 %. When tested on CRC-VAL-HE-7K, D1 and D2 achieved 95 % accuracy (J, MCC, Kp, and CSI of 0.8845, 0.9455, 0.9452, and 0.8745) and 96 % accuracy (J, MCC, Kp, and CSI of 0.8926, 0.9504, 0.9503, and 0.8798), respectively, outperforming previously reported results and aligning closely with others. Lastly, training D2 on just 10 % of NCT-CRC-HE-100K and testing on CRC-VAL-HE-7K resulted in significant outperformance of InceptionV3, Xception, and DenseNet201 benchmarks, achieving an accuracy rate of 82.98 % (J, MCC, Kp, and CSI of 0.7227, 0.8095, 0.8081, and 0.6671). Finally, using explainable AI algorithms such as Grad-CAM, Grad-CAM++, Score-CAM, and Faster Score-CAM, along with their emphasized versions, we visualized the features from the last layer of DenseNet201 for histopathological as well as CT-scan image samples. The proposed dense models, with their multi-modality, robustness, and efficiency in cancer image classification, hold the promise of significant advancements in medical diagnostics. They have the potential to revolutionize early cancer detection and improve healthcare accessibility worldwide.

Optimizing Appearance-Based Localization with Catadioptric Cameras: Small-Footprint Models for Real-Time Inference on Edge Devices.

This paper considers the task of appearance-based localization: visual place recognition from omnidirectional images obtained from catadioptric cameras. The focus is on designing an efficient neural network architecture that accurately and reliably recognizes indoor scenes on distorted images from a catadioptric camera, even in self-similar environments with few discernible features. As the target application is the global localization of a low-cost service mobile robot, the proposed solutions are optimized toward being small-footprint models that provide real-time inference on edge devices, such as Nvidia Jetson. We compare several design choices for the neural network-based architecture of the localization system and then demonstrate that the best results are achieved with embeddings (global descriptors) yielded by exploiting transfer learning and fine tuning on a limited number of catadioptric images. We test our solutions on two small-scale datasets collected using different catadioptric cameras in the same office building. Next, we compare the performance of our system to state-of-the-art visual place recognition systems on the publicly available COLD Freiburg and Saarbrücken datasets that contain images collected under different lighting conditions. Our system compares favourably to the competitors both in terms of the accuracy of place recognition and the inference time, providing a cost- and energy-efficient means of appearance-based localization for an indoor service robot.

Multi-Scale Hybrid Spectral Network for Feature Learning and Hyperspectral Image Classification

Hyperspectral image (HSI) classification is an important concern in remote sensing, but it is complex since few numbers of labelled training samples and the high-dimensional space with many spectral bands. Hence, it is essential to develop a more efficient neural network architecture to improve performance in the HSI classification task. Deep learning models are contemporary techniques for pixel-based hyperspectral image (HSI) classification. Deep feature extraction from both spatial and spectral channels has led to high classification accuracy. Meanwhile, the effectiveness of these spatial-spectral methods relies on the spatial dimension of every patch, and there is no feasible method to determine the best spatial dimension to take into consideration. It makes better sense to retrieve spatial properties through examination at different neighborhood scales in spatial dimensions. In this context, this paper presents a multi-scale hybrid spectral convolutional neural network (MS-HybSN) model that uses three distinct multi-scale spectral-spatial patches to pull out properties in spectral and spatial domains. The presented deep learning framework uses three patches of different sizes in spatial dimension to find these possible features. The process of Hybrid convolution operation (3D-2D) is done on each selected patch and is repeated throughout the image. To assess the effectiveness of the presented model, three benchmark datasets that are openly accessible (Pavia University, Indian Pines, and Salinas) and new Indian datasets (Ahmedabad-1 and Ahmedabad-2) are being used in experimental studies. Empirically, it has been demonstrated that the presented model succeeds over the remaining state-of-the-art approaches in terms of classification performance.

Integration of Fuzzy Ontologies and Neural Networks in the Detection of Time Series Anomalies

This paper explores an approach to solving the problem of detecting time series anomalies, taking into account the specifics of the subject area. We propose a method based on the integration of a neural network with long short-term memory (LSTM) and Fuzzy OWL (Fuzzy Web Ontology Language) ontology. A LSTM network is used for the mathematical search for anomalies in the first stage. The fuzzy ontology filters the detection results and draws an inference for decision making in the second stage. The ontology contains a formalized representation of objects in the subject area and inference rules that select only those anomaly values that correspond to this subject area. In the article, we propose the architecture of a software system that implements this approach. Computational experiments were carried out on free data of technical characteristics of drilling rigs. The experiments showed high efficiency, but not the maximum efficiency of the proposed approach. In the future, we plan to select a more efficient neural network architecture for mathematical anomaly detection. We also plan to develop an algorithm for automatically filling the rules of inference into the ontology when analyzing text sources.

Using Deep Learning with Bayesian–Gaussian Inspired Convolutional Neural Architectural Search for Cancer Recognition and Classification from Histopathological Image Frames

We propose a neural architectural search model which examines histopathological images to detect the presence of cancer in both lung and colon tissues. In recent times, deep artificial neural networks have made tremendous impacts in healthcare. However, obtaining an optimal artificial neural network model that could yield excellent performance during training, evaluation, and inferencing has been a bottleneck for researchers. Our method uses a Bayesian convolutional neural architectural search algorithm in collaboration with Gaussian processes to provide an efficient neural network architecture for efficient colon and lung cancer classification and recognition. The proposed model learns by using the Gaussian process to estimate the required optimal architectural values by choosing a set of model parameters through the exploitation of the expected improvement (EI) values, thereby minimizing the number of sampled trials and suggesting the best model architecture. Several experiments were conducted, and a landmark performance was obtained in both validation and test data through the evaluation of the proposed model on a dataset consisting of 25,000 images of five different classes with convergence and F1‐score matrices.

DeepGlow: An efficient neural network emulator of physical afterglow models for gamma-ray bursts and gravitational-wave events

Abstract Gamma-ray bursts (GRBs) and double neutron star merger gravitational-wave events are followed by afterglows that shine from X-rays to radio, and these broadband transients are generally interpreted using analytical models. Such models are relatively fast to execute, and thus easily allow estimates of the energy and geometry parameters of the blast wave, through many trial-and-error model calculations. One problem, however, is that such analytical models do not capture the underlying physical processes as well as more realistic relativistic numerical hydrodynamic (RHD) simulations do. Ideally, those simulations are used for parameter estimation instead, but their computational cost makes this intractable. To this end, we present DeepGlow, a highly efficient neural network architecture trained to emulate a computationally costly RHD-based model of GRB afterglows, to within a few percent accuracy. As a first scientific application, we compare both the emulator and a different analytical model calibrated to RHD simulations, to estimate the parameters of a broadband GRB afterglow. We find consistent results between these two models, and also give further evidence for a stellar wind progenitor environment around this GRB source. DeepGlow fuses simulations that are otherwise too complex to execute over all parameters, to real broadband data of current and future GRB afterglows.

Dynamic single-photon 3D imaging with a sparsity-based neural network.

Deep learning is emerging as an important tool for single-photon light detection and ranging (LiDAR) with high photon efficiency and image reconstruction quality. Nevertheless, the existing deep learning methods still suffer from high memory footprint and low inference speed, which undermine their compatibility when it comes to dynamic and long-range imaging with resource-constrained devices. By exploiting the sparsity of the data, we proposed an efficient neural network architecture which significantly reduces the storage and computation overhead by skipping the inactive sites with no photon counts. In contrast with the state-of-the-art deep learning methods, our method supports one-shot processing of data frames with high spatial resolution, and achieves over 90% acceleration in computation speed without sacrificing the reconstruction quality. In addition, the speed of our method is not sensitive to the detection distance. The experiment results on public real-world dataset and our home-built system have demonstrated the outstanding dynamic imaging capability of the algorithm, which is orders of magnitude faster than the competing methods and does not require any data pruning for hardware compatibility.

Flood video segmentation on remotely sensed UAV using improved Efficient Neural Network

Semantic segmentation can be used to analyze the video data taken by UAV in the flood monitoring system. An accurate analysis can help rescue teams to assess and mitigate flood disasters. This paper proposed an improved Efficient Neural Network architecture to segment the UAV video of flood disaster. The proposed method consists of atrous separable convolution as the encoder and depth-wise separable convolution as the decoder. The experimental results reveal that the proposed method outperforms Efficient Neural Networks’ other architecture and gives the highest frame per second.

Multiclass Classification of UML Diagrams from Images Using Deep Learning

Unified Modeling Language (UML) diagrams are a recognized standard modeling language for representing design of software systems. For academic research, large cases containing UML diagrams are needed. One of the challenges in collecting such datasets is automatically determining whether an image is a UML diagram or not and what type of UML diagram an image contains. In this work, we collected UML diagrams from open datasets and manually labeled them into 10 types of UML diagrams (i.e. class diagrams, activity diagrams, use case diagrams, sequence diagrams, communication diagrams, component diagrams, deployment diagrams, object diagrams, package diagrams, and state machine diagrams) and non-UML images. We evaluated the performance of seven popular neural network architectures using transfer learning on the dataset of 4706 images, including 700 class diagrams, 454 activity diagrams, 651 use case diagrams, 706 sequence diagrams, 204 communication diagrams, 208 component diagrams, 287 deployment diagrams, 207 object diagrams, 246 package diagrams, 323 state machine diagrams, and 720 non-UML images, respectively. We also proposed our neural network architecture for multiclass classification of UML diagrams. The experiment results show that Xception achieved the best performance amongst the algorithms we evaluated with a precision of 93.03%, a recall of 92.44%, and an F1-score of 92.73%. Moreover, it is possible to develop small and almost the same efficient neural network architectures, that our proposed architecture has the least parameters (around 2.4 millions) and spends the least time per image (0.0135[Formula: see text]s per image using graphics processing unit) for classifying UML diagrams with a precision of 91.25%, a recall of 90.34%, and an F1-score of 90.79%.

Big bang–big crunch-CNN: an optimized approach towards rice crop protection and disease detection

Convolutional neural networks (CNNs) are successfully used for solving image classification tasks. Manually designing efficient neural network architecture requires expertise in CNN and domain knowledge. In this paper, a novel approach is proposed for automatically evolving CNN architecture using big bang–big crunch (BB–BC) algorithm. The proposed approach can be used to automatically evolve CNN architecture and obtain tuned hyper-parameters for CNN. The CNN architecture is validated using the 5932 on-field rice leaf images infected with bacterial leaf blight, rice blast, brown leaf spot and tungro diseases. The proposed approach performed better than CNN evolved using genetic algorithm, SVM, KNN, decision tree, random forest with the test accuracy of 98.7%. The experimental results illustrate that the BB–BC CNN uses fewer trainable parameters compared to popular pre-trained CNN models like ResNet50, MobileNetV2, VGG16, VGG19, InceptionV3, etc.

Learning Hierarchical Representations with Spike-and-Slab Inception Network.

Recently, deep convolutional neural networks (CNN) with inception modules have attracted much attention due to their excellent performances on diverse domains. Nevertheless, the basic CNN can only capture a univariate feature, which is essentially linear. It leads to a weak ability in feature expression, further resulting in insufficient feature mining. In view of this issue, researchers incessantly deepened the network, bringing parameter redundancy and model over-fitting. Hence, whether we can employ this efficient deep neural network architecture to improve CNN and enhance the capacity of image recognition task still remains unknown. In this paper, we introduce spike-and-slab units to the modified inception module, enabling our model to capture dual latent variables and the average and covariance information. This operation further enhances the robustness of our model to variations of image intensity without increasing the model parameters. The results of several tasks demonstrated that dual variable operations can be well-integrated into inception modules, and excellent results have been achieved.

Comparison of Different Neural Network Architectures for Plasmonic Inverse Design.

The merge between nanophotonics and a deep neural network has shown unprecedented capability of efficient forward modeling and accurate inverse design if an appropriate network architecture and training method are selected. Commonly, an iterative neural network and a tandem neural network can both be used in the inverse design process, where the latter is well known for tackling the nonuniqueness problem at the expense of more complex architecture. However, we are curious to compare these two networks’ performance when they are both applicable. Here, we successfully trained both networks to inverse design the far-field spectrum of plasmonic nanoantenna, and the results provide some guidelines for choosing an appropriate, sufficiently accurate, and efficient neural network architecture.

Bioacoustic classification of avian calls from raw sound waveforms with an open-source deep learning architecture

The use of autonomous recordings of animal sounds to detect species is a popular conservation tool, constantly improving in fidelity as audio hardware and software evolves. Current classification algorithms utilise sound features extracted from the recording rather than the sound itself, with varying degrees of success. Neural networks that learn directly from the raw sound waveforms have been implemented in human speech recognition but the requirements of detailed labelled data have limited their use in bioacoustics. Here we test SincNet, an efficient neural network architecture that learns from the raw waveform using sinc-based filters. Results using an off-the-shelf implementation of SincNet on a publicly available bird sound dataset (NIPS4Bplus) show that the neural network rapidly converged reaching accuracies of over 65% with limited data. Their performance is comparable with traditional methods after hyperparameter tuning but they are more efficient. Learning directly from the raw waveform allows the algorithm to select automatically those elements of the sound that are best suited for the task, bypassing the onerous task of selecting feature extraction techniques and reducing possible biases. We use publicly released code and datasets to encourage others to replicate our results and to apply SincNet to their own datasets; and we review possible enhancements in the hope that algorithms that learn from the raw waveform will become useful bioacoustic tools.

Compute and Memory Efficient Universal Sound Source Separation

Recent progress in audio source separation lead by deep learning has enabled many neural network models to provide robust solutions to this fundamental estimation problem. In this study, we provide a family of efficient neural network architectures for general purpose audio source separation while focusing on multiple computational aspects that hinder the application of neural networks in real-world scenarios. The backbone structure of this convolutional network is the SUccessive DOwnsampling and Resampling of Multi-Resolution Features (SuDoRM-RF) as well as their aggregation which is performed through simple one-dimensional convolutions. This mechanism enables our models to obtain high fidelity signal separation in a wide variety of settings where variable number of sources are present and with limited computational resources (e.g. floating point operations, memory footprint, number of parameters and latency). Our experiments show that SuDoRM-RF models perform comparably and even surpass several state-of-the-art benchmarks with significantly higher computational resource requirements. The causal variation of SuDoRM-RF is able to obtain competitive performance in real-time speech separation of around 10dB scale-invariant signal-to-distortion ratio improvement (SI-SDRi) while remaining up to 20 times faster than real-time on a laptop device.

Non-fusion time-resolved depth image reconstruction using a highly efficient neural network architecture.

Single-photon avalanche diodes (SPAD) are powerful sensors for 3D light detection and ranging (LiDAR) in low light scenarios due to their single-photon sensitivity. However, accurately retrieving ranging information from noisy time-of-arrival (ToA) point clouds remains a challenge. This paper proposes a photon-efficient, non-fusion neural network architecture that can directly reconstruct high-fidelity depth images from ToA data without relying on other guiding images. Besides, the neural network architecture was compressed via a low-bit quantization scheme so that it is suitable to be implemented on embedded hardware platforms. The proposed quantized neural network architecture achieves superior reconstruction accuracy and fewer parameters than previously reported networks.

CovSegNet: A Multi Encoder-Decoder Architecture for Improved Lesion Segmentation of COVID-19 Chest CT Scans.

Automatic lung lesion segmentation of chest computer tomography (CT) scans is considered a pivotal stage toward accurate diagnosis and severity measurement of COVID-19. Traditional U-shaped encoder-decoder architecture and its variants suffer from diminutions of contextual information in pooling/upsampling operations with increased semantic gaps among encoded and decoded feature maps as well as instigate vanishing gradient problems for its sequential gradient propagation that result in suboptimal performance. Moreover, operating with 3-D CT volume poses further limitations due to the exponential increase of computational complexity making the optimization difficult. In this article, an automated COVID-19 lesion segmentation scheme is proposed utilizing a highly efficient neural network architecture, namely CovSegNet, to overcome these limitations. Additionally, a two-phase training scheme is introduced where a deeper 2-D network is employed for generating region-of-interest (ROI)-enhanced CT volume followed by a shallower 3-D network for further enhancement with more contextual information without increasing computational burden. Along with the traditional vertical expansion of Unet, we have introduced horizontal expansion with multistage encoder-decoder modules for achieving optimum performance. Additionally, multiscale feature maps are integrated into the scale transition process to overcome the loss of contextual information. Moreover, a multiscale fusion module is introduced with a pyramid fusion scheme to reduce the semantic gaps between subsequent encoder/decoder modules while facilitating the parallel optimization for efficient gradient propagation. Outstanding performances have been achieved in three publicly available datasets that largely outperform other state-of-the-art approaches. The proposed scheme can be easily extended for achieving optimum segmentation performances in a wide variety of applications. Impact Statement-With lower sensitivity (60-70%), elongated testing time, and a dire shortage of testing kits, traditional RTPCR based COVID-19 diagnostic scheme heavily relies on postCT based manual inspection for further investigation. Hence, automating the process of infected lesions extraction from chestCT volumes will be major progress for faster accurate diagnosis of COVID-19. However, in challenging conditions with diffused, blurred, and varying shaped edges of COVID-19 lesions, conventional approaches fail to provide precise segmentation of lesions that can be deleterious for false estimation and loss of information. The proposed scheme incorporating an efficient neural network architecture (CovSegNet) overcomes the limitations of traditional approaches that provide significant improvement of performance (8.4% in averaged dice measurement scale) over two datasets. Therefore, this scheme can be an effective, economical tool for the physicians for faster infection analysis to greatly reduce the spread and massive death toll of this deadly virus through mass-screening.