Compact Neural Network Research Articles

Efficient Limb Range of Motion Analysis from a Monocular Camera for Edge Devices

Traditional limb kinematic analysis relies on manual goniometer measurements. With computer vision advancements, integrating RGB cameras can minimize manual labor. Although deep learning-based cameras aim to offer the same ease as manual goniometers, previous approaches have prioritized accuracy over efficiency and cost on PC-based devices. Nevertheless, healthcare providers require a high-performance, low-cost, camera-based tool for assessing upper and lower limb range of motion (ROM). To address this, we propose a lightweight, fast, deep learning model to estimate a human pose and utilize predicted joints for limb ROM measurement. Furthermore, the proposed model is optimized for deployment on resource-constrained edge devices, balancing accuracy and the benefits of edge computing like cost-effectiveness and localized data processing. Our model uses a compact neural network architecture with 8-bit quantized parameters for enhanced memory efficiency and reduced latency. Evaluated on various upper and lower limb tasks, it runs 4.1 times faster and is 15.5 times smaller than a state-of-the-art model, achieving satisfactory ROM measurement accuracy and agreement with a goniometer. We also conduct an experiment on a Raspberry Pi, illustrating that the method can maintain accuracy while reducing equipment and energy costs. This result indicates the potential for deployment on other edge devices and provides the flexibility to adapt to various hardware environments, depending on diverse needs and resources.

Developing an EEG-based model to predict awakening after cardiac arrest using partial processing with the BIS Engine.

Accurate prognostication in comatose survivors of cardiac arrest is a challenging and high-stakes endeavor. We sought to determine whether internal EEG subparameters extracted by the Bispectral Index (BIS) monitor, a device commonly used to estimate depth-of-anesthesia intraoperatively, could be repurposed to predict recovery of consciousness after cardiac arrest. In this retrospective cohort study, we trained a 3-layer neural network to predict recovery of consciousness to the point of command following versus not based on 48 hours of continuous EEG recordings in 315 comatose patients admitted to a single US academic medical center after cardiac arrest (Derivation cohort: N=181; Validation cohort: N=134). Continuous EEGs were partially processed into subparameters using virtualized emulation of the BIS Engine (i.e., the internal software of the BIS monitor) applied to signals from the frontotemporal leads of the standard 10-20 EEG montage. Our model was trained on hourly-averaged measurements of these internal subparameters. We compared this model's performance to the modified Westhall qualitative EEG scoring framework. Maximum prognostic accuracy in the Derivation Cohort was achieved using a network trained on only four BIS subparameters (inverse burst suppression ratio, mean spectral power density, gamma power, and theta/delta power). In a held-out sample of 134 patients, our model outperformed current state-of-the-art qualitative EEG assessment techniques at predicting recovery of consciousness (area under the receiver operating characteristic curve: 0.86, accuracy: 0.87, sensitivity: 0.83, specificity: 0.88, positive predictive value: 0.71, negative predictive value: 0.94). Gamma band power has not been previously reported as a correlate of recovery potential after cardiac arrest. In patients comatose after cardiac arrest, four EEG features calculated internally by the BIS Engine were repurposed by a compact neural network to achieve a prognostic accuracy superior to the current clinical qualitative gold-standard, with high sensitivity for recovery. These features hold promise for assessing patients after cardiac arrest.

A high performance heterogeneous hardware architecture for brain computer interface.

Brain-computer interface (BCI) has been widely used in human-computer interaction. The introduction of artificial intelligence has further improved the performance of BCI system. In recent years, the development of BCI has gradually shifted from personal computers to embedded devices, which boasts lower power consumption and smaller size, but at the cost of limited device resources and computing speed, thus can hardly improve the support of complex algorithms. This paper proposes a heterogeneous BCI architecture based on ARM + FPGA, enabling real-time processing of electroencephalogram (EEG) signals. Adopting data quantization, layer fusion and data augmentation to optimize the compact neural network model EEGNet, and design dedicated hardware engines to accelerate the network. Experimental results show that the system achieves 93.3% classification accuracy for steady-state visual evoked potential signals, with a time delay of 0.2ms per trail, and a power consumption of approximately (1.91W). That is 31.5 times faster acceleration is realized at the cost of only 0.7% lower accuracy compared with the conventional processor. The results show that the BCI architecture proposed in this study has strong practicability and high research significance.

Design of scatterometry with optoelectronic machine learning for discriminating nanohole cross-sectional structure.

This paper presents a system for discriminating the verticality of nanohole sidewalls on dielectric substrates. The proposed system comprises optical filters and a compact neural network with only two input ports. The weak scattered field from the nanohole passes through the filters, and the neural network processes the intensity of the focused field. Numerical simulations demonstrate that this system achieves significantly lower error rates compared to conventional systems that use an optical microscope and a neural network. Additionally, we discuss the minimum aperture size of nanoholes that can be effectively discriminated.

Improving I-ELM structure through optimal addition of hidden nodes: Compact I-ELM

Incremental extreme learning machines (I-ELMs) can automatically determine the structure of neural networks and achieve high learning speeds. However, during the process of adding hidden nodes, unnecessary hidden nodes that have little relevance to the target may be added. Several studies have proposed methods to overcome this problem by measuring the relevance between hidden nodes and outputs and adding or removing hidden nodes accordingly. Random hidden nodes have the advantage of creating diverse patterns, but they encounter a problem in which hidden nodes that generate patterns with little or no relevance to the target can be added, thereby increasing the number of hidden nodes. Unlike in existing I-ELMs, which use random hidden nodes, we propose a compact I-ELM algorithm that initially adds linear regression nodes and subsequently applies a method to ensure that the hidden nodes have patterns differing from the existing ones. Based on benchmark data, we confirmed that the proposed method constructs a compact neural network structure with fewer hidden nodes compared to the existing I-ELM systems.

Retinal Connectomics: A Review.

The retina is an ideal model for understanding the fundamental rules for how neural networks are constructed. The compact neural networks of the retina perform all of the initial processing of visual information before transmission to higher visual centers in the brain. The field of retinal connectomics uses high-resolution electron microscopy datasets to map the intricate organization of these networks and further our understanding of how these computations are performed by revealing the fundamental topologies and allowable networks behind retinal computations. In this article, we review some of the notable advances that retinal connectomics has provided in our understanding of the specific cells and the organization of their connectivities within the retina, as well as how these are shaped in development and break down in disease. Using these anatomical maps to inform modeling has been, and will continue to be, instrumental in understanding how the retina processes visual signals.

Recovering Mullins damage hyperelastic behaviour with physics augmented neural networks

The aim of this work is to develop a neural network for modelling incompressible hyperelastic behaviour with isotropic damage, the so-called Mullins effect. This is obtained through the use of feed-forward neural networks with special attention to the architecture of the network in order to fulfil several physical restrictions such as objectivity, polyconvexity, non-negativity, material symmetry and thermodynamic consistency. The result is a compact neural network with few parameters that is able to reconstruct the hyperelastic behaviour with Mullins-type damage. The network is trained with artificially generated plane stress data and even correctly captures the full 3D behaviour with much more complex loading conditions. The energy and stress responses are correctly captured, as well as the evolution of the damage. The resulting neural network can be seamlessly implemented in widely used simulation software. Implementation details are provided and all numerical examples are performed in Abaqus.

Towards Non-Region Specific Large-Scale Inundation Modelling with Machine Learning Methods

Traditional flood inundation modelling methods are computationally expensive and not suitable for near-real time inundation prediction. In this study we explore a data-driven machine learning method to complement and, in some cases, replace existing methods. Given sufficient training data and model capacity, our design enables a single neural network instance to approximate the flow characteristics of any input region, opening the possibility of applying the model to regions without available training data. To demonstrate the method we apply it to a very large >8000 km2 region of the Fitzroy river basin in Western Australia with a spatial resolution of 30 m × 30 m, placing an emphasis on efficiency and scalability. In this work we identify and address a range of practical limitations, e.g., we develop a novel water height regression method and cost function to address extreme class imbalances and by carefully constructing the input data, we introduce some natural physical constraints. Furthermore, a compact neural network design and training method was developed to enable the training problem to fit within GPU memory constraints and a novel dataset was constructed from the output of a calibrated two-dimensional hydrodynamic model. A good correlation between the predicted and groundtruth water heights was observed.

Neural SSS: Lightweight Object Appearance Representation

AbstractWe present a method for capturing the BSSRDF (bidirectional scattering‐surface reflectance distribution function) of arbitrary geometry with a neural network. We demonstrate how a compact neural network can represent the full 8‐dimensional light transport within an object including heterogeneous scattering. We develop an efficient rendering method using importance sampling that is able to render complex translucent objects under arbitrary lighting. Our method can also leverage the common planar half‐space assumption, which allows it to represent one BSSRDF model that can be used across a variety of geometries. Our results demonstrate that we can render heterogeneous translucent objects under arbitrary lighting and obtain results that match the reference rendered using volumetric path tracing.

Sharing massive biomedical data at magnitudes lower bandwidth using implicit neural function

Efficient storage and sharing of massive biomedical data would open up their wide accessibility to different institutions and disciplines. However, compressors tailored for natural photos/videos are rapidly limited for biomedical data, while emerging deep learning-based methods demand huge training data and are difficult to generalize. Here, we propose to conduct Biomedical data compRession with Implicit nEural Function (BRIEF) by representing the target data with compact neural networks, which are data specific and thus have no generalization issues. Benefiting from the strong representation capability of implicit neural function, BRIEF achieves 2[Formula: see text]3 orders of magnitude compression on diverse biomedical data at significantly higher fidelity than existing techniques. Besides, BRIEF is of consistent performance across the whole data volume, and supports customized spatially varying fidelity. BRIEF's multifold advantageous features also serve reliable downstream tasks at low bandwidth. Our approach will facilitate low-bandwidth data sharing and promote collaboration and progress in the biomedical field.

Visual Analytics in Explaining Neural Networks with Neuron Clustering

Deep learning (DL) models have achieved state-of-the-art performance in many domains. The interpretation of their working mechanisms and decision-making process is essential because of their complex structure and black-box nature, especially for sensitive domains such as healthcare. Visual analytics (VA) combined with DL methods have been widely used to discover data insights, but they often encounter visual clutter (VC) issues. This study presents a compact neural network (NN) view design to reduce the visual clutter in explaining the DL model components for domain experts and end users. We utilized clustering algorithms to group hidden neurons based on their activation similarities. This design supports the overall and detailed view of the neuron clusters. We used a tabular healthcare dataset as a case study. The design for clustered results reduced visual clutter among neuron representations by 54% and connections by 88.7% and helped to observe similar neuron activations learned during the training process.

Deep smoothness weighted essentially non-oscillatory method for two-dimensional hyperbolic conservation laws: A deep learning approach for learning smoothness indicators

In this work, we enhance the fifth-order Weighted Essentially Non-Oscillatory (WENO) shock-capturing scheme by integrating deep learning techniques. We improve the established WENO algorithm by training a compact neural network to dynamically adjust the smoothness indicators within the WENO scheme. This modification boosts the accuracy of the numerical results, particularly in proximity to abrupt shocks. Notably, our approach eliminates the need for additional post-processing steps, distinguishing it from previous deep learning-based methods. We substantiate the superiority of our new approach through the examination of multiple examples from the literature concerning the two-dimensional Euler equations of gas dynamics. Through a thorough investigation of these test problems, encompassing various shocks and rarefaction waves, our novel technique consistently outperforms the traditional fifth-order WENO scheme. This superiority is especially evident in cases where numerical solutions exhibit excessive diffusion or overshoot around shocks.

Monolithically Integrated Complementary Ferroelectric FET XNOR Synapse for the Binary Neural Network.

Neuromorphic computing, which mimics the structure and principles of the human brain, has the potential to facilitate the hardware implementation of next-generation artificial intelligence systems and process large amounts of data with very low power consumption. Among them, the XNOR synapse-based Binary Neural Network (BNN) has been attracting attention due to its compact neural network parameter size and low hardware cost. The previous XNOR synapse has drawbacks, such as a trade-off between cell density and accuracy. In this work, we show nonvolatile XNOR synapses with high density and accuracy using a monolithically stacked complementary ferroelectric field-effect transistor (C-FeFET) composed of a p-type Si MFMIS-FeFET at the bottom and a 3D stackable n-type Al:IZTO MFS-FeTFT, achieving 60F2 per cell (2C-FeFET). For adjusting the threshold voltage and improving the switching speed (100 ns) of n-type ferroelectric TFT, we employed a dual-gate configuration and a unique operation scheme, making it comparable to those of Si-based FeFETs. We performed array-level simulation with a 512 × 512 subarray size and a 3-bit flash ADC, demonstrating that the image recognition accuracies using the MNIST and CIFAR-10 data sets were increased by 3.17 and 14.07%, respectively, in comparison to other nonvolatile XNOR synapses. In addition, we performed system-level analysis on a 512 × 512 XNOR C-FeFET, exhibiting an outstanding throughput of 717.37 GOPS and an energy efficiency of 196.7 TOPS/W. We expect that our approach would contribute to the high-density memory systems, logic-in-memory technology, and hardware implementation of neural networks.

Compact Neural Network via Stacking Hybrid Units.

As an effective tool for network compression, pruning techniques have been widely used to reduce the large number of parameters in deep neural networks (NNs). Nevertheless, unstructured pruning has the limitation of dealing with the sparse and irregular weights. By contrast, structured pruning can help eliminate this drawback but it requires complex criteria to determine which components to be pruned. Therefore, this paper presents a new method termed BUnit-Net, which directly constructs compact NNs by stacking designed basic units, without requiring additional judgement criteria anymore. Given the basic units of various architectures, they are combined and stacked systematically to build up compact NNs which involve fewer weight parameters due to the independence among the units. In this way, BUnit-Net can achieve the same compression effect as unstructured pruning while the weight tensors can still remain regular and dense. We formulate BUnit-Net in diverse popular backbones in comparison with the state-of-the-art pruning methods on different benchmark datasets. Moreover, two new metrics are proposed to evaluate the trade-off of compression performance. Experiment results show that BUnit-Net can achieve comparable classification accuracy while saving around 80% FLOPs and 73% parameters. That is, stacking basic units provides a new promising way for network compression.

Learning Kernel-Modulated Neural Representation for Efficient Light Field Compression.

Light fields capture 3D scene information by recording light rays emitted from a scene at various orientations. They offer a more immersive perception, compared with classic 2D images, but at the cost of huge data volumes. In this paper, we design a compact neural network representation for the light field compression task. In the same vein as the deep image prior, the neural network takes randomly initialized noise as input and is trained in a supervised manner in order to best reconstruct the target light field Sub-Aperture Images (SAIs). The network is composed of two types of complementary kernels: descriptive kernels (descriptors) that store scene description information learned during training, and modulatory kernels (modulators) that control the rendering of different SAIs from the queried perspectives. To further enhance compactness of the network meanwhile retain high quality of the decoded light field, we propose modulator allocation and apply kernel tensor decomposition techniques, followed by non-uniform quantization and lossless entropy coding. Extensive experiments demonstrate that our method outperforms other state-of-the-art (SOTA) methods by a significant margin in the light field compression task. Moreover, after adapting descriptors, the modulators learned from one light field can be transferred to new light fields for rendering dense views, showing the potential of the solution for view synthesis.

Neural Global Illumination: Interactive Indirect Illumination Prediction under Dynamic Area Lights.

We propose neural global illumination, a novel method for fast rendering full global illumination in static scenes with dynamic viewpoint and area lighting. The key idea of our method is to utilize a deep rendering network to model the complex mapping from each shading point to global illumination. To efficiently learn the mapping, we propose a neural-network-friendly input representation including attributes of each shading point, viewpoint information, and a combinational lighting representation that enables high-quality fitting with a compact neural network. To synthesize high-frequency global illumination effects, we transform the low-dimension input to higher-dimension space by positional encoding and model the rendering network as a deep fully-connected network. Besides, we feed a screen-space neural buffer to our rendering network to share global information between objects in the screen-space to each shading point. We have demonstrated our neural global illumination method in rendering a wide variety of scenes exhibiting complex and all-frequency global illumination effects such as multiple-bounce glossy interreflection, color bleeding, and caustics.

A Lightweight Neural Network for Spectroscopic Ellipsometry Analysis

AbstractEllipsometry is a widely used technique in thin film characterizations. To extract the optical properties of the films from the measured data, regression data fitting techniques have been developed that iteratively find a set of optical parameters that best fit the observations. However, this iterative process often faces challenges in converging to the correct solution, and it can be time‐consuming. To address these issues, an 8‐bit quantized lightweight method of neural network analysis for spectroscopic ellipsometry are proposed. This method features compact neural network modules to enhance speed and efficiency. The effectiveness of the approach through experimental verification is validated on a diverse range of material films, including metals, semiconductors, and dielectrics. The approach is fully automatic and lightweight, which offers a new perspective on balancing predictive accuracy with limited computational resources. This method holds the potential to achieve automatic, rapid, and high‐throughput optical characterization of films, facilitating real‐time quality monitoring for repeatable high‐precision film manufacturing.

Neural Shading Fields for Efficient Facial Inverse Rendering

AbstractGiven a set of unstructured photographs of a subject under unknown lighting, 3D geometry reconstruction is relatively easy, but reflectance estimation remains a challenge. This is because it requires disentangling lighting from reflectance in the ambiguous observations. Solutions exist leveraging statistical, data‐driven priors to output plausible reflectance maps even in the under‐constrained single‐view, unknown lighting setting. We propose a very low‐cost inverse optimization method that does not rely on data‐driven priors, to obtain high‐quality diffuse and specular, albedo and normal maps in the setting of multi‐view unknown lighting. We introduce compact neural networks that learn the shading of a given scene by efficiently finding correlations in the appearance across the face. We jointly optimize the implicit global illumination of the scene in the networks with explicit diffuse and specular reflectance maps that can subsequently be used for physically‐based rendering. We analyze the veracity of results on ground truth data, and demonstrate that our reflectance maps maintain more detail and greater personal identity than state‐of‐the‐art deep learning and differentiable rendering methods.

HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networks

Low-rank compression is an important model compression strategy for obtaining compact neural network models. In general, because the rank values directly determine the model complexity and model accuracy, proper selection of layer-wise rank is very critical and desired. To date, though many low-rank compression approaches, either selecting the ranks in a manual or automatic way, have been proposed, they suffer from costly manual trials or unsatisfied compression performance. In addition, all of the existing works are not designed in a hardware-aware way, limiting the practical performance of the compressed models on real-world hardware platforms. To address these challenges, in this paper we propose HALOC, a hardware-aware automatic low-rank compression framework. By interpreting automatic rank selection from an architecture search perspective, we develop an end-to-end solution to determine the suitable layer-wise ranks in a differentiable and hardware-aware way. We further propose design principles and mitigation strategy to efficiently explore the rank space and reduce the potential interference problem. Experimental results on different datasets and hardware platforms demonstrate the effectiveness of our proposed approach. On CIFAR-10 dataset, HALOC enables 0.07% and 0.38% accuracy increase over the uncompressed ResNet-20 and VGG-16 models with 72.20% and 86.44% fewer FLOPs, respectively. On ImageNet dataset, HALOC achieves 0.9% higher top-1 accuracy than the original ResNet-18 model with 66.16% fewer FLOPs. HALOC also shows 0.66% higher top-1 accuracy increase than the state-of-the-art automatic low-rank compression solution with fewer computational and memory costs. In addition, HALOC demonstrates the practical speedups on different hardware platforms, verified by the measurement results on desktop GPU, embedded GPU and ASIC accelerator.

Principles for coding associative memories in a compact neural network.

A major goal in neuroscience is to elucidate the principles by which memories are stored in a neural network. Here, we have systematically studied how four types of associative memories (short- and long-term memories, each as positive and negative associations) are encoded within the compact neural network of Caenorhabditis elegans worms. Interestingly, sensory neurons were primarily involved in coding short-term, but not long-term, memories, and individual sensory neurons could be assigned to coding either the conditioned stimulus or the experience valence (or both). Moreover, when considering the collective activity of the sensory neurons, the specific training experiences could be decoded. Interneurons integrated the modulated sensory inputs and a simple linear combination model identified the experience-specific modulated communication routes. The widely distributed memory suggests that integrated network plasticity, rather than changes to individual neurons, underlies the fine behavioral plasticity. This comprehensive study reveals basic memory-coding principles and highlights the central roles of sensory neurons in memory formation.