Accuracy Degradation Research Articles

Performance Benchmarking of Different Methods to Solve Gauss-Newton Trust Region Subproblems

Summary The Gauss-Newton (GN) trust region optimization methods perform robustly but may introduce significant overhead cost when using the conventional matrix factorization method to solve the associated GN trust region subproblem (GNTRS). Solving a GNTRS involves solving a nonlinear equation using an iterative Newton-Raphson (NR) method. In each NR iteration, a symmetric linear system can be solved by different matrix factorization methods, including Cholesky decomposition (CD), eigenvalue decomposition (EVD), and singular value decomposition (SVD). Because CD fails to factorize a singular symmetric matrix, we propose solving a GNTRS using the robust EVD method. In this paper, we analyze the performances of different methods to solve a GNTRS using different matrix factorization subroutines in LAPACK with different options and settings. The cost of solving a GNTRS mainly depends on the number of observed data (m) and the number of uncertainty parameters (n). When n≤m, we recommend directly solving the original GNTRS with n variables. When n>m, we propose an indirect method that transforms the original GNTRS with n variables to a new problem with m unknowns. The proposed indirect method can significantly reduce the computational cost by dimension reduction. However, dimension reduction may introduce numerical errors, which, in turn, may result in accuracy degradation and cause failure of convergence using the popular iterative NR method. To further improve the overall performance, we introduce a numerical error indicator to terminate the iterative NR process when numerical errors become dominant. Finally, we benchmarked the performances of different approaches on a set of testing problems with different settings. Our results confirm that the GNTRS solver using the EVD method together with the modified NR method performs the best, being both robust (no failure for all testing problems) and efficient (consuming comparable CPU time to other methods).

Study on the interaction between shaking table and eccentric load

The control-structure interaction (CSI) between shaking table and eccentric load is one of the most important reasons causing the accuracy degradation of shaking table test. At present, the eccentric ratio (ER) of load and the coupling between actuators pose challenges to study the CSI. Thus, this paper establishes an analytical transfer function matrix of a shaking table and eccentric load. Based on the transfer function matrix, a comprehensive study is conducted to analyse the CSI effect under different eccentric ratio conditions. The analysis proves the influence of the CSI, and the CSI amplifies the actuator coupling more than 22 times at 20.00 Hz. Furthermore, a real-time CSI compensation strategy considering the actuator coupling is proposed. With the adopting of the proposed strategy, the coupling between two actuators is fully eliminated, and the correlation coefficients of ground motion records of two actuators are improved by 14.75% and 5.48%, respectively.

Rapid and high-accuracy forming of ceramic parts by DLP technology based on optimization of shear stress

Stereolithography technology shows great potential for manufacturing ceramic parts with complex structures. However, it seems difficult to achieve both high forming accuracy and speed. Excess cured areas caused by light scattering is one of the main reasons for the degradation of accuracy. This paper presents a special strategy to remove the excess cured areas by utilizing the shear stress that is generated during the forming process. By adjusting the feeding speed, feeding height, feeding temperature and their combinations, the shear stress can be controlled to be beneficial. The relative forming error greatly decreased by 51.66% for small features. As the feeding speed was increased to 3 or 4 times, the forming accuracy of the structures on different surfaces was increased to 9 times or 2.8 times. This work provides a simple, general and low-cost method for rapid and high-accuracy manufacturing of ceramic parts, especially for those with small structural features.

Overcomplete graph convolutional denoising autoencoder for noisy skeleton action recognition

AbstractCurrent skeleton‐based action recognition methods usually assume the input skeleton is complete and noise‐free. However, it is inevitable that the captured skeletons are incomplete due to occlusions or noisy due to changes in the environment. When dealing with these data, even State Of The Art (SOTA) recognition backbones experience significant degradation in recognition accuracy. Though a few methods have been proposed to address this issue, they still lack flexibility, efficiency and interpretability. In this work, an overcomplete Graph Convolutional Denoising Autoencoder (GCDAE) is proposed which can act as a flexible preprocessing module for pretrained recognition backbones and improve their robustness. Taking advantages of the overcomplete and fully graph convolutional structure, GCDAE is able to rectify noisy joints while keeping information of unspoiled details efficiently. On two large scale skeleton datasets NTU RGB+D 60 and 120, the introducing of GCDAE brings significant robustness improvements to SOTA backbones towards different types of noises.

Capturing spatial–temporal correlations with Attention based Graph Convolutional Network for network traffic prediction

Network traffic prediction is essential and significant to network management and network security. Existing prediction methods cannot well capture the temporal–spatial correlations hidden in the network traffic and suffer from a prediction accuracy degradation for two reasons. First, the common recurrent neural network models which are leveraged to learn the temporal relations in network traffic exhibit a poor performance in long-term prediction for its limited receptive field. Second, the existing methods for modeling the spatial relationship of network traffic focus on capturing the distance relationship between nodes in the network topology, which cannot reflect the implicit correlation between network nodes. To tackle these two problems, we propose an Attention-based Graph Convolutional Network model (AGCN) for capturing both the spatial and temporal correlations in network traffic. To catch the hidden spatial dependencies in network traffic, we combine graph attention network with graph convolutional network to mine the spatial relationships of network traffic. To efficiently learn the temporal long-term relations embedded in network traffic, we design a dilated convolution module to enable an exponentially growing receptive field for handling long sequences. Experimental results on three network traffic datasets show that AGCN has excellent performance in terms of prediction accuracy and inference time compared to current mainstream methods.

Recursive Probability Mass Function Method to Calculate Probability Distributions of Pulse-Shaped Signals

This paper proposes an accurate and efficient method to calculate probability distributions of pulse-shaped complex signals. We show that the distribution over the in-phase and quadrature-phase (I/Q) complex plane is obtained by a recursive probability mass function of the accumulator for a pulse-shaping filter. In contrast to existing analytical methods, the proposed method provides complex-plane distributions in addition to instantaneous power distributions. Since digital signal processing generally deals with complex amplitude rather than power, the complex-plane distributions are more useful when considering digital signal processing. In addition, our approach is free from the derivation of signal-dependent functions. This fact results in its easy application to arbitrary constellations and pulse-shaping filters like Monte Carlo simulations. Since the proposed method works without numerical integrals and calculations of transcendental functions, the accuracy degradation caused by floating-point arithmetic is inherently reduced. Even though our method is faster than Monte Carlo simulations, the obtained distributions are more accurate. These features of the proposed method realize a novel framework for evaluating the characteristics of pulse-shaped signals, leading to new modulation, predistortion and peak-to-average power ratio (PAPR) reduction schemes.

Diverse Sample Generation: Pushing the Limit of Generative Data-Free Quantization.

Generative data-free quantization emerges as a practical compression approach that quantizes deep neural networks to low bit-width without accessing the real data. This approach generates data utilizing batch normalization (BN) statistics of the full-precision networks to quantize the networks. However, it always faces the serious challenges of accuracy degradation in practice. We first give a theoretical analysis that the diversity of synthetic samples is crucial for the data-free quantization, while in existing approaches, the synthetic data completely constrained by BN statistics experimentally exhibit severe homogenization at distribution and sample levels. This paper presents a generic Diverse Sample Generation (DSG) scheme for the generative data-free quantization, to mitigate detrimental homogenization. We first slack the statistics alignment for features in the BN layer to relax the distribution constraint. Then, we strengthen the loss impact of the specific BN layers for different samples and inhibit the correlation among samples in the generation process, to diversify samples from the statistical and spatial perspectives, respectively. Comprehensive experiments show that for large-scale image classification tasks, our DSG can consistently quantization performance on different neural architectures, especially under ultra-low bit-width. And data diversification caused by our DSG brings a general gain to various quantization-aware training and post-training quantization approaches, demonstrating its generality and effectiveness.

PATCH

Recent advancements in deep learning have shown that multimodal inference can be particularly useful in tasks like autonomous driving, human health, and production line monitoring. However, deploying state-of-the-art multimodal models in distributed IoT systems poses unique challenges since the sensor data from low-cost edge devices can get corrupted, lost, or delayed before reaching the cloud. These problems are magnified in the presence of asymmetric data generation rates from different sensor modalities, wireless network dynamics, or unpredictable sensor behavior, leading to either increased latency or degradation in inference accuracy, which could affect the normal operation of the system with severe consequences like human injury or car accident. In this paper, we propose PATCH, a framework of speculative inference to adapt to these complex scenarios. PATCH serves as a plug-in module in the existing multimodal models, and it enables speculative inference of these off-the-shelf deep learning models. PATCH consists of 1) a Masked-AutoEncoder-based cross-modality imputation module to impute missing data using partially-available sensor data, 2) a lightweight feature pair ranking module that effectively limits the searching space for the optimal imputation configuration with low computation overhead, and 3) a data alignment module that aligns multimodal heterogeneous data streams without using accurate timestamp or external synchronization mechanisms. We implement PATCH in nine popular multimodal models using five public datasets and one self-collected dataset. The experimental results show that PATCH achieves up to 13% mean accuracy improvement over the state-of-art method while only using 10% of training data and reducing the training overhead by 73% compared to the original cost of retraining the model.

Real-Time Traffic Flow Statistics Based on Dual-Granularity Classification

Article Real-Time Traffic Flow Statistics Based on Dual-Granularity Classification Yanchao Bi, Yuyan Yin, Xinfeng Liu *, Xiushan Nie, Chenxi Zou, and Junbiao Du 1 School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101, China * Correspondence: liuxinfeng18@sdjzu.edu.cn Received: 17 June 2023 Accepted: 13 September 2023 Published: 26 September 2023 Abstract: Traffic detection devices can cause accuracy degradation over time. Considering problems such as time-consuming and laborious manual statistics, high misdetection probabilities, and model tracking failures, there is an urgent need to develop a deep learning model (which can stably achieve detection accuracy over 90%) to evaluate whether the device accuracy still satisfies the requirements or not. In this study, based on dual-granularity classification, a real-time traffic flow statistics method is proposed to address the above problems. The method is divided into two stages. The first stage uses YOLOv5 to acquire all the motorized and non-motorized vehicles appearing in the scene. The second stage uses EfficientNet to acquire the motorized vehicles obtained in the previous stage and classify such vehicles into six categories. Through this dual-granularity classification, the considered problem is simplified and the probability of false detection is reduced significantly. To correlate the front and back frames of the video, vehicle tracking is implemented using DeepSORT, and vehicle re-identification is implemented in conjunction with the ResNet50 model to improve the tracking accuracy. The experimental results show that the method used in this study solves the problems of misdetection and tracking effectively. Moreover, the proposed method achieves 98.7% real-time statistical accuracy by combining the two-lane counting method.

Information geometry based extreme low-bit neural network for point cloud

Deep learning has significantly advanced three-dimensional computer vision applied to point clouds. Nevertheless, the substantial consumption of time, storage, and energy substantially limits its deployment on edge devices with constrained resources. Extremely low bit quantization has received wide attention due to its extremely high compression ratio, but the problem of a significant drop in accuracy cannot be ignored. To alleviate the obvious accuracy degradation for extreme low-bit quantization, this paper proposes a novel compression framework for binary and ternary neural networks applied to point clouds, which introduces information geometry to model quantization to compensate the severe feature manifold distortion. It applies differential geometry on manifolds to study the implicit information of the point cloud feature data. Based on the theoretical analysis from the novel perspective of information geometry, two optimization modules are proposed to alleviate severe geometry distortions on differential manifolds. The first module, scaling recovery, provides layer-wise scaling parameters to reduce geometric distortion caused by quantization. The second module, Pooling Recovery, is specially designed to alleviate more severe pooling geometry distortions in point clouds. These two modules benefit both binary and ternary neural networks with ignored overheads. For ternary quantization, optimizations on convolution weights and gradients are additionally introduced. The proposed self-adaptive gradient estimation provides a more accurate approximation to the non-differential ternary staircase function. Convolution weight optimization is implemented on an information-geometry optimized model to achieve even higher accuracy and less memory consumption. Experimental results validate that the proposed models significantly outperform state-of-the-art methods and demonstrate better scalability. Overall, this compression framework has the potential to facilitate the deployment of deep learning models on edge devices with limited resources, opening up new opportunities for applications of three-dimensional computer vision.

SANA: Sensitivity-Aware Neural Architecture Adaptation for Uniform Quantization

Uniform quantization is widely taken as an efficient compression method in practical applications. Despite its merit of having a low computational overhead, uniform quantization fails to preserve sensitive components in neural networks when applied with ultra-low bit precision, which could lead to a non-trivial accuracy degradation. Previous works have applied mixed-precision quantization to address this problem. However, finding the correct bit settings for different layers always demands significant time and resource consumption. Moreover, mixed-precision quantization is not well supported on current general-purpose machines such as GPUs and CPUs and, thus, will cause intolerable overheads in deployment. To leverage the efficiency of uniform quantization while maintaining accuracy, in this paper, we propose sensitivity-aware network adaptation (SANA), which automatically modifies the model architecture based on sensitivity analysis to make it more compatible with uniform quantization. Furthermore, we formulated four different channel initialization strategies to accelerate the quantization-aware fine-tuning process of SANA. Our experimental results showed that SANA can outperform standard uniform quantization and other state-of-the-art quantization methods in terms of accuracy, with comparable or even smaller memory consumption. Notably, ResNet-50-SANA (24.4 MB) with W4A8 quantization achieved 77.8% top-one accuracy on ImageNet, which even surpassed the 77.6% of the full-precision ResNet-50 (97.8 MB) baseline.

Long-term stability of molecular doped epigraphene quantum Hall standards: single elements and large arrays (R K/236 ≈ 109 Ω)

In this work we investigate the long-term stability of epitaxial graphene (epigraphene) quantum Hall resistance standards, including single devices and an array device composed of 236 elements providing R K/236 ≈ 109 Ω, with R K the von Klitzing constant. All devices utilize the established technique of chemical doping via molecular dopants to achieve homogenous doping and control over carrier density. However, optimal storage conditions and the long-term stability of molecular dopants for metrological applications have not been widely studied. In this work we aim to identify simple storage techniques that use readily available and cost-effective materials which provide long-term stability for devices without the need for advanced laboratory equipment. The devices are stored in glass bottles with four different environments: ambient, oxygen absorber, silica gel desiccant, and oxygen absorber/desiccant mixture. We have tracked the carrier densities, mobilities, and quantization accuracies of eight different epigraphene quantum Hall chips for over two years. We observe the highest stability (i.e. lowest change in carrier density) for samples stored in oxygen absorber/desiccant mixture, with a relative change in carrier density below 0.01% per day and no discernable degradation of quantization accuracy at the part-per-billion level. This storage technique yields a comparable stability to the currently established best storage method of inert nitrogen atmosphere, but it is much easier to realize in practice. It is possible to further optimize the mixture of oxygen absorber/desiccant for even greater stability performance in the future. We foresee that this technique can allow for simple and stable long-term storage of polymer-encapsulated molecular doped epigraphene quantum Hall standards, removing another barrier for their wide-spread use in practical metrology.

Overflow-free Compute Memories for Edge AI Acceleration

Compute memories are memory arrays augmented with dedicated logic to support arithmetic. They support the efficient execution of data-centric computing patterns, such as those characterizing Artificial Intelligence (AI) algorithms. These architectures can provide computing capabilities as part of the memory array structures (In-Memory Computing, IMC) or at their immediate periphery (Near-Memory Computing, NMC). By bringing the processing elements inside (or very close to) storage, compute memories minimize the cost of data access. Moreover, highly parallel (and, hence, high-performance) computations are enabled by exploiting the regular structure of memory arrays. However, the regular layout of memory elements also constrains the data range of inputs and outputs, since the bitwidths of operands and results stored at each address cannot be freely varied. Addressing this challenge, we herein propose a HW/SW co-design methodology combining careful per-layer quantization and inter-layer scaling with lightweight hardware support for overflow-free computation of dot-vector operations. We demonstrate their use to implement the convolutional and fully connected layers of AI models. We embody our strategy in two implementations, based on IMC and NMC, respectively. Experimental results highlight that an area overhead of only 10.5% (for IMC) and 12.9% (for NMC) is required when interfacing with a 2KB subarray. Furthermore, inferences on benchmark CNNs show negligible accuracy degradation due to quantization for equivalent floating-point implementations.

Explaining Cross-Domain Recognition with Interpretable Deep Classifier

The recent advances in deep learning predominantly construct models in their internal representations, and it is opaque to explain the rationale behind and decisions to human users. Such explainability is especially essential for domain adaptation, whose challenges require developing more adaptive models across different domains. In this paper, we ask the question: how much each sample in source domain contributes to the network’s prediction on the samples from target domain. To address this, we devise a novel Interpretable Deep Classifier (IDC) that learns the nearest source samples of a target sample as evidence upon which the classifier makes the decision. Technically, IDC maintains a differentiable memory bank for each category and the memory slot derives a form of key-value pair. The key records the features of discriminative source samples and the value stores the corresponding properties, e.g., representative scores of the features for describing the category. IDC computes the loss between the output of IDC and the labels of source samples to back-propagate to adjust the representative scores and update the memory banks. Extensive experiments on Office-Home and VisDA-2017 datasets demonstrate that our IDC leads to a more explainable model with almost no accuracy degradation and effectively calibrates classification for optimum reject options. More remarkably, when taking IDC as a prior interpreter, capitalizing on 0.1% source training data selected by IDC still yields superior results than that uses full training set on VisDA-2017 for unsupervised domain adaptation.

Prediction of pulsating turbulent pipe flow by deep learning with generalization capability

The spatiotemporal development of turbulent pipe flows with various conditions of pulsation is predicted by deep learning. In the present model, convolutional autoencoder (CAE) extracts the spatial features of the input as latent space vectors and long short-term memory (LSTM) evolves them in time. Time-delay neural network (TDNN) is accompanied by LSTMs to treat general unsteady variations of the spatial mean pressure gradient of the pulsatile flows. Temporal evolution is processed in a sequence-to-sequence manner. Note that the pulsating turbulent pipe flows are statistically unsteady flows for which the spatiotemporal development has been seldom predicted by the deep learning technique. The flow field obtained by direct numerical simulation (DNS) was used to train the model. When the pulsating parameters for the training and prediction are the same, several model parameters are validated. To examine further generalization capability, the present model was trained by data from four pulsation conditions. The model was tested afterward by test data from twenty different pulsation conditions of which the range of parameters is either within or out of the range covered by the training data. The model successfully predicts the various pulsating flow fields with reasonable accuracy. The change in the period yields a larger influence on prediction accuracy than the change in the amplitude. The degradation of the prediction accuracy of the bulk velocity for the case with a long period and large amplitude is not caused by the latent space vectors themselves, but by the decoder which reconstructs the flow field from the latent space vectors.

Routing Optimization of Unmanned AVS for Localization of Wireless Emitters in Outside Locations

To control illegal radio waves, methods of locating unknown emitters are used. Location methods using ground sensors suffer from location accuracy degradation in environments where the distance between the emitter and the sensor is not line of sight (NLoS). Therefore, research is being done to improve location accuracy by using Unmanned Aerial Vehicles (UAVs) as sensors to ensure a line-of-sight (LoS) condition. However, UAVs can fly freely in the sky, making it difficult to optimize flight paths based on particle swarm optimization (PSO) for efficient and accurate localization. This article examines the optimization of UAV flight paths to achieve highly efficient and accurate external localization of unknown emitters through two approaches, a circular orbit and a free path, respectively. Our numerical results reveal the improved location estimation error performance of our proposed approach. In particular, when evaluating at the 90th percentile of the cumulative error distribution function (CDF), the proposed approach can achieve an error of 28.59 m with circular orbit and 12.91 m with free path orbit, compared to the case of the conventional fixed sensor. whose location estimation error is 55.02 m.

Privacy-Preserving Fleet-Wide Learning of Wind Turbine Conditions with Federated Learning

A wealth of data is constantly being collected by manufacturers from their wind turbine fleets. And yet, a lack of data access and sharing impedes exploiting the full potential of the data. Our study presents a privacy-preserving machine learning approach for fleet-wide learning of condition information without sharing any data locally stored on the wind turbines. We show that through federated fleet-wide learning, turbines with little or no representative training data can benefit from accuracy gains from improved normal behavior models. Customizing the global federated model to individual turbines yields the highest fault detection accuracy in cases where the monitored target variable is distributed heterogeneously across the fleet. We demonstrate this for bearing temperatures, a target variable whose normal behavior can vary widely depending on the turbine. We show that no member of the fleet is affected by a degradation in model accuracy by participating in the collaborative learning procedure, resulting in superior performance of the federated learning strategy in our case studies. Distributed learning increases the normal behavior model training times by about a factor of ten due to increased communication overhead and slower model convergence.

AOS: An Automated Overclocking System for High-Performance CNN Accelerator Through Timing Delay Measurement on FPGA

With the inherent algorithmic error resilience of conventional neural networks (CNN) and the worst-case design methodologies of current electronic design automation tools, overclocking-based timing speculation is a promising technique to improve the performance of CNN accelerators on FPGA by removing unnecessary timing margins. To avoid potential timing errors, timing delay measurement should be used during overclocking. However, current approaches are not yet good at measuring paths with more intense variability factors such as jitter, and lack an automated process for testing circuit delays. In this paper, we first propose 2-dimension multi-frame fusion to deal with the sampling jitter, then present a timing delay measurement-based automatic overclocking system (AOS) running on heterogeneous FPGA for high-performance CNN accelerators. On the FPGA side, AOS is composed of timing delay monitors (TDM) that can measure all types of timing paths, a TDM controller that converts the sampled values of TDMs into timing delay in terms of the ratio of path delay to the clock period. On the CPU side, AOS converts the path delay from clock period ratio to absolute delay value and decides the frequency of the accelerator in the next iteration. We demonstrate AOS with a SkyNet accelerator on the Xilinx ZCU104 board and achieve 657FPS at 436MHz without accuracy degradation, which is 1.41× performance compared to the baseline.

SCANet: Securing the Weights With Superparamagnetic-MTJ Crossbar Array Networks.

Deep neural networks (DNNs) form a critical infrastructure supporting various systems, spanning from the iPhone neural engine to imaging satellites and drones. The design of these neural cores is often proprietary or a military secret. Nevertheless, they remain vulnerable to model replication attacks that seek to reverse engineer the network's synaptic weights. In this article, we propose SCANet (Superparamagnetic-MTJ Crossbar Array Networks), a novel defense mechanism against such model stealing attacks by utilizing the innate stochasticity in superparamagnets. When used as the synapse in DNNs, superparamagnetic magnetic tunnel junctions (s-MTJs) are shown to be significantly more secure than prior memristor-based solutions. The thermally induced telegraphic switching in the s-MTJs is robust and uncontrollable, thus thwarting the attackers from obtaining sensitive data from the network. Using a mixture of both superparamagnetic and conventional MTJs in the neural network (NN), the designer can optimize the time period between the weight updation and the power consumed by the system. Furthermore, we propose a modified NN architecture that can prevent replication attacks while minimizing power consumption. We investigate the effect of the number of layers in the deep network and the number of neurons in each layer on the sharpness of accuracy degradation when the network is under attack. We also explore the efficacy of SCANet in real-time scenarios, using a case study on object detection.

Optimal Information Fusion-Based Strong Tracking Filter for State of Charge Estimation of Lithium-Ion Batteries

State of charge (SOC) plays a crucial role in battery management systems, which is of paramount importance in safety of lithium-ion batteries. However, incorrect charging/discharging, electromagnetic interference, electrochemical rebound characteristics of the battery, or battery faults can lead to sudden and unexpected variations in SOC, posing hazards on systems with lithium-ion batteries. To achieve rapid and accurate tracking of such variations, this paper proposes a robust strong tracking filter based on optimal information fusion, which can address the issue of estimation accuracy degradation caused by the over-adjustment of the fading factor in traditional strong tracking filters, while maintaining strong tracking capability for SOC variations. The effectiveness of the proposed method has been demonstrated by discharge experiments and dynamic stress testing.