Hard Thresholding Research Articles

Comparing penalization methods for linear models on large observational health data.

This study evaluates regularization variants in logistic regression (L1, L2, ElasticNet, Adaptive L1, Adaptive ElasticNet, Broken adaptive ridge [BAR], and Iterative hard thresholding [IHT]) for discrimination and calibration performance, focusing on both internal and external validation. We use data from 5 US claims and electronic health record databases and develop models for various outcomes in a major depressive disorder patient population. We externally validate all models in the other databases. We use a train-test split of 75%/25% and evaluate performance with discrimination and calibration. Statistical analysis for difference in performance uses Friedman's test and critical difference diagrams. Of the 840 models we develop, L1 and ElasticNet emerge as superior in both internal and external discrimination, with a notable AUC difference. BAR and IHT show the best internal calibration, without a clear external calibration leader. ElasticNet typically has larger model sizes than L1. Methods like IHT and BAR, while slightly less discriminative, significantly reduce model complexity. L1 and ElasticNet offer the best discriminative performance in logistic regression for healthcare predictions, maintaining robustness across validations. For simpler, more interpretable models, L0-based methods (IHT and BAR) are advantageous, providing greater parsimony and calibration with fewer features. This study aids in selecting suitable regularization techniques for healthcare prediction models, balancing performance, complexity, and interpretability.

Hybrid Thresholding for Image Deconvolution in Expectation Maximization framework

ABSTRACT In this study, we proposed an image deconvolution method in the expectation maximization (EM) framework. This method involves two steps: (i) E-step: utilizing the fast Fourier transform (FFT) for computationally efficient inversion of the convolution operator and (ii) M-step: employing the discrete wavelet transform (DWT) for estimating the original image from the image obtained in the E-step. In M-step, we proposed a modified L1-clipped penalty resulting in a hybrid thresholding scheme that integrates conventional hard and soft thresholds. This hybrid threshold ameliorates the inherent bias-variance trade-offs associated with traditional hard and soft thresholding schemes. The mathematical expressions for the risk, bias, and variance of the proposed hybrid threshold are derived and the performance is evaluated through simulation. Experimental results show that the proposed method achieves optimal values for variance and bias, thereby minimizing the risk. Moreover, the proposed method outperforms state-of-the-art methods in terms of performance metrics: PSNR, ISNR, and SSIM.

Distributed Spatially Non-Stationary Channel Estimation for Extremely-Large Antenna Systems

This paper aims to develop a distributed channel estimation (CE) algorithm for spatially non-stationary (SNS) channels in extremely large aperture array systems, addressing the issues of high communication cost and computational complexity associated with traditional centralized algorithms. However, SNS channels differ from conventional spatially stationary channels, presenting new challenges such as varying sparsity patterns for different antennas. To overcome these challenges, we propose a novel distributed CE algorithm accompanied by a simple yet effective hard thresholding scheme. The proposed algorithm is not only suitable for uniform antenna arrays but also for irregularly deployed antennas. Simulation results demonstrate the advantages of the proposed algorithm in terms of estimation accuracy, communication cost, and computational complexity.

M-band wavelet network for machine anomaly detection from a frequency perspective

The autoencoder (AE) is widely utilized in deep anomaly detection, but it lacks explainability due to the complexity of nonlinear mapping. One approach to address this issue is incorporating wavelet theory, which shares similarities in decomposition and reconstruction procedures. However, the perfect reconstruction property of wavelet theory conflicts with AE-based anomaly detection. To tackle this problem, we introduce a novel deep anomaly detection method from a frequency perspective. A learnable M-band wavelet network (MWNet) is designed to offer a flexible frequency band structure for signal representation. Subsequently, with the aid of sparsity constraint, MWNet dynamically focuses on key components within each frequency band. A learnable hard threshold function with a threshold maximization constraint is proposed to retain the essential frequency band of normal signals. After training, the MWNet is exclusively capable of well reconstructing normal signals, thereby producing a noticeable reconstruction error difference between normal and abnormal signals. Extensive experiments on both simulated and experimental datasets validate the effectiveness of the proposed method. The corresponding Python codes are available at https://github.com/albertszg/MbandWaveletNet.

Multiple Choice Hard Thresholding Pursuit (MCHTP) for simultaneous sparse recovery and sparsity order estimation

In this work, we propose a new greedy algorithm, referred to as the Multiple Choice Hard Thresholding Pursuit (MCHTP) which recovers a sparse vector from compressed linear measurements without explicit knowledge of its sparsity. MCHTP achieves this by suitably combining the steps of hard thresholding pursuit (HTP) with a novel sparsity selection criterion to iteratively select one among two possible sparsity values and subsequently recovering the unknown sparse vector along with its unknown sparsity order. We provide provable theoretical guarantees which ensure that MCHTP can estimate the sparsity order as well as the unknown sparse vector exactly from linear measurements. We also propose an extension of MCHTP, referred to as MCHTP−d, which can provide finer control on the estimated sparsity by tuning the parameter d. The simulation results corroborate the theoretical findings demonstrating the superior performance of MCHTP and MCHTP−d compared to state-of-the-art techniques in terms of sparse recovery and sparsity order estimation.

Towards robust and sparse linear discriminant analysis for image classification

Linear discriminant analysis (LDA) is a popular dimensionality reduction technique that has been widely used in pattern recognition. However, there exist a large number of redundant features and corrupted noise in real-world applications, which makes the performance of existing LDA methods degrade and thus leads to a decrease in classification accuracy. To address the above issues, we propose a novel robust and sparse LDA formulation dubbed RSLDA+. The key idea is introducing the mixed sparse regularization, i.e., ℓ0-norm plus ℓ2,0-norm, for feature representation and enforce ℓ0-norm for noise reduction. Furthermore, an optimization algorithm based on the alternating direction method of multipliers (ADMM) is developed in combination with hard thresholding operators. Extensive experiments on six common image datasets verify that the proposed RSLDA+ outperforms state-of-the-art LDA variants in classification accuracy. In addition, the ablation, robustness, convergence, stability, and sparsity are analyzed in detail. The results suggest that the proposed RSLDA+ provides an effective and robust method for image classification.

Research on Complementary Filtered Attitude Solution Method for Quadcopter Based on Double Filter Preprocessing

In the attitude calculation process of quadrotor, the traditional pre-filtering method has degraded the attitude solving accuracy due to incomplete denoising of accelerometer and gyroscope measurements. Therefore, a complementary filtered attitude solution method based on double filter pre-processing is proposed to address this problem. First, the signal decomposition, hard threshold denoising, and signal reconstruction of the accelerometer and gyroscope acquired data are performed using the Haar real-time wavelet filtering algorithm. The reconstructed signal is then filtered by the infinite impulse response ( IIR ) low-pass filtering algorithm to remove the residual high-frequency noise and complete the dual filtering preprocessing. Next, the gyroscope data is corrected with accelerometer data according to the complementary filtering algorithm. Finally, the corrected data are used to solve the quaternion and thus the attitude angle by combining the Longacurta method. The results show that the dual filtering preprocessing method can further reduce the noise in the measurements of accelerometer and gyroscope. The attitude angle results calculated by the proposed method in the static and dynamic hovering states of the four-rotor aircraft have a small degree of dispersion, which can effectively improve the accuracy of attitude calculation.

Broiler sound signal filtering method based on improved wavelet denoising and effective pulse extraction

There is little attention paid to signal filtering in existing broiler health monitoring or broiler sound signal classification research, and the only few studies still have issues such as lack of specific, in-depth, and specialized details. In response to these problems, the authors conducted a depth analysis of the signal characteristics of effective signal components and noises, and proposed a broiler sound signal filtering method based on improved wavelet denoising and effective pulse extraction. This method consists of two parts. The first part is wavelet denoising, which is used to remove the widely distributed noises in broiler sound signals. Specifically, a wavelet denoising algorithm based on continuous differentiable threshold function and scale threshold has been proposed. The disadvantages in existing hard threshold function, soft threshold function, and semi-soft threshold function have been effectively addressed, and the smoothness of broiler sound signals has been ensured. The second part is pulse extraction processing, which is used to remove sudden noises that appears as independent pulses in broiler sound signals. Specifically, a two-stage variable multi-threshold pulse extraction algorithm has been proposed. The main advantages of existing three-threshold pulse extraction algorithm and double-threshold pulse extraction algorithm have been referenced, the threshold setting standard has been clarified, the continuous multi-pulse problem has been given special attention, and the effective signal components in broiler sound signals have been accurately extracted. In this way, the noises in broiler sound signals were removed, and the effective signal components were highlighted. The signal-to-noise ratio (SNR) and root mean square error (RMSE) from signal detection field, as well as the classification accuracy and F1-score from machine learning field, were used to comprehensively evaluate the effect of the signal filtering method proposed in this paper. A large number of tests shown that, compared to existing signal filtering methods, the broiler sound signal processed by the signal filtering method proposed in this paper achieved the maximum SNR and the minimum RMSE. On feature data sets created from broiler sound signals processed by different signal filtering methods, three classifiers all achieved the highest classification accuracy and F1-score on the feature data set corresponding to the signal filtering method proposed in this paper. The feasibility, practicality, and superiority of the signal filtering method proposed in this paper have been widely verified. This study is an important supplement to the broiler health monitoring and broiler sound signal classification research, providing important references for signal filtering research in similar fields.

A high speed and sensitive protection for distribution network with DGs using ferrite rings as line boundary

AbstractThere exist problems of hard coordination and threshold setting as well as low sensitivity in overcurrent relay due to the bidirectional and small fault current when distributed generators are integrated into the distribution network. This paper introduces ferrite rings (FRs) to act as boundaries of distribution feeders, and a peak time of voltage traveling wave‐based protection is proposed. The characteristics of FRs are analyzed and their feasibility as line boundary is validated. The fault characteristics of internal and external faults are analyzed when FRs are installed at both ends of the protected feeder. The results show that the peak time of the voltage travelling wave of the internal fault is far smaller than that of the external faults. According to the difference, the fault detection method and the protection scheme are proposed. Simulations in PSCAD and experiment validate that the proposed protection principle can reliably distinguish between internal and external faults. Besides, the proposed protection scheme has higher sensitivity compared to ROCOV‐based protection.

DEFR-net: A decompose-enhance fourier residual network for fault diagnosis of rotating machine with high noise immunity

The actual operating environment of rotating mechanical device contains a large number of noisy interference sources, leading to complex components, strong coupling, and low signal to noise ratio for vibration. It becomes a big challenge for intelligent fault diagnosis from high-noise vibration signals. Thus, this paper proposes a new deep learning approach, namely decomposition-enhance Fourier residual network (DEFR-net), to achieve high noise immunity for vibration signal and learn effective features to discriminate between different types of rotational machine faults. In the proposed DEFR-net, a novel algorithm is proposed to explicitly model high-noise signals for noisy data filtering and effective feature enhancement based on a hard threshold decomposition function and muti-channel self-attention mechanism. Furthermore, it deeply integrates complementary analysis based on fast Fourier transform in the time-frequency domain and extends the breadth of network. The performance of the proposed model is verified by comparison with five state-of-the-art algorithms on two public datasets. Moreover, the noise experimental results show that the fault diagnosis accuracy is still 85.91% when the signal-to-noise-ratio reaches extreme noise of –8 dB. The results demonstrate that the proposed method is a valuable study for intelligent fault diagnosis of rotating machines in high-noise environments.

Reliability assessment models for competing failure processes with two types of correlative thresholds

AbstractThe two dependent competing risks are soft failure due to aging degradation and fragmentation caused by shocks and hard failure due to spring breakage caused by the same shock process. Considering the complexity of the product itself and the instability of the working environment, this study proposed a generalized reliability model for systems experiencing dependent competing failure processes (DCFPs) of degradation and random shocks, which considered two kinds of DCFP: (1) shock process could affect soft failure thresholds; (2) degradation process could affect hard failure thresholds. In case (1), we considered the effect that a cumulative number of shocks above a certain magnitude could have on the change in the soft failure threshold. In case (2), we considered not only the shock process's impact on the soft failure threshold but also the total degradation (including continuous degradation and sudden degradation caused by shock) on the hard failure threshold. The model captures the features that the shocks experienced by the system affect the degradation process, accelerating the system degradation and causing soft failures; the degradation process of the system affects the shock process, making the system more susceptible to failure from external shocks. Finally, an example using micro‐electro‐mechanical systems devices illustrates the effectiveness of the proposed approach with sensitivity analysis.

Phase retrieval from integrated intensity of auto-convolution

Ultra-fast optical pulses are the most ephemeral sensing paradigm ever devised, examining events over incredibly brief timescales with broadband illumination. A consequence of sensing at timescales lower than a picosecond is that pulse characterization cannot be done with traditional analog-to-digital samplers and must be ascertained from integrating intensity sensors. Techniques for pulse characterization have been constructed using combinations of time-invariant and time-variant filter responses to create non-linear but invertible intensity datasets (Walmsley & Dorrer, 2009). In this paper, we develop a novel high-order phase retrieval technique to perform pulse characterization from a single-pixel integrating sensor measuring integrated intensity of auto-convolution (IIAC). We examine gradient descent’s ability to recover signals as a function of signal dimension and measurement count, and we demonstrate the effective use of iterative hard tensor thresholding as an initializer. Finally, we demonstrate IIAC recovery in a laboratory setting to recover the time profile of a complex laser pulse. We assert that the IIAC recovery solution demonstrated here simultaneously provides the optics community with a pulse characterization technique that scales to low-power microscopy systems and provides the optimization community with a physically motivated high-order phase retrieval problem enhanced by low-rank tensor processing.

Thermo-oxidative aging state detection of rubber sandwich structure using synchrosqueezing transform-assisted feature extraction and customized detection indicator

Rubber sandwich structures (RSSs) are used extensively in mechanical engineering. The aging state detection of such structures is urgently required to avoid disastrous accidents. However, this is still a challenging task owing to the weakness of the rubber layer aging feature information contained in the vibration signal of the RSS and the lack of effective aging feature information extraction techniques. Thus, an aging state detection method for the RSS using synchrosqueezing transform (SST)-assisted feature extraction and a customized detection indicator was proposed in this study. First, the SST was used to decompose the vibration signal of the RSS, and a time-frequency (TF) spectrum with an enhanced aging state feature was obtained. Second, a TF bandpass filter was constructed and used to filter the information unrelated to the aging state feature from the TF spectrum. Subsequently, a hard threshold denoising method was applied to reduce noise in the filtered TF spectrum. Then, the aging state signal was reconstructed using the inverse SST. Finally, a customized detection indicator was constructed and its value was calculated to detect the aging state of the RSS. A thermo-oxidative aging dataset of the RSS from Xi’an Jiaotong University was used to validate the proposed method. The experimental results showed that the proposed method was more effective than other methods.

Cardinality-Constrained Multi-objective Optimization: Novel Optimality Conditions and Algorithms

In this paper, we consider multi-objective optimization problems with a sparsity constraint on the vector of variables. For this class of problems, inspired by the homonymous necessary optimality condition for sparse single-objective optimization, we define the concept of L-stationarity and we analyze its relationships with other existing conditions and Pareto optimality concepts. We then propose two novel algorithmic approaches: the first one is an iterative hard thresholding method aiming to find a single L-stationary solution, while the second one is a two-stage algorithm designed to construct an approximation of the whole Pareto front. Both methods are characterized by theoretical properties of convergence to points satisfying necessary conditions for Pareto optimality. Moreover, we report numerical results establishing the practical effectiveness of the proposed methodologies.

Sparse basis covariance matrix estimation for high dimensional compositional data via hard thresholding

Motivated by the high-dimensional compositional data appearing in many fields, this paper addresses the problem of large covariance estimation for compositional data. Firstly, we introduce the hard thresholding estimator to approximate the sparse basis covariance matrix which is relevant with compositional data. Then the upper error bounds are measured by the general matrix lv,w-norm and the general entrywise Lv,w-norm respectively with v,w∈[1,∞] in terms of probability. Finally, numerical simulations and real datasets application demonstrate that our estimator is close to the oracle estimator and outperforms the COAT estimator proposed by Cao et al. (2019).

Efficient and Fast Joint Sparse Constrained Canonical Correlation Analysis for Fault Detection.

The canonical correlation analysis (CCA) has attracted wide attention in fault detection (FD). To improve the detection performance, we propose a new joint sparse constrained CCA (JSCCCA) model that integrates the l2,0 -norm joint sparse constraints into classical CCA. The key idea is that JSCCCA can fully exploit the joint sparse structure to determine the number of extracted variables. We then develop an efficient alternating minimization algorithm using the improved iterative hard thresholding and manifold constrained gradient descent method. More importantly, we establish the convergence guarantee with detailed analysis. Finally, we provide extensive numerical studies on the simulated dataset, the benchmark Tennessee Eastman process, and a practical cylinder-piston process. In some cases, the computing time is reduced by 600 times, and the FD rate is increased by 12.62% compared with classical CCA. The results suggest that the proposed approach is efficient and fast.

A Systematic Evaluation of Recurrent Neural Network Models for Edge Intelligence and Human Activity Recognition Applications

The Recurrent Neural Networks (RNNs) are an essential class of supervised learning algorithms. Complex tasks like speech recognition, machine translation, sentiment classification, weather prediction, etc., are now performed by well-trained RNNs. Local or cloud-based GPU machines are used to train them. However, inference is now shifting to miniature, mobile, IoT devices and even micro-controllers. Due to their colossal memory and computing requirements, mapping RNNs directly onto resource-constrained platforms is arcane and challenging. The efficacy of edge-intelligent RNNs (EI-RNNs) must satisfy both performance and memory-fitting requirements at the same time without compromising one for the other. This study’s aim was to provide an empirical evaluation and optimization of historic as well as recent RNN architectures for high-performance and low-memory footprint goals. We focused on Human Activity Recognition (HAR) tasks based on wearable sensor data for embedded healthcare applications. We evaluated and optimized six different recurrent units, namely Vanilla RNNs, Long Short-Term Memory (LSTM) units, Gated Recurrent Units (GRUs), Fast Gated Recurrent Neural Networks (FGRNNs), Fast Recurrent Neural Networks (FRNNs), and Unitary Gated Recurrent Neural Networks (UGRNNs) on eight publicly available time-series HAR datasets. We used the hold-out and cross-validation protocols for training the RNNs. We used low-rank parameterization, iterative hard thresholding, and spare retraining compression for RNNs. We found that efficient training (i.e., dataset handling and preprocessing procedures, hyperparameter tuning, and so on, and suitable compression methods (like low-rank parameterization and iterative pruning) are critical in optimizing RNNs for performance and memory efficiency. We implemented the inference of the optimized models on Raspberry Pi.

Compressive Perception Image Reconstruction Technology for Basic Mixed Sparse Basis in Metal Surface Detection

<p>Applying Compressed Sensing (CS) technology to robot vision image transmission, an effective method for image reconstruction in robot imaging is proposed to improve the accuracy of reconstruction. Reconstructing images using a mixed sparse representation of DCT and circularly symmetric contour wave transform, the basic algorithm used is the Smoothed Projection Landweber (SPL) algorithm, which optimizes the coefficients under different sparse transformations by incorporating hard thresholding and binary thresholding methods for different sparse bases during iterations. The experiment shows that compared with single sparse base image reconstruction, the proposed reconstruction method has improved reconstruction accuracy.</p> <p> </p>

Sufficient condition based on nearly optimal order RIC for IHT algorithm

Iterative hard thresholding (IHT) is one class of sparse signal recovery algorithms widely used in compressed sensing. The well-known restricted isometry constant (RIC) is one of the most popular frameworks used to ensure the convergence of recovery algorithms. In previous literature, sufficient conditions to guarantee the recovery performance of the IHT algorithm have usually been established based on RICs with orders 3 s and 2 s , while no such condition in terms of RIC with order less than 2 s has been reported yet. In this study, based on the theoretical optimal step-length, we develop the RIC condition γ < ( 5 − 1 ) / 8 ≈ 0.1545 for successful recovery via the IHT algorithm. Here, γ = δ s if s is an even number, otherwise γ = δ s + 1 . To the best of our knowledge, this is the first result based on RIC with nearly optimal order for the IHT algorithm.

Breast ultrasound image despeckling using multi-filtering DFrFT and adaptive fast BM3D

Improving the quality of breast ultrasound images is of great significance for clinical diagnosis which can greatly boost the diagnostic accuracy of ultrasonography. However, due to the influence of ultrasound imaging principles and acquisition equipment, the collected ultrasound images naturally contain a large amount of speckle noise, which leads to a decrease in image quality and affects clinical diagnosis. To overcome this problem, we propose an improved denoising algorithm combining multi-filter DFrFT (Discrete Fractional Fourier Transform) and the adaptive fast BM3D (Block Matching and 3D collaborative filtering) method. Firstly, we provide the multi-filtering DFrFT method for preprocessing the original breast ultrasound image so as to remove some speckle noise early in fractional transformation domain. Based on the fractional frequency spectrum characteristics of breast ultrasound images, three types of filters are designed correspondingly in low, medium, and high frequency domains. And by integrating filtered images, the enhanced images are obtained which not only remove some speckle noise in background but also preserve the details of breast lesions. Secondly, for further enhancing the image quality on the basis of multi-filter DFrFT, we propose the adaptive fast BM3D method by introducing the DBSCAN-based super pixel segmentation to block matching process, which utilizes super pixel segmentation labels to provide a reference on how similar it is between target block and retrieval blocks. It reduces the number of blocks to be retrieved and make the matched blocks with more similar features. At last, the local noise parameter estimation is also adopted in the hard threshold filtering process of traditional BM3D algorithm to achieve local adaptive filtering and further improving the denoising effect. The synthetic data and real breast ultrasound data examples show that this combined method can improve the speckle suppression level and keep the fidelity of structure effectively without increasing time cost.