Pruning Strategy Research Articles

PAMPred: A hierarchical evolutionary ensemble framework for identifying plant antimicrobial peptides

Antimicrobial peptides (AMPs) play a crucial role in plant immune regulation, growth and development stages, which have attracted significant attentions in recent years. As the wet-lab experiments are laborious and cost-prohibitive, it is indispensable to develop computational methods to discover novel plant AMPs accurately. In this study, we presented a hierarchical evolutionary ensemble framework, named PAMPred, which consisted of a multi-level heterogeneous architecture to identify plant AMPs. Specifically, to address the existing class imbalance problem, a cluster-based resampling method was adopted to build multiple balanced subsets. Then, several peptide features including sequence information-based and physicochemical properties-based features were fed into the different types of basic learners to increase the ensemble diversity. For boosting the predictive capability of PAMPred, the improved particle swarm optimization (PSO) algorithm and dynamic ensemble pruning strategy were used to optimize the weights at different levels adaptively. Furthermore, extensive ten-fold cross-validation and independent testing experimental results demonstrated that PAMPred achieved excellent prediction performance and generalization ability, and outperformed the state-of-the-art methods. It also indicated that the proposed method could serve as an effective auxiliary tool to identify plant AMPs, which would be conducive to explore the immune regulatory mechanism of plants.

Multiple dependence representation of attention graph convolutional network relation extraction model

AbstractDependency analysis can better help neural network to capture semantic features in sentences, so as to extract entity relation. Currently, hard pruning strategies and soft pruning strategies based on dependency tree structure coding have been proposed to balance beneficial additional information and adverse interference in extraction tasks. A new model based on graph convolutional networks, which uses a variety of representations describing dependency trees from different perspectives and combining these representations to obtain a better sentence representation for relation classification is proposed. A newly defined module is added, and this module uses the attention mechanism to capture deeper semantic features from the context representation as the global semantic features of the input text, thus helping the model to capture deeper semantic information at the sentence level for relational extraction tasks. In order to get more information about a given entity pair from the input sentence, the authors also model implicit co‐references (references) to entities. This model can extract semantic features related to the relationship between entities from sentences to the maximum extent. The results show that the model in this paper achieves good results on SemEval2010‐Task8 and KBP37 datasets.

Convexity and global optimisation of lane-based fixed-time signal model for delay minimisation at an isolated intersection

Lane-based fixed-time signal is basic to various signal control strategies. It performs well in maximizing road capacity, but is faced with significant challenge in minimizing traffic delay. This study validates the convexity of lane-based fixed-time signal model for delay minimization, when lane markings are determined as well as flow factors. Thus Breadth first search algorithm is developed to enumerate the feasible lane markings, which are then screened with flow factors. Cutting plane algorithm is applied to the CMINLP for each feasible lane markings, where the non-linear delay function is converted to a series of linear ones, until the relaxed delay converges to the actual delay. Branch pruning strategy is established for efficiency, to eliminate the lane markings with uncompetitive delay. Numerical analyses follow to validate the proposed algorithm. This research promotes the redesign of lane-based fixed-time signal control.

V-LSTM: An Efficient LSTM Accelerator Using Fixed Nonzero-Ratio Viterbi-Based Pruning

Long Short-Term Memory (LSTM) has been widely adopted in tasks with sequence data, such as speech recognition and language modeling. LSTM brought significant accuracy improvement by introducing additional parameters to Recurrent Neural Network (RNN). However, increasing number of parameters and computations also led to inefficiency in computing LSTM on edge devices with limited on-chip memory size and DRAM bandwidth. In order to reduce the latency and energy of LSTM computations, there has been a pressing need for model compression schemes and suitable hardware accelerators. In this paper, we first propose the Fixed Nonzero-ratio Viterbi-based Pruning, which can reduce the memory footprint of LSTM models by 96% with negligible accuracy loss. By applying additional constraints on the distribution of surviving weights in Viterbi-based Pruning, the proposed pruning scheme mitigates the load-imbalance problem and thereby increases the processing engine utilization rate. Then, we propose the V-LSTM, an efficient sparse LSTM accelerator based on the proposed pruning scheme. High compression ratio of the proposed pruning scheme allows the proposed accelerator to achieve 24.9% lower per-sample latency than that of state-of-the-art accelerators. The proposed accelerator is implemented on Xilinx VC-709 FPGA evaluation board running at 200MHz for evaluation.

Effective and efficient content-based similarity retrieval of large lung CT images based on WSSLN model.

The in-depth combination and application of AI technology and medical imaging, especially high- definition CT imaging technology, make accurate diagnosis and treatment possible. Retrieving similar CT image(CI)s to an input one from the large-scale CI database of labeled diseases is helpful to realize a precise computer-aided diagnosis. In this paper, we take lung CI as an example and propose progressive content-based similarity retrieval(CBSR) method of the lung CIs based on a Weakly Supervised Similarity Learning Network (WSSLN) model. Two enabling techniques (i.e., the WSSLN model and the distance- based pruning scheme) are proposed to facilitate the CBSR processing of the large lung CIs. The main result of our paper is that, our approach is about 45% more effective than the state-of-the-art methods in terms of the mean average precision(mAP). Moreover, for the retrieval efficiency, the WSSLN-based CBSR method is about 150% more efficient than the sequential scan.

Mesh-Informed Neural Networks for Operator Learning in Finite Element Spaces

Thanks to their universal approximation properties and new efficient training strategies, Deep Neural Networks are becoming a valuable tool for the approximation of mathematical operators. In the present work, we introduce Mesh-Informed Neural Networks (MINNs), a class of architectures specifically tailored to handle mesh based functional data, and thus of particular interest for reduced order modeling of parametrized Partial Differential Equations (PDEs). The driving idea behind MINNs is to embed hidden layers into discrete functional spaces of increasing complexity, obtained through a sequence of meshes defined over the underlying spatial domain. The approach leads to a natural pruning strategy which enables the design of sparse architectures that are able to learn general nonlinear operators. We assess this strategy through an extensive set of numerical experiments, ranging from nonlocal operators to nonlinear diffusion PDEs, where MINNs are compared against more traditional architectures, such as classical fully connected Deep Neural Networks, but also more recent ones, such as DeepONets and Fourier Neural Operators. Our results show that MINNs can handle functional data defined on general domains of any shape, while ensuring reduced training times, lower computational costs, and better generalization capabilities, thus making MINNs very well-suited for demanding applications such as Reduced Order Modeling and Uncertainty Quantification for PDEs.

UVS: underwater visual SLAM—a robust monocular visual SLAM system for lifelong underwater operations

AbstractIn this paper, a visual simultaneous localization and mapping (VSLAM/visual SLAM) system called underwater visual SLAM (UVS) system is presented, specifically tailored for camera-only navigation in natural underwater environments. The UVS system is particularly optimized towards precision and robustness, as well as lifelong operations. We build upon Oriented features from accelerated segment test and Rotated Binary robust independent elementary features simultaneous localization and mapping (ORB-SLAM) and improve the accuracy by performing an exact search in the descriptor space during triangulation and the robustness by utilizing a unified initialization method and a motion model. In addition, we present a scale-agnostic station-keeping detection, which aims to optimize the map and poses during station-keeping, and a pruning strategy, which takes into account the point’s age and distance to the active keyframe. An exhaustive evaluation is presented to the reader, using a total of 38 in-air and underwater sequences.

Modeling construction and demolition waste quantities in Tanta City, Egypt: a synergistic approach of remote sensing, geographic information system, and hybrid fuzzy neural networks

A waste management strategy needs accurate data on the generation rates of construction and demolition waste (CDW). The objective of this study is to provide a robust methodology for predicting CDW generation in Tanta City, one of the largest and most civilized cities in Egypt, based on socioeconomic and waste generation statistics from 1965 to 2021. The main contribution of this research involves the fusion of remote sensing and geographic information systems to construct a geographical database, which is employed using machine learning for modeling and predicting the quantities of generated waste. The land use/land cover map is determined by integrating topographic maps and remotely sensed data to extract the built-up, vacant, and agricultural areas. The application of a self-organizing fuzzy neural network (SOFNN) based on an adaptive quantum particle swarm optimization algorithm and a hierarchical pruning scheme is introduced to predict the waste quantities. The performance of the proposed models is compared against that of the FNN with error backpropagation and the group method of data handling using five evaluation measures. The results of the proposed models are satisfactory, with mean absolute percentage error (MAPE), normalized root mean square error (NRMSE), determination coefficient, Kling–Gupta efficiency, and index of agreement ranging between 0.70 and 1.56%, 0.01 and 0.03, 0.99 and 1.00, 0.99, and 1.00. Compared to other models, the proposed models reduce the MAPE and NRMSE by more than 92.90% and 90.64% based on fivefold cross-validation. The research findings are beneficial for utilizing limited data in developing effective strategies for quantifying waste generation. The simulation outcomes can be applied to monitor the urban metabolism, measure carbon emissions from the generated waste, develop waste management facilities, and build a circular economy in the study area.

Targeted mining of top-[formula omitted] high utility itemsets

Finding high-importance patterns in data is an emerging data mining task known as High-utility itemset mining (HUIM). Given a minimum utility threshold, a HUIM algorithm extracts all the high-utility itemsets (HUIs) whose utility values are not less than the threshold. This can reveal a wealth of useful information, but the precise needs of users are not well taken into account. In particular, users often want to focus on patterns that have some specific items rather than find all patterns. To overcome that difficulty, targeted mining has emerged, focusing on user preferences, but only preliminary work has been conducted. For example, the targeted high-utility itemset querying algorithm (TargetUM) was proposed, which uses a lexicographic tree to query itemsets containing a target pattern. However, selecting the minimum utility threshold is difficult when the user is not familiar with the processed database. As a solution, this paper formulates the task of targeted mining of the top-k high-utility itemsets and proposes an efficient algorithm called TMKU based on the TargetUM algorithm to discover the top-k target high-utility itemsets (top-k THUIs). At the same time, several pruning strategies are used to reduce memory consumption and execution time. Extensive experiments show that the proposed TMKU algorithm has good performance on real and synthetic datasets.

Efficient mining of concise and informative representations of frequent high utility itemsets

The discovery of frequent closed high utility itemsets (FCHUIs) and frequent generators of high utility itemsets (FGHUIs) is significant because they serve as important and concise representations of frequent high utility itemsets (FHUIs), offering a brief but essential summary that can be considerably smaller. Besides, they facilitate the generation of nonredundant high utility association rules that are crucial for decision-makers. However, the challenge lies in the difficulty of mining these representations due to scalability issues, high memory usage, and long runtimes, particularly when dealing with dense and large datasets. To address this issue, this paper proposes a novel approach for efficiently mining FCHUIs and FGHUIs using a novel weak lower bound named wlbu on the utility. The approach includes effective pruning strategies for early eliminating non-closed and/or non-generator high utility branches in the prefix search tree based on wlbu. These pruning strategies allow faster execution with lower memory usage. In addition, the paper presents two novel algorithms, FCGHUI-Miner and FGHUI-Miner, which can simultaneously discover both FGHUIs and FCHUIs or solely mine FGHUIs, respectively. The experimental results demonstrate that the proposed algorithms outperform state-of-the-art algorithms in terms of efficiency and effectiveness.

A Biomedical Relation Extraction Method Based on Graph Convolutional Network with Dependency Information Fusion

Biomedical texts are relatively obscure in describing relations between specialized entities, and the automatic extraction of drug–drug or drug–disease relations from massive biomedical texts presents a challenge faced by many researchers. To this end, this paper designs a relation extraction method based on dependency information fusion to improve the predictive power of the model for the relations between given biomedical entities. Firstly, we propose a local–global pruning strategy for the dependency syntax tree. Next, we propose the construction of a dependency type matrix for the pruned dependency tree to incorporate sentence dependency information into the model to feature extraction. We then incorporate attention mechanism into the graph convolutional model by calculating the attention weights of word–word dependencies, thus improving the traditional graph convolutional network. The model distinguishes the importance of different dependency information by attention weights, thus weakening the influence of interfering information such as word-to-word dependencies that are unrelated to entities in long sentences. In this paper, our proposed Dependency Information Fusion Attention Graph Convolutional Network (DIF-A-GCN) is evaluated on two biomedical datasets, DDI and CIVIC. The experimental results show that our proposed method based on dependency information fusion outperforms current state-of-the-art biomedical relation extraction models.

Random pruning: channel sparsity by expectation scaling factor.

Pruning is an efficient method for deep neural network model compression and acceleration. However, existing pruning strategies, both at the filter level and at the channel level, often introduce a large amount of computation and adopt complex methods for finding sub-networks. It is found that there is a linear relationship between the sum of matrix elements of the channels in convolutional neural networks (CNNs) and the expectation scaling ratio of the image pixel distribution, which is reflects the relationship between the expectation change of the pixel distribution between the feature mapping and the input data. This implies that channels with similar expectation scaling factors () cause similar expectation changes to the input data, thus producing redundant feature mappings. Thus, this article proposes a new structured pruning method called EXP. In the proposed method, the channels with similar are randomly removed in each convolutional layer, and thus the whole network achieves random sparsity to obtain non-redundant and non-unique sub-networks. Experiments on pruning various networks show that EXP can achieve a significant reduction of FLOPs. For example, on the CIFAR-10 dataset, EXP reduces the FLOPs of the ResNet-56 model by 71.9% with a 0.23% loss in Top-1 accuracy. On ILSVRC-2012, it reduces the FLOPs of the ResNet-50 model by 60.0% with a 1.13% loss of Top-1 accuracy. Our code is available at: https://github.com/EXP-Pruning/EXP_Pruning and DOI: 10.5281/zenodo.8141065.

A Pruning and Feedback Strategy for Locating Reliability-Critical Gates in Combinational Circuits

In nanometric integrated circuits, to harden reliability-critical gates (RCGs) is an important step to improve overall circuit reliability at a low cost. To locate RCGs quickly and efficiently is a key prerequisite for selective hardening at the early stage of circuit design. This article develops a new approach for locating RCGs for multiple input vectors in combinational circuits, using an input vector-oriented pruning technology to identify RCGs, and a sensitivity-based algorithm to measure the criticality of gate reliability (CGR) for each identified RCG. To accelerate the location of RCGs, a feedback-based algorithm mines the accumulated simulation data for each RCG, and a grouping algorithm handles RCGs with similar CGR in the stage of convergence checking. Simulations on 74-series and ISCAS 85 benchmark circuits show that the average accuracy of the proposed method is 0.986 with Monte–Carlo (MC) as the reference and it is 7181 times faster than the MC model. Also, this method performs better than other approximate algorithms in terms of location accuracy and time overhead.

Efficient design of neural networks for the classification of acoustic spectra.

A previous paper by Paul and Nelson [(2021). J. Acoust. Soc. Am. 149(6), 4119-4133] presented the application of the singular value decomposition (SVD) to the weight matrices of multilayer perceptron (MLP) networks as a pruning strategy to remove weight parameters. This work builds on the previous technique and presents a method of reducing the size of a hidden layer by applying a similar SVD algorithm. Results show that by reducing the neurons in the hidden layer, a significant amount of training time is saved compared to the algorithm presented in the previous paper while no or little accuracy is being lost compared to the original MLP model.

An Adaptive Sparse Channel Estimation Algorithm for OFDM Systems Based on Distributed Compressive Sensing

The current greedy iterative pursuit algorithms for sparse channel estimation in an orthogonal frequency division multiplexing (OFDM) system based on compressive sensing (CS) or distributed CS (DCS) have the disadvantages of relying on channel priori information as halting condition and having a low support searching efficiency. Under DCS framework, this letter proposes a unique halting condition for greedy algorithms by exploiting the delay correlation between adjacent symbol channels. Additionally, we present a segmented pruning strategy that supports to select multiple atoms in a single iteration to improve the support searching efficiency. Simulation results show that our algorithm can achieve more robust sparsity-adaptive channel estimation with reduced computational complexity compared to the traditional methods.

A Novel AGV Path Planning Approach for Narrow Channels Based on the Bi-RRT Algorithm with a Failure Rate Threshold.

The efficiency of the rapidly exploring random tree (RRT) falls short when efficiently guiding targets through constricted-passage environments, presenting issues such as sluggish convergence speed and elevated path costs. To overcome these algorithmic limitations, we propose a narrow-channel path-finding algorithm (named NCB-RRT) based on Bi-RRT with the addition of our proposed research failure rate threshold (RFRT) concept. Firstly, a three-stage search strategy is employed to generate sampling points guided by real-time sampling failure rates. By means of the balance strategy, two randomly growing trees are established to perform searching, which improves the success rate of the algorithm in narrow channel environments, accelerating the convergence speed and reducing the number of iterations required. Secondly, the parent node re-selection and path pruning strategy are integrated. This shortens the path length and greatly reduces the number of redundant nodes and inflection points. Finally, the path is optimized by utilizing segmented quadratic Bezier curves to achieve a smooth trajectory. This research shows that the NCB-RRT algorithm is better able to adapt to the complex narrow channel environment, and the performance is also greatly improved in terms of the path length and the number of inflection points. Compared with the RRT, RRT* and Bi-RRT algorithms, the success rate is increased by 2400%, 1900% and 11.11%, respectively.

Single-shot pruning and quantization for hardware-friendly neural network acceleration

Applying CNN on embedded systems is challenging due to model size limitations. Pruning and quantization can help, but are time-consuming to apply separately. Our Single-Shot Pruning and Quantization strategy addresses these issues by quantizing and pruning in a single process. We evaluated our method on CIFAR-10 and CIFAR-100 datasets for image classification. Our model is 69.4% smaller with little accuracy loss, and runs 6–8 times faster on NVIDIA Xavier NX hardware.

A Real-Time Detection and Maturity Classification Method for Loofah

Fruit maturity is a crucial index for determining the optimal harvesting period of open-field loofah. Given the plant’s continuous flowering and fruiting patterns, fruits often reach maturity at different times, making precise maturity detection essential for high-quality and high-yield loofah production. Despite its importance, little research has been conducted in China on open-field young fruits and vegetables and a dearth of standards and techniques for accurate and non-destructive monitoring of loofah fruit maturity exists. This study introduces a real-time detection and maturity classification method for loofah, comprising two components: LuffaInst, a one-stage instance segmentation model, and a machine learning-based maturity classification model. LuffaInst employs a lightweight EdgeNeXt as the backbone and an enhanced pyramid attention-based feature pyramid network (PAFPN). To cater to the unique characteristics of elongated loofah fruits and the challenge of small target detection, we incorporated a novel attention module, the efficient strip attention module (ESA), which utilizes long and narrow convolutional kernels for strip pooling, a strategy more suitable for loofah fruit detection than traditional spatial pooling. Experimental results on the loofah dataset reveal that these improvements equip our LuffaInst with lower parameter weights and higher accuracy than other prevalent instance segmentation models. The mean average precision (mAP) on the loofah image dataset improved by at least 3.2% and the FPS increased by at least 10.13 f/s compared with Mask R-CNN, Mask Scoring R-CNN, YOLACT++, and SOLOv2, thereby satisfying the real-time detection requirement. Additionally, a random forest model, relying on color and texture features, was developed for three maturity classifications of loofah fruit instances (M1: fruit setting stage, M2: fruit enlargement stage, M3: fruit maturation stage). The application of a pruning strategy helped attain the random forest model with the highest accuracy (91.47% for M1, 90.13% for M2, and 92.96% for M3), culminating in an overall accuracy of 91.12%. This study offers promising results for loofah fruit maturity detection, providing technical support for the automated intelligent harvesting of loofah.

Federated learning with network pruning and rebirth for remaining useful life prediction of engineering systems

Remaining useful life (RUL) prediction has achieved considerable success through centralized learning methods. However, traditional data aggregation may cause privacy disclosure, and existing prediction models are often too large to be trained efficiently. This paper proposes a RUL prediction method in the federated learning (FL) framework, which aims to develop a lightweight model using network pruning and rebirth strategies. First, a deep convolutional neural network (DCNN) is designed as the prediction model. Next, the Taylor expansion and the l2 norm pruning criteria are executed on the convolutional and fully-connected layers of DCNN to prune some unimportant feature maps and neurons, respectively. After each pruning operation, the network rebirth strategies, including model relocation, federated averaging (FedAvg), and selective retraining, are used to fine-tune the pruned model in the FL. Finally, the network pruning and rebirth occur alternately to produce a compact RUL prediction model with fewer parameters, which can achieve the same good performance as the original one. Experiments study on the C-MAPSS dataset demonstrates the effectiveness of the proposed method.

Performance-Aware Approximation of Global Channel Pruning for Multitask CNNs.

Global channel pruning (GCP) aims to remove a subset of channels (filters) across different layers from a deep model without hurting the performance. Previous works focus on either single task model pruning or simply adapting it to multitask scenario, and still face the following problems when handling multitask pruning: 1) Due to the task mismatch, a well-pruned backbone for classification task focuses on preserving filters that can extract category-sensitive information, causing filters that may be useful for other tasks to be pruned during the backbone pruning stage; 2) For multitask predictions, different filters within or between layers are more closely related and interacted than that for single task prediction, making multitask pruning more difficult. Therefore, aiming at multitask model compression, we propose a Performance-Aware Global Channel Pruning (PAGCP) framework. We first theoretically present the objective for achieving superior GCP, by considering the joint saliency of filters from intra- and inter-layers. Then a sequentially greedy pruning strategy is proposed to optimize the objective, where a performance-aware oracle criterion is developed to evaluate sensitivity of filters to each task and preserve the globally most task-related filters. Experiments on several multitask datasets show that the proposed PAGCP can reduce the FLOPs and parameters by over 60% with minor performance drop, and achieves 1.2x ∼3.3x acceleration on both cloud and mobile platforms. Our code is available at http://www.github.com/HankYe/PAGCP.git.