Subset Of Training Data Research Articles

Point-of-care breath sample analysis by semiconductor-based E-Nose technology discriminates non-infected subjects from SARS-CoV-2 pneumonia patients: a multi-analyst experiment.

Metal oxide sensor-based electronic nose (E-Nose) technology provides an easy to use method for breath analysis by detection of volatile organic compound (VOC)-induced changes of electrical conductivity. Resulting signal patterns are then analyzed by machine learning (ML) algorithms. This study aimed to establish breath analysis by E-Nose technology as a diagnostic tool for severe acute respiratory syndrome coronavirus type 2 (SARS-CoV-2) pneumonia within a multi-analyst experiment. Breath samples of 126 subjects with (n=63) or withoutSARS-CoV-2 pneumonia (n=63) were collected using the ReCIVA® Breath Sampler, enriched and stored on Tenax sorption tubes, and analyzed using an E-Nose unit with 10 sensors. ML approaches were applied by three independent data analyst teams and included a wide range of classifiers, hyperparameters, training modes, and subsets of training data. Within the multi-analyst experiment, all teams successfully classified individuals as infected or uninfected with an averaged area under the curve (AUC) larger than 90% and misclassification error lower than 19%, and identified the same sensor as most relevant to classification success. This new method using VOC enrichment and E-Nose analysis combined with ML can yield results similar to polymerase chain reaction (PCR) detection and superior to point-of-care (POC) antigen testing. Reducing the sensor set to the most relevant sensor may prove interesting for developing targeted POC testing.

AutoPeptideML: a study on how to build more trustworthy peptide bioactivity predictors.

Automated machine learning (AutoML) solutions can bridge the gap between new computational advances and their real-world applications by enabling experimental scientists to build their own custom models. We examine different steps in the development life-cycle of peptide bioactivity binary predictors and identify key steps where automation cannot only result in a more accessible method, but also more robust and interpretable evaluation leading to more trustworthy models. We present a new automated method for drawing negative peptides that achieves better balance between specificity and generalization than current alternatives. We study the effect of homology-based partitioning for generating the training and testing data subsets and demonstrate that model performance is overestimated when no such homology correction is used, which indicates that prior studies may have overestimated their performance when applied to new peptide sequences. We also conduct a systematic analysis of different protein language models as peptide representation methods and find that they can serve as better descriptors than a naive alternative, but that there is no significant difference across models with different sizes or algorithms. Finally, we demonstrate that an ensemble of optimized traditional machine learning algorithms can compete with more complex neural network models, while being more computationally efficient. We integrate these findings into AutoPeptideML, an easy-to-use AutoML tool to allow researchers without a computational background to build new predictive models for peptide bioactivity in a matter of minutes. Source code, documentation, and data are available at https://github.com/IBM/AutoPeptideML and a dedicated web-server at http://peptide.ucd.ie/AutoPeptideML. A static version of the software to ensure the reproduction of the results is available at https://zenodo.org/records/13363975.

Adaptive K values and training subsets selection for optimal K-NN performance on FPGA

This study introduces an Adaptive K-Nearest Neighbors methodology designed for FPGA platforms, offering substantial improvements over traditional K-Nearest Neighbors implementations. By integrating a dynamic classifier selection system, our approach enhances adaptability, enabling on-the-fly adjustments of K values and subsets of training data. This flexibility results in up to a 10.66% improvement in accuracy and significantly reduces latency, rendering our system up to 3.918 times more efficient than conventional K-Nearest Neighbors techniques. The methodology’s efficacy is validated through experiments across multiple datasets, demonstrating its potential in optimizing both classification accuracy and system efficiency. The adaptive approach’s ability to improve response times, along with its flexibility, positions it as an ideal solution for real-time applications and highlights the advantages of the adaptive K-Nearest Neighbors methodology in overcoming the constraints of hardware-accelerated machine learning.

Semantically redundant training data removal and deep model classification performance: A study with chest X-rays

Deep learning (DL) has demonstrated its innate capacity to independently learn hierarchical features from complex and multi-dimensional data. A common understanding is that its performance scales up with the amount of training data. However, the data must also exhibit variety to enable improved learning. In medical imaging data, semantic redundancy, which is the presence of similar or repetitive information, can occur due to the presence of multiple images that have highly similar presentations for the disease of interest. Also, the common use of augmentation methods to generate variety in DL training could limit performance when indiscriminately applied to such data. We hypothesize that semantic redundancy would therefore tend to lower performance and limit generalizability to unseen data and question its impact on classifier performance even with large data. We propose an entropy-based sample scoring approach to identify and remove semantically redundant training data and demonstrate using the publicly available NIH chest X-ray dataset that the model trained on the resulting informative subset of training data significantly outperforms the model trained on the full training set, during both internal (recall: 0.7164 vs 0.6597, p<0.05) and external testing (recall: 0.3185 vs 0.2589, p<0.05). Our findings emphasize the importance of information-oriented training sample selection as opposed to the conventional practice of using all available training data.

I-CEE: Tailoring Explanations of Image Classification Models to User Expertise

Effectively explaining decisions of black-box machine learning models is critical to responsible deployment of AI systems that rely on them. Recognizing their importance, the field of explainable AI (XAI) provides several techniques to generate these explanations. Yet, there is relatively little emphasis on the user (the explainee) in this growing body of work and most XAI techniques generate "one-size-fits-all'' explanations. To bridge this gap and achieve a step closer towards human-centered XAI, we present I-CEE, a framework that provides Image Classification Explanations tailored to User Expertise. Informed by existing work, I-CEE explains the decisions of image classification models by providing the user with an informative subset of training data (i.e., example images), corresponding local explanations, and model decisions. However, unlike prior work, I-CEE models the informativeness of the example images to depend on user expertise, resulting in different examples for different users. We posit that by tailoring the example set to user expertise, I-CEE can better facilitate users' understanding and simulatability of the model. To evaluate our approach, we conduct detailed experiments in both simulation and with human participants (N = 100) on multiple datasets. Experiments with simulated users show that I-CEE improves users' ability to accurately predict the model's decisions (simulatability) compared to baselines, providing promising preliminary results. Experiments with human participants demonstrate that our method significantly improves user simulatability accuracy, highlighting the importance of human-centered XAI.

Improving classification accuracy of fine-tuned CNN models: Impact of hyperparameter optimization

The immense popularity of convolutional neural network (CNN) models has sparked a growing interest in optimizing their hyperparameters. Discovering the ideal values for hyperparameters to achieve optimal CNN training is a complex and time-consuming task, often requiring repetitive numerical experiments. As a result, significant attention is currently being devoted to developing methods aimed at tailoring hyperparameters for specific CNN models and classification tasks. While existing optimization methods often yield favorable image classification results, they do not provide guidance on which hyperparameters are worth optimizing, the appropriate value ranges for those hyperparameters, or whether it is reasonable to use a subset of training data for the optimization process. This work is focused on the optimization of hyperparameters during transfer learning, with the goal of investigating how different optimization methods and hyperparameter selections impact the performance of fine-tuned models. In our experiments, we assessed the importance of various hyperparameters and identified the ranges within which optimal CNN training can be achieved. Additionally, we compared four hyperparameter optimization methods—grid search, random search, Bayesian optimization, and the Asynchronous Successive Halving Algorithm (ASHA). We also explored the feasibility of fine-tuning hyperparameters using a subset of the training data. By optimizing the hyperparameters, we observed an improvement in CNN classification accuracy of up to 6%. Furthermore, we found that achieving a balance in class distribution within the subset of data used for parameter optimization is crucial in establishing the optimal set of hyperparameters for CNN training. The results we obtained demonstrate that hyperparameter optimization is highly dependent on the specific task and dataset at hand.

Predicting the flowability of UHPC and identifying its significant influencing factors using an accurate ANN model

In this research, a one-hidden layer artificial neural network paradigm (ANN) was created to forecast the slump flow of ultra-high-performance concrete (UHPC). To achieve this goal, 3,200 ANNs were evaluated to estimate the fresh UHPC’s slump flow utilizing 793 observations. The performance metrics measured on training and test data subsets were in the same order of magnitude, thereby pointing out the proper work of the k-fold validation procedure. The results of the connection weight approach analysis (CWA) indicated that water dosage had the highest positive importance in slump flow, preceding the superplasticizer volume ratio. Other factors that positively influenced slump flow were the water-to-powder ratio, the dosage of high-alkali glass powder, the water-to-binder ratio, and limestone concentration. The most negative influences on rheology were the high-alumina FC3R and metakaolin. The ANN accurately predicted the slump flow of UHPC, while the results of the CWA analysis were well-correlated with previous research.

Partial Multilabel Learning Using Noise-Tolerant Broad Learning System With Label Enhancement and Dimensionality Reduction.

Partial multilabel learning (PML) addresses the issue of noisy supervision, which contains an overcomplete set of candidate labels for each instance with only a valid subset of training data. Using label enhancement techniques, researchers have computed the probability of a label being ground truth. However, enhancing labels in the noisy label space makes it impossible for the existing partial multilabel label enhancement methods to achieve satisfactory results. Besides, few methods simultaneously involve the ambiguity problem, the feature space's redundancy, and the model's efficiency in PML. To address these issues, this article presents a novel joint partial multilabel framework using broad learning systems (namely BLS-PML) with three innovative mechanisms: 1) a trustworthy label space is reconstructed through a novel label enhancement method to avoid the bias caused by noisy labels; 2) a low-dimensional feature space is obtained by a confidence-based dimensionality reduction method to reduce the effect of redundancy in the feature space; and 3) a noise-tolerant BLS is proposed by adding a dimensionality reduction layer and a trustworthy label layer to deal with PML problem. We evaluated it on six real-world and seven synthetic datasets, using eight state-of-the-art partial multilabel algorithms as baselines and six evaluation metrics. Out of 144 experimental scenarios, our method significantly outperforms the baselines by about 80%, demonstrating its robustness and effectiveness in handling partial multilabel tasks.

Deep learning for automatic organ and tumor segmentation in nanomedicine pharmacokinetics.

Rationale: Multimodal imaging provides important pharmacokinetic and dosimetry information during nanomedicine development and optimization. However, accurate quantitation is time-consuming, resource intensive, and requires anatomical expertise. Methods: We present NanoMASK: a 3D U-Net adapted deep learning tool capable of rapid, automatic organ segmentation of multimodal imaging data that can output key clinical dosimetry metrics without manual intervention. This model was trained on 355 manually-contoured PET/CT data volumes of mice injected with a variety of nanomaterials and imaged over 48 hours. Results: NanoMASK produced 3-dimensional contours of the heart, lungs, liver, spleen, kidneys, and tumor with high volumetric accuracy (pan-organ average %DSC of 92.5). Pharmacokinetic metrics including %ID/cc, %ID, and SUVmax achieved correlation coefficients exceeding R = 0.987 and relative mean errors below 0.2%. NanoMASK was applied to novel datasets of lipid nanoparticles and antibody-drug conjugates with a minimal drop in accuracy, illustrating its generalizability to different classes of nanomedicines. Furthermore, 20 additional auto-segmentation models were developed using training data subsets based on image modality, experimental imaging timepoint, and tumor status. These were used to explore the fundamental biases and dependencies of auto-segmentation models built on a 3D U-Net architecture, revealing significant differential impacts on organ segmentation accuracy. Conclusions: NanoMASK is an easy-to-use, adaptable tool for improving accuracy and throughput in imaging-based pharmacokinetic studies of nanomedicine. It has been made publicly available to all readers for automatic segmentation and pharmacokinetic analysis across a diverse array of nanoparticles, expediting agent development.

Analysis of Service Science Design through Customer Experience to Increase PosAja Application User Satisfaction

The PosAja mobile application is a digital service owned by Pos Indonesia in the field of delivery. Despite being downloaded by more than 100K users, the PosAja application has experienced a decline in rating and application productivity. This research intends to represent user complaints through review data analysis and develop a more effective and efficient service science design based on user experience. The research method applied is descriptive quantitative with data analysis applying SVM to classify positive and negative sentiments from review data. Model evaluation was carried out by training the training and test data subsets 3 times with variations of k = 5, k = 7, and k = 10. The results showed that the best accuracy was obtained with an accuracy of 89.98% on the SVM_kernel Sigmoid parameter. Analysis of review data revealed that the sentiment of PosAja application users tends to be negative compared to positive sentiment on the Google Play Store. To improve user experience and satisfaction, service science design development should focus on continuous improvement and evaluation of the PosAja application.

PolyID: Artificial Intelligence for Discovering Performance-Advantaged and Sustainable Polymers.

A necessary transformation for a sustainable economy is the transition from fossil-derived plastics to polymers derived from biomass and waste resources. While renewable feedstocks can enhance material performance through unique chemical moieties, probing the vast material design space by experiment alone is not practically feasible. Here, we develop a machine-learning-based tool, PolyID, to reduce the design space of renewable feedstocks to enable efficient discovery of performance-advantaged, biobased polymers. PolyID is a multioutput, graph neural network specifically designed to increase accuracy and to enable quantitative structure-property relationship (QSPR) analysis for polymers. It includes a novel domain-of-validity method that was developed and applied to demonstrate how gaps in training data can be filled to improve accuracy. The model was benchmarked with both a 20% held-out subset of the original training data and 22 experimentally synthesized polymers. A mean absolute error for the glass transition temperatures of 19.8 and 26.4 °C was achieved for the test and experimental data sets, respectively. Predictions were made on polymers composed of monomers from four databases that contain biologically accessible small molecules: MetaCyc, MINEs, KEGG, and BiGG. From 1.4 × 106 accessible biobased polymers, we identified five poly(ethylene terephthalate) (PET) analogues with predicted improvements to thermal and transport performance. Experimental validation for one of the PET analogues demonstrated a glass transition temperature between 85 and 112 °C, which is higher than PET and within the predicted range of the PolyID tool. In addition to accurate predictions, we show how the model's predictions are explainable through analysis of individual bond importance for a biobased nylon. Overall, PolyID can aid the biobased polymer practitioner to navigate the vast number of renewable polymers to discover sustainable materials with enhanced performance.

An active learning convolutional neural network for predicting river flow in a human impacted system

The South Platte river system contains a mixture of natural streams, reservoirs, and pipeline projects that redirect water to front range communities in Colorado. At many timepoints, a simple persistence model is the best predictor for flow from pipelines and reservoirs but at other times, flows change based on snowmelt and inputs such as reservoir fill rates, local weather, and anticipated demand. Here we find that a convolutional Long Short-Term Memory (LSTM) network is well suited to modeling flow in parts of this basin that are strongly impacted by water projects as well as ones that are relatively free from direct human modifications. Furthermore, it is found that including an active learning component in which separate Convolutional Neural Networks (CNNs) are used to classify and then select the data that is then used for training a convolutional LSTM network is advantageous. Models specific for each gauge are created by transfer of parameter from a base model and these gauge-specific models are then fine-tuned based a curated subset of training data. The result is accurate predictions for both natural flow and human influenced flow using only past river flow, reservoir capacity, and historical temperature data. In 14 of the 16 gauges modeled, the error in the prediction is reduced when using the combination of on-the-fly classification by CNN followed by analysis by either a persistence or convolutional LSTM model. The methods designed here could be applied broadly to other basins and to other situations where multiple models are needed to fit data at different times and locations.

Leveraging global binary masks for structure segmentation in medical images

Deep learning (DL) models for medical image segmentation are highly influenced by intensity variations of input images and lack generalization due to primarily utilizing pixels’ intensity information for inference. Acquiring sufficient training data is another challenge limiting models’ applications. Here, we proposed to leverage the consistency of organs’ anatomical position and shape information in medical images. We introduced a framework leveraging recurring anatomical patterns through global binary masks for organ segmentation. Two scenarios were studied: (1) global binary masks were the only input for the U-Net based model, forcing exclusively encoding organs’ position and shape information for rough segmentation or localization. (2) Global binary masks were incorporated as an additional channel providing position/shape clues to mitigate training data scarcity. Two datasets of the brain and heart computed tomography (CT) images with their ground-truth were split into (26:10:10) and (12:3:5) for training, validation, and test respectively. The two scenarios were evaluated using full training split as well as reduced subsets of training data. In scenario (1), training exclusively on global binary masks led to Dice scores of 0.77 ± 0.06 and 0.85 ± 0.04 for the brain and heart structures respectively. Average Euclidian distance of 3.12 ± 1.43 mm and 2.5 ± 0.93 mm were obtained relative to the center of mass of the ground truth for the brain and heart structures respectively. The outcomes indicated encoding a surprising degree of position and shape information through global binary masks. In scenario (2), incorporating global binary masks led to significantly higher accuracy relative to the model trained on only CT images in small subsets of training data; the performance improved by 4.3%–125.3% and 1.3%–48.1% for 1–8 training cases of the brain and heart datasets respectively. The findings imply the advantages of utilizing global binary masks for building models that are robust to image intensity variations as well as an effective approach to boost performance when access to labeled training data is highly limited.

Semi-supervised machine learning for motor eccentricity fault diagnosis

Eccentricity is one major indicator of mechanical faults in electric machines and needs to be detected early to avoid machine failures. Data-driven techniques based on machine learning and deep learning algorithms have been proposed in recent years for motor fault detection. However, majority of these methods use supervised learning algorithms and require large, labelled datasets, which can be challenging to obtain. In this paper, we propose a semi-supervised learning method based on a deep generative model using variational auto-encoder for eccentricity fault quantification. Good prediction accuracy can be achieved when only a small subset of training data has labels.

A Genetic Algorithm-based sequential instance selection framework for ensemble learning

The accumulation of large amounts of historical data has led to the wide application of ensemble learning over the past few decades, but the balance between the individual accuracy of base classifiers (BCs) and the diversity among these BCs is rarely considered in the construction of ensemble models. Since such a balance is crucial to the success of ensemble models, this paper proposes a Genetic Algorithm-based sequential instance selection framework to address this research gap. The novelties of the proposed framework include: transforming the balance between the individual accuracy of BCs and the diversity among BCs into a general combinatorial optimization model and designing a Genetic Algorithm-based evolutionary instance selection method to solve this model. The proposed framework not only overcomes the inherent limitations of the Genetic Algorithm in some high-dimensional tasks but also provides an explicit and automatic way to balance the accuracy and diversity by searching appropriate training data subsets for different component BCs. Based on obtained training data subsets, the component BCs of the ensemble model are generated sequentially, and their predictions are further combined with the weighted majority voting rule. Using 30 real datasets collected from various practical applications, such as medicine, business, and industry, the effectiveness of the proposed framework in constructing powerful ensemble models is examined and compared with six benchmark ensemble learning methods. In addition, the capability of the proposed framework to improve convergence performances is also examined by the comparison with the traditional Genetic Algorithm.

Adversarial Coreset Selection for Efficient Robust Training

It has been shown that neural networks are vulnerable to adversarial attacks: adding well-crafted, imperceptible perturbations to their input can modify their output. Adversarial training is one of the most effective approaches to training robust models against such attacks. Unfortunately, this method is much slower than vanilla training of neural networks since it needs to construct adversarial examples for the entire training data at every iteration. By leveraging the theory of coreset selection, we show how selecting a small subset of training data provides a principled approach to reducing the time complexity of robust training. To this end, we first provide convergence guarantees for adversarial coreset selection. In particular, we show that the convergence bound is directly related to how well our coresets can approximate the gradient computed over the entire training data. Motivated by our theoretical analysis, we propose using this gradient approximation error as our adversarial coreset selection objective to reduce the training set size effectively. Once built, we run adversarial training over this subset of the training data. Unlike existing methods, our approach can be adapted to a wide variety of training objectives, including TRADES, ell _p-PGD, and Perceptual Adversarial Training. We conduct extensive experiments to demonstrate that our approach speeds up adversarial training by 2–3 times while experiencing a slight degradation in the clean and robust accuracy.

Hit the gym or hit the hay: can evening exercise characteristics predict compromised sleep in healthy adults?

Introduction: Recent sleep guidelines regarding evening exercise have shifted from a conservative (i.e., do not exercise in the evening) to a more nuanced approach (i.e., exercise may not be detrimental to sleep in circumstances). With the increasing popularity of wearable technology, information regarding exercise and sleep are readily available to the general public. There is potential for these data to aid sleep recommendations within and across different population cohorts. Therefore, the aim of this study was to examine if sleep, exercise, and individual characteristics can be used to predict whether evening exercise will compromise sleep. Methods: Data regarding evening exercise and the subsequent night's sleep were obtained from 5,250 participants (1,321F, 3,929M, aged 30.1 ± 5.2yrs) using a wearable device (WHOOP 3.0). Data for females and males were analysed separately. The female and male datasets were both randomly split into subsets of training and testing data (training:testing = 75:25). Algorithms were trained to identify compromised sleep (i.e., sleep efficiency <90%) for females and males based on factors including the intensity, duration and timing of evening exercise. Results: When subsequently evaluated using the independent testing datasets, the algorithms had sensitivity for compromised sleep of 87% for females and 90% for males, specificity of 29% for females and 20% for males, positive predictive value of 32% for females and 36% for males, and negative predictive value of 85% for females and 79% for males. If these results generalise, applying the current algorithms would allow females to exercise on ~ 25% of evenings with ~ 15% of those sleeps being compromised and allow males to exercise on ~ 17% of evenings with ~ 21% of those sleeps being compromised. Discussion: The main finding of this study was that the models were able to predict a high percentage of nights with compromised sleep based on individual characteristics, exercise characteristics and habitual sleep characteristics. If the benefits of exercising in the evening outweigh the costs of compromising sleep on some of the nights when exercise is undertaken, then the application of the current algorithms could be considered a viable alternative to generalised sleep hygiene guidelines.

Predicting uniaxial compressive strength from drilling variables aided by hybrid machine learning

Awareness of uniaxial compressive strength (UCS) as a key rock formation parameter for the design and development of gas and oil field plays. It plays an essential role in the selection of the drill bits and stability of the wellbore’s wall. Precise prediction of UCS before or during the drilling, especially in exploration wellbores, is necessary to improve the drilling speed and reduce the instability of the wellbore walls. UCS predictor machine-learning (ML) models are developed in this study using drilling parameters recorded during drilling using least-squares support-vector machine (LSSVM) and multi-layer extreme learning machine (MELM) algorithms hybridized with cuckoo optimization algorithm (COA), particle swarm optimization (PSO) and genetic algorithm (GA) optimizers. In addition, stand-alone LSSVM and convolutional neural network (CNN) models without optimizer enhancements are evaluated. Drilling and petrophysical data recorded for two wells (A and B) from the Rag-e-Safid oil field in southwest Iran were compiled to form the studied dataset. UCS was initially calculated numerically based on data from laboratory tests from petrophysical logs. The Well A dataset was pre-processed to remove outlying data records by applying the quantile regression algorithm. That analysis indicated that 9 data records should be removed from the Well A dataset. A decision tree model was employed for feature selection purposes to identify the more influential variables with respect to UCS. Depth, weight on the drill bit (WOB), drill-string rotation speed (RPM), rate of penetration (ROP), and torque (Trq) were the variables identified as being highly influential on UCS values. Application of the ML models on the training data subset (75% of Well A data records) revealed that the MELM-COA algorithm achieved the lowest root mean squared error (4.6945 MPa) and a higher coefficient of determination (0.9873) value than the other models when predicting UCS in the Well A training and validation data subsets. The Well-A-trained MELM-COA model confirmed its generalizability within the studied field by generating low UCS prediction errors when applied to the independent Well B testing dataset.

A statistical modeling approach based on the small-scale field trial and meteorological data for preliminary prediction of the impact of low temperature on Eucalyptus globulus trees

Eucalyptus trees are important for industrial forestry plantations because of their high potential for biomass production, but their susceptibility to damage at low temperatures restricts their plantation areas. In this study, a 6-year field trial of Eucalyptus globulus was conducted in Tsukuba, Japan, which is the northernmost reach of Eucalyptus plantations, and leaf damage was quantitatively monitored over four of six winters. Leaf photosynthetic quantum yield (QY) levels, an indicator of cold stress-induced damage, fluctuated synchronously with temperature in the winters. We performed a maximum likelihood estimation of the regression model explaining leaf QY using training data subsets for the first 3 years. The resulting model explained QY by the number of days when the daily maximum temperature was below 9.5 °C over approximately the last 7 weeks as an explanatory variable. The correlation coefficient and coefficient of determination of prediction by the model between the predicted and observed values were 0.84 and 0.70, respectively. The model was then used to perform two kinds of simulations. Geographical simulations of potential Eucalyptus plantation areas using global meteorological data from more than 5,000 locations around the world successfully predicted an area that generally agreed with the global Eucalyptus plantation distribution reported previously. Another simulation based on meteorological data of the past 70 years suggested that global warming will increase the potential E. globulus plantation area in Japan approximately 1.5-fold over the next 70 years. These results suggest that the model developed herein would be applicable to preliminary predictions of E. globulus cold damage in the field.

Loss-aware automatic selection of structured pruning criteria for deep neural network acceleration

Structured pruning is a well-established technique for compressing neural networks, making them suitable for deployment in resource-limited edge devices. This study presents an efficient loss-aware automatic selection of structured pruning (LAASP) criteria for slimming and accelerating deep neural networks. The majority of pruning methods employ a sequential process consisting of three stages, 1) training, 2) pruning, and 3) fine-tuning, whereas the proposed pruning technique adopts a pruning-while-training approach that eliminates the first stage and integrates the second and third stages into a single cycle. The automatic selection of magnitude or similarity-based filter pruning criteria from a specified pool of criteria and the specific pruning layer at each pruning iteration is guided by the network's overall loss on a small subset of training data. To mitigate the abrupt accuracy drop due to pruning, the network is retrained briefly after each reduction of a predefined number of floating-point operations (FLOPs). The optimal pruning rates for each layer in the network are automatically determined, eliminating the need for manual allocation of fixed or variable pruning rates for each layer. Experiments on the VGGNet, ResNet, and MobileNet models on the CIFAR-10 and ImageNet benchmark datasets demonstrate the effectiveness of the proposed method. In particular, the ResNet56 and ResNet110 models on the CIFAR-10 dataset significantly improve the top-1 accuracy compared to state-of-the-art methods while reducing the network FLOPs by 52%. Furthermore, pruning the ResNet50 model on the ImageNet dataset reduces FLOPs by more than 42% with a negligible 0.33% drop in the top-5 accuracy. The source code of this study is publicly available on GitHub: https://github.com/ghimiredhikura/laasp.