Types Of Input Features Research Articles

Exploring interactive effects of environmental and microbial factors on food waste anaerobic digestion performance: Interpretable machine learning models

Biogas yield in anaerobic digestion (AD) involves continuous and complex biological reactions. The traditional linear models failed to quantitatively assess the interactive effects of these factors on AD performance. To further explore the internal relationship between target variables and AD performance, this study developed four machine learning models to predict biogas yield and consider the interaction among various factors. Results indicated that the highest prediction accuracy of AD performance was achieved by adding bacterial genera dataset with environmental factors. Random forest model exhibited the highest accuracy, with the testing coefficient of determination equal to 0.9879. Among two types of input features, the bacterial genera accounted for 89.9 % of the impact on biogas yield, followed by environmental factors. The results revealed Keratinibaculum and Acetomicrobium as critical bacteria. The volatile fatty acid controlled below 2000 mg/L and the improved stirring system in AD process were recommended to achieve maximum biogas yield.

Development of learning-based predictive models for radiation-induced atrial fibrillation in non-small cell lung cancer patients by integrating patient-specific clinical, dosimetry, and diagnostic information

Background and purposeRadiotherapy (RT) in non-small cell lung cancer (NSCLC) can induce cardiac adverse events, including atrial fibrillation (AF), despite advanced RT. This study integrates patient-specific information to develop learning-based models to predict the incidence of AF following NSCLC chemoradiotherapy (CRT) and evaluates these models using institutional and external datasets. Materials and methodsInstitutional and external patient cohorts consisted of 321 and 187 NSCLC datasets who received definitive CRT, including 17 and 6 AF incidences, respectively. The network input had 159 features with clinical, dosimetry, and diagnostic. The class imbalance was mitigated by synthetic minority oversampling technique. To handle various types of input features, machine learning-based model adopted an intervention technique that chose one feature with the largest weight at each dosimetry sub-group in feature selection process, while deep learning-based model employed a hybrid architecture assigning different types of networks to corresponding input paths. Performance was assessed by area under the curve (AUC). The key features were investigated for the machine and deep learning-based models. ResultsThe hybrid deep learning model outperformed the machine learning-based algorithm in internal validation (AUC: 0.817 vs. 0.801) and produced more consistent performance in external validation (AUC: 0.806 vs. 0.776). Importantly, maximum dose to heart and sinoatrial node (SAN) were found to be the key features for both learning-based models in external and internal validations. ConclusionsThe learning-based predictive models showed consistent prediction performance across internal and external cohorts, identifying maximum heart and SAN dose as key features for the incidence of AF.

Applications and Performance of Machine Learning Algorithms in Emergency Medical Services: A Scoping Review.

The aim of this study was to summarize the literature on the applications of machine learning (ML) and their performance in Emergency Medical Services (EMS). Four relevant electronic databases were searched (from inception through January 2024) for all original studies that employed EMS-guided ML algorithms to enhance the clinical and operational performance of EMS. Two reviewers screened the retrieved studies and extracted relevant data from the included studies. The characteristics of included studies, employed ML algorithms, and their performance were quantitively described across primary domains and subdomains. This review included a total of 164 studies published from 2005 through 2024. Of those, 125 were clinical domain focused and 39 were operational. The characteristics of ML algorithms such as sample size, number and type of input features, and performance varied between and within domains and subdomains of applications. Clinical applications of ML algorithms involved triage or diagnosis classification (n = 62), treatment prediction (n = 12), or clinical outcome prediction (n = 50), mainly for out-of-hospital cardiac arrest/OHCA (n = 62), cardiovascular diseases/CVDs (n = 19), and trauma (n = 24). The performance of these ML algorithms varied, with a median area under the receiver operating characteristic curve (AUC) of 85.6%, accuracy of 88.1%, sensitivity of 86.05%, and specificity of 86.5%. Within the operational studies, the operational task of most ML algorithms was ambulance allocation (n = 21), followed by ambulance detection (n = 5), ambulance deployment (n = 5), route optimization (n = 5), and quality assurance (n = 3). The performance of all operational ML algorithms varied and had a median AUC of 96.1%, accuracy of 90.0%, sensitivity of 94.4%, and specificity of 87.7%. Generally, neural network and ensemble algorithms, to some degree, out-performed other ML algorithms. Triaging and managing different prehospital medical conditions and augmenting ambulance performance can be improved by ML algorithms. Future reports should focus on a specific clinical condition or operational task to improve the precision of the performance metrics of ML models.

EEG Emotion Classification Based on Graph Convolutional Network

EEG-based emotion recognition is a task that uses scalp-EEG data to classify the emotion states of humans. The study of EEG-based emotion recognition can contribute to a large spectrum of application fields including healthcare and human–computer interaction. Recent studies in neuroscience reveal that the brain regions and their interactions play an essential role in the processing of different stimuli and the generation of corresponding emotional states. Nevertheless, such regional interactions, which have been proven to be critical in recognizing emotions in neuroscience, are largely overlooked in existing machine learning or deep learning models, which focus on individual channels in brain signals. Motivated by this, in this paper, we present RGNet, a model that is designed to learn the regional level representation of EEG signal for accurate emotion recognition. Specifically, after applying preprocessing and feature extraction techniques on raw signals, RGNet adopts a novel region-wise encoder to extract the features of channels located within each region as input to compute the regional level features, enabling the model to effectively explore the regional functionality. A graph is then constructed by considering each region as a node and connections between regions as edges, upon which a graph convolutional network is designed with spectral filtering and learned adjacency matrix. Instead of focusing on only the spatial proximity, it allows the model to capture more complex functional relationships. We conducted experiments from the perspective of region division strategies, region encoders and input feature types. Our model has achieved 98.64% and 99.33% for Deap and Dreamer datasets, respectively. The comparison studies show that RGNet outperforms the majority of the existing models for emotion recognition from EEG signals.

Indoor semantic segmentation based on Swin-Transformer

In recent years, with the rapid development of Transformer in the field of natural language processing, many researchers have realized its potential and gradually applied it to the field of computer vision, with a proliferation of theoretical approaches represented by Vision Transformer (ViT) and Data-efficient image Transformer (DeiT). On the basis of ViT, the famous Swin-Transformer was proposed as one of the best computer vision neural network backbones, which can be widely used in tasks such as image classification, target detection and video recognition. However, in the field of image segmentation, the semantic segmentation of indoor scenes is still very challenging due to the wide variety of objects, large differences in object sizes, and a large number of overlapping objects with occlusion. Aiming at the problem that the existing semantic segmentation of RGB-D indoor scenes cannot effectively fuse multimodal features, in this paper, we propose a novel indoor semantic segmentation algorithm based on Swin-Transformer. It attempts to apply Swin-Transformer to the field of indoor RGBD semantic segmentation, and tests the performance of the model by conducting extensive experiments on the mainstream indoor semantic segmentation datasets NYU-Depth V2 and SUN RGB-D. The experimental results show that the Swin-L RGB+Depth setting achieves 52.44% MIoU on the NYU-Depth V2 data and 51.15% MIoU on the SUN RGB-D data set, which reflects an excellent performance in the field of indoor semantic segmentation. The improved performance of the Depth features on the indoor semantic segmentation model has also been demonstrated in the experiments by controlling the type of input features. Our source code is publicly available at https://github.com/YunpingZheng/ISSSW.

Alteration assemblage characterization using machine learning applied to high-resolution drill-core images, hyperspectral data and geochemistry

Integration of multiple data types is beneficial for prediction of geological characteristics. From the perspective that geochemistry characterizes the composition of a rock mass, hyperspectral data characterizes alteration mineralogy and image feature extraction characterizes texture, most geological classifications would be well-informed by the combination of these three features. The process of meaningfully integrating distinctly sourced datasets and producing scale-relevant predictions for geological classifications involves several steps. We demonstrate a workflow to comprehensively structure and integrate these three feature families, refine training data, predict alteration classes and mitigate noise derived from scale mismatch in output predictions. The dataset, compiled from the Josemaria porphyry copper–gold deposit in Argentina, is comprised of more than 14 000 intervals of approximately 2 m, taken from 36 drillholes, where geochemistry was merged with hyperspectral mineralogy represented as tabular pixel abundances, and textural metrics extracted from core imagery, structured into the geochemical interval. Feature engineering and principal component analysis provided insights into the behaviour of the ore system during intermediate steps, as well as providing uncorrelated feature inputs for a random forest predictor. Training data were refined by producing an initial prediction, thresholding the predictions to >70% dominant class probability and using those (high-probability) samples to produce a final model encoding better constrained separation between alteration assemblages. Prediction using the final model returned an accuracy of 82.5%, as a function of model discrepancy combined with logging ambiguity and a scale mismatch between generalized logged intervals and much more granular (2 m) feature inputs. Noise reduction and generalization to the desired resolution of output was achieved by applying the multiscale multivariate continuous wavelet transform tessellation method to class membership probabilities. Ultimately, a large database of logged drill core was homogenized using empirical methodologies. The described workflow is adaptable to distinct scenarios with some modification and is apt for integrating multiple input feature types and using them to systematically define geological classifications in drillhole data.

Improving classification accuracy for separation of area under crops based on feature selection from multi-temporal images and machine learning algorithms

Timeliness, accurate, and frequent monitoring of area under crops are necessary for achieving food security. Therefore, the present study was conducted to improve the classification accuracy for the area under crops based on feature selection (FS) technique from multi-temporal Sentinel-1 and Sentinel-2 images in Shush County, Iran. To this end, the required pre-processing was performed on the images, and then features were extracted and categorized into four types of datasets with 78, 53, 65 features, for dataset 1 to 3 respectively. Dataset 4 is the result of integrating datasets 1, 2, and 3 with 194 features. The Sequential Forward Selection (SFS) method was used for the optimal FS, and Support Vector Machine (SVM) and Random Forest (RF) algorithms were used for the image classification. The results of FS by SFS method and SVM algorithm for datasets 1, 2, 3, and 4 showed that 15 features were optimal. The SFS method and RF algorithm determined the optimum feature number as 18, 19, 11, and 14 for datasets 1, 2, 3, and 4, respectively. The results of the RF algorithm performed on dataset 4 showed the highest Overall Accuracy (OA) and Kappa coefficient of 91.23% and 0.9, respectively. The results from this algorithm indicated that the type of input features affects the classification accuracy, so dataset 4 (compared to dataset 2) improved the classification with a difference in OA and kappa coefficient of approximately 2.5% and 0.03, respectively. According to the findings, it can be concluded that selecting the optimal number of features using the SFS method in the separation of area under crops is of high importance and improves the computational efficiency of the models and classification accuracy. The SFS result indicated that the higher accuracy could be achieved when less than 20 features recruited in classification.

Breast Cancer Classification using Random Forest Algorithm

This study uses detect breast cancer based on Random Forest (RF). It is crucial to diagnose the illness to identify treatment solutions closely linked to patient safety. Breast cancer is diagnosed using past medical records and various classification methods used in data mining fields today. Each technique performs differently depending on the input feature types and model parameters. Neutral Networks have been proven to be more effective in data analysis and pre-diagnosis without requiring medical knowledge. The study reduces diagnostic variance and increases diagnostic accuracy by overcoming the limitation of individual models. The Random Forest model had a training and validation accuracy of 90% and 91%.

Assessing workload in using electromyography (EMG)-based prostheses

Using prosthetic devices requires a substantial cognitive workload. This study investigated classification models for assessing cognitive workload in electromyography (EMG)-based prosthetic devices with various types of input features including eye-tracking measures, task performance, and cognitive performance model (CPM) outcomes. Features selection algorithm, hyperparameter tuning with grid search, and k-fold cross-validation were applied to select the most important features and find the optimal models. Classification accuracy, the area under the receiver operation characteristic curve (AUC), precision, recall, and F1 scores were calculated to compare the models’ performance. The findings suggested that task performance measures, pupillometry data, and CPM outcomes, combined with the naïve bayes (NB) and random forest (RF) algorithms, are most promising for classifying cognitive workload. The proposed algorithms can help manufacturers/clinicians predict the cognitive workload of future EMG-based prosthetic devices in early design phases. Practitioner summary: This study investigated the use of machine learning algorithms for classifying the cognitive workload of prosthetic devices. The findings suggested that the models could predict workload with high accuracy and low computational cost and could be used in assessing the usability of prosthetic devices in the early phases of the design process. Abbreviations: 3d: 3 dimensional; ADL: Activities for daily living; ANN: Artificial neural network; AUC: Area under the receiver operation characteristic curve; CC: Continuous control; CPM: Cognitive performance model; CPM-GOMS: Cognitive-Perceptual-Motor GOMS; CRT: Clothespin relocation test; CV: Cross validation; CW: Cognitive workload; DC: Direct control; DOF: Degrees of freedom; ECRL: Extensor carpi radialis longus; ED: Extensor digitorum; EEG: Electroencephalogram; EMG: Electromyography; FCR: Flexor carpi radialis; FD: Flexor digitorum; GOMS: Goals, Operations, Methods, and Selection Rules; LDA: Linear discriminant analysis; MAV: Mean absolute value; MCP: Metacarpophalangeal; ML: Machine learning; NASA-TLX: NASA task load index; NB: Naïve Bayes; PCPS: Percent change in pupil size; PPT: Purdue Pegboard Test; PR: Pattern recognition; PROS-TLX: Prosthesis task load index; RF: Random forest; RFE: Recursive feature selection; SHAP: Southampton hand assessment protocol; SFS: Sequential feature selection; SVC: Support vector classifier

Sensor Fusion Approach for Multiple Human Motion Detection for Indoor Surveillance Use-Case.

Multi-human detection and tracking in indoor surveillance is a challenging task due to various factors such as occlusions, illumination changes, and complex human-human and human-object interactions. In this study, we address these challenges by exploring the benefits of a low-level sensor fusion approach that combines grayscale and neuromorphic vision sensor (NVS) data. We first generate a custom dataset using an NVS camera in an indoor environment. We then conduct a comprehensive study by experimenting with different image features and deep learning networks, followed by a multi-input fusion strategy to optimize our experiments with respect to overfitting. Our primary goal is to determine the best input feature types for multi-human motion detection using statistical analysis. We find that there is a significant difference between the input features of optimized backbones, with the best strategy depending on the amount of available data. Specifically, under a low-data regime, event-based frames seem to be the preferred input feature type, while higher data availability benefits the combined use of grayscale and optical flow features. Our results demonstrate the potential of sensor fusion and deep learning techniques for multi-human tracking in indoor surveillance, although it is acknowledged that further studies are needed to confirm our findings.

Privacy-preserving Federated Learning and its application to natural language processing

State-of-the-art edge devices are capable of not only inferring machine learning (ML) models but also training them on the device with local data. When this local data is sensitive, privacy becomes a crucial property that must be addressed. This implies that sharing data with a server for training a model is undesirable and should be avoided. The Federated Learning (FL) approach can help in these situations, however, FL alone is still not the ultimate tool to solve all challenges, especially when privacy is a major concern. We propose a privacy-preserving FL framework, which leverages the concepts of bitwise quantization, local differential privacy (LDP), and feature hashing for input representation in the collaborative training of ML models. In our approach, the local model updates are first quantized, then a randomized-response technique is applied on the resulting update vector.Although our proposed framework functions with arbitrary types of input features, we emphasize its usability with natural language data. The text input on the client-side is encoded using a rolling-hash-based representation, which provides a combined solution for the high resource demands of embedding algorithms and the privacy concerns of sharing sensitive data. We evaluate our method in a sentiment analysis task using the IMDB Movie Reviews dataset as well as a rating prediction task with the MovieLens dataset augmented with additional movie keywords. We demonstrate that our approach is a feasible solution for private language processing tasks on edge devices without the use of resource-hungry language models or privacy-violating collection of client data.

Machine learning for predicting the dynamic extraction of multiple substances by emulsion liquid membranes

As of now, the models are only able to predict emulsion liquid membranes (ELM) for a single contaminant. In this study, to develop models for predicting the extraction efficiency of ELM for multiple substances, we used the linear model, random forest model, extreme gradient boosting model (XGB), and artificial neural network model. A total of 1010 data points were collected, which contained two main types of input features, the preparation conditions and operating parameters of ELM. Among these models, XGB showed excellent prediction accuracy (R2: 0.942, RMSE: 6.478, MAPE: 1.587). These evaluations showed that this model had a powerful ability to predict the extraction efficiency of ELM, and the results show that the predicted extraction efficiency was very close to the real ones. By quantifying the weightiness and marginal effect of the parameters, the models were interpreted using the Shapley additive explanation method. This work showed that the extraction time, volume ratio of organic phase to internal phase, emulsification time and feed concentration had important effects on the extraction efficiency of ELM, and the results indicated that the extraction efficiency can be promoted by increasing the values of the parameters before the optimum value, while it was inhibited after the optimum value. Our work suggested that the proposed model can be helpful for making more rational decisions in determining the preparation conditions and operating parameters of ELM.

Short-term Lake Erie algal bloom prediction by classification and regression models

The recent outbreaks of harmful algal blooms in the western Lake Erie Basin (WLEB) have drawn tremendous attention to bloom prediction for better control and management. Many weekly to annual bloom prediction models have been reported, but they only employ small datasets, have limited types of input features, build linear regression or probabilistic models, or require complex process-based computations. To address these limitations, we conducted a comprehensive literature review, complied a large dataset containing chlorophyll-a index (from 2002 to 2019) as the output and a novel combination of riverine (the Maumee & Detroit Rivers) and meteorological (WLEB) features as the input, and built machine learning-based classification and regression models for 10-d scale bloom predictions. By analyzing the feature importance, we identified 8 most important features for the HAB control, including nitrogen loads, time, water levels, soluble reactive phosphorus load, and solar irradiance. Here, both long- and short-term nitrogen loads were for the first time considered in HAB models for Lake Erie. Based on these features, the 2-, 3-, and 4-level random forest classification models achieved an accuracy of 89.6%, 77.0%, and 66.7%, respectively, and the regression model achieved an R2 value of 0.69. In addition, long-short term memory (LSTM) was implemented to predict temporal trends of four short-term features (N, solar irradiance, and two water levels) and achieved the Nash-Sutcliffe efficiency of 0.12–0.97. Feeding the LSTM model predictions for these features into the 2-level classification model reached an accuracy of 86.0% for predicting the HABs in 2017–2018, suggesting that we can provide short-term HAB forecasts even when the feature values are not available.

Leveraging data driven approaches for enhanced tsunami damage modelling: Insights from the 2011 Great East Japan event

This study aims at developing an empirical, multi-variable tsunami damage model for buildings, based on machine-learning algorithms which leverage about 250.000 ex-post data surveyed by the Japanese Ministry of Land, Infrastructure and Transportation after the 2011 Great East Japan event in the Tōhoku region. By implementing simple geospatial tools, the dataset is integrated with additional explanatory variables, including, among others, factors accounting for the mutual interaction between the inundated structures. Tests on models’ sensitivity to the number and type of input features used for model development reveal the importance, on the predictive performance, of considering usually neglected mechanisms like the shielding effect and the debris impact generation. The analysis for the potential spatial transferability indicates a reduction in the accuracy, thus suggesting a better suitability of empirical models for descriptive purposes, limiting their predictive ability only to region-specific cases.

Accelerating 3D Convolutional Neural Network with Channel Bottleneck Module for EEG-Based Emotion Recognition.

Deep learning-based emotion recognition using EEG has received increasing attention in recent years. The existing studies on emotion recognition show great variability in their employed methods including the choice of deep learning approaches and the type of input features. Although deep learning models for EEG-based emotion recognition can deliver superior accuracy, it comes at the cost of high computational complexity. Here, we propose a novel 3D convolutional neural network with a channel bottleneck module (CNN-BN) model for EEG-based emotion recognition, with the aim of accelerating the CNN computation without a significant loss in classification accuracy. To this end, we constructed a 3D spatiotemporal representation of EEG signals as the input of our proposed model. Our CNN-BN model extracts spatiotemporal EEG features, which effectively utilize the spatial and temporal information in EEG. We evaluated the performance of the CNN-BN model in the valence and arousal classification tasks. Our proposed CNN-BN model achieved an average accuracy of 99.1% and 99.5% for valence and arousal, respectively, on the DEAP dataset, while significantly reducing the number of parameters by 93.08% and FLOPs by 94.94%. The CNN-BN model with fewer parameters based on 3D EEG spatiotemporal representation outperforms the state-of-the-art models. Our proposed CNN-BN model with a better parameter efficiency has excellent potential for accelerating CNN-based emotion recognition without losing classification performance.

A universal similarity based approach for predictive uncertainty quantification in materials science

Immense effort has been exerted in the materials informatics community towards enhancing the accuracy of machine learning (ML) models; however, the uncertainty quantification (UQ) of state-of-the-art algorithms also demands further development. Most prominent UQ methods are model-specific or are related to the ensembles of models; therefore, there is a need to develop a universal technique that can be readily applied to a single model from a diverse set of ML algorithms. In this study, we suggest a new UQ measure known as the Δ-metric to address this issue. The presented quantitative criterion was inspired by the k-nearest neighbor approach adopted for applicability domain estimation in chemoinformatics. It surpasses several UQ methods in accurately ranking the predictive errors and could be considered a low-cost option for a more advanced deep ensemble strategy. We also evaluated the performance of the presented UQ measure on various classes of materials, ML algorithms, and types of input features, thus demonstrating its universality.

A machine learning method for predicting the chloride migration coefficient of concrete

• Extensive model optimizations and performance evaluations are carried out. • XGBoost predicts the chloride migration coefficient with the highest accuracy. • XGBoost outperforms other five tree-based algorithms. • Fresh density and strength tests are predictors of the chloride migration coefficient. • The models can replace traditional laboratory-based concrete chloride migration tests. This work adopts a state-of-the-art machine learning algorithm, XGBoost, to predict the chloride migration coefficient (D nssm ) of concrete. An extensive database of experimental data covering various concrete types is created by gathering from research projects and previously published studies. A total of four D nssm models are developed depending on the number and type of input features. All models are verified with unseen data using four statistical performance indicators and compared to other five tree-based algorithms. The verification results confirm that the XGBoost model predicts the D nssm with high accuracy. The model has the potential to replace cumbersome, time-consuming and resource-intensive laboratory testing.

A survey of sound source localization with deep learning methods.

This article is a survey of deep learning methods for single and multiple sound source localization, with a focus on sound source localization in indoor environments, where reverberation and diffuse noise are present. We provide an extensive topography of the neural network-based sound source localization literature in this context, organized according to the neural network architecture, the type of input features, the output strategy (classification or regression), the types of data used for model training and evaluation, and the model training strategy. Tables summarizing the literature survey are provided at the end of the paper, allowing a quick search of methods with a given set of target characteristics.

Drug response prediction in patient-derived xenografts with data augmentation and multimodal deep learning.

e13572 Background: Prediction of drug response is a critical research area in precision oncology and has been previously explored with large drug screening studies of cancer cell lines (CCLs). Patient-derived xenografts (PDXs) are an appealing platform for preclinical drug studies because the in vivo environment of PDXs helps preserve tumor heterogeneity and usually better mimics drug response of patients with cancer compared to CCLs. Methods: We investigate multimodal neural network (NN) and data augmentation for drug response prediction in PDXs. The multimodal NN learns to predict response using drug descriptors, gene expressions (GE), and histology whole-slide images (WSIs) where the multi-modality refers to tumor features only. The NN uses late integration where separate subnetworks are used to encode the input feature types before concatenation and prediction layers. Median tumor volume per treatment group is assessed relative to the control group to create a binary variable representing response. The data include twelve single-drug and 36 drug-pair treatments resulting in 2,556 single-drug and 2,203 drug-pair response values. Pathology and omics data from 487 PDXs from NCI's Patient Derived Models Repository are used as tumor feature model inputs. We explore whether the integration of WSIs with GE improves predictions as compared with models that use GE alone. We use two methods to address the limited number of response values in the dataset: 1) homogenize drug representations which allows to combine single-drug and drug-pairs into a single dataset, 2) augment drug-pair samples by switching the order of drug features which doubles the sample size of all drug-pair samples. These methods enable us to combine single-drug and drug-pair treatments which results in 6,962 responses, allowing us to train multimodal and unimodal NNs without changing architectures or the dataset. Results: Prediction performance of three unimodal NNs which use GE (um1, um2, and um3) are compared to assess the contribution of data augmentation methods. NN um1 that uses the full dataset which includes the original and the augmented drug-pair treatments as well as single-drug treatments significantly outperforms NNs (p-values < 0.01) that ignore either the augmented drug-pairs (um2) or the single-drug treatments (um3). In assessing the contribution of multimodal learning, results show that the multimodal NN (mm) outperforms both unimodal NNs that ignore either the GE (um4) or the WSIs (um1). However, the improvement of mm over um1 is not statistically significant (p-value < 0.26). Conclusions: Our results show that data augmentation and integration of histology images and GE can help improve prediction performance of drug response in PDXs.[Table: see text]

Compressed video ensemble based pseudo-labeling for semi-supervised action recognition

Some recent studies have focused on deep learning based semi-supervised learning for action recognition. However, it is difficult to scale up their training because their input is RGB frames, the obtainment of which incurs computational and storage costs. In this paper, we propose a semi-supervised action recognition method that makes it easy to scale up the training by using features stored in compressed videos. Our method directly extracts multiple types of input features from compressed videos without any decoding and generates artificial labels of unlabeled videos through the ensembling of the predictions from these features. In addition to the standard supervised training on labeled videos, our models are trained to predict the artificial labels from strongly augmented features in unlabeled compressed videos. We show that our method is more efficient and achieves a better classification performance on some widely used datasets than conventional semi-supervised learning methods applying RGB frames.