Hand-crafted Feature Extraction Research Articles

SMOTE-Based Automated PCOS Prediction Using Lightweight Deep Learning Models

Background: The reproductive age of women is particularly vulnerable to the effects of polycystic ovarian syndrome (PCOS). High levels of testosterone and other male hormones are frequent contributors to PCOS. It is believed that miscarriages and ovulation problems are majorly caused by PCOS. A recent study found that 31.3% of Asian women have been afflicted with PCOS. Healing women with life-threatening disorders associated with PCOS requires more research. In prior research, methods have involved autonomously classified PCOS using a number of different machine learning techniques. ML-based approaches involve hand-crafted feature extraction and suffer from low performance issues, which cannot be ignored for the accurate prediction and identification of PCOS. Objective: Hence, predicting PCOS using cutting-edge deep learning methods for automated feature engineering with better performance is the prime focus of this study. Methods: The proposed method suggests three lightweight (LSTM-based, CNN-based, and CNN-LSTM-based) deep learning models, incorporating SMOTE for dataset balancing to obtain a valid performance. Results: The proposed three models tend to offer an accuracy of 92.04%, 96.59%, and 94.31%, an ROC-AUC of 92.0%, 96.6%, and 94.3%, the number of parameters of 6689, 297, and 13285, and a training time of 67.27 s, 10.02 s, and 18.51 s, respectively. In addition, the DeLong test is also performed to compare AUCs to assess the statistical significance of all three models. Among all three models, the SMOTE + CNN models performs better in terms of accuracy, precision, recall, AUC, number of parameters, training time, DeLong’s p-value over the other. Conclusions: Moreover, a performance comparison is also carried out with other state-of-the-art PCOS detection studies and methods, which validates the better performance of the proposed model. Thus, the proposed model provides the greatest performance, which can lead to a reduction in the number of failed pregnancies and help in finding PCOS in the early stages.

HemoFuse: multi-feature fusion based on multi-head cross-attention for identification of hemolytic peptides.

Hemolytic peptides are therapeutic peptides that damage red blood cells. However, therapeutic peptides used in medical treatment must exhibit low toxicity to red blood cells to achieve the desired therapeutic effect. Therefore, accurate prediction of the hemolytic activity of therapeutic peptides is essential for the development of peptide therapies. In this study, a multi-feature cross-fusion model, HemoFuse, for hemolytic peptide identification is proposed. The feature vectors of peptide sequences are transformed by word embedding technique and four hand-crafted feature extraction methods. We apply multi-head cross-attention mechanism to hemolytic peptide identification for the first time. It captures the interaction between word embedding features and hand-crafted features by calculating the attention of all positions in them, so that multiple features can be deeply fused. Moreover, we visualize the features obtained by this module to enhance its interpretability. On the comprehensive integrated dataset, HemoFuse achieves ideal results, with ACC, SP, SN, MCC, F1, AUC, and AP of 0.7575, 0.8814, 0.5793, 0.4909, 0.6620, 0.8387, and 0.7118, respectively. Compared with HemoDL proposed by Yang et al., it is 3.32%, 3.89%, 5.93%, 10.6%, 8.17%, 5.88%, and 2.72% higher. Other ablation experiments also prove that our model is reasonable and efficient. The codes and datasets are accessible at https://github.com/z11code/Hemo .

Development and validation of a CT-based deep learning radiomics signature to predict lymph node metastasis in oropharyngeal squamous cell carcinoma: a multicenter study.

Lymph node metastasis (LNM) is a pivotal determinant that influences the treatment strategies and prognosis for oropharyngeal squamous cell carcinoma (OPSCC) patients. This study aims to establish and verify a deep learning (DL) radiomics model for the prediction of LNM in OPSCCs using contrast-enhanced computed tomography (CECT). A retrospective analysis included 279 OPSCC patients from three institutions. CECT images were used for handcrafted (HCR) and DL feature extraction. Dimensionality reduction for HCR features used recursive feature elimination and least absolute shrinkage and selection operator algorithms, whereas DL feature dimensionality reduction used variance-threshold and recursive feature elimination algorithms. Radiomics signatures were constructed using support vector machine, decision tree, random forest, k-nearest neighbor, gaussian naive bayes classifiers and light gradient boosting machine. A combined model was then constructed using the screened DL, HCR, and clinical features. The area under the receiver operating characteristic curve (AUC) served to quantify the model's performance, and calibration curves were utilized to assess its calibration. The combined model exhibited robust performance, achieving AUC values of 0.909 (95% CI: 0.861-0.957) in the training cohort, 0.884 (95% CI: 0.800-0.968) in the internal validation cohort, and 0.865 (95% CI: 0.791-0.939) in the external validation cohort. It outperformed both the clinical model and best-performing radiomics model. Moreover, calibration was deemed satisfactory. The combined model based on CECT demonstrates the potential to predict LNM in OPSCCs preoperatively, offering a valuable tool for more precise and tailored treatment strategies. This study presents a novel combined model integrating clinical factors with deep learning radiomics, significantly enhancing preoperative LNM prediction in OPSCC.

Phasor-Based Myoelectric Synergy Features: A Fast Hand-Crafted Feature Extraction Scheme for Boosting Performance in Gait Phase Recognition.

Gait phase recognition systems based on surface electromyographic signals (EMGs) are crucial for developing advanced myoelectric control schemes that enhance the interaction between humans and lower limb assistive devices. However, machine learning models used in this context, such as Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM), typically experience performance degradation when modeling the gait cycle with more than just stance and swing phases. This study introduces a generalized phasor-based feature extraction approach (PHASOR) that captures spatial myoelectric features to improve the performance of LDA and SVM in gait phase recognition. A publicly available dataset of 40 subjects was used to evaluate PHASOR against state-of-the-art feature sets in a five-phase gait recognition problem. Additionally, fully data-driven deep learning architectures, such as Rocket and Mini-Rocket, were included for comparison. The separability index (SI) and mean semi-principal axis (MSA) analyses showed mean SI and MSA metrics of 7.7 and 0.5, respectively, indicating the proposed approach's ability to effectively decode gait phases through EMG activity. The SVM classifier demonstrated the highest accuracy of 82% using a five-fold leave-one-trial-out testing approach, outperforming Rocket and Mini-Rocket. This study confirms that in gait phase recognition based on EMG signals, novel and efficient muscle synergy information feature extraction schemes, such as PHASOR, can compete with deep learning approaches that require greater processing time for feature extraction and classification.

NeuroPpred-SHE: An interpretable neuropeptides prediction model based on selected features from hand-crafted features and embeddings of T5 model

Neuropeptides are the most ubiquitous neurotransmitters in the immune system, regulating various biological processes. Neuropeptides play a significant role for the discovery of new drugs and targets for nervous system disorders. Traditional experimental methods for identifying neuropeptides are time-consuming and costly. Although several computational methods have been developed to predict the neuropeptides, the accuracy is still not satisfactory due to the representability of the extracted features. In this work, we propose an efficient and interpretable model, NeuroPpred-SHE, for predicting neuropeptides by selecting the optimal feature subset from both hand-crafted features and embeddings of a protein language model. Specially, we first employed a pre-trained T5 protein language model to extract embedding features and twelve other encoding methods to extract hand-crafted features from peptide sequences, respectively. Secondly, we fused both embedding features and hand-crafted features to enhance the feature representability. Thirdly, we utilized random forest (RF), Max-Relevance and Min-Redundancy (mRMR) and eXtreme Gradient Boosting (XGBoost) methods to select the optimal feature subset from the fused features. Finally, we employed five machine learning methods (GBDT, XGBoost, SVM, MLP, and LightGBM) to build the models. Our results show that the model based on GBDT achieves the best performance. Furthermore, our final model was compared with other state-of-the-art methods on an independent test set, the results indicate that our model achieves an AUROC of 97.8 % which is higher than all the other state-of-the-art predictors. Our model is available at: https://github.com/wenjean/NeuroPpred-SHE.

Evaluation of Hand-Crafted Feature Extraction for Fault Diagnosis in Rotating Machinery: A Survey.

This article presents a comprehensive collection of formulas and calculations for hand-crafted feature extraction of condition monitoring signals. The documented features include 123 for the time domain and 46 for the frequency domain. Furthermore, a machine learning-based methodology is presented to evaluate the performance of features in fault classification tasks using seven data sets of different rotating machines. The evaluation methodology involves using seven ranking methods to select the best ten hand-crafted features per method for each database, to be subsequently evaluated by three types of classifiers. This process is applied exhaustively by evaluation groups, combining our databases with an external benchmark. A summary table of the performance results of the classifiers is also presented, including the percentage of classification and the number of features required to achieve that value. Through graphic resources, it has been possible to show the prevalence of certain features over others, how they are associated with the database, and the order of importance assigned by the ranking methods. In the same way, finding which features have the highest appearance percentages for each database in all experiments has been possible. The results suggest that hand-crafted feature extraction is an effective technique with low computational cost and high interpretability for fault identification and diagnosis.

From sleep patterns to heart rhythm: Predicting atrial fibrillation from overnight polysomnograms

BackgroundAtrial fibrillation (AF) is often asymptomatic and thus under-observed. Given the high risks of stroke and heart failure among patients with AF, early prediction and effective management are crucial. Given the prevalence of obstructive sleep apnea among AF patients, electrocardiogram (ECG) analysis from polysomnography (PSG) offers a unique opportunity for early AF prediction. Our aim is to identify individuals at high risk of AF development from single‑lead ECGs during standard PSG. MethodsWe analyzed 18,782 single‑lead ECG recordings from 13,609 subjects undergoing PSG at the Massachusetts General Hospital sleep laboratory. AF presence was identified using ICD-9/10 codes. The dataset included 15,913 recordings without AF history and 2054 recordings from patients diagnosed with AF between one month to fifteen years post-PSG. Data were partitioned into training, validation, and test cohorts ensuring that individual patients remained exclusive to each cohort. The test set was held out during the training process.We employed two different methods for feature extraction to build a final model for AF prediction: Extraction of hand-crafted ECG features and a deep learning method. For extraction of ECG-hand-crafted features, recordings were split into 30-s windows, and those with a signal quality index (SQI) below 0.95 were discarded. From each remaining window, 150 features were extracted from the time, frequency, time-frequency domains, and phase-space reconstructions of the ECG. A compilation of 12 statistical features summarized these window-specific features per recording, resulting in 1800 features (12 × 150).A pre-trained deep neural network from the PhysioNet Challenge 2021 was updated using transfer learning to discriminate recordings with and without AF. The model processed PSG ECGs in 16-s windows to generate AF probabilities, from which 13 statistical features were extracted. Combining 1800 features from feature extraction with 13 from the deep learning model, we performed a feature selection and subsequently trained a shallow neural network to predict future AF and evaluated its performance on the test cohort. ResultsOn the test set, our model exhibited sensitivity, specificity, and precision of 0.67, 0.81, and 0.3, respectively, for AF prediction. Survival analysis revealed a hazard ratio of 8.36 (p-value: 1.93 × 10−52) for AF outcomes using the log-rank test. ConclusionsOur proposed ECG analysis method, utilizing overnight PSG data, shows promise in AF prediction despite modest precision, suggesting false positives. This approach could enable low-cost screening and proactive treatment for high-risk patients. Refinements, including additional physiological parameters, may reduce false positives, enhancing clinical utility and accuracy.

RoBERTa, ResNeXt and BiLSTM with self-attention: The ultimate trio for customer sentiment analysis

Sentiment analysis of customer feedback is a pivotal component of Natural Language Processing (NLP), enabling businesses to gauge consumer emotions towards their products and services. The inherent variability of language introduces substantial challenges in the analysis of unstructured customer data. Sentiment analysis techniques generally fall into two categories: traditional methods, which involve the extraction of hand-crafted features and the application of machine learning algorithms, and end-to-end deep learning approaches that process raw data, transforming them layer by layer into high-level textual features. Each method presents its own balance between simplicity and speed versus analytical power and flexibility, and they all grapple with the need for extensive labeled data and computational resources. To address these challenges, this paper presents an innovative hybrid model that amalgamates RoBERTa, ResNeXt, BiLSTM, and self-attention mechanisms. This integration capitalizes on the collective strengths of these models to surmount the limitations inherent in each. The proposed model proves to be a powerful and efficient tool for sentiment analysis, demonstrating proficiency across various data types and tasks. We undertake a comprehensive evaluation of the proposed model’s accuracy and training efficiency using four benchmark datasets. Our investigation also explores the impact of different similarity measures and convolutional neural network architectures on the model’s performance. The results affirm that our model not only achieves high accuracy but also significantly reduces training time compared to RoBERTa’s fine-tuning. Furthermore, the model exhibits exceptional domain adaptability, particularly when fine-tuned on the IMDb dataset following initial training on the Yelp Review Full dataset. The practicality of the proposed model is underscored by its reduced computational complexity and its adeptness at navigating semantic and syntactic nuances.

A self-supervised embedding of cell migration features for behavior discovery over cell populations

Background and objective:Recent studies point out that the dynamics and interaction of cell populations within their environment are related to several biological processes in immunology. Hence, single-cell analysis in immunology now relies on spatial omics. Moreover, recent literature suggests that immunology scenarios are hierarchically organized, including unknown cell behaviors appearing in different proportions across some observable control and therapy groups. These dynamic behaviors play a crucial role in identifying the causes of processes such as inflammation, aging, and fighting off pathogens or cancerous cells. In this work, we use a self-supervised learning approach to discover these behaviors associated with cell dynamics in an immunology scenario. Materials and methods:Specifically, we study the different responses of control group and therapy groups in a scenario involving inflammation due to infarct, with a focus on neutrophil migration within blood vessels. Starting from a set of hand-crafted spatio-temporal features, we use a recurrent neural network to generate embeddings that properly describe the dynamics of the migration processes. The network is trained using a novel multi-task contrastive loss that, on the one hand, models the hierarchical structure of our scenario (groups-behaviors-samples) and, on the other, ensures temporal consistency within the embedding, enforcing that subsequent temporal samples obtained from a given cell stay close in the latent space. Results:Our experimental results demonstrate that the resulting embeddings improve the separability of cell behaviors and log-likelihood of the therapies, when compared to the hand-crafted feature extraction and recent methods from the state of the art, even with dimensionality reduction (16 vs. 21 hand-crafted features). Conclusions:Our approach enables single-cell analyses at a population level, being able to automatically discover shared behaviors among different groups. This, in turn, enables the prediction of the therapy effectiveness based on their proportions within a study group.

An image processing approach for fatigue crack identification in cellulose acetate replicas

The cellulose acetate replication technique is an important method for studying material fatigue. However, extracting accurate information from pictures of cellulose replicas poses challenges because of distortions and numerous artifacts. This paper presents an image processing procedure for effective fatigue crack identification in plastic replicas. The approach employs thresholding, adaptive Gaussian thresholding, and Otsu binarization to convert gray-scale images into binary ones, enhancing crack visibility. Morphological operations refine object shapes, and Connected Components Analysis facilitates crack identification. Despite limited data, the handcrafted feature extraction algorithm proves robust, addressing challenges. The algorithm shows efficacy in detecting cracks as small as 30 μm, even in the presence of cellulose replication artifacts. The results highlight ability to capture significant cracks’ orientation, length, and growth stages, essential for understanding fatigue mechanisms. Analysis of results, especially evaluation metrics encompassing false positives and false negatives, provides a comprehensive understanding of the algorithm’s strengths and limitations. The proposed tool is available on GitHub.

User identification across online social networks based on gated multi-feature extraction

User identification is an essential technical support for downstream tasks such as recommendation systems, information retrieval, and collaborative filtering. Computing the similarity between user display names through classifiers is an effective solution for user identification across social networks. However, there are two problems with existing methods. Applying expert domain knowledge to extract handcrafted features of display names ignores the semantic information, resulting in poor performance of these methods. Selecting influential display names, and handcrafted features in user identification problems are also one of the difficulties. To solve these two problems, we propose a method based on the multi-feature fusion of display names using gated units. First, we extract the deep semantic features of display names through the BERT pre-trained multi-language model. Then, the gated mechanism is applied to select the handcrafted features we extracted to retain the essential features. Then, the adaptive factors are used to fuse handcrafted features and deep features to obtain user identification results across social networks. Finally, the efficiency of our model is verified on three constructed real-world multilingual display names datasets across multiple online social networks and compared with existing state-of-the-art methods. Experimental results show that the proposed algorithm outperforms the compared algorithms.

A Dataset for Kinship Estimation from Image of Hand Using Machine Learning

Kinship (Familial relationships) detection is crucial in many fields and has applications in biometric security, adoption, forensic investigations, and more. It is also essential during wars and natural disasters like earthquakes since it may aid in reunion, missing person searches, establishing emergency contacts, and providing psychological support. The most common method of determining kinship is DNA analysis which is highly accurate. Another approach, which is noninvasive, uses facial photos with computer vision and machine learning algorithms for kinship estimation. Each part of the Human -body has its own embedded information that can be extracted and adopted for identification, verification, or classification of that person. Kinship recognition is based on finding traits that are shared by every family. We investigate the use of hand geometry for kinship detection, which is a new approach. Because of the available hand image Datasets do not contain kinship ground truth; therefore, we created our own dataset. This paper describes the tools, methodology, and details of the collected MKH, which stands for the Mosul Kinship Hand, images dataset. The images of MKH dataset were collected using a mobile phone camera with a suitable setup and consisted of 648 images for 81 individuals from 14 families (8 hand situations per person). This paper also presents the use of this dataset in kinship prediction using machine learning. Google MdiaPipe was used for hand detection, segmentation, and geometrical key points finding. Handcraft feature extraction was used to extract 43 distinctive geometrical features from each image. A neural network classifier was designed and trained to predict kinship, yielding about 93% prediction accuracy. The results of this novel approach demonstrated that the hand possesses biometric characteristics that may be used to establish kinship, and that the suggested method is a promising way as a kinship indicator.

Computer-Aided Detection and Diagnosis of Breast Cancer: a Review

Statistics across different countries point to breast cancer being among severe cancers with a high mortality rate. Early detection is essential when it comes to reducing the severity and mortality of breast cancer. Researchers proposed many computer-aided diagnosis/detection (CAD) techniques for this purpose. Many perform well (over 90% of classification accuracy, sensitivity, specificity, and f-1 sore), nevertheless, there is still room for improvement. This paper reviews literature related to breast cancer and the challenges faced by the research community. It discusses the common stages of breast cancer detection/ diagnosis using CAD models along with deep learning and transfer learning (TL) methods. In recent studies, deep learning models outperformed the handcrafted feature extraction and classification task and the semantic segmentation of ROI images achieved good results. An accuracy of up to 99.8% has been obtained using these techniques. Furthermore, using TL, researchers combine the power of both, pre-trained deep learning-based networks and traditional feature extraction approaches.

Reducing False Alarms in Wearable Seizure Detection With EEGformer: A Compact Transformer Model for MCUs.

The long-term, continuous analysis of electroencephalography (EEG) signals on wearable devices to automatically detect seizures in epileptic patients is a high-potential application field for deep neural networks, and specifically for transformers, which are highly suited for end-to-end time series processing without handcrafted feature extraction. In this work, we propose a small-scale transformer detector, the EEGformer, compatible with unobtrusive acquisition setups that use only the temporal channels. EEGformer is the result of a hardware-oriented design exploration, aiming for efficient execution on tiny low-power micro-controller units (MCUs) and low latency and false alarm rate to increase patient and caregiver acceptance.Tests conducted on the CHB-MIT dataset show a 20% reduction of the onset detection latency with respect to the state-of-the-art model for temporal acquisition, with a competitive 73% seizure detection probability and 0.15 false-positive-per-hour (FP/h). Further investigations on a novel and challenging scalp EEG dataset result in the successful detection of 88% of the annotated seizure events, with 0.45 FP/h.We evaluate the deployment of the EEGformer on three commercial low-power computing platforms: the single-core Apollo4 MCU and the GAP8 and GAP9 parallel MCUs. The most efficient implementation (on GAP9) results in as low as 13.7 ms and 0.31 mJ per inference, demonstrating the feasibility of deploying the EEGformer on wearable seizure detection systems with reduced channel count and multi-day battery duration.

Multi-task contrastive learning for change detection in remote sensing images

ABSTRACT This letter proposes a novel multi-task contrastive learning (MTCL) approach for change detection of high-resolution remote sensing images. Current self-supervised learning approaches have limitations in utilizing multiview information of remote sensing images, and they cannot directly train the change detection backbone network for feature learning. The investigated method exploits multiview information contained in remote sensing images to directly train the change detection backbone network. Specifically, multiview views are constructed from high-resolution remote sensing images by handcrafted feature extraction approach, which contain complementary information from different domains. Then each constructed view is utilized for contrastive learning, and the multi-task learning strategy of parameter sharing is used to learn more rich and robust information. Furthermore, instead of learning feature representations from single remote sensing image, the proposed approach directly utilizes image pairs to train the change detection backbone, further improving the performance of change detection network. Two benchmark datasets are employed for comparative experiments, and experimental results certify the effectiveness and superiority of the investigated approach.

Multi-head CNN-based activity recognition and its application on chest-mounted sensor-belt

In recent years, a great deal of research has been done on the identification, monitoring, and classification of human activities. Human activity recognition (HAR) is a term commonly used to describe the automatic identification of physical activities. For activity recognition, there are primarily vision-based and sensor-based methods available. The computer vision-based method is generally effective in lab settings, but because of clutter, fluctuating light levels, and contrast, it may not perform well in real-world scenarios. Continuous monitoring and analysis of physiological signals obtained from heterogeneous sensors attached to an individual’s body is required to realise sensor-based HAR systems. Most of the previous research in human activity recognition (HAR) is biased along with feature engineering and pre-processing which requires a good amount of domain knowledge. Application-specific modelling and time-taking methods are involved in these approaches. In this work, the multi-head convolutional neural network-based human activity recognition framework is proposed where automatic feature extraction and classification are involved in the form of an end-to-end classification approach. Experiments of this approach are performed by taking raw wearable sensor data with few pre-processing steps and without the involvement of a handcrafted feature extraction technique. 99.23% and 93.55% accuracy are obtained on the WISDM and UCI-HAR datasets which denoted the much improvement in the assessment of HAR over other similar approaches. The model is also tested on locally collected data from a chest mounted belt with fabric sensors and an accuracy of 87.14% has been achieved on that data.

Content-based fabric recommendation system integrating image and text information

The increasing popularity of e-commerce and online fashion platforms has led to a growing demand for effective fabric recommendation systems. Choosing the suitable fabric is crucial to achieving desired aesthetic outcomes, ensuring product quality, and enhancing user satisfaction in the fashion design and textile manufacturing industries. Although our research group has made significant progress in fabric retrieval, we have encountered several challenges in practical collaborations with industry partners. These challenges include users’ difficulty expressing their preferences and the lack of personalization in fabric retrieval. We proposed a content-based fabric recommendation system that combines image and text information to address these challenges. Our approach employed various methods to extract hand-crafted features, such as image color and texture, to characterize the visual communication of fabrics. We combined this with text information to provide a more comprehensive fabric representation. By mining user preferences and employing personalized recommendation algorithms, our system delivered personalized fabric recommendations to cater to individual needs. Experimentation with comparison texts, colors, and textures separately or together. According to the assessment results, the highest F1 value of 83% can be achieved when the color, texture, and text combine with the color: texture: text = 1:1:1 and the filter value = 2.2. Precision and recall rates are also 83% at this point. Our proposed fabric recommendation system is useful and efficient.

SNC_Net: Skin Cancer Detection by Integrating Handcrafted and Deep Learning-Based Features Using Dermoscopy Images

The medical sciences are facing a major problem with the auto-detection of disease due to the fast growth in population density. Intelligent systems assist medical professionals in early disease detection and also help to provide consistent treatment that reduces the mortality rate. Skin cancer is considered to be the deadliest and most severe kind of cancer. Medical professionals utilize dermoscopy images to make a manual diagnosis of skin cancer. This method is labor-intensive and time-consuming and demands a considerable level of expertise. Automated detection methods are necessary for the early detection of skin cancer. The occurrence of hair and air bubbles in dermoscopic images affects the diagnosis of skin cancer. This research aims to classify eight different types of skin cancer, namely actinic keratosis (AKs), dermatofibroma (DFa), melanoma (MELa), basal cell carcinoma (BCCa), squamous cell carcinoma (SCCa), melanocytic nevus (MNi), vascular lesion (VASn), and benign keratosis (BKs). In this study, we propose SNC_Net, which integrates features derived from dermoscopic images through deep learning (DL) models and handcrafted (HC) feature extraction methods with the aim of improving the performance of the classifier. A convolutional neural network (CNN) is employed for classification. Dermoscopy images from the publicly accessible ISIC 2019 dataset for skin cancer detection is utilized to train and validate the model. The performance of the proposed model is compared with four baseline models, namely EfficientNetB0 (B1), MobileNetV2 (B2), DenseNet-121 (B3), and ResNet-101 (B4), and six state-of-the-art (SOTA) classifiers. With an accuracy of 97.81%, a precision of 98.31%, a recall of 97.89%, and an F1 score of 98.10%, the proposed model outperformed the SOTA classifiers as well as the four baseline models. Moreover, an Ablation study is also performed on the proposed method to validate its performance. The proposed method therefore assists dermatologists and other medical professionals in early skin cancer detection.

Speech-driven head motion generation from waveforms

Head motion generation task for speech-driven virtual agent animation is commonly explored with handcrafted audio features, such as MFCCs as input features, plus additional features, such as energy and F0 in the literature. In this paper, we study the direct use of speech waveform to generate head motion. We claim that creating a task-specific feature from waveform to generate head motion leads to better performance than using standard acoustic features to generate head motion overall. At the same time, we completely abandon the handcrafted feature extraction process, leading to more effectiveness. However, the difficulty of creating a task-specific feature from waveform is their staggering quantity of irrelevant information, implicating potential cumbrance for neural network training. Thus, we apply a canonical-correlation-constrained autoencoder (CCCAE), where we are able to compress the high-dimensional waveform into a low-dimensional embedded feature, with the minimal error in reconstruction, and sustain the relevant information with the maximal cannonical correlation to head motion. We extend our previous research by including more speakers in our dataset and also adapt with a recurrent neural network, to show the feasibility of our proposed feature. Through comparisons between different acoustic features, our proposed feature, WavCCCAE, shows at least a 20% improvement in the correlation from the waveform, and outperforms the popular acoustic feature, MFCC, by at least 5% respectively for all speakers. Through the comparison in the feedforward neural network regression (FNN-regression) system, the WavCCCAE-based system shows comparable performance in objective evaluation. In long short-term memory (LSTM) experiments, LSTM-models improve the overall performance in normalised mean square error (NMSE) and CCA metrics, and adapt the WavCCCAEfeature better, which makes the proposed LSTM-regression system outperform the MFCC-based system. We also re-design the subjective evaluation, and the subjective results show the animations generated by models where WavCCCAEwas chosen to be better than the other models by the participants of MUSHRA test.

Deep Image Feature Learning With Fuzzy Rules

Feature extraction methods are key to many image processing tasks. At present, the most popular method is to use a deep neural network to automatically extract features through end-to-end training instead of the traditional hand-crafted feature extraction. However, the training of deep neural network relies heavily on data quality and quantity, and the network is a black-box model that has poor interpretability. Human intelligence can be leveraged here to improve the deep neural network model, where the human decision process can be integrated in feature learning and object classification to enhance robustness and interpretability. In this paper, the method Deep Image Feature Learning with Fuzzy Rules (DIFL-FR) is proposed, where human decision process is embedded in feature extraction by combining fuzzy logic rule-based modeling with deep-stacked learning strategy. The proposed method has the following distinctive characteristics. First, since the method is based on fuzzy sets and fuzzy inference, it can extract more robust features from noisy data scenes. Second, the method progressively learns the image features through a layer-by-layer approach based on fuzzy rules, so that the feature learning process can be better explained by the rules generated. Third, the learning process of the method has a high efficiency since it is only based on forward propagation without back propagation and iterative learning. Finally, while the method is based on unsupervised learning, it can be easily extended to supervised and semi-supervised learning cases. The results of extensive experiments conducted on image datasets of different scales clearly show the effectiveness of the proposed method.