Lower Training Time Research Articles

Application of machine learning based models in computer network data

Abstract Computer networks, as one of the indispensable infrastructures in today’s world, play an extremely important role in industrial production and daily life. This paper revolves around the intelligent diagnosis of computer network data faults using machine learning methods. Firstly, the support vector machine network fault minimization principle is proposed based on the machine learning model, which leads to under-learning or over-learning when the samples are limited, or there is noise. Then, a loss function is introduced for the under-learning or over-learning problem to ensure that the support vector machine can accurately achieve fault diagnosis. Finally, the evaluation index of computational network fault diagnosis is constructed for the experimental purpose, and four algorithms are selected as the experimental control group to analyze the data. It is obtained that SVM and ANN models have high DR and low FAR. Their DR is 87.9% and 84.5%, respectively, while their FAR is only 5.4%. This further validates the superiority of SVM in computer network data fault detection. This study possesses low training time complexity and can overcome the problem of uneven distribution of the number of faulty and normal samples in network fault diagnosis to some extent.

Toward IoT Node Authentication Mechanism in Next Generation Networks

Although the next generation networks (5G-NGNs) provide a flexible infrastructure to support latency-sensitive and bandwidth-hungry mission-critical Internet of Things (IoT) applications, however, the 5G-IoT integration in NGNs has increased the threat surface. Unfortunately, IoT devices are resource constrained, and the traditional Intrusion Detection Systems (IDS) approaches based on cryptography are not effective on 5G-IoT ecosystems. In this paper, we propose an effective 5GIoT node authentication approach that leverages unique RF (Radio Frequency) fingerprinting data to train the Deep learning model to detect legitimate and non-legitimate IoT nodes. Our approach is based on Mahalanobis Distance theory and Chisquare distribution theories. The proposed approach achieves a higher detection accuracy (99.35%) as well as lower training time compared to other existing approaches which is a key benefit of our approach in NGNs. The experiments are conducted using ETSI-Open Source NFV Management and Orchestration (OSM-MANO) platform on AWS (Amazon Web Services) cloud platform to verify how the proposed approach would fit in reallife scenarios. The method can be used as a standalone security system or as a part of multi-factor authentication.

V-A3tS: A rapid text steganalysis method based on position information and variable parameter multi-head self-attention controlled by length

Text length varies in social networks, such as IMDB long text, Twitter short text, and long–short mixed text. For these complex situations, the time series, convolution, or fine-tuned BERT models are used in existing text steganalysis methods almost. However, these methods do not simultaneously consider higher detection accuracy and lower training time at the same time. To alleviate this dilemma, this paper proposes a novel text steganalysis method. First, the proposed method maps words into a semantic space containing position information. Second, a variable parameter attention layer scaled appropriately according to text length is designed, it achieves the purpose that the entire parameter amount of the model is not redundant and can ensure effective detection. Finally, the steganalysis features are enhanced by the residual linear layer. For long, short, and mixed text datasets, comparing experiments show that the proposed method has higher detection accuracy, fewer parameters, and shorter training time than existing methods. Among them, the advantage of this method is more obvious for long and mixed texts.

Unbalance Detection in Induction Motors through Vibration Signals Using Texture Features

The detection of faults in induction motors has been one of the main challenges to the industry in recent years. An effective fault detection method is fundamental to ensure operational security and productivity. Different models for intelligent fault diagnosis have been recently proposed. However, not all of them are accessible for some manufacturing processes because of the black-box approach, the complexity of hyperparameter tuning, high-dimensionality feature vectors, and the need for sophisticated computational resources. In this paper, a method for the detection of an unbalance fault in induction motors based on a low-dimensional feature vector and a low-complexity classification approach is proposed. The feature vector presented in this manuscript is based on texture features, which are a basic tool for image processing and image understanding. Nevertheless, texture features have not been explored as a powerful instrument for induction motor fault analysis. In this approach, texture features are used to analyze a set of vibration signals belonging to two different classes: an unbalanced motor and a healthy motor. Training-validation and testing stages are developed to build and evaluate the performance of the classifier, respectively. The results show higher accuracy and lower training time in comparison with different state-of-the-art approaches.

Fault diagnosis of reciprocating machinery based on improved MEEMD-SqueezeNet

Reciprocating machinery structure is complex, and its signal is nonlinear and nonstationary. Because of the complex working environment, the signal usually contains noise interference. Modified Ensemble Empirical Mode Decomposition(MEEMD) reduces residual noise by introducing permutation entropy. But it directly eliminates components larger than the threshold, which may lose the effective information of the original signal. Deep learning models can solve nonlinear problems. But with the number of network layers increases, it increases the number of parameters, resulting in long model training time. Based on the above analysis, this paper proposes a fault diagnosis of reciprocating machinery based on improved MEEMD-SqueezeNet. By introducing multi-scale permutation entropy to replace the original single-scale permutation entropy, the calculated entropy contains more scale information. Singular value decomposition (SVD) is carried out to denoise the signal larger than the threshold to retain more effective information in the original signal. Convert the noise reduced 1D time series to 2D images by Gram Angle Field(GAF) to preserve the temporal correlation of 1D vibration data. Combined with the idea of Inception series model, an InFire module is proposed to replace the original Fire model of SqueezeNet. Use the improved SqueezeNet to extract features from the generated 2D images, and finally input the softmax classifier for classification to obtain fault diagnosis results. The simulation experiment and noise reduction comparison experiment prove that the improved MEEMD method proposed in this paper can effectively achieve signal noise reduction and improve the overall average accuracy of diagnosis from 89.49% to 94.35%. The fault classification comparison experiments prove that the accuracy of improved SqueezeNet is basically the same as that of VGG19. Compared with SqueezeNet, the accuracy of improved Squeezenet is improved by 1.77%, and the number of model parameters is reduced by nearly 29.4%. In terms of training time, it is proved that the training time of the improved SqueezeNet is basically the same as that of SqueezeNet, accounting for only one-third of the training time of VGG19. The improved SqueezeNet not only ensures lower training time, but also improves the accuracy of SqueezeNet. The above experimental results show that the proposed method reduces the model training time and improves the fault diagnosis accuracy of reciprocating machinery.

Localization of spleen and kidney organs from CT scans based on classification of slices in rotational views

This article presents a novel multiple organ localization and tracking technique applied to spleen and kidney regions in computed tomography images. The proposed solution is based on a unique approach to classify regions in different spatial projections (e.g., side projection) using convolutional neural networks. Our procedure merges classification results from different projection resulting in a 3D segmentation. The proposed system is able to recognize the contour of the organ with an accuracy of 88–89% depending on the body organ. Research has shown that the use of a single method can be useful for the detection of different organs: kidney and spleen. Our solution can compete with U-Net based solutions in terms of hardware requirements, as it has significantly lower demands. Additionally, it gives better results in small data sets. Another advantage of our solution is a significantly lower training time on an equally sized data set and more capabilities to parallelize calculations. The proposed system enables visualization, localization and tracking of organs and is therefore a valuable tool in medical diagnostic problems.

Generalized stacked LSTM for the seismic damage evaluation of ductile reinforced concrete buildings

AbstractTo organize accurate and effective emergency responses after an earthquake, it is vital to conduct an early and precise assessment of damage to structures. The use of fragility/vulnerability curves is an advanced evaluation approach for structural damage assessments. However, the analysis based on fragility curves significantly varies depending on soil conditions, ground motion, and structural characteristics. To overcome this issue, a stacked long short‐term memory network was proposed in this research. Unlike previous studies, two input features (acceleration time history in the form of vector and the number of stories in the scalar) are utilized to generalize the results for the same plan building frames with different stories. Three different approaches are presented in this work to link the ground motion time history with the number of stories (2, 4, 8, 12, and 20 stories) in the reinforced concrete building frame, and the networks were tested for unknown ground motions. Of the three approaches, those providing good results were selected for further analysis. For the approaches chosen, the network architectures were changed to a diamond shape and an autoencoder‐like shape with more hidden units (to obtain higher accuracy), which were tested for unknown same plan layout frames. The accuracy obtained using these approaches was significantly high (80%–90%) with a low training time. The proposed model is compared with other techniques and shows significant accuracy. The suggested networks exhibited a number of scenarios for estimating the damage state for unknown ground motions, as well as for unknown frames with various stories. Moreover, the capability of the networks to handle more scalar input features is examined by adding them probabilistically; with additional input variables, the networks predicted the damage state with higher accuracy.

A Deep Learning-based Fast Fake News Detection Model for Cyber-Physical Social Services

With the prevalence of social network service in cyber-Physical space, flow of various fake news has been a rough issue for operators of social service. Although many theoretical outcomes have been produced in recent years, they are generally challenged by processing speed of semantic modeling. To solve this issue, this paper presents a deep learning-based fast fake news detection model for cyber-physical social services. Taking Chinese text as the objective, each character in Chinese text is directly adopted as the basic processing unit. Considering the fact that the news are generally short texts and can be remarkably featured by some keywords, convolution-based neural computing framework is adopted to extract feature representation for news texts. Such design is able to ensure both processing speed and detection ability in scenes of Chinese short texts. At last, some experiments are conducted for evaluation on a real-world dataset collected from a Chinese social media. The results show that the proposal possesses lower training time cost as well as higher classification accuracy compared with baseline methods.

Improving binary crow search algorithm for feature selection

Abstract The feature selection (FS) process has an essential effect in solving many problems such as prediction, regression, and classification to get the optimal solution. For solving classification problems, selecting the most relevant features of a dataset leads to better classification accuracy with low training time. In this work, a hybrid binary crow search algorithm (BCSA) based quasi-oppositional (QO) method is proposed as an FS method based on wrapper mode to solve a classification problem. The QO method was employed in tuning the value of flight length in the BCSA which is controlling the ability of the crows to find the optimal solution. To evaluate the performance of the proposed method, four benchmark datasets have been used which are human intestinal absorption, HDAC8 inhibitory activity (IC50), P-glycoproteins, and antimicrobial. Accordingly, the experimental results are discussed and compared against other standard algorithms based on the accuracy rate, the average number of selected features, and running time. The results have proven the robustness of the proposed method relied on the high obtained value of accuracy (84.93–95.92%), G-mean (0.853–0.971%), and average selected features (4.36–11.8) with a relatively low computational time. Moreover, to investigate the effectiveness of the proposed method, Friedman test was used which declared that the performance supremacy of the proposed BCSA-QO with four datasets was very evident against BCSA and CSA by selecting the minimum relevant features and producing the highest accuracy classification rate. The obtained results verify the usefulness of the proposed method (BCSA-QO) in the FS with classification in terms of high classification accuracy, a small number of selected features, and low computational time.

Multi-Reservoir Echo State Networks with Hodrick–Prescott Filter for nonlinear time-series prediction

The Echo State Network (ESN) is a representative model for reservoir computing, which is capable of high-speed model training for machine learning tasks with time series data. Extended models of the ESN, such as Multi-Reservoir ESNs (MRESNs), have been intensively studied for performance improvement in recent years. In this study, we propose a new model called an HP-MRESN by combining an MRESN with the Hodrick–Prescott (HP) filter for nonlinear time series prediction. The proposed HP-MRESN comprises three basic components: a time series decomposer, a reservoir state extractor, and an ensemble decoder. In the time series decomposer, we recursively leverage the HP filter to decompose original time-series data into multiple trend and cycle components. In the reservoir state extractor, each time series component is fed into a corresponding reservoir-state encoder for generating a reservoir state which is extracted as it is or through the principal component analysis. In the ensemble decoder, the states of multiple reservoirs are collected and processed to produce model outputs. Moreover, we propose a greedy algorithm to automatically find the best model architectures under designated hyperparameters for different prediction tasks. Experimental results on a total of 24 nonlinear time-series prediction tasks with 6 real-world datasets demonstrate that our proposed HP-MRESN not only can outperform some existing representative MRESN models and fully-trained RNN models but also can have relatively low training time. In addition, performance comparisons between the HP-MRESN and related MRESN models with other prepossessing methods show the benefit of time series decompositions using the HP filter. The codes of the proposed method are publicly available on https://github.com/Ziqiang-IRCN/HP-MRESN.

Efficient Transformer based Sentiment Classification Models

Recently, transformer models have gained significance as a state-of-the art technique for sentiment prediction based on text. Attention mechanism of transformer model speeds up the training process by allowing modelling of dependencies without regard to their distance in the input or output sequences. There are two types of transformer models – transformer base models and transformer large models. Since the implementation of large transformer models need better hardware and more training time, we propose new simpler models or weak learners with lower training time for sentiment classification in this work. These models enhance the speed of performance without compromising the classification accuracy. The proposed Efficient Transformer-based Sentiment Classification (ETSC) models are built by setting configuration of large models as minimum, shuffling dataset randomly and experimenting with various percentages of training data. Early stopping and smaller batch size in training techniques improve the accuracy of the proposed model. The proposed models exhibit promising performance in comparison with existing transformer-based sentiment classification models in terms of speed and accuracy.

SGD-Based Cascade Scheme for Higher Degrees Wiener Polynomial Approximation of Large Biomedical Datasets

The modern development of the biomedical engineering area is accompanied by the availability of large volumes of data with a non-linear response surface. The effective analysis of such data requires the development of new, more productive machine learning methods. This paper proposes a cascade ensemble that combines the advantages of using a high-order Wiener polynomial and Stochastic Gradient Descent algorithm while eliminating their disadvantages to ensure a high accuracy of the approximation of such data with a satisfactory training time. The work presents flow charts of the learning algorithms and the application of the developed ensemble scheme, and all the steps are described in detail. The simulation was carried out based on a real-world dataset. Procedures for the proposed model tuning have been performed. The high accuracy of the approximation based on the developed ensemble scheme was established experimentally. The possibility of an implicit approximation by high orders of the Wiener polynomial with a slight increase in the number of its members is shown. It ensures a low training time for the proposed method during the analysis of large datasets, which provides the possibility of its practical use in the biomedical engineering area.

Comparative Evaluation of Data-Driven Approaches to Develop an Engine Surrogate Model for NOx Engine-Out Emissions under Steady-State and Transient Conditions

In this paper, a methodology based on data-driven models is developed to predict the NOx emissions of an internal combustion engine using, as inputs, a set of ECU channels representing the main engine actuations. Several regressors derived from the machine learning and deep learning algorithms are tested and compared in terms of prediction accuracy and computational efficiency to assess the most suitable for the aim of this work. Six Real Driving Emission (RDE) cycles performed at the roll bench were used for the model training, while another two RDE cycles and a steady-state map of NOx emissions were used to test the model under dynamic and stationary conditions, respectively. The models considered include Polynomial Regressor (PR), Support Vector Regressor (SVR), Random Forest Regressor (RF), Light Gradient Boosting Regressor (LightGBR) and Feed-Forward Neural Network (ANN). Ensemble methods such as Random Forest and LightGBR proved to have similar performances in terms of prediction accuracy, with LightGBR requiring a much lower training time. Afterwards, LightGBR predictions are compared with experimental NOx measurements in steady-state conditions and during two RDE cycles. Coefficient of determination (R2), normalized root mean squared error (nRMSE) and mean average percentage error (MAPE) are the main metrics used. The NOx emissions predicted by the LightGBR show good coherence with the experimental test set, both with the steady-state NOx map (R2 = 0.91 and MAPE = 6.42%) and with the RDE cycles (R2 = 0.95 and nRMSE = 0.04).

Intelligent Monitoring Model for Fall Risks of Hospitalized Elderly Patients.

Early detection of high fall risk is an important process of fall prevention in hospitalized elderly patients. Hospitalized elderly patients can face several falling risks. Monitoring systems can be utilized to protect health and lives, and monitoring models can be less effective if the alarm is not invoked in real time. Therefore, in this paper we propose a monitoring prediction system that incorporates artificial intelligence. The proposed system utilizes a scalable clustering technique, namely the Catboost method, for binary classification. These techniques are executed on the Snowflake platform to rapidly predict safe and risky incidence for hospitalized elderly patients. A later stage employs a deep learning model (DNN) that is based on a convolutional neural network (CNN). Risky incidences are further classified into various monitoring alert types (falls, falls with broken bones, falls that lead to death). At this phase, the model employs adaptive sampling techniques to elucidate the unbalanced overfitting in the datasets. A performance study utilizes the benchmarks datasets, namely SERV-112 and SV-S2017 of the image sequences for assessing accuracy. The simulation depicts that the system has higher true positive counts in case of all health-related risk incidences. The proposed system depicts real-time classification speed with lower training time. The performance of the proposed multi-risk prediction is high at 87.4% in the SERV-112 dataset and 98.71% in the SV-S2017 dataset.

Forecasting the crowd: An effective and efficient neural network for citywide crowd information prediction at a fine spatio-temporal scale

Modelling and forecasting citywide crowd information (e.g., crowd volume of a region, the inflow of crowds into a region, outflow of crowds from a region) at a fine spatio-temporal scale is crucial for urban and transport planning, city management, public safety, and traffic management. However, this is a challenging task due to its complex spatial and temporal dependences. This paper proposes an effective and efficient model to reduce the training time cost while maintaining predictive accuracy in forecasting citywide crowd information at a fine spatio-temporal scale. Our model integrates Gated Recurrent Unit (GRU), convolutional neural network (CNN), and k-nearest neighbors (k-NN) to jointly capture the spatial and temporal dependences between two regions in a city. The evaluation with two different datasets in two different cities shows that compared to the state-of-the-art baselines, our model has better predictive accuracy (reducing the mean absolute errors MAEs by 20.99% on average) and a lower training time cost (reducing the time cost to only 26.16% on average of that of the baselines). Our model also has better abilities in making accurate predictions with low time cost under the influences of large-scale special events (when massive crowds of people are gathering in a short time) and for regions with high and irregular crowd changes. In summary, our model is an effective, efficient, and reliable method for forecasting citywide crowd information at a fine spatio-temporal scale, and has a high potential for many applications, such as city management, public safety, and transportation.

Design and Performance Analyses of Evacuated U-Tube Solar Collector Using Data-Driven Machine Learning Models

Abstract In this study, the implementation of data-driven machine learning (ML) models for design analysis and performance prediction of evacuated U-tube solar collectors (ETSCs) is investigated for the first time. Although evacuated U-tube solar collectors are widely investigated both numerically and experimentally, the implementation of data-driven machine learning models as an efficient predictive tool was not explored for the ETSC. So, to fill this literature gap, seven ML models such as linear regression with repeated K-fold cross-validation (LR), K-nearest neighbors (KNNs), principal component analysis (PCA), partial least-square regression-I (PLSR-I), partial least-square regression-II (PLSR-II), support vector regression (SVR), and stochastic gradient descent regression (SGDR) are employed using three hundred experimental data points and are reported in the literature. The heat transfer fluid outlet temperature (Thtf,o), thermal energy gained by heat transfer fluid (Q˙htf), and ETSC efficiency (ɳETSC) are considered as output/performance parameters. The outcome of the predicted results suggests that the SGDR ML model is superior in predicting all the performance parameters showing R2 values of 0.98, 0.981, and 0.99 for “Thtf,o,” “Q˙htf,” and “ɳETSC,” respectively. Moreover, the KNN ML model showed weaker performance for predicting the output parameters. In addition, it is observed that the SGDR ML model has a low training time of 0.45 s when compared to other ML models. For the given operating range, the predicted optimal performance parameters such as “Thtf,o,” “Q˙htf,” and “ɳETSC” obtained from the SGDR ML model are 45 °C, 0.44 kW, and 71%, respectively. Furthermore, the recommendations and shortcomings associated with the ML models for the design and performance optimization of ETSC are also presented in detail.

Rethinking ImageNet Pre-training for Computational Histopathology.

Transfer learning from ImageNet pretrained weights is widely used when training Deep Learning models on a Histopathology dataset. However, the visual features of the two domains are different. Rather than ImageNet pretrained weights, pre-training on a Histopathology dataset may provide better initialization. To prove this hypothesis, we train two commonly used Deep Learning model architectures - ResNet and DenseNet on a complex Histopathology classification dataset, and compare transfer learning performance with ImageNet pretrained weights. Based on the fine-tuning on three histopathology datasets including two different stains (H&E and IHC), we show that the domain specific pretrained weights are better suited for transfer learning. This is reflected by higher performance, lower training time as well as better feature reuse. Clinical Relevance - The paper establishes merit of using Histopathology domain specific pretrained weights rather than ImageNet pretrained weights.

Text to speech using Mel-Spectrogram with deep learning algorithms

The purpose of text to speech (TTS), sometimes called speech synthesis, is to synthesize a natural and intelligible speech for a given text. A wide range of applications uses TTS technologies in media, chatbots, and entertainment, among other fields, making it a hot topic for the research community. Recently, the progress achieved by artificial intelligence, especially in deep learning and neural networks, enables TTS to produce a high-quality synthesized speech. However, despite the success achieved, currently, available works suffer from the need for very long training and inference time, which makes it dominated by big tech companies. This paper proposes a model based on convolutional neural networks (CNN) and gated recurrent units (GRU). The proposed model can work even in low computational environments and requires low training time. The MOS achieved is 4.26, higher than the MOS performed by state-of-the-art methods.

Transfer Learning-Based Artificial Neural Network for Forward Kinematic Estimation of 6-DOF Robot

Transfer Learning (TL) can significantly lower training time and reduce dependency on a large number of target domain datasets. Such an approach is still not exploited for robotic prediction tasks. Currently, a TL based Artificial Neural Network (ANN) is explored and validated for robotic forward kinematics estimation of a 6-DOF robot. The robotic positions are estimated from the available joint angle information. The 6-R MTAB Aristo-XT robot is selected as a case study to generate the target experimental training and testing data for validation of ML techniques. While, the PUMA 560 6-DOF robot is used as a source for prior training of the ANN model. Standard performance measures such as learning error, deviation error and Mean Square Error (MSE) are evaluated and graphical illustrations are presented for fair comparison of the results. Experimental results reveal that, instead of ANN, the TL-ANN is strongly suggested to improve the training time of ANN regressor, and it also reduces the randomness and improves the accuracy as compared to its counterpart.

Trainable segmentation for transmission electron microscope images of inorganic nanoparticles.

We present a trainable segmentation method implemented within the python package ParticleSpy. The method takes user labelled pixels, which are used to train a classifier and segment images of inorganic nanoparticles from transmission electron microscope images. This implementation is based on the trainable Waikato Environment for Knowledge Analysis (WEKA) segmentation, but is written in python, allowing a large degree of flexibility and meaning it can be easily expanded using other python packages. We find that trainable segmentation offers better accuracy than global or local thresholding methods and requires as few as 100 user-labelled pixels to produce an accurate segmentation. Trainable segmentation presents a balance of accuracy and training time between global/local thresholding and neural networks, when used on transmission electron microscope images of nanoparticles. We also quantitatively investigate the effectiveness of the components of trainable segmentation, its filter kernels and classifiers, in order to demonstrate the use cases for the different filter kernels in ParticleSpy and the most accurate classifiers for different data types. A set of filter kernels is identified that are effective in distinguishing particles from background but that retain dissimilar features. In terms of classifiers, we find that different classifiers perform optimally for different image contrast; specifically, a random forest classifier performs best for high-contrast ADF images, but that QDA and Gaussian Naïve Bayes classifiers perform better for low-contrast TEM images.