Hyperparameter Selection Research Articles

An ensemble CNN-LSTM and GRU adaptive weighting model based improved sparrow search algorithm for predicting runoff using historical meteorological and runoff data as input

Accurate prediction of river runoff is of great significance for water resources management, flood prevention and mitigation. The causes of runoff are complex and the mechanisms behind them are difficult to grasp. Building a data-driven deep learning model for runoff prediction is an effective solution. To achieve the fusion of multi-source information, prediction accuracy and wide applicability, a hybrid model based on CNN-LSTM & GRU-ISSA is proposed in this study. In this paper, meteorological data, hydrological data and runoff data are selected and the maximum information coefficient (MIC) is used to calculate the relationship between each variable and runoff in order to reduce the dimensionality of the data. A convolutional neural network (CNN) is used to extract features of the long time series of runoff and a long short-term memory network (LSTM) is used for the prediction of the long time series of runoff. A gated recurrent unit (GRU) is also used for the short time series prediction of runoff. In order to extract the advantages of both prediction models, an adaptive weighting module (AWM) is proposed to dynamically learn the outputs of both models and combine them into the final prediction results. To be able to solve the selection of model hyperparameters, we use the improved sparrow search algorithm (ISSA). This algorithm introduces two improvement points, Lévy Flight and Sine Cosine Algorithm, based on the Sparrow Search Algorithm (SSA), to achieve fast convergence of the algorithm with better global optimal solutions. The proposed model was validated using watersheds with different runoff ranges, e.g., in the Bailong River watershed, the NSE value was 0.90 and the RMSE value was 2.17. The results showed that the proposed model significantly outperformed the other baseline models.

Inverse estimation of a vertical current velocity profile using motions of an FPSO and artificial neural network

This study utilizes artificial neural network (ANN) models to inversely estimate the near-real-time 3D ocean vertical current profile (magnitudes and directions) using floater-mounted motion sensors with (or without) two bi-axial inclinometers on the top portion of a riser. A coupled time-domain dynamics model that takes into account the interaction between the FPSO (Floating Production Storage and Offloading platform), mooring lines, risers, and cables is established to provide inputs and outputs for ANN models. The dynamics simulations are performed using the 2-year-long measured environmental (wind, wave, current) conditions. Then, three ANN models with different combinations of input variables are evaluated, and the process of hyperparameter selection is conducted to increase the prediction accuracy compared to the actually measured current profiles. The root-mean-square errors (RMSEs) and R-squared of current speed, the contour plots of measured and estimated current speed and direction, representative motions of the FPSO, and inclination and azimuthal angles of a riser are systematically presented. The results demonstrate that the proposed ANN model adequately estimates the 3D current profile from FPSO motions while the accuracy especially at deeper depth is improved by considering bi-axial inclinometers on the riser as additional inputs.

Bayesian optimized GoogLeNet based respiratory signal prediction model from empirically decomposed gammatone visualization

Discrimination of normal and adventitious respiratory sounds from stethoscope auscultation by human ears is challenging owing to low frequency characteristics and varying frequency range for inspiration and expiration. This makes the diagnosis of pulmonary disorders a subjective one relying on the experience and hearing capability of the physician. Most computer assisted diagnosis systems formulated to address these limitations fail to capture the inherent acoustic property of respiratory sounds close to human auditory system. To circumvent this problem, this study exploits the gammatone filter banks to evaluate the distribution of all frequency components present in the signal. To categorize the respiratory signals, GoogLeNet based Convolutional Neural Networks (CNN) prediction model is developed through Time-Frequency (TF) visualization of gammatone cepstral coefficients acquired from decomposed Intrinsic Mode Functions (IMFs) in an empirical manner. As the performance of the CNN model is greatly dependent on the learning environment, this study also tends to optimize the values of the hyper parameters to enhance the classification performance of the CNN model. Accordingly, the optimal values of initial learning rate, L2 regularization and momentum are identified for both Stochastic Gradient Descent Momentum (SGDM) and Adaptive Momentum (ADAM) optimizer using Bayesian optimization technique. Experimentation is carried out for three TF visualization methods viz. spectrograms, scalograms and Constant Q spectrograms. Evaluated results show that scalogram method of classification yields high accuracy of 87.37% and 88.32% for default selection of hyperparameters using SGDM and ADAM optimizers and with optimal selection of hyperparameters based on objective function reveal an improved accuracy of 93.68 % and 95.67% for SGDM and ADAM optimizers.

Pelican Optimization Algorithm with Deep Learning for Aspect based Sentiment Analysis on Asian Low Resource Languages

Recently, sentiment analysis (SA) has become more popular as it is crucial to moderate and examine the data from the internet. It contains several applications, such as social media monitoring, market research, and opinion mining. Aspect Based Sentiment Analysis (ABSA) is a domain of SA that manages sentiment at a better level. ABSA classifies sentiment in terms of all the aspects for obtaining superior insights as sentiment expressed. A major contribution has been developed in ABSA, and then this progress can be restricted only to some languages with suitable resources. One common method is to utilize machine learning (ML) approaches, namely Neural Networks (NN), Support Vector Machines (SVM), and Naive Bayes (NB), together with Asian and low language-specific resources. These resources offer data on the sentiment polarity (neutral, positive, or negative) of phrases and words that are generally utilized in low-resource languages. In this aspect, this study develops a new Pelican Optimization Algorithm with Deep Learning for ABSA (POADL-ABSA) on Asian and Low Resource Languages. The proposed POADL-ABSA technique focuses on the detection and classification of sentiments. To accomplish this, the POADL-ABSA technique encompasses various levels of operations such as pre-processed, feature vector conversion, and classification. In addition, the POADL-ABSA technique employs the BERT model for feature vector extraction. Besides, attention-based bi-directional long short-term memory (ABiLSTM) system was used for the recognition and classification of sentiments. Finally, the POA was utilized for optimum hyperparameter selection of the ABiLSTM model, and it helps in attaining enhanced sentiment classification results. To ensure the improvised performance of the IAOADL-ABSA technique, an extensive experimental outcome the IAOADL-ABSA technique surpassed other models with acc{u}_y , pre{c}_n , rec{a}_l , and {F}_{score} of 98.72%, 98.71%, 98.72%, and 98.71%, respectively. Therefore, the IAOADL-ABSA technique can be employed for accurate classification results.

Manifold learning for fMRI time-varying functional connectivity.

Whole-brain functional connectivity (FC) measured with functional MRI (fMRI) evolves over time in meaningful ways at temporal scales going from years (e.g., development) to seconds [e.g., within-scan time-varying FC (tvFC)]. Yet, our ability to explore tvFC is severely constrained by its large dimensionality (several thousands). To overcome this difficulty, researchers often seek to generate low dimensional representations (e.g., 2D and 3D scatter plots) hoping those will retain important aspects of the data (e.g., relationships to behavior and disease progression). Limited prior empirical work suggests that manifold learning techniques (MLTs)-namely those seeking to infer a low dimensional non-linear surface (i.e., the manifold) where most of the data lies-are good candidates for accomplishing this task. Here we explore this possibility in detail. First, we discuss why one should expect tvFC data to lie on a low dimensional manifold. Second, we estimate what is the intrinsic dimension (ID; i.e., minimum number of latent dimensions) of tvFC data manifolds. Third, we describe the inner workings of three state-of-the-art MLTs: Laplacian Eigenmaps (LEs), T-distributed Stochastic Neighbor Embedding (T-SNE), and Uniform Manifold Approximation and Projection (UMAP). For each method, we empirically evaluate its ability to generate neuro-biologically meaningful representations of tvFC data, as well as their robustness against hyper-parameter selection. Our results show that tvFC data has an ID that ranges between 4 and 26, and that ID varies significantly between rest and task states. We also show how all three methods can effectively capture subject identity and task being performed: UMAP and T-SNE can capture these two levels of detail concurrently, but LE could only capture one at a time. We observed substantial variability in embedding quality across MLTs, and within-MLT as a function of hyper-parameter selection. To help alleviate this issue, we provide heuristics that can inform future studies. Finally, we also demonstrate the importance of feature normalization when combining data across subjects and the role that temporal autocorrelation plays in the application of MLTs to tvFC data. Overall, we conclude that while MLTs can be useful to generate summary views of labeled tvFC data, their application to unlabeled data such as resting-state remains challenging.

Optimizing EMG Classification through Metaheuristic Algorithms

This work proposes a metaheuristic-based approach to hyperparameter selection in a multilayer perceptron to classify EMG signals. The main goal of the study is to improve the performance of the model by optimizing four important hyperparameters: the number of neurons, the learning rate, the epochs, and the training batches. The approach proposed in this work shows that hyperparameter optimization using particle swarm optimization and the gray wolf optimizer significantly improves the performance of a multilayer perceptron in classifying EMG motion signals. The final model achieves an average classification rate of 93% for the validation phase. The results obtained are promising and suggest that the proposed approach may be helpful for the optimization of deep learning models in other signal processing applications.

Prediction using multi-objective slime mould algorithm optimized support vector regression model

Different from the traditional neural network models based on empirical risk minimization principle, support vector regression (SVR) minimizes the upper limit of generalization error by the structural risk minimization principle. SVR is suitable for solving nonlinear, small-sample, high-dimensional modelling, and pattern recognition problems. However, the selection of hyperparameters of SVR can significantly affect the prediction accuracy and computational time. This paper proposes a method to optimize the SVR hyperparameters through an intelligent optimization algorithm, specifically multi-objective slime mould algorithm (MOSMA). MOSMA is a new multi-objective optimization algorithm with strong global search ability and fast convergence. The principle and calculation process of MOSMA are first described in detail. The analytical method for transforming the SVR hyperparameter optimization problem into a multi-objective problem is then presented. The process of optimizing two SVR hyperparameters (the penalty coefficient and radial basis function kernel parameter) based on MOSMA is described in steps. Meanwhile, the procedures of algorithm parameter setting, fitness function design, and population update are given. To investigate the characteristics and performance of MOSMA-SVR, two datasets are used to predict the vibration trend of the spindle of CNC milling machine in one and the maximum bending normal stress in the cutting process in the other. The performance of MOSMA-SVR is evaluated by multiple statistical indexes and compared with seven other prediction models (GA-SVR, PSO-SVR, ABC-SVR, GWO-SVR, SSA-SVR, SMA-SVR and MOWOA-SVR).

APPLICATION OF MACHINE LEARNING ALGORITMS FOR HEART DISEASE PREDICTION

This paper focuses on the application of machine learning algorithms to predict cardiovascular diseases (CVDs). Every year a large number of deaths are registered all over the world. According to the World Health Organisation, CVDs are the leading cause of high mortality in the world. One of the necessary preventive measures to reduce mortality from CVDs is the timely prediction of diseases in people at high risk of such diseases. Specially developed scales and machine learning algorithms are now being used for the timely prediction of CVDs.
 Background. To predict heart disease, algorithms are often used: naive Bayesian classifier (Gaussian Naïve Bayes Classificator, GNBC), k-nearest neighbours (K-Nearest Neghboors, KNN), decision tree (Decision Tree, DT). In domestic literature, there are known works devoted to the application of SWD prediction using Adam gradient algorithm in deep neural network training. One of the necessary conditions for increasing the predictive ability of a machine learning model (MLM) is the optimal selection of hyperparameters. The choice of the optimal hyperparameters is often made on the basis of empirical experience.
 Purpose. To explore the specific application of machine algorithms to the prediction of heart disease.
 Materials and methods. Scientific novelty of the work. In this research we analyse machine learning algorithms for predicting the risk of CVDs using the approach of automatic search for hyperparameters MMO. The following algorithms were used to construct MMOs: NBS, KNN, DT, Logistic Regression, Support Vector Machine (SVM), Random Foorest (RF), Complement Naïve Bayes Classificator (CNBC), Linear Discriminant Analysis (LDA), Radial Basic Function (RBF), Gradient Boost (XGBoost). To evaluate the accuracy of machine learning models we used the following indicators: mean absolute error (MAE), precision, completeness (recall), F-measure (F-beta), False Positive Rate (FPR), False Negative Rate (FNR). Additionally, visual analysis of ROC curve (receiver operating characteristic) and areas under the curve (areas under the curve, AUC) were used to analyse the results of MMO. Using AUC value allows to estimate prognostic ability of MLM.
 Results. The training results showed that RF and XGBoost algorithms are characterized by higher accuracy. With optimal selection of MMO parameters, the overall classification accuracy was 0.88 and 0.94 respectively.
 Conclusion. The application of machine learning algorithms allows predictive models to be built with high accuracy. This requires the construction of a machine learning model. The ensemble machine learning algorithms RF and XGBoost have higher accuracy rates than the following algorithms: decision trees, Bayesian classification methods, logistic regression, linear discriminant analysis.

IMPLEMENTATION OF TECHNOLOGY FOR IMPROVING THE QUALITY OF SEGMENTATION OF MEDICAL IMAGES BY SOFTWARE ADJUSTMENT OF CONVOLUTIONAL NEURAL NETWORK HYPERPARAMETERS

Background. The scientists have built effective convolutional neural networks in their research, but the issue of optimal setting of the hyperparameters of these neural networks remains insufficiently researched. Hyperparameters affect model selection. They have the greatest impact on the number and size of hidden layers. Effective selection of hyperparameters improves the speed and quality of the learning algorithm. It is also necessary to pay attention to the fact that the hyperparameters of the convolutional neural network are interconnected. That is why it is very difficult to manually select the effective values of hyperparameters, which will ensure the maximum efficiency of the convolutional neural network. It is necessary to automate the process of selecting hyperparameters, to implement a software mechanism for setting hyperparameters of a convolutional neural network. The author has successfully implemented the specified task.
 Objective. The purpose of the paper is to develop a technology for selecting hyperparameters of a convolutional neural network to improve the quality of segmentation of medical images..
 Methods. Selection of a convolutional neural network model that will enable effective segmentation of medical images, modification of the Keras Tuner library by developing an additional function, use of convolutional neural network optimization methods and hyperparameters, compilation of the constructed model and its settings, selection of the model with the best hyperparameters.
 Results. A comparative analysis of U-Net and FCN-32 convolutional neural networks was carried out. U-Net was selected as the tuning network due to its higher quality and accuracy of image segmentation. Modified the Keras Tuner library by developing an additional function for tuning hyperparameters. To optimize hyperparameters, the use of the Hyperband method is justified. The optimal number of epochs was selected - 20. In the process of setting hyperparameters, the best model with an accuracy index of 0.9665 was selected. The hyperparameter start_neurons is set to 80, the hyperparameter net_depth is 5, the activation function is Mish, the hyperparameter dropout is set to False, and the hyperparameter bn_after_act is set to True.
 Conclusions. The convolutional neural network U-Net, which is configured with the specified parameters, has a significant potential in solving the problems of segmentation of medical images. The prospect of further research is the use of a modified network for the diagnosis of symptoms of the coronavirus disease COVID-19, pneumonia, cancer and other complex medical diseases.

The Poisson distribution model fits UMI-based single-cell RNA-sequencing data

BackgroundModeling of single cell RNA-sequencing (scRNA-seq) data remains challenging due to a high percentage of zeros and data heterogeneity, so improved modeling has strong potential to benefit many downstream data analyses. The existing zero-inflated or over-dispersed models are based on aggregations at either the gene or the cell level. However, they typically lose accuracy due to a too crude aggregation at those two levels.ResultsWe avoid the crude approximations entailed by such aggregation through proposing an independent Poisson distribution (IPD) particularly at each individual entry in the scRNA-seq data matrix. This approach naturally and intuitively models the large number of zeros as matrix entries with a very small Poisson parameter. The critical challenge of cell clustering is approached via a novel data representation as Departures from a simple homogeneous IPD (DIPD) to capture the per-gene-per-cell intrinsic heterogeneity generated by cell clusters. Our experiments using real data and crafted experiments show that using DIPD as a data representation for scRNA-seq data can uncover novel cell subtypes that are missed or can only be found by careful parameter tuning using conventional methods.ConclusionsThis new method has multiple advantages, including (1) no need for prior feature selection or manual optimization of hyperparameters; (2) flexibility to combine with and improve upon other methods, such as Seurat. Another novel contribution is the use of crafted experiments as part of the validation of our newly developed DIPD-based clustering pipeline. This new clustering pipeline is implemented in the R (CRAN) package scpoisson.

Leveraging the meta-embedding for text classification in a resource-constrained language

This paper proposes an intelligent text classification framework for a resource-constrained language like Bengali, which is considered a challenging task due to the lack of standard corpora, appropriate hyper-parameter tuning method, and pre-trained language-specific embedding. The proposed framework comprises an average meta-embedding feature fusion module and a convolutions neural network module called AVG-M+CNN. This work also proposes an algorithm, i.e., automatic hyperparameter tuning and selection, for enhancing the performance of the AVG-M+CNN technique. All meta-embedding models are evaluated using the intrinsic, e.g., semantic, syntactic, relatedness word similarity, analogy tasks and extrinsic evaluators. The intrinsic evaluator evaluates 200 Bengali semantic, syntactic and relatedness word pairs. Spearman (ρˆ), Pearson (rˆ) and cosine similarity correlations are used to evaluate 18 individual embedding and 9 meta-embedding models. The 3COSADD and 3COSMUL evaluators evaluate the 300 analogy tasks. The extrinsic evaluator evaluates a total of 156 classification models on four corpora: BARD, IndicNLP, Prothom-Alo and BTCC11 (a newly developed corpus having eleven distinct categories). Among these, the AVG-M+CNN model achieves the highest accuracy regarding four Bengali corpora: 95.92±.001% for BARD, 93.10±.001% for Prothom-Alo, 90.07±.001% for BTCC11 and 87.44±.001% for IndicNLP, respectively.

Bayesian optimization of multi-layer perceptron models for power distribution system state estimation

Abstract Feedforward multilayer perceptron models (MLPs) have been applied to power distribution system state estimation (DSSE) in the past. Existing methods usually employ an ad-hoc or trial and error approach to MLP hyperparameter selection, and thus a systematic way of selecting the optimal hyperparameters including the number of neurons per hidden layer, learning rate, number of training epochs and training batch size is desirable and needed. This paper presents an approach based on Bayesian Optimization with Gaussian Processes for selecting MLP model hyperparameters for state estimation purposes. Results of the optimized MLP models are presented alongside the unoptimized models to compare performance of training, testing, and validation in terms of root-mean-squared-error (RMSE). Additionally, machine learning pipelines were employed and total execution time (seconds) for each trial is presented. The study shows that the MLP models obtained through the proposed optimization method outperform unoptimized models in terms of generalization capability for unseen, new cases.

Improved bacterial foraging optimization with deep learning based anomaly detection in smart cities

The Internet of Things (IoT) contains many smart devices that collect, store, communicate, and process data. IoT implementation has performed novel opportunities in industries, environments, businesses, and homes. Anomaly detection (AD) is helpful in IoT platforms that can recognize and prevent potential system failures, decrease downtime, enhance the quality of products and services, and improve overall operational efficacy. AD systems for IoT data contain statistical modelling, deep learning (DL), and machine learning (ML) approaches which detect patterns and anomalies in the data. This article introduces an Improved Bacterial Foraging Optimization with optimum deep learning for Anomaly Detection (IBFO-ODLAD) in the IoT network. The presented IBFO-ODLAD technique performs data normalization using Z-score normalization approach. For the feature selection process, the IBFO-ODLAD technique designs the IBFO algorithm to choose an optimal subset of features. In addition, the IBFO-ODLAD technique uses multiplicative long short term memory (MLSTM) model for intrusion detection and classification process. Furthermore, the Bayesian optimization algorithm (BOA) was executed for the optimum hyperparameter selection of the MLSTM model. The experimental outcome of the IBFO-ODLAD method was validated on the UNSW NB-15 dataset and UCI SECOM dataset. The experimental outcomes signified the improved performance of the IBFO-ODLAD algorithm with maximum accuracy of 98.89 % and 98.66 % validated on the UNSW NB-15 dataset and UCI SECOM dataset respectively.

Stochastic Gradient Descent Introduces an Effective Landscape-Dependent Regularization Favoring Flat Solutions.

Generalization is one of the most important problems in deep learning, where there exist many low-loss solutions due to overparametrization. Previous empirical studies showed a strong correlation between flatness of the loss landscape at a solution and its generalizability, and stochastic gradient descent (SGD) is crucial in finding the flat solutions. To understand the effects of SGD, we construct a simple model whose overall loss landscape has a continuous set of degenerate (or near-degenerate) minima and the loss landscape for a minibatch is approximated by a random shift of the overall loss function. By direct simulations of the stochastic learning dynamics and solving the underlying Fokker-Planck equation, we show that due to its strong anisotropy the SGD noise introduces an additional effective loss term that decreases with flatness and has an overall strength that increases with the learning rate and batch-to-batch variation. We find that the additional landscape-dependent SGD loss breaks the degeneracy and serves as an effective regularization for finding flat solutions. As a result, the flatness of the overall loss landscape increases during learning and reaches a higher value (flatter minimum) for a larger SGD noise strength before the noise strength reaches a critical value when the system fails to converge. These results, which are verified in realistic neural network models, elucidate the role of SGD for generalization, and they may also have important implications for hyperparameter selection for learning efficiently without divergence.

Enhanced bare-bones particle swarm optimization based evolving deep neural networks

In this research, we propose a variant of the Bare-Bones Particle Swarm Optimization (BBPSO) algorithm for hyper-parameter selection and deep architecture generation for image, audio and video classification tasks. Since the search process of the original BBPSO model is guided by a single leader and the particles’ personal best experiences, there is a lack of interactions pertaining to the neighbouring elite solutions. To overcome this limitation, we propose a versatile search process for a modified BBPSO model that incorporates a number of effective components and operations. These include the neighbouring and global best signals, search actions with Cauchy/Levy scale factors, sub-dimension operations guided by the local and global elite solutions, and a Levy-driven local search mechanism. Moreover, root-finding algorithms are employed which use informative mathematical principles to estimate new root offspring for leader/particle enhancement. A reinforcement learning algorithm is subsequently used to identify the optimal sequential deployment of these numerical analysis methods to increase robustness. Several medical imaging data sets, i.e., ISIC 2017, PH2 and Dermofit skin lesion databases, the ALL-IDB2 microscopic blood image data set, the MURA musculoskeletal radiographic database, the CK + facial expression data set, as well as the Coswara respiratory audio data set and UCF101 video action data set, are employed for evaluation. The proposed BBPSO-optimized Convolutional Neural Network (CNN), bidirectional Long Short-Term Memory (BiLSTM) with attention mechanism, and CNN-BiLSTM models outperform those devised by other PSO and BBPSO variants, as well as state-of-the-art existing studies, significantly, for image, audio respiratory abnormality and realistic video action recognition.

Contextual Representation in NLP to Improve Success in Accident Classification of Mine Safety Narratives

Contextual representation has taken center stage in Natural Language Processing (NLP) in the recent past. Models such as Bidirectional Encoder Representations from Transformers (BERT) have found tremendous success in the arena. As a first attempt in the mining industry, in the current work, BERT architecture is adapted in developing the MineBERT model to accomplish the classification of accident narratives from the US Mine Safety and Health Administration (MSHA) data set. In the past multi-year research, several machine learning (ML) methods were used by authors to improve classification success rates in nine significant MSHA accident categories. Out of nine, for four major categories (“Type Groups”) and five “narrow groups”, Random Forests (RF) registered 75% and 42% classification success rates, respectively, on average, while keeping the false positives under 5%. Feature-based innovative NLP methods such as accident-specific expert choice vocabulary (ASECV) and similarity score (SS) methods were developed to improve upon the RF success rates. A combination of all these methods (“Stacked” approach) is able to slightly improve success over RF (71%) to 73.28% for the major category “Caught-in”. Homographs in narratives are identified as the major problem that was preventing further success. Their presence was creating ambiguity problems for classification algorithms. Adaptation of BERT effectively solved the problem. When compared to RF, MineBERT implementation improved success rates among major and narrow groups by 13% and 32%, respectively, while keeping the false positives under 1%, which is very significant. However, BERT implementation in the mining industry, which has unique technical aspects and jargon, brought a set of challenges in terms of preparation of data, selection of hyperparameters, and fine-tuning the model to achieve the best performance, which was met in the current research.

Blockchain-Assisted Homomorphic Encryption Approach for Skin Lesion Diagnosis using Optimal Deep Learning Model

Blockchain (BC) and Machine learning (ML) technologies have been investigated for potential applications in medicine with reasonable success to date. On the other hand, as accurate and early diagnosis of skin lesion classification is essential to gradually increase the survival rate of the patient, Deep-Learning (DL) and ML technologies were introduced for supporting dermatologists to overcome these challenges. This study designed a Blockchain Assisted Homomorphic Encryption Approach for Skin Lesion Diagnosis using an Optimal Deep Learning (BHESKD-ODL) model. The presented BHESKD-ODL model achieves security and proper classification of skin lesion images using BC to store the medical images of the patients to restrict access to third-party users or intruders. In addition, the BHESKD-ODL method secures the medical images using the mayfly optimization (MFO) algorithm with the Homomorphic Encryption (HE) technique. For skin lesion diagnosis, the proposed BHESKD-ODL method uses pre-processing and the Adam optimizer with a Fully Convolutional Network (FCN) based segmentation process. Furthermore, a radiomics feature extraction with a Bidirectional Recurrent Neural Network (BiRNN) model was employed for skin lesion classification. Finally, the Red Deer Optimization (RDO) algorithm was used for the optimal hyperparameter selection of the BiRNN approach. The experimental results of the BHESKD-ODL system on a benchmark skin dataset proved its promising performance in terms of different measures.

Exploring TeV Candidates of Fermi Blazars through Machine Learning

In this work, we make use of a supervised machine-learning algorithm based on Logistic Regression (LR) to select TeV blazar candidates from the 4FGL-DR2/4LAC-DR2, 3FHL, 3HSP, and 2BIGB catalogs. LR constructs a hyperplane based on a selection of optimal parameters, named features, and hyperparameters whose values control the learning process and determine the values of features that a learning algorithm ends up learning, to discriminate TeV blazars from non-TeV blazars. In addition, it gives the probability (or logistic) that a source may be considered a TeV blazar candidate. Non-TeV blazars with logistics greater than 80% are considered high-confidence TeV candidates. Using this technique, we identify 40 high-confidence TeV candidates from the 4FGL-DR2/4LAC-DR2 blazars and we build the feature hyperplane to distinguish TeV and non-TeV blazars. We also calculate the hyperplanes for the 3FHL, 3HSP, and 2BIGB. Finally, we construct the broadband spectral energy distributions for the 40 candidates, testing for their detectability with various instruments. We find that seven of them are likely to be detected by existing or upcoming IACT observatories, while one could be observed with extensive air shower particle detector arrays.

Systematic Evaluation of Deep Neural Network Based Dynamic Modeling Method for AC Power Electronic System

Since the high penetration of renewable energy complicates the dynamic characteristics of the AC power electronic system (ACPES), it is essential to establish an accurate dynamic model to obtain its dynamic behavior for ensure the safe and stable operation of the system. However, due to the no or limited internal control details, the state-space modeling method cannot be realized. It leads to the ACPES system becoming a black-box dynamic system. The dynamic modeling method based on deep neural network can simulate the dynamic behavior using port data without obtaining internal control details. However, deep neural network modeling methods are rarely systematically evaluated. In practice, the construction of neural network faces the selection of massive data and various network structure parameters. However, different sample distributions make the trained network performance quite different. Different network structure hyperparameters also mean different convergence time. Due to the lack of systematic evaluation and targeted suggestions, neural network modeling with high precision and high training speed cannot be realized quickly and conveniently in practical engineering applications. To fill this gap, this paper systematically evaluates the deep neural network from sample distribution and structural hyperparameter selection. The influence on modeling accuracy is analyzed in detail, then some modeling suggestions are presented. Simulation results under multiple operating points verify the effectiveness of the proposed method.

Dap-FL: Federated Learning Flourishes by Adaptive Tuning and Secure Aggregation

Federated learning (FL), an attractive and promising distributed machine learning paradigm, has sparked extensive interest in exploiting tremendous data stored on ubiquitous mobile devices. However, conventional FL suffers severely from resource heterogeneity, as clients with weak computational and communication capabilities may be unable to complete local training using the same local training hyper-parameters. In this paper, we propose Dap-FL, a deep deterministic policy gradient (DDPG)-assisted adaptive FL system, in which local learning rates and local training epochs are adaptively adjusted by all resource-heterogeneous clients through locally deployed DDPG-assisted adaptive hyper-parameter selection schemes. Particularly, the rationality of the proposed hyper-parameter selection scheme is confirmed through rigorous mathematical proof. Besides, due to the thoughtlessness of security consideration of adaptive FL systems in previous studies, we introduce the Paillier cryptosystem to aggregate local models in a secure and privacy-preserving manner. Rigorous analyses show that the proposed Dap-FL system could protect clients' private local models against chosen-plaintext attacks and chosen-message attacks in a widely used honest-but-curious participants and active adversaries security model. More importantly, through ingenious and extensive experiments, the proposed Dap-FL achieves higher model prediction accuracy than two state-of-the-art RL-assisted FL methods, i.e., 6.03% higher than DDPG-based FL and 7.85% higher than DQN-based FL. In addition, experimental results also show that the proposed Dap-FL achieves higher global model prediction accuracy and faster convergence rates than conventional FL, and the comprehensiveness of the adjusted local training hyper-parameters is validated.