Multilayer Neural Network Model Research Articles

Detecting network-based internet censorship via latent feature representation learning

Internet censorship is a phenomenon of societal importance and attracts investigation from multiple disciplines. Several research groups, such as Censored Planet, have deployed large scale Internet measurement platforms to collect network reachability data. However, existing studies generally rely on manually designed rules (i.e., using censorship fingerprints) to detect network-based Internet censorship from the data. While this rule-based approach yields a high true positive detection rate, it suffers from several challenges: it requires human expertise, is laborious, and cannot detect any censorship not captured by the rules. Seeking to overcome these challenges, we design and evaluate a classification model based on latent feature representation learning and an image-based classification model to detect network-based Internet censorship. To infer latent feature representations from network reachability data, we propose a sequence-to-sequence autoencoder to capture the structure and the order of data elements in the data. To estimate the probability of censorship events from the inferred latent features, we rely on a densely connected multi-layer neural network model. Our image-based classification model encodes a network reachability data record as a gray-scale image and classifies the image as censored or not using a dense convolutional neural network. We compare and evaluate both approaches using data sets from Censored Planet via a hold-out evaluation. Both classification models are capable of detecting network-based Internet censorship as we were able to identify instances of censorship not detected by the known fingerprints. Latent feature representations likely encode more nuances in the data since the latent feature learning approach discovers a greater quantity, and a more diverse set, of new censorship instances.

Comparative evaluation of classical and SARIMA-BL time series hybrid models in predicting monthly qualitative parameters of Maroon river

In the present study, seasonal autoregressive integrated moving average (SARIMA) time series models, nonlinear BL model, multi-layer perceptron artificial neural network and SARIMA-bilinear hybrid models were employed to predict the quality parameters of total dissolved solids (TDS), sodium adsorption ratio (SAR), electrical conductivity (EC) in the Maroon basin in Khuzestan Province. For fitting the mentioned models, the monthly data were used for the calibration (1970–2000), confirmation (2001–2011) and prediction of the model (2012–2018). The appropriate models of SARIMA, the bilinear BL and SARIMA-BL hybrid models of the rationalized parameters of the above-mentioned quality parameters were selected based on the adequacy tests, such as the Akaike criterion and the independence test of the model residuals (Ljung–Box). To determine the effective input parameters of the network, the partial mutual information (PMI) algorithm was used to model the three monthly EC, TDS, and SAR parameters. Also, for input and output layers linear transfer function and for hidden layer various active functions with the back-propagation learning algorithm were used for modeling and predicting this water quality parameters. Comparison of the models showed the tangible superiority of SARIMA-bilinear hybrid models than the artificial neural network with effective input parameters based on PMI algorithm, SARIMA and bl models in performed prediction of the all three monthly qualitative parameters in all three stages, training-validation-test in Maroon basin.

Prediction of Compressive Strength of Fly Ash-Slag Based Geopolymer Paste Based on Multi-Optimized Artificial Neural Network.

The fly ash-slag geopolymer is regarded as one of the new green cementitious materials that can replace cement, but it is difficult to predict its mechanical properties by conventional methods. Therefore, in the present study, the back propagation (BP) artificial neural network technique is used to predict the compressive strength of the fly ash-slag geopolymer. In this paper, data from the published literature were collected as the training set and the experimental results from laboratory experiments were used as the test set. Eight input parameters were determined, as follows: the percentage of fly ash, the percentage of slag, the water-cement ratio, the curing age, the modulus of alkali activator, the mass ratio of NaOH to Na2SiO3 and the moles of Na2O and SiO2 in the alkali activator. Three multilayer artificial neural network models were constructed using the Levenberg-Marquardt (LM), Bayesian regularization (BR) and scaled conjugate gradient (SCG) algorithms to compare the prediction accuracy of the compressive strength of the fly ash-slag geopolymer paste at different ages (3, 7, and 28 d). It was concluded that the training set error of the BR-BP neural network was the smallest. Ultimately, the hyperparameter optimization of the BR-BP neural network was carried out to compare the training set and the test set errors before and after the optimization, and the results show that the BR-BP neural network model with hyperparameter optimization had the highest prediction accuracy.

A Modified Artificial Neural Network (ANN)-Based Time Series Prediction of COVID-19 Cases from Multi-Country Data

The deadly Corona virus that first appeared in a seafood market in the Wuhan city of China in December 2019 has been causing global distress by claiming lives and collapsing economies. Given its serious nature, there is an urgent need to understand the virus’s future trajectory. The current study predicts the next day confirmed, death and recovery cases of COVID-19 pandemic for India, Italy, Spain, and the USA by using a modified multilayer neural network (MMLNN) model. The spread of the COVID-19 data is collected from the Kaggle website for the period of 22nd January 2020 to 20th April 2020 (i.e., for 90 days). The predicted figures of the spread of the disease have been estimated and compared with the actual values. Higher precision of the estimates has been observed from the MMLNN model compared to the conventional multilayer neural network (MLANN) model. Specifically, the MMLNN model does faster and more efficient training of the data resulting in less error. The paper forecasts the next day figures (i.e., for 21st April) for all the three cases and does the comparison of the results with the actual values reported. A deviation of 6% is obtained for India, and for the other three countries the deviation is below 3.5%. Given the high accuracy predictive power, the authors recommend that the MMLNN model can be integrated into the health policy of the countries that are struggling with the spread of the virus. Specifically, a decision on health policies such as restrictions on movement can be based on the short-range predictions of the spread of the virus infection.

Prediction of CI Engine Emissions Fueled with Multiwalled Carbon Nanotube-Doped Waste Cooking Oil Biodiesel using Multilayer Neural Network

Nanocatalysts play a significant role to improve the thermal and physical properties of biodiesel. In the present work, the multiwalled carbon nanotubes (MWCNTs) as an additive with the fraction of 30, 40, and 50 ppm are dispersed with the different biodiesel–diesel blends of 10%, 30%, and 50% of waste cooking oil (WCO)-based biodiesel (B10, B30, B50) for the prediction of four-stroke compression ignition (CI) engine emissions using multilayer neural network (MLNN) model. An MLNN model uses a backpropagation algorithm to map input and output parameters. The input parameters to MLNN are load, blends, and MWCNTs in ppm. On the other hand, the output parameters are HC, CO, and NOx. The results for the optimum topological structure of 3-10-3 denoted mean square error (MSE) equal to 0.095 that are capable of predicting the emissions for different operating conditions. Thereafter, the developed MLNN model is tested on an experimental setup consisting of a single-cylinder four-stroke CI engine and emission analyzer. The emission characteristics predicted by MLNN are called to be nearly experimental measurements with reasonable accuracy as it depicts the good “R” values as 0.95, 0.96, and 0.976 for HC, CO, and NOx, respectively, and also gives the reasonable average relative error values as 0.83%, 1.01%, and 1.05%, for HC, CO, and NOx, respectively. Further, the developed model is suitable for predicting emissions of CI engines, thus minimizing the cost, time, and labor effort.

Estimation and prediction of the prevalence rate of COVID-19 disease based on multilayer perceptron artificial neural networks model

Estimation and prediction of the prevalence rate of COVID-19 disease based on multilayer perceptron artificial neural networks model

Prediction and Optimization of the Thermal Properties of TiO2/Water Nanofluids in the Framework of a Machine Learning Approach

In this study, comparing multiple models of machine learning, a multiple linear regression (MLP), multilayer feed-forward artificial neural network (BP) model, and a radial-basis feed-forward artificial neural network (RBF-BP) model are selected for the optimization of the thermal properties of TiO₂/water nanofluids. In particular, the least squares support vector machine (LS-SVM) method and radial basis support vector machine (RB-SVM) method are implemented. First, curve fitting is performed by means of multiple linear regression in order to obtain bivariate correlation functions for thermal conductivity and viscosity of the nanofluid. Then the aforementioned models are used for a predictive analysis of the dependence of its thermal conductivity and viscosity on temperature and volume fraction. The results show that the least squares support vector machine (LS-SVM) has a prediction accuracy higher than the other models. The model predicts the thermal conductivity of TiO₂/water MSE = 1.0853 × 10⁻⁶, R²= 0.99864, MAE = 0.00092, RMSE = 0.00104, and the viscosity of TiO₂/water MSE = 8.1397 × 10⁻⁶, R²= 0.99995, MAE = 0.00074, RMSE = 0.0009.

Bitcoin monthly return forecast: A comparison of ARIMA and multi layer Perceptron Artificial Neural Network

In this paper, we compare the predictive power of Auto Regressive Integrated Moving Averages (ARIMA) and Multi-Layer Perceptron Artificial Neural Networks (MLP ANN) model to short-term forecast the monthly returns of Bitcoin cryptocurrency. We evaluate the performance of two models using time series with monthly data from January 2018 to December 2021. The key parameters for the final assessment of prognostic models are the values of Root Mean Square Error-RMSE and Forecast Error-FE. The results of the short-term BTC return forecast showed better properties of composite compared to univariate time series forecasting models, i.e., higher prognostic power of the MLP ANN model compared to the selected ARIMA (1,1,3) model (lower RMSE and FE). The results point to further comparative research of prognostic models and the possibility of forming more complex and hybrid structures of neural network models in order to predict economic phenomena as accurately as possible.

Hybrid Architecture using CNN and LSTM for Image Captioning in Hindi Language

With the advent of deep learning in recent years, the integration of computer vision with natural language processing has garnered a lot of attention. Generating descriptions from images is one of the most intriguing and focused areas of Machine Learning that faces a number of obstacles, particularly when describing images in languages other than English. In image captioning, a computer is trained to understand the visual information of an image and then generate a description based on the image features and reference sentences. Having an application that automatically describes events in their environment and then translates them into a caption or message can make a significant contribution to society. This paper presents a multi-layered CNN-LSTM neural network model that is utilized to recognize and generate Hindi captions for the objects in images. In addition, a variety of models were trained by adjusting hyperparameters and the number of hidden layers to find the optimum model and maximize the likelihood of the resultant Hindi description. Moreover, after testing the effectiveness of our models, it was observed that our model has shown an increase of 34.64% and 29.13% in terms of BLEU score (Unigram) and BLEU score (Bigram) respectively when compared to the existing work done in this field. Image captioning in Hindi can have a multitude of applications in today's society, and it can also provide a user-friendly interface for Hindi speakers.

An Experimental Investigation on Workability and Bleeding Behaviors of Cement Pastes Doped with Nano Titanium Oxide (n-TiO2) Nanoparticles and Fly Ash

In this study, the workability of cement-based grouts containing n-TiO2 nanoparticles and fly ash has been investigated experimentally. Several characteristic quantities (including, but not limited to, the marsh cone flow time, the mini slump spreading diameter and the plate cohesion meter value) have been measured for different percentages of these additives. The use of fly ash as a mineral additive has been found to result in improvements in terms of workability behavior as expected. Moreover, if nano titanium oxide is also used, an improvement can be obtained regarding the bleeding values for the cement-based grout mixes. Using such experimental data, a multi-layer perceptron artificial neural network model has been developed (5 neurons in the hidden layer of the network model have been developed using a total of 42 experimental data). 70% of the data employed in this model have been used for training, 15% for validation and 15% for the test phase. The results demonstrate that the arti- ficial neural network model can predict Marsh cone flow time, mini slump spreading diameter and plate cohesion meter values with an average error of 0.15%.

Prediction of Electric Power Demand of HVACs for Operating Rooms in Case of Dynamic Set Points of Temperature: A Case Study

In healthcare settings, particularly in areas such as operating rooms and intensive care units, there is a need for a dynamically controlled temperature environment that can adapt to the changing needs of both patients and healthcare workers. This is due to the fact that the desired temperature can vary depending on the condition of the patient and the specific requirements of surgical and treatment procedures. To address this need, our objective is to develop a tool for predicting the electric power needed to maintain a desired temperature in these critical care areas. Previous research has employed artificial learning algorithms and mathematical equations to predict electric power for various types and sizes of buildings, with promising results. However, our study focuses specifically on critical care areas within hospitals and utilizes fluctuating temperature set-points to predict power demand using historical weather data and Building Management System (BMS) data. We employed both Multi-Layer Artificial Neural Network (ML-ANN) and Long short-term memory (LSTM) models for this purpose and found that ML-ANN outperformed LSTM. The results showed that the ML-ANN model performed better than the LSTM model, with a testing accuracy of 96% compared to 78% for the LSTM model. This indicates that the ML-ANN model was more accurate in predicting the power consumption for the desired temperature in the operating room.

Multitask representations in the human cortex transform along a sensory-to-motor hierarchy.

Human cognition recruits distributed neural processes, yet the organizing computational and functional architectures remain unclear. Here, we characterized the geometry and topography of multitask representations across the human cortex using functional magnetic resonance imaging during 26 cognitive tasks in the same individuals. We measured the representational similarity across tasks within a region and the alignment of representations between regions. Representational alignment varied in a graded manner along the sensory-association-motor axis. Multitask dimensionality exhibited compression then expansion along this gradient. To investigate computational principles of multitask representations, we trained multilayer neural network models to transform empirical visual-to-motor representations. Compression-then-expansion organization in models emerged exclusively in a rich training regime, which is associated with learning optimized representations that are robust to noise. This regime produces hierarchically structured representations similar to empirical cortical patterns. Together, these results reveal computational principles that organize multitask representations across the human cortex to support multitask cognition.

Application of deep learning and spectral deconvolution for estimating mineral abundances of zeolite, Mg-sulfate and montmorillonite mixtures and its implications for Mars

A technique for estimating clinoptilolite, montmorillonite, and epsomite mineral abundances from a reflectance spectrum of mineral mixtures using spectral deconvolution and a deep neural network is proposed. Sixty-six ternary mineral mixtures were physically prepared with <150 μ m grain size with different weight percentages of minerals. A combination of normal and skewed Gaussian curves was fitted to the absorption bands at 1.4 μ m , 1.9 μ m , and 2.2 μ m of the acquired reflectance spectra of these mineral mixtures. Six Gaussian curve parameters with maximum absorption band depth ∼1.9 μ m , and wavelength at the maximum band depth, were used (along with mineral abundances) to train multilayer perceptron deep neural network (MLP-DNN) models. Forty-eight models with different DNN architectures and different hyperparameters were trained and the results were validated to find the best models. Winning models were tested using twenty-five samples including fourteen library spectra from RELAB and USGS spectral databases, a spectrum from a different sample, five different amounts of noise-added spectra simulating CRISM (Compact Reconnaissance Imaging Spectrometer for Mars) orbital spectral data, and five mixed spectra derived from linear mixtures of laboratory minerals. The best model was able to predict mineral mixture composition with higher accuracies; three of five montmorillonite and four of the five epsomite library spectra were identified with more than 90% accuracy. However, clinoptilolite samples show less than 50% accuracy and always predicted as mixtures of clinoptilolite and epsomite. This shows the difficulties of distinguishing non-analcime zeolites (e.g., clinoptilolite) from Mg-polyhydrated sulfate minerals, as discussed by authors who mapped hydrous minerals on Mars using hyperspectral image data. Also, the presence of at least ∼10% of montmorillonite in a clinoptilolite-montmorillonite mixture can entirely mask the presence of clinoptilolite in SWIR spectral data. Random artifacts introduced by noise sometime lead to predictions of completely different and incorrect mineral abundances. The study also discusses the possible reasons for the incorrect prediction of mineral abundances and how to overcome these difficulties. Overall, the study shows the advantage of spectral deconvolution with deep neural network for calculating mineral abundance from mixed mineral spectra. • Deep neural network estimating mineral abundances from spectroscopy is proposed. • Presence of montmorillonite mask the spectral identification of clinoptilolite. • Montmorillonite will also mask the presence of epsomite in SWIR spectral data.

Towards Predictive Maintenance of a Heavy-Duty Gas Turbine A New Hybrid Intelligent Methodology for Performance Simulation

As the increasing demands of global clean power for the main purpose of lowering environmental pollution, heavy-duty gas turbines are playing an increasingly important role in energy fields because of their low emission, high thermal efficiency and flexible start-up capacity. Accurate modeling and simulation of the turbine performance is extremely needed to precisely design a large-watt industrial heavy-duty gas turbine, which is still very challenging due to the high nonlinearity, system complexity, varying conditions, and strong coupling interaction of high-dimension parameters under the harsh operation environment of the turbine. In order to improve the accuracy and efficiency of the performance simulation model for a heavy-duty gas turbine at various operating conditions, this paper presents a new hybrid intelligent methodology to adeptly integrate system thermodynamic balance mechanism with multivariate data by seamlessly combing advanced signal processing, machine learning and artificial neural network modeling techniques. The thermodynamic model of a complicated single-shaft gas turbine is first created based on the balances of both flow and power in various subsystems including inlet, compressor, turbine, combustor and exhaust. The characteristic curves of compressor and turbine are utilized to accurately represent the physical mechanism and effectively simulate the high nonlinear behaviors of subsystems. Wavelet signal processing and machine learning based feature extraction are employed to preprocess the multivariate raw data of the turbine. Multilayer artificial neural network models are explored to efficiently simulate the start-up transient process of the turbine, thus improve the simulation efficiency and accuracy of the complicated system. Multivariate data collected from a real-world industrial heavy-duty gas turbine is employed to illustrate the effectiveness and feasibility of the proposed methodology.

Multi-workflow scheduling and resource provisioning in Mobile Edge Computing using opposition-based Marine-Predator Algorithm

Workflow Scheduling in Mobile Edge Computing (MEC) tries to allocate the best possible set of resources for the workflows, considering objectives such as deadline, cost, energy, Quality of Service (QoS), and so on. However, MEC may be under different workloads from the IoT and this may not have the required amount of resources to efficiently handle the workflows. To mitigate this problem, in this paper, we use proactive resource provisioning and workload prediction methods. For this purpose, we present a workload prediction method using a multilayer feed-forward Artificial Neural Network (ANN) model and apply its results for resource provisioning. Afterward, we present an opposition-based version of the Marine-Predator Algorithm (MPA) algorithm, denoted as OMPA. In this algorithm, we present a probabilistic opposition-based learning (OBL) method, which benefits from the OBL, Quasi OBL, and dynamic OBL methods. Afterward, the OMPA algorithm is used for training the multi-layer feed-forward ANN model and multi-workflow scheduling by taking into account factors such as the makespan and number of Virtual Machines (VMs). Extensive experiments conducted in the iFogSim simulator and on the NASA and Saskatchewan datasets indicate that the proposed scheme can achieve better results compared to other metaheuristic algorithms and scheduling schemes.

Prediction of the Compressive Strength of Waste-Based Concretes Using Artificial Neural Network.

In the 21st century, numerous numerical calculation techniques have been discovered and used in several fields of science and technology. The purpose of this study was to use an artificial neural network (ANN) to forecast the compressive strength of waste-based concretes. The specimens studied include different kinds of mineral additions: metakaolin, silica fume, fly ash, limestone filler, marble waste, recycled aggregates, and ground granulated blast furnace slag. This method is based on the experimental results available for 1303 different mixtures gathered from 22 bibliographic sources for the ANN learning process. Based on a multilayer feedforward neural network model, the data were arranged and prepared to train and test the model. The model consists of 18 inputs following the type of cement, water content, water to binder ratio, replacement ratio, the quantity of superplasticizer, etc. The ANN model was built and applied with MATLAB software using the neural network module. According to the results forecast by the proposed neural network model, the ANN shows a strong capacity for predicting the compressive strength of concrete and is particularly precise with satisfactory accuracy (R² = 0.9888, MAPE = 2.87%).

Ndist2vec: Node with Landmark and New Distance to Vector Method for Predicting Shortest Path Distance along Road Networks

The ability to quickly calculate or query the shortest path distance between nodes on a road network is essential for many real-world applications. However, the traditional graph traversal shortest path algorithm methods, such as Dijkstra and Floyd–Warshall, cannot be extended to large-scale road networks, or the traversal speed on large-scale networks is very slow, which is computational and memory intensive. Therefore, researchers have developed many approximate methods, such as the landmark method and the embedding method, to speed up the processing time of graphs and the shortest path query. This study proposes a new method based on landmarks and embedding technology, and it proposes a multilayer neural network model to solve this problem. On the one hand, we generate distance-preserving embedding for each node, and on the other hand, we predict the shortest path distance between two nodes of a given embedment. Our approach significantly reduces training time costs and is able to approximate the real distance with a relatively low Mean Absolute Error (MAE). The experimental results on a real road network confirm these advantages.

Official logo recognition based on multilayer convolutional neural network model

Deep learning has gained high popularity in the field of image processing and computer vision applications due to its unique feature extraction property. For this characteristic, deep learning networks used to solve different issues in computer vision applications. In this paper the issue has been raised is classification of logo of formal directors in Iraqi government. The paper proposes a multi-layer convolutional neural network (CNN) to classify and recognize these official logos by train the CNN model on several logos. The experimental show the effectiveness of the proposed method to recognize the logo with high accuracy rate about 99.16%. The proposed multi-layers CNN model proves the effectiveness to classify different logos with various conditions.

Dermoscopy lesion classification based on GANs and a fuzzy rank-based ensemble of CNN models

Background and Objective. Skin lesion classification by using deep learning technologies is still a considerable challenge due to high similarity among classes and large intraclass differences, serious class imbalance in data, and poor classification accuracy with low robustness. Approach. To address these issues, a two-stage framework for dermoscopy lesion classification using adversarial training and a fuzzy rank-based ensemble of multilayer feature fusion convolutional neural network (CNN) models is proposed. In the first stage, dermoscopy dataset augmentation based on generative adversarial networks is proposed to obtain realistic dermoscopy lesion images, enabling significant improvement for balancing the number of lesions in each class. In the second stage, a fuzzy rank-based ensemble of multilayer feature fusion CNN models is proposed to classify skin lesions. In addition, an efficient channel integrating spatial attention module, in which a novel dilated pyramid pooling structure is designed to extract multiscale features from an enlarged receptive field and filter meaningful information of the initial features. Combining the cross-entropy loss function with the focal loss function, a novel united loss function is designed to reduce the intraclass sample distance and to focus on difficult and error-prone samples to improve the recognition accuracy of the proposed model. Main results. In this paper, the common dataset (HAM10000) is selected to conduct simulation experiments to evaluate and verify the effectiveness of the proposed method. The subjective and objective experimental results demonstrate that the proposed method is superior over the state-of-the-art methods for skin lesion classification due to its higher accuracy, specificity and robustness. Significance. The proposed method effectively improves the classification performance of the model for skin diseases, which will help doctors make accurate and efficient diagnoses, reduce the incidence rate and improve the survival rates of patients.

A novel adaptive predictive control strategy of hybrid radiant-air cooling systems operating in desert climates

Hydronic radiant-air cooling systems have proved to be highly energy-efficient compared to all-air systems. However, such systems require advanced control strategies to overcome their slow thermal response and potential water vapor condensation on the radiant surfaces. Yet, there is a clear lack of proposed adaptive controllers in the literature to collaboratively manage both radiant and air sides of the system. Such adaptive controllers are necessary for reliable and stable operation with minimal energy consumption. In this study, a novel offline adaptive predictive control strategy is proposed for the advantages of maximizing the energy savings of hybrid radiant-air systems, located in desert climate zones, while maintaining a higher quality indoor environment. A transient simulation model is developed in TRNSYS v.18 based on a typical medium-office space, located in Cairo, Egypt, with two air-cooled chiller loops serving a thermally activated building system and a dedicated outdoor air system. A multi-input multi-output multi-layered perceptron artificial neural network model has been developed in MATLAB based on batch simulations, performed in TRNSYS, to predict the electricity consumption and Predicted Percentage Dissatisfied (PPD) for different setpoints. The trained model was used as a fitness function of a multi-objective genetic algorithm, coupled with a decision-making criterion, to determine chillers’ setpoint temperatures that minimize the electricity consumption and thermal discomfort on a half-hourly simulation basis. The proposed controller was compared to a baseline three-step feedback controller, which turns the water circulation pumps on or off depending on the difference between the setpoint temperature and the indoor air temperature, as well as the initial state of the pumps (on or off). The results show that the proposed strategy substantially improves the indoor thermal environment, where the percentage of comfortable occupied hours is increased by 25.1%. The proposed controller also reduced the peak electricity consumption by 18.8%. However, the total energy consumption of the cooling season was lower by only 2.7% due to the longer operating hours of the chillers and the circulation pumps with this controller. Different strategies for further reduction of total energy consumption with a minimal tuning of the proposed controller are recommended for future studies.