Shallow Neural Network Model Research Articles

Assessing the detection of lateral movement through unsupervised learning techniques

Lateral movement (LM) is an umbrella term for techniques through which attackers spread from an entry point to the rest of the network. Typically, LM involves both pivoting through multiple systems and privilege escalation. As LM techniques proliferate and evolve, there is a need for advanced security controls able to detect and possibly nip such attacks in the bud. Based on the published literature, we argue that although LM-focused intrusion detection systems have received considerable attention, a prominent issue remains largely unaddressed. This concerns the detection of LM through unsupervised machine learning (ML) techniques. This work contributes to this field by capitalizing on the LMD-2023 dataset containing traces of 15 diverse LM attack techniques as they were logged by the system monitor (Sysmon) service of the MS Windows platform. We provide a panorama of this sub-field and associated methodologies, exploring the potential of standard ML-based detection. In further detail, in addition to analyzing feature selection and preprocessing, we detail and evaluate a plethora of unsupervised ML techniques, both shallow and deep. The derived scores for the best performer in terms of the AUC and F1 metrics are quite promising, around 94.7%/93% and 95.2%/93.8%, for the best shallow and deep neural network model, respectively. On top of that, in an effort to further improve on those metrics, we devise and evaluate a two-stage ML model, surpassing the previous best score by approximately 3.5%. Overall, to our knowledge, this work provides the first full-blown study on LM detection via unsupervised learning techniques, therefore it is anticipated to serve as a groundwork for anyone working in this timely field.

Fusion of convolutional neural network with XGBoost feature extraction for predicting multi-constituents in corn using near infrared spectroscopy

Near-infrared (NIR) spectroscopy has been widely utilized to predict multi-constituents of corn in agriculture. However, directly extracting constituent information from the NIR spectra is challenging due to many issues such as broad absorption band, overlapping and non-specific nature. To solve these problems and extract implicit features from the raw data of NIR spectra to improve performance of quantitative models, a one-dimensional shallow convolutional neural network (CNN) model based on an eXtreme Gradient Boosting (XGBoost) feature extraction method was proposed in this paper. The leaf node feature information in the XGBoost was encoded and reconstructed to obtain the implicit features of raw data in the NIR spectra. A two-parametric Swish (TSwish or TS) activation function was proposed to improve the performance of CNN, and the elastic net (EN) was also applied to avoid the overfitting problem of the CNN model. Performance of the developed XGBoost-CNN-TS-EN model was evaluated using two public NIR spectroscopy datasets of corn and soil, and the obtained determination coefficients (R2) for moisture, oil, protein, and starch of the corn on test set were 0.993, 0.991, 0.998, and 0.992, respectively, with that of the soil organic matter being 0.992. The XGBoost-CNN-TS-EN model exhibits superior stability, good prediction accuracy, and generalization ability, demonstrating its great potentials for quantitative analysis of multi-constituents in spectroscopic applications.

ShallowBKGC: a BERT-enhanced shallow neural network model for knowledge graph completion

Knowledge graph completion aims to predict missing relations between entities in a knowledge graph. One of the effective ways for knowledge graph completion is knowledge graph embedding. However, existing embedding methods usually focus on developing deeper and more complex neural networks, or leveraging additional information, which inevitably increases computational complexity and is unfriendly to real-time applications. In this article, we propose an effective BERT-enhanced shallow neural network model for knowledge graph completion named ShallowBKGC. Specifically, given an entity pair, we first apply the pre-trained language model BERT to extract text features of head and tail entities. At the same time, we use the embedding layer to extract structure features of head and tail entities. Then the text and structure features are integrated into one entity-pair representation via average operation followed by a non-linear transformation. Finally, based on the entity-pair representation, we calculate probability of each relation through multi-label modeling to predict relations for the given entity pair. Experimental results on three benchmark datasets show that our model achieves a superior performance in comparison with baseline methods. The source code of this article can be obtained from https://github.com/Joni-gogogo/ShallowBKGC.

AN IN-DEPTH ANALYSIS OF ARTIFICIAL INTELLIGENCE APPROACHES FOR RAINFALL PREDICTION

Natural disasters and floods brought on by heavy rainfall pose serious threats to human health and lives every year on a global scale. The intricacy of meteorological data makes it difficult to provide accurate rainfall predictions, despite their critical importance in nations like India where agriculture is the primary occupation. Rainfall forecasting has recently benefited from Artificial Intelligence (AI) developments such as Deep Learning (DL) and Machine Learning (ML) techniques. This article provides a comprehensive survey of recent studies that use AI techniques for rainfall prediction, analyzing them based on the ML algorithms and DL methods used, organized by publication year. The findings show that DL approaches are more effective than traditional ML methods and shallow neural network models. This research is important as it has significant impacts on agriculture, disaster preparedness, and water resource management. Finally, it outlines future research directions for further advancements in rainfall prediction through AI methodologies.

Demographics and socioeconomic determinants of health predict continued participation in a CT lung cancer screening program

PurposeWe developed machine learning (ML) models to assess demographic and socioeconomic status (SES) variables’ value in predicting continued participation in a low-dose CT lung cancer screening (LCS) program. Materials and Methods480 LCS subjects were retrospectively examined for the following outcomes: (#1) no follow-up (single LCS scan) vs. multiple follow-ups (220 and 260 subjects respectively) and (#2) absent or delayed (>1 month past the due date) follow-up vs timely follow-up (356 and 124 subjects respectively). We quantified the contributions of 14 socioeconomic, demographic, and clinical predictors to LCS adherence, and validated and compared prediction performances of multivariate logistic regression (MLR), support vector machine (SVM) and shallow neural network (NN) models. ResultsFor outcome #1, age, sex, race, insurance status, personal cancer history, and median household income were found to be associated with returning for follow-ups. For outcome #2, age, sex, race, and insurance status were significant predictor of absent/delayed LCS follow-up. Across 5-fold cross-validation, the MLR model achieved an average AUC of 0.732 (95% CI, 0.661-0.803) for outcome #1 and 0.633 (95% CI, 0.602-0.664) for outcome #2 and is the model with best predictive performance overall, whereas NN and SVM tended to overfit training data and fell short on testing data performance for either outcome. ConclusionsWe identified significant predictors of LCS adherence, and our ML models can predict which subjects are at higher risk of receiving no or delayed LCS follow-ups. Our results could inform data-driven interventions to engage vulnerable populations and extend the benefits of LCS.

Performance of convolutional neural networks on CIFAR and Dog Breed classification dataset

This research aims to study the deep learning applications in image recognition and classification tasks. The advantages and limitations of deep learning are explored by analysing existing deep learning algorithms and their applications on tasks such as using images to do classification, image segmentation, and target detection. In the experimental part, this paper evaluates the performance of deep learning in image recognition by using classical neural network models, including training and testing models on a large size of image datasets. The results show that neural network has good classification and detection capabilities in image recognition tasks and also achieves good results for image segmentation tasks. However, the model training process of shallow neural network models is time-consuming and performs poorly for small-scale datasets. Based on these situations, this paper proposes some optimization strategies to achieve high performance and efficiency of neural networks in image recognition and analyze the performance differences between different strategies .

Shallow hybrid quantum-classical convolutional neural network model for image classification

Shallow hybrid quantum-classical convolutional neural network model for image classification

Detection and mitigation of false data injection attack in DC–DC synchronous boost converter: A real‐time implementation using shallow neural network model

AbstractWith implementing cyber‐physical systems, modern power systems are at continuous risk of cyber‐attacks. Sensor data that is communicated through the communication links is the most vulnerable entity. The attacker breaches the confidentiality of the data and affects the controller's performance, destabilizing the system. This article proposes a neural network‐based cyber‐attack detection and mitigation scheme to detect and mitigate false data injection attacks on the sensors. Neural networks are used to predict duty; a combination of a data sampler and binary attack detector detects the presence of an attack. Attack mitigation is performed in the final step by analysing the outputs of prediction and detection networks. The designed control scheme is demonstrated at the DC–DC converter level by considering the synchronous boost converter as a plant with an input voltage sensor, output voltage sensor, input current sensor, and output current sensor. The shallow neural network model is trained with appropriate hyperparameters to obtain a root mean square error of 0.003. The control scheme is designed and implemented in MATLAB Simulink platform and realized in real‐time hardware setup by deploying the neural network into the microcontroller and its results are explored.

Estimation of Lightning Flash Rate in Precipitation Features by Applying Shallow AI Neural Network Models to the GMI Passive Microwave Brightness Temperatures

AbstractThe satellite‐constellation passive‐microwave Brightness Temperature (TB) observations, with global coverage, and more additions from upcoming CubeSats, have been mainly used in surface precipitation retrievals. However, these observations can also be used to indicate the intensity of convective systems. This study attempts to relate Global Precipitation Mission (GPM) Microwave Imager (GMI) TBs to Geostationary Lightning Mapper (GLM) lightning flashes by using 4 years (02/2018–04/2022) of GPM Precipitation Feature (PF) database. GMI TBs are collocated to the GLM lightning counts, and to the ERA5 reanalysis 2‐m air temperature in PFs. Three Artificial Intelligent Neural Network Models (AI‐NN) are trained to classify PFs producing lightning in a 20‐min window respectively for land, ocean, or coast. The flash rate of the determined Lightning producing PFs (LPFs) is then quantified by three other AI‐NNs, each trained for one of the three regions. Though the models clearly capture the global geographical distribution of LPFs with a Probability Of Detection over 90%, high False Alarm Rates are found, ranging from 49.9% over land to 91.5% over the ocean. The importance of TB at each passive microwave channel varies regionally, corresponding to the different microphysical properties in various types of precipitation systems. The global lightning distribution is derived by applying the AI models to global PFs and is well compared to the lightning climatology from Lightning Imaging Sensor and Optical Transient Detector. This suggests that the use of passive microwave TBs can help to fill the gaps in lightning monitoring thanks to their global coverage.

A SHALLOW NEURAL NETWORK MODEL FOR URBAN LAND COVER CLASSIFICATION USING VHR SATELLITE IMAGE FEATURES

Abstract. Recently, image classification techniques using neural networks have received considerable attention in sustainable urban development, since their applications have an extreme effect on building distribution, infrastructural networks, and water resource management. In this research, a back-propagation shallow neural network model is presented for very high resolution satellite image classification in urban environments. Workflow procedures consider selecting and collecting data, preparing required study areas, extracting distinctive features, and applying the classification process. Visual interpretation is performed to identify observed land cover classes and detect distinctive features in the urban environment. Pre-processing techniques are implemented to present the used images in a more suited form for the classification techniques. A shallow neural network model (supported by MathWorks MATLAB environment) is successfully applied and results are evaluated. The proposed model is tested for classifying both WorldView-2 and WorldView-3 multispectral images with different spatial and spectral characteristics to check the model’s applicability to various kinds of satellite imagery and different study areas. Model outcomes are compared to two well-known classification methods; the Nearest Neighbour object-based method and the Maximum Likelihood pixel-based classifier, to validate and check the model stability. The overall accuracy achieved by the proposed model is 86.25% and 83.25%, while the nearest neighbour approach has obtained 84.50% and 82.75%, and the maximum likelihood classifier has accomplished 82.50% and 80.25% for study area 1 and study area 2 respectively. Obtained results indicate that the developed shallow neural network model achieves a promising accuracy for urban land cover classification in comparison with the standard techniques.

Prediction of ultimate tensile strength of friction stir welding joint using deep learning-based-multilayer perceptron and long short term memory networks

Friction stir welding (FSW) is a solid-state joining technique where the joint strength is mainly influenced by three process parameters, namely, spindle speed (N), welding speed (V), and plunge force (Fz). The modelling of complex relationships between the process parameters and joint strength requires many experiments, which is a challenging, time-consuming, and non-economical affair. To tackle this problem, computational mathematical models such as deep learning (DL) can be employed to predict the joint strength reliably. In this paper, DL techniques, namely, deep multilayer perceptron (DMLP) and long short-term memory (LSTM) networks have been proposed for such a purpose. The DL networks were first trained with the FSW experimental data and then, the pre-trained models were used for predicting the weld strength. It was found that the DMLP and LSTM models provided lower prediction errors, which are RMSE of 3.30 and 7.63, respectively, and can be effectively utilized for determining weld quality. The proposed DL-based techniques were further compared with the traditional models – the shallow artificial neural network (SANN) model having an RMSE of 27.11 and the ANFIS model having an RMSE of 5.31. DMLP was found to be superior in determining the weld strength most accurately.

Task-Driven CT Image Quality Optimization for Low-Contrast Lesion Detectability with Tunable Neural Networks.

Low-contrast lesions are difficult to detect in noisy low-dose CT images. Improving CT image quality for this detection task has the potential to improve diagnostic accuracy and patient outcomes. In this work, we use tunable neural networks for CT image restoration with a hyperparameter to control the variance/bias tradeoff. We use clinical images from a super-high-resolution normal-dose CT scan to synthesize low-contrast low-dose CT images for supervised training of deep learning CT reconstruction models. Those models are trained using with multiple noise realizations so that variance and bias can be penalized separately. We use a training loss function with one hyperparameter called the denoising level, which controls the variance/bias tradeoff. Finally, we evaluate the CT image quality to find the optimal denoising level for low-contrast lesion detectability. We evaluate performance using a shallow neural network model classification model to represent a suboptimal image observer. Our results indicate that the optimal networks for low-contrast lesion detectability are those that prioritize bias reduction rather than mean-squared error, which demonstrates the potential clinical benefit of our proposed tunable neural networks.

Interpretable selective learning in credit risk

Forecasting credit default risk has been an important research field for several decades. Traditionally, logistic regression has been widely recognized as a solution because of its accuracy and interpretability. Although complex machine learning models may improve accuracy over simple logistic regressions, their interpretability has prevented their use in credit risk assessment. We introduce a neural network with a selective option to increase interpretability by distinguishing whether linear models can explain the dataset. Our methods are tested on two datasets: 25,000 samples from the Taiwan payment system collected in October 2005 and 250,000 samples from the 2011 Kaggle competition. We find that, for most of samples, logistic regression will be sufficient, with reasonable accuracy; meanwhile, for some specific data portions, a shallow neural network model leads to much better accuracy without significantly sacrificing interpretability.

A Deep Learning Framework for Day Ahead Wind Power Short-Term Prediction

Due to the increasing proportion of wind power connected to the grid, day-ahead wind power prediction plays a more and more important role in the operation of the power system. This paper proposes a day-ahead wind power short-term prediction model based on deep learning (DWT_AE_BiLSTM). Firstly, discrete wavelet transform (DWT) is used to denoise the data, then an autoencoder (AE) technology is used to extract the data features, and finally, bidirectional long short-term memory (BiLSTM) is used for prediction. To verify the effectiveness of the proposed DWT_AE_BiLSTM model, we studied three different power stations and compared their performance with the shallow neural network model. Experimental analysis shows that this model is more competitive in forecasting accuracy and stability. Compared with the BP model, the proposed model has increased by 3.86%, 3.22% and 3.42% in three wind farms, respectively.

DSPNet: A Self-ONN Model for Robust DSPN Diagnosis From Temperature Maps

Diabetic sensorimotor polyneuropathy (DSPN) leads to pain, diabetic foot ulceration (DFU), amputation, and death. The diagnosis of advanced DSPN to identify those at risk is key to preventing DFU and amputation. Alterations in foot pressure and temperature may help to detect DSPN and the risk of DFU. We have applied a robust machine-learning approach to identify patients with severe DSPN using standing foot temperature maps generated using temperature sensor data. A robust shallow operational neural network model DSPNet is proposed. The study utilized a labeled dataset from the University Hospital Magdeburg, Magdeburg, Germany, consisting of temperature sensor data from eight different points on the foot in seating and standing positions in patients with severe DSPN ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n =25$ </tex-math></inline-formula> ) and healthy controls ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n =18$ </tex-math></inline-formula> ). The proposed network achieved an F1 score of 90.3% for identifying patients with DSPN and outperformed current state-of-the-art deep-learning network methods. This is the first of its kind of research where the results confirm that temperature maps are not only effective in the detection of those at high risk of DFU but also in identifying patients with severe DSPN. Such sensors could easily be incorporated into smart insoles.

An Impact Localization Solution Using Embedded Intelligence-Methodology and Experimental Verification via a Resource-Constrained IoT Device.

Recent advances both in hardware and software have facilitated the embedded intelligence (EI) research field, and enabled machine learning and decision-making integration in resource-scarce IoT devices and systems, realizing "conscious" and self-explanatory objects (smart objects). In the context of the broad use of WSNs in advanced IoT applications, this is the first work to provide an extreme-edge system, to address structural health monitoring (SHM) on polymethyl methacrylate (PPMA) thin-plate. To the best of our knowledge, state-of-the-art solutions primarily utilize impact positioning methods based on the time of arrival of the stress wave, while in the last decade machine learning data analysis has been performed, by more expensive and resource-abundant equipment than general/development purpose IoT devices, both for the collection and the inference stages of the monitoring system. In contrast to the existing systems, we propose a methodology and a system, implemented by a low-cost device, with the benefit of performing an online and on-device impact localization service from an agnostic perspective, regarding the material and the sensors' location (as none of those attributes are used). Thus, a design of experiments and the corresponding methodology to build an experimental time-series dataset for impact detection and localization is proposed, using ceramic piezoelectric transducers (PZTs). The system is excited with a steel ball, varying the height from which it is released. Based on TinyML technology for embedding intelligence in low-power devices, we implement and validate random forest and shallow neural network models to localize in real-time (less than 400 ms latency) any occurring impacts on the structure, achieving higher than 90% accuracy.

Identification of Type of a Fault in Distribution System Using Shallow Neural Network with Distributed Generation

A distributed generation system (DG) has several benefits over a traditional centralized power system. However, the protection area in the case of the distributed generator requires special attention as it encounters stability loss, failure re-closure, fluctuations in voltage, etc. And thereby, it demands immediate attention in identifying the location & type of a fault without delay especially when occurred in a small, distributed generation system, as it would adversely affect the overall system and its operation. In the past, several methods were proposed for classification and localisation of a fault in a distributed generation system. Many of those methods were accurate in identifying location, but the accuracy in identifying the type of fault was not up to the acceptable mark. The proposed work here uses a shallow artificial neural network (sANN) model for identifying a particular type of fault that could happen in a specific distribution network when used in conjunction with distributed generators. Firstly, a distribution network consisting of two similar distributed generators (DG1 and DG2), one grid, and a 100 Km distribution line is modeled. Thereafter, different voltages and currents corresponding to various faults (line to line, line to ground) at different locations are tabulated, resulting in a matrix of 500 × 18 inputs. Secondly, the sANN is formulated for identifying the types of faults in the system in which the above-obtained data is used to train, validate, and test the neural network. The overall result shows an unprecedented almost zero percent error in identifying the type of the faults.

MODENN: A Shallow Broad Neural Network Model Based on Multi-Order Descartes Expansion.

Deep neural networks have achieved great success in almost every field of artificial intelligence. However, several weaknesses keep bothering researchers due to its hierarchical structure, particularly when large-scale parallelism, faster learning, better performance, and high reliability are required. Inspired by the parallel and large-scale information processing structures in the human brain, a shallow broad neural network model is proposed on a specially designed multi-order Descartes expansion operation. Such Descartes expansion acts as an efficient feature extraction method for the network, improve the separability of the original pattern by transforming the raw data pattern into a high-dimensional feature space, the multi-order Descartes expansion space. As a result, a single-layer perceptron network will be able to accomplish the classification task. The multi-order Descartes expansion neural network (MODENN) is thus created by combining the multi-order Descartes expansion operation and the single-layer perceptron together, and its capacity is proved equivalent to the traditional multi-layer perceptron and the deep neural networks. Three kinds of experiments were implemented, the results showed that the proposed MODENN model retains great potentiality in many aspects, including implementability, parallelizability, performance, robustness, and interpretability, indicating MODENN would be an excellent alternative to mainstream neural networks.

A Binaural MFCC-CNN Sound Quality Model of High-Speed Train

The high-speed train (HST) is one of the most important transport tools in China, and the sound quality of its interior noise affects passengers’ comfort. This paper proposes a HST sound quality model. The model combines Mel-scale frequency cepstral coefficients (MFCCs), the most popular spectral-based input parameter in deep learning models, with convolutional neural networks (CNNs) to evaluate the sound quality of HSTs. Meanwhile, two input channels are applied to simulate binaural hearing so that the different sound signals can be processed separately. The binaural MFCC-CNN model achieves an accuracy of 96.2% and outperforms the traditional shallow neural network model because it considers the time-varying characteristics of noise. The MFCC features are capable of capturing the characteristics of noise and improving the accuracy of sound quality evaluations. Besides, the results suggest that the time and level differences in sound signals are important factors affecting sound quality at low annoyance levels. The proposed model is expected to optimize the comfort of the interior acoustic environment of HSTs.

Ensuring Earthquake-Proof Development in a Swiftly Developing Region through Neural Network Modeling of Earthquakes Using Nonlinear Spatial Variables

Northern Pakistan, the center of major construction projects due to the commencement of the China Pakistan Economic Corridor, is among the most earthquake-prone regions globally owing to its tectonic settings. The area has experienced several devastating earthquakes in the past, and these earthquakes pose a severe threat to infrastructure and life. Several researchers have previously utilized advanced tools such as Machine Learning (ML) and Deep Learning (DL) algorithms for earthquake predictions. This technological advancement helps with construction innovation, for instance, by designing earthquake-proof buildings. However, previous studies have focused mainly on temporal rather than spatial variables. The present study examines the impact of spatial variables to assess the performance of the different ML and DL algorithms for predicting the magnitude of short-term future earthquakes in North Pakistan. Two ML methods, namely Modular Neural Network (MNN) and Shallow Neural Network (SNN), and two DL methods, namely Recurrent Neural Network (RNN) and Deep Neural Network (DNN) algorithms, were used to meet the research objectives. The performance of the techniques was assessed using statistical measures, including accuracy, information gain analysis, sensitivity, specificity, and positive and negative predictive values. These metrics were used to evaluate the impact of including a new variable, Fault Density (FD), and the standard seismic variables in the predictions. The performance of the proposed models was examined for different patterns of variables and different classes of earthquakes. The accuracy of the models for the training data ranged from 73% to 89%, and the accuracy for the testing data ranged from 64% to 85%. The analysis outcomes demonstrated an improved performance when using an additional variable of FD for the earthquakes of low and high magnitudes, whereas the performance was less for moderate-magnitude earthquakes. DNN, and SNN models, performed relatively better than other models. The results provide valuable insights about the influence of the spatial variable. The outcome of the present study adds to the existing pool of knowledge about earthquake prediction, fostering a safer and more secure regional development plan involving innovative construction.