Performance Of Neural Network Research Articles

Predicting some Milk Constituents from Linear Body Measurements and Udder Characteristics of Bunaji Cows Utilizing Machine Learning Techniques

Machine Learning techniques are capable of being used in agriculture since they are powerful, fast and flexible tools for classification and predictions of traits of economic importance, particularly those involving nonlinear systems. This study aimed to show how well machine learning methods could predict milk's crude protein and fat content based on the Bunaji cows' linear body measurements and udder characteristics. Forty (40) lactating Bunaji cows from two dairy farms in Yola South Local Government Area, Adamawa State, were purposefully selected for the study. Milk samples were collected from the cows (one-off measurement). The crude protein and fat content of the milk were determined in percentage. An ensemble model consisting of the random forest (RF), support vector machine (SVM) and neural network (NN) algorithms were created using the Orange Software. The models were cross-validated using five folds to assess their robustness. Mean squared error (MSE), root mean squared error (RMSE), mean absolute error (MAE) and coefficient of determination (R2 ) are the metrics utilized for the evaluation. Using the udder features, the approach was used to construct demonstrative models for the prediction of milk protein content. The SVM and RF learners had MSE that was lower than that of NN, showing that the two were superior to NN. The lower performance of NN in this study can be explained on the basis of extremely low data that was available for this study. RF learners exhibited a low likelihood of mistakes in predicting crude protein from udder traits despite the study's small data set. Large data sets were needed for machine learning techniques. As a result, these models must be improved with lots of data.

Comparative Assessment of the Performances of GPS‐TEC Assisted NTCM, NeQuick2 and Neural Network Models to Describe the East‐African Equatorial Ionosphere

AbstractDifferent ionospheric climatological models such as NeQuick2 and NTCM have been developed to mitigate the ionosphere impact on the trans‐ionosphere propagating radio wave. Moreover, the Neural Network (NN) is used to model and characterize the ionosphere. However, no one has compared the performances of NeQuick2, NTCM, and NN after adapting to GPS TEC. This study evaluates their performances in the East‐African region in 2013 and 2015. It has been done by computing their drivers (effective ionization level, Az for NeQuick2 and ionization driving index, Id for NTCM) through least‐square fitting to TEC observation sense. NN‐algorithm has also been trained and tested with observed TEC used for NeQuick2 and NTCM adaptation. The annual performance test has shown that the correlation coefficient (R) values between observed and NTCM modeled TEC, after an adaption, are better than the corresponding values obtained from NeQuick2 and NN. It also shows that the standard deviations (STD) and root‐mean‐square errors (RMSE) obtained for NTCM are smaller than the STD and RMSE computed for NeQuick2 and NN. On the other hand, the daily performance test of now‐casting and predicting showed that the NN performs the best, followed by NTCM. However, the 1‐hr prediction test showed that NTCM performs the best among the models considered in this study.

Fault Location of Doukan-Erbil 132kv Double Transmission Lines Using Artificial Neural Network ANN

Transmission lines are generally subjected to faults, so it is advantageous to determine these faults as quickly as possible. This study uses an Artificial Neural Network technique to locate a fault as soon as it happens on the Doukan-Erbil of 132kv double Transmission lines network. CYME 7.1-Programming/Simulink utilized simulation to model the suggested network. A multilayer perceptron feed-forward artificial neural network with a back propagation learning algorithm is used for the intelligence locator's training, testing, assessment, and validation. Voltages and currents were applied as inputs during the neural network's training. The pre-fault and post-fault values determined the scaled values. The neural network's performance was evaluated, and tests were run. Line-to-ground faults were examined. The study demonstrates how effective, rapid, and precise this method is at locating faults. The neural network's performance was examined, and tests were run on it. The overall performance of the mean square error in the trained network execution was 0.11792 at 35 epochs. The correlation coefficient at the entire target was 0.99987 percent of an error on the Doukan-Erbil double transmission lines.

Effect of traditional and automated sorting on some tomato's properties

This study was conducted in a laboratory experiment at the University of Baghdad, College of Science, computing Department, 5 km from the center of Baghdad city, in 2021 to evaluate the sorting method for the tomato crop. The experiments were conducted in a factorial experiment under a complete randomized design with three replications and using SAS analysis, artificial neural network, image processing, the study of external characteristics, and physical features; fruit surface area and fruit circumference were 1334.46 cm2,57.53 cm2 and free diseases. The error value was less than zero, while training with outputs recorded the highest value and which was 5. The neural network's performance between the input and the mean square of the regression, as recorded as the best value to validate the performance, was 58.11 in the second period. The importance of fruit circumference is attributed to the sorting and grading of fruits, especially in packing boxes and marketing. Keywords: Automated grading and sorting, traditional grading and sorting, image processing, computer vision, Artificial neural networks

Neural networks to retrieve in water constituents applied to radiative transfer models simulating coastal water conditions

Estimation of chlorophyll (CHL) using ocean colour remote sensing (OCRS) signals in coastal waters is difficult due to the presence of two other constituents altering the light signal: coloured dissolved organic material (CDOM) and mineral suspended sediments (MSS). Artificial neural networks (NNs) have the capacity to deal with signal complexity and are a potential solution to the problem. Here NNs are developed to operate on two datasets replicating MODIS Aqua bands simulated using Hydrolight 5.2. Artificial noise is added to the simulated signal to improve realism. Both datasets use the same ranges of in water constituent concentrations, and differ by the type of logarithmic concentration distributions. The first uses a Gaussian distribution to simulate samples from natural water conditions. The second uses a flat distribution and is intended to allow exploration of the impact of undersampling extremes at both high and low concentrations in the Gaussian distribution. The impact of the concentration distribution structure is assessed and no benefits were found by switching to a flat distribution. The normal distribution performs better because it reduces the number of low concentration samples that are relatively difficult to resolve against varying concentrations of other constituents. In this simulated environment NNs have the capacity to estimate CHL with outstanding performance compared to real in situ algorithms, except for low values when other constituents dominate the light signal in coastal waters. CDOM and MSS can also be predicted with very high accuracies using NNs. It is found that simultaneous retrieval of all three constituents using multitask learning (MTL) does not provide any advantage over single parameter retrievals. Finally it is found that increasing the number of wavebands generally improves NN performance, though there appear to be diminishing returns beyond ∼8 bands. It is also shown that a smaller number of carefully selected bands performs better than a uniformly distributed band set of the same size. These results provide useful insight into future performance for NNs using hyperspectral satellite sensors and highlight specific wavebands benefits.

Area-Efficient Mapping of Convolutional Neural Networks to Memristor Crossbars Using Sub-Image Partitioning.

Memristor crossbars can be very useful for realizing edge-intelligence hardware, because the neural networks implemented by memristor crossbars can save significantly more computing energy and layout area than the conventional CMOS (complementary metal-oxide-semiconductor) digital circuits. One of the important operations used in neural networks is convolution. For performing the convolution by memristor crossbars, the full image should be partitioned into several sub-images. By doing so, each sub-image convolution can be mapped to small-size unit crossbars, of which the size should be defined as 128 × 128 or 256 × 256 to avoid the line resistance problem caused from large-size crossbars. In this paper, various convolution schemes with 3D, 2D, and 1D kernels are analyzed and compared in terms of neural network's performance and overlapping overhead. The neural network's simulation indicates that the 2D + 1D kernels can perform the sub-image convolution using a much smaller number of unit crossbars with less rate loss than the 3D kernels. When the CIFAR-10 dataset is tested, the mapping of sub-image convolution of 2D + 1D kernels to crossbars shows that the number of unit crossbars can be reduced almost by 90% and 95%, respectively, for 128 × 128 and 256 × 256 crossbars, compared with the 3D kernels. On the contrary, the rate loss of 2D + 1D kernels can be less than 2%. To improve the neural network's performance more, the 2D + 1D kernels can be combined with 3D kernels in one neural network. When the normalized ratio of 2D + 1D layers is around 0.5, the neural network's performance indicates very little rate loss compared to when the normalized ratio of 2D + 1D layers is zero. However, the number of unit crossbars for the normalized ratio = 0.5 can be reduced by half compared with that for the normalized ratio = 0.

Intelligent Prediction Model on Soil Bamboo Fibre Mix for Road Construction

Manufacturing with natural materials has recently become popular as a reaction to global temperature challenges and the need for a more sustainable society. Concrete is used as a basic material in modern building projects. Steel is utilized as reinforcement to create tensile strength since concrete has a brittle tensile strength. When bamboo loses water, it shrinks far more than any other species of timber. Before being used for architectural purposes, bamboo should be suitably treated against insect or fungus assault. To overcome the above-mentioned drawbacks this research tends to introduce an intelligent prediction model for road construction. The input parameters, such as “mix type, soil, percentage of fiber, and fiber length”, are initially supplied as input to the optimal Neural Network (NN) model. A combination of +1 percent fiber, +2 percent fiber, +3 percent fiber, and +4 percent fiber is considered in the soil. The NN with Levy Updated Shark Smell Optimization (LU-SSO) based optimization provides the predicted output on “Maximum dry Density (MDD), California Bearing Ratio (CBR), and Optimum moisture content (OMC)”. Finally, error metrics are computed to analyze the performance of NN with the LU-SSO scheme. A less error of 0.1 is achieved in the proposed model when compared to the other existing approaches.

Swallow Neural Network-Empowered High-Speed Brillouin Optical Correlation-Domain Reflectometry: Optimization and Real-Time Operation

In this article, a fiber-optic sensor based on Brillouin optical correlation-domain reflectometry (BOCDR) with a high repetition rate and real-time signal processing function is demonstrated with the assistance of swallow neural networks (NNs) constituted of no more than three hidden layers. In the proposed scheme, the signal processing time for every single Brillouin spectrum is compressed to less than the acquisition period of the spectrum by designing the structure of the NNs, on the basis of the knowledge of the relationship between the implementation time and the preliminary calculation count involved in the NNs. In the experiments with both simulated data and the real-world system, the performances of NNs with different sizes are studied from the perspectives of timing and accuracy. By training NN models with the data acquired from the experiments, real-time dynamic strain measurement is realized with a repetition rate of up to 20 kHz and a dynamic range of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4000 \mu \varepsilon $ </tex-math></inline-formula> . Different from other works regarding machine learning-empowered measurement acceleration in distributed optical fiber sensors with an offline signal processing phase, the method proposed in this article enables consecutive monitoring of the parameters under test.

Novel Spectral Loss Function for Unsupervised Hyperspectral Image Segmentation

Neural networks (NNs) have gained importance in hyperspectral image (HSI) segmentation for earth observation due to its unparalleled data-driven feature extraction capability. However, in many real-life situations, ground truth is not available, and the performance of unsupervised NNs is still susceptible to enhancement. To overcome this challenge, this letter presents a new loss function to improve the performance of unsupervised HSI segmentation models. The spectral loss function, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Sl</i> , which can be included in different models, is based on the purity of the unmixing endmembers and the spectral similarity of the clusters provided by the NN to determine the classes. It is incorporated into a 3D convolutional autoencoder to validate its performance on four standard HSI benchmarks. Furthermore, its performance has been qualitatively examined in a real case study, an oil spill without ground truth. The results show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Sl</i> is a breakthrough in unsupervised HS segmentation, obtaining the best overall performance and highlighting the importance of spectral signatures. Additionally, dimensional reduction is also vital in compacting the spectral information, which facilitates its segmentation. Source code available at [https://github.com/mhaut/HSI-3DSpLoss].

Artificial Neural Network Hyperparameters Optimization: A Survey

Machine-learning (ML) methods often utilized in applications like computer vision, recommendation systems, natural language processing (NLP), as well as user behavior analytics. Neural Networks (NNs) are one of the most es-sential ways to ML; the most challenging element of designing a NN is de-termining which hyperparameters to employ to generate the optimal model, in which hyperparameter optimization improves NN performance. This study includes a brief explanation regarding a few types of NN as well as some methods for hyperparameter optimization, as well as previous work results in enhancing ANN performance using optimization methods that aid research-ers and data analysts in developing better ML models via identifying the ap-propriate hyperparameter configurations.

Robust Composite H∞ Synchronization of Markov Jump Reaction-Diffusion Neural Networks via a Disturbance Observer-Based Method.

This article focuses on the composite H∞ synchronization problem for jumping reaction-diffusion neural networks (NNs) with multiple kinds of disturbances. Due to the existence of disturbance effects, the performance of the aforementioned system would be degraded; therefore, improving the control performance of closed-loop NNs is the main goal of this article. Notably, for these disturbances, one of them can be described as a norm-bounded, and the other is generated by an exogenous model. In order to reject the above one kind of disturbance, a disturbance observer is developed. Furthermore, combining the disturbance observer approach and conventional state-feedback control scheme, a composite disturbance rejection controller is specifically designed to compensate for the influences of the disturbances. Then, some criteria are established based on the general Lyapunov stability theory, which can ensure that the synchronization error system is stochastically stable and satisfies a fixed H∞ performance level. A simulation example is finally presented to verify the availability of our developed method.

Predicting AT(N) pathologies in Alzheimer's disease from blood-based proteomic data using neural networks.

Blood-based biomarkers represent a promising approach to help identify early Alzheimer's disease (AD). Previous research has applied traditional machine learning (ML) to analyze plasma omics data and search for potential biomarkers, but the most modern ML methods based on deep learning has however been scarcely explored. In the current study, we aim to harness the power of state-of-the-art deep learning neural networks (NNs) to identify plasma proteins that predict amyloid, tau, and neurodegeneration (AT[N]) pathologies in AD. We measured 3,635 proteins using SOMAscan in 881 participants from the European Medical Information Framework for AD Multimodal Biomarker Discovery study (EMIF-AD MBD). Participants underwent measurements of brain amyloid β (Aβ) burden, phosphorylated tau (p-tau) burden, and total tau (t-tau) burden to determine their AT(N) statuses. We ranked proteins by their association with Aβ, p-tau, t-tau, and AT(N), and fed the top 100 proteins along with age and apolipoprotein E (APOE) status into NN classifiers as input features to predict these four outcomes relevant to AD. We compared NN performance of using proteins, age, and APOE genotype with performance of using age and APOE status alone to identify protein panels that optimally improved the prediction over these main risk factors. Proteins that improved the prediction for each outcome were aggregated and nominated for pathway enrichment and protein-protein interaction enrichment analysis. Age and APOE alone predicted Aβ, p-tau, t-tau, and AT(N) burden with area under the curve (AUC) scores of 0.748, 0.662, 0.710, and 0.795. The addition of proteins significantly improved AUCs to 0.782, 0.674, 0.734, and 0.831, respectively. The identified proteins were enriched in five clusters of AD-associated pathways including human immunodeficiency virus 1 infection, p53 signaling pathway, and phosphoinositide-3-kinase-protein kinase B/Akt signaling pathway. Combined with age and APOE genotype, the proteins identified have the potential to serve as blood-based biomarkers for AD and await validation in future studies. While the NNs did not achieve better scores than the support vector machine model used in our previous study, their performances were likely limited by small sample size.

An algorithm to improve the performance of convolutional neural networks for TSC tasks

AbstractThe convolutional neural network (CNN) is an important network model for processing time series classification tasks. However, there can be a lot of noise in the data itself. The clarity of spatial feature expression of time‐series samples and the freeness and balance of data samples both have a big effect on how well CNNs can handle time series classification (TSC) tasks. This paper proposes a multi‐pose marking learning algorithm (MPML), which optimizes the spatial representation of data samples through multiple graphical representations of time series data and then solves the problems of freeness and heterogeneity of data samples through labeling and preferential learning of poorly trained sample data. The algorithm improves the model's robustness to noise by improving the spatial expression sentiment of the time series and the labeling learning of the feature data, thereby improving the convolutional neural network's performance to perform the TSC task. We show how well the algorithm works by testing it on 46 datasets from the UCR Time Series Classification Archive and comparing the results.

Accelerating Neural Network Inference With Processing-in-DRAM: From the Edge to the Cloud

Neural networks (NNs) are growing in importance and complexity. An NN’s performance (and energy efficiency) can be bound either by computation or memory resources. The processing-in-memory (PIM) paradigm, where computation is placed near or within memory arrays, is a viable solution to accelerate memory-bound NNs. However, PIM architectures vary in form, where different PIM approaches lead to different tradeoffs. Our goal is to analyze, discuss, and contrast dynamic random-access memory (DRAM)-based PIM architectures for NN performance and energy efficiency. To do so, we analyze three state-of-the-art PIM architectures: 1) UPMEM, which integrates processors and DRAM arrays into a single 2-D chip, 2) Mensa, a 3-D-stacking-based PIM architecture tailored for edge devices, and 3) SIMDRAM, which uses the analog principles of DRAM to execute bit-serial operations. Our analysis reveals that PIM greatly benefits memory-bound NNs: 1) UPMEM provides 23× the performance of a high-end graphics processing unit (GPU) when the GPU requires memory oversubscription for a general matrix–vector multiplication kernel, 2) Mensa improves energy efficiency and throughput by 3.0× and 3.1× over the baseline Edge tensor processing unit for 24 Google edge NN models, and 3) SIMDRAM outperforms a central processing unit/graphics processing unit by 16.7×/1.4× for three binary NNs. We conclude that the ideal PIM architecture for NN models depends on a model's distinct attributes, due to the inherent architectural design choices.

Facilitating the implementation of neural network-based predictive control to optimize building heating operation

Simple neural network (NN) architecture is a reliable tool to transform reactive rule-based systems into predictive systems. Thermal comfort is of utmost importance in office buildings, which need the activation of heating systems at an optimal time. A high-performance NN predictive system requires a large training dataset. This can limit system efficiency due to the lack of enough historical data derived from thermal controllers. To address this issue, we generated, trained and tested a dataset of eight sizes using a calibrated building model. A set of key performance indicators (KPIs) was improved by studying the output performance. The effect of normalization and standardization preprocessing techniques on NN prediction ability was studied. Learning curves showed that a minimum of 1–4 months of data are required to obtain enough accuracy. Two heating seasons provide the optimal data size to calibrate the NN properly with high prediction accuracy. The results also revealed that building data from ≥two years slightly improve NN performance. The most accurate results in KPIs (≥ 90%) were obtained with preprocessed data. The effect of preprocessing on large training patterns was less than that of training patterns <100. Finally, NN model performance was less accurate in cold climate zones.

Short-Term Weather Forecast Skill of Artificial Neural Networks

Abstract We evaluate the short-term weather forecast performance of three flavors of artificial neural networks (NNs): feed forward back propagation, radial basis function, and generalized regression. To prepare the application of the NNs to an operational setting, we tune NN hyperparameters using over two years of historical data. Five objective guidance products serve as predictors to the NNs: North American Mesoscale and Global Forecast System model output statistics (MOS) forecasts, the High-Resolution Rapid Refresh (HRRR) model, National Weather Service forecasts, and the National Blend of Models product. We independently test NN performance using 96 real-time forecasts of temperature, wind, and precipitation across 11 U.S. cities made during the WxChallenge, a weather forecasting competition. We demonstrate that all NNs significantly improve short-range weather forecasts relative to the traditional objective guidance aids used to train the networks. For example, 1-day maximum and minimum temperature forecast error is 20%–30% lower than MOS. However, NN improvement over multiple linear regression for short-term forecasts is not significant. We suggest this may be attributed to the small number of training samples, the operational nature of the experiment, and the short forecast lead times. Regardless, our results are consistent with previous work suggesting that applying NNs to model forecasts can have a positive impact on operational forecast skill and will become valuable tools when integrated into the forecast enterprise. Significance Statement We used approximately two years of historical weather data and objective forecasts for a number of cities to tune a series of artificial neural networks (NNs) to forecast 1-day values of maximum and minimum temperature, maximum sustained wind speed, and quantitative precipitation. We compare forecast error against common objective guidance and multiple linear regression. We found that the NNs exhibit about 25% lower error than common objective guidance for temperature forecasting and 50% lower error for wind speed. Our results suggest that NNs will be a valuable contributor to improving weather forecast skill when adopted into the existing forecast enterprise.

Neural network-based method for diagnosis and severity assessment of Graves’ orbitopathy using orbital computed tomography

Computed tomography (CT) has been widely used to diagnose Graves’ orbitopathy, and the utility is gradually increasing. To develop a neural network (NN)-based method for diagnosis and severity assessment of Graves’ orbitopathy (GO) using orbital CT, a specific type of NN optimized for diagnosing GO was developed and trained using 288 orbital CT scans obtained from patients with mild and moderate-to-severe GO and normal controls. The developed NN was compared with three conventional NNs [GoogleNet Inception v1 (GoogLeNet), 50-layer Deep Residual Learning (ResNet-50), and 16-layer Very Deep Convolutional Network from Visual Geometry group (VGG-16)]. The diagnostic performance was also compared with that of three oculoplastic specialists. The developed NN had an area under receiver operating curve (AUC) of 0.979 for diagnosing patients with moderate-to-severe GO. Receiver operating curve (ROC) analysis yielded AUCs of 0.827 for GoogLeNet, 0.611 for ResNet-50, 0.540 for VGG-16, and 0.975 for the oculoplastic specialists for diagnosing moderate-to-severe GO. For the diagnosis of mild GO, the developed NN yielded an AUC of 0.895, which is better than the performances of the other NNs and oculoplastic specialists. This study may contribute to NN-based interpretation of orbital CTs for diagnosing various orbital diseases

Constructing a physics-guided machine learning neural network to predict tonal noise emitted by a propeller

Artificial neural networks offer a highly nonlinear and adaptive model for predicting complex interactions between input-output parameters. However, these networks require large datasets which often exceed practical considerations in modeling experimental results. To alleviate the dataset size requirement, a method known as physics-guided machine learning has been applied to construct several neural networks for predicting propeller tonal noise in the time domain over a broad range of flight conditions. Three space-filling designs, namely, Latin-Hypercube, Sphere-Packing, and Grid-Space, were used to distribute points throughout the input parameter space encompassing nondimensional flight conditions and observer geometry. Each neural network's performance was validated by conditions outside of the training set and compared to the Propeller Analysis System tool from the NASA Aircraft Noise Prediction Program. The Latin-Hypercube and the Sphere-Packing designs provided a uniform representation of the domain, which improved the tonal noise prediction in comparison to the Grid design. The black-box nature of these neural networks was explored, and post-network functions were developed to remove discontinuities in the acoustic signal. Overall, the methods herein show a notable improvement in prediction performance in comparison to a multilayer perceptron. Additional loss functions are necessary for ensuring reasonable accuracy of predictive networks on small datasets and will be investigated for the final paper.

Multivariate optimization of the electrochemical degradation for COD and TN removal from wastewater: An inverse computation machine learning approach

An inverse computation machine learning (ML) framework that couples genetic algorithm (GA) to feedforward neural network was developed to optimize the operating parameters for maximizing electrochemical removal of chemical oxygen demand (COD) and total nitrogen (TN) from wastewater. Conventional neural networks (NN) are generally unable to implement inverse computation from a desired abatement to feasible input conditions, and NN trained from small-scale datasets tends to be unreliable for multivariate simulation. To address this issue, we employed GA for tuning the weights and biases of the network to enhance the stability and generalization performance of NN, the optimization of multiple operating conditions was performed by the GA strategy of iterative evolution and global search.In this work, we investigated and analyzed the performance of multivariate optimization approaches such as orthogonal design, response surface methodology (RSM), traditional NN, and the developed neuro-genetic model (GANN). Significance and error analysis demonstrate that GANN exhibits superior stability and generalization ability with the highest R2 of 0.946 (COD), 0.874 (TN), and the lowest RMSE of 0.022 (COD), 0.028 (TN). The introduction of GA is significant for improving the prediction accuracy of NN derived from small-scale datasets. The average error of the GANN trained by 25 data points is lower than that of the RSM model derived from 54 data points. According to the validation outcomes, the scheme suggested by GANN achieves the best COD (93.6%) and TN (62.8%) degradation efficiencies, which are 6.1% and 9.9% higher than the optimal values in the original dataset, respectively. The proposed multivariate global optimization strategy can be extended to other cases, and the computational framework of GANN also contributes to the progress of more reliable ML models.

VARIANTS OF NEURAL NETWORKS: A REVIEW

Machine learning (ML) techniques are part of artificial intelligence. ML involves imitating human behavior in solving different problems, such as object detection, text handwriting recognition, and image classification. Several techniques can be used in machine learning, such as Neural Networks (NN). The expansion in information technology enables researchers to collect large amounts of various data types. The challenging issue is to uncover neural network parameters suitable for object detection problems. Therefore, this paper presents a literature review of the latest proposed and developed components in neural network techniques to cope with different sizes and data types. A brief discussion is also introduced to demonstrate the different types of neural network parameters, such as activation functions, loss functions, and regularization methods. Moreover, this paper also uncovers parameter optimization methods and hyperparameters of the model, such as weight, the learning rate, and the number of iterations. From the literature, it is notable that choosing the activation function, loss function, number of neural network layers, and data size is the major factor affecting NN performance. Additionally, utilizing deep learning NN resulted in a significant improvement in model performance for a variety of issues, which became the researcher's attention.