Deep Neural Network Algorithm Research Articles

The Potential Application of Innovative Methods in Neural Networks for Surface Crack Recognition of Unshelled Hazelnut

In some countries, most hazelnuts are cracked using semi-industrial or hand-crafted machines and marketed as open-shell. In the process of hazelnut cracking, because of the different sizes and shapes of hazelnuts, many hazelnuts leave the cracking machine in the form of a cracked or closed-shell. The presence of cracked or closed-shell hazelnuts reduces the marketability of the product. Therefore, after the cracking operation, the separation of cracked or closed-shells from whole hazelnuts has largely been conducted by visual inspection, which is time-consuming, labor-intensive, and lacks accuracy. So, the purpose of this study was to use the deep convolutional neural network (DCNN) algorithm to classify hazelnuts into two classes: open-shell and closed-shell or cracked hazelnuts. To compare the proposed method with pretrained DCNN models, three models including ResNet-50, Inception-V3, and VGG-19 were investigated. The results of the proposed model (accuracy of 98% and F 1 -score of 96.8) showed that the proposed DCNN has good capability in predicting hazelnut classes. Compared with pretrained models, because of the small size and simple architecture of the proposed model, this model can be a good substitute for a complex and large model such as Inception-V3. Overall, the results indicate that crack on the hazelnut surface can be successfully detected automatically, and the proposed DCNN has a high potential to facilitate the development of a hazelnut sorter based on surface crack.

Deep learning algorithms in predicting Cr(VI) removal performance of S-ZVI: Models building and optimal parameters prediction

The Cr(VI) removal by sulfidated zero-valent iron (S-ZVI) is a complex process, and reaction conditions can greatly affect Cr(VI) removal performance. To obtain the optimal experimental conditions for removing Cr(VI) by S-ZVI, a significant amount of experimental data is needed which is very laborious. Deep learning is an effective method for predicting unknown or unobservable data and reducing experimental workload. It can build highly complex data models based on existing experimental data through multiple layers of nonlinear transformations. In this study, deep learning was used to establish regression models based on experimental data to predict Cr(VI) removal performance of S-ZVI. Deep neural network (DNN) and long short-term memory (LSTM) algorithms were selected, initial pH and reaction time were input features, and C/C0 of Cr(VI) removal performance was output value. The performance of models built by DNN and LSTM algorithms was investigated by comparing R2 and mean square error. Both models built by DNN and LSTM algorithms exhibited accurate predictive ability. Then DNN and LSTM models were used to predict Cr(VI) removal trend by S-ZVI at different pH and determine optimal pH. As predicted by DNN and LSTM, the optimal removal of Cr(VI) by S-ZVI was achieved when pH was 5.2 and 5.1, respectively. This study provides an effective and promising method for using deep learning modeling to predict Cr(VI) removal performance of S-ZVI. It is expected to simplify the preliminary experiments and reduce labor, time and chemical costs.

A novel approach for predicting Lockout/Tagout safety procedures for smart maintenance strategies

A novel approach for predicting Lockout/Tagout safety procedures for smart maintenance strategies

Deep Self-Supervised Graph Attention Convolution Autoencoder for Networks Clustering

In recent years, graph-based deep learning algorithms have attracted widespread attention in the field of consumer electronics. Still, most of the current graph neural networks are based on supervised learning or semi-supervised learning, which often relies on the true labels of the given samples as auxiliary information. To solve this problem, we propose a Deep Self-Supervised Attention Convolution Autoencoder Graph Clustering (DSAGC) model and use it for social networks clustering. We divide the proposed model into two parts: a pretext task and a downstream task. In the pretext task, we obtain pseudo-labels of samples by graph attention autoencoder and clustering, then adopt the proposed reliable sample selection mechanism to gain a high confidential sample. These samples and corresponding pseudo-labels are selected as input of downstream task for helping the downstream task to finish training. The model obtains the predictive labels of the sample from the downstream task. This model does not depend on the true label of the sample for training and uses the pseudo-labels produced by unsupervised learning to improve the clustering performance. We apply the proposed model to three commonly used public social network datasets to test its performance. Compared with the presented unsupervised clustering algorithms and other deep graph neural network algorithms, various metrics have shown that the proposed model consistently outperforms the state-of-the-art. The proposed method can be used for consumer behavior analysis. We release the source code at https://github.com/hulu88/DSAGC.

Prediction of geomechanical bearing capacity using autoregressive deep neural network in carbon capture and storage systems

Carbon Capture and Storage (CCS) field is growing rapidly as a means to mitigate the accumulation of greenhouse gas emissions. However, the geomechanical stability of CCS systems, particularly related to bearing capacity, remains a critical challenge that requires accurate prediction models. In this research paper, we investigate the efficacy of employing an Autoregressive Deep Neural Network (ARDNN) algorithm to predict the geomechanical bearing capacity in CCS systems through shear wave velocity prediction as an index for bearing capacity evaluation of deep rock formations. The model utilizes a dataset consisting of 23,000 data points to train and test the ARDNN algorithm. Its scalability, use of deep learning techniques, automatic feature extraction, adaptability to changes in data, and versatility in various prediction tasks make it an attractive option for accurate predictions. The results demonstrate exceptional performance, as evidenced by an R-squared value of 0.9906 and a mean squared error of 0.0438 for the test data compared to the measured data. This research has significant practical implications for effectively predicting geomechanical stability in CCS systems, thus mitigating potential risks associated with their operation.

Retracted: Application of Deep Neural Network Algorithm in the Analysis of Legal Precedent Citation Basis

Retracted: Application of Deep Neural Network Algorithm in the Analysis of Legal Precedent Citation Basis

Semantic Segmentation of Portuguese Agri-Forestry Using High-Resolution Orthophotos

The Montado ecosystem is an important agri-forestry system in Portugal, occupying about 8% of the total area of the country. However, this biodiverse ecosystem is threatened due to factors such as shrub encroachment. In this context, the development of tools for characterizing and monitoring Montado areas is crucial for their conservation. In this study, we developed a deep convolutional neural network algorithm based on the U-net architecture to identify regions with trees, shrubs, grass, bare soil, or other areas in Montado areas using high-resolution RGB and near-infrared orthophotos (with a spatial resolution of 25 cm) from seven experimental sites in the Alentejo region of Portugal (six used for training/validation and one for testing). To optimize the model’s performance, we performed hyperparameter tuning, which included adjusting the number of filters, dropout rate, and batch size. The best model achieved an overall classification performance of 0.88 and a mean intersection of the union of 0.81 on the test set, indicating high accuracy and reliability of the model in identifying and delineating land cover classes in the Montado ecosystem. The developed model is a powerful tool for identifying the status of the Montado ecosystem regarding shrub encroachment and facilitating better future management.

인공신경망(ANN)을 이용한 옵션가격결정 및 딥헤징 (DH)에 대한 SNN 및 RNN 결과의 비교·분석

The purpose of this study is to present alternative methods to overcome market limitations such as the assumption of fixing the dynamics of underlying assets and market friction for the traditional Black-Scholes option pricing model. As for the research method of this paper, Adam is used as the gradient descent optimizer for the hedging model of the call option short portfolio using artificial neural network models SNN (simple neural network) and RNN (recurrent neural network) as analysis models, and deep neural network (deep neural network) is used as the hedging model. neural network) methodology was applied.
 The research results of this paper are as follows. Regarding the ATM call option price, the BS model showed 10.245, the risk neutral model showed 10.268, the SNN-DH model showed 11.834, and the RNN-DH model showed 11.882. Therefore, it appears that there is a slight difference in the call option price according to each analysis model. At this time, the DH analysis data sample is 100,000 (1×) training samples generated by Monte Carlo Simulation, and the number of testing samples is the training sample. 20% (20,000 pieces) was used, and the option payoff function is lambda(), which is -max(, 0). In addition, the option price and P&L (P&L) of SNN-DH and RNN-DH appear linearly on the basis, and the longer the remaining maturity, the closer the delta value of SNN-DH and BS to the distribution of the S-curve, and the closer the expiration date, the closer the two models are. Delta is densely distributed between 0 and 1. As for the total loss of delta hedging of the short call option position, SNN-DH was -0.0027 and RNN-DH was -0.0061, indicating that the total hedged profit and loss was close to 0.
 The implications of this study are to present DL techniques based on AI methods as an alternative way to overcome limitations such as fixing underlying asset dynamics and market friction of the traditional Black-Scholes model. In addition, it is an independent analysis model that considers valid robust tools by applying deep neural network algorithms for portfolio hedging problems. One limitation of the research is that it analyzed the model under the assumption of zero transaction costs, so future studies should consider this aspect.

Enhancing pavement maintenance: A deep learning model for accurate prediction and early detection of pavement structural damage

Pavement distress, which is exacerbated by structural damage, results in massive carbon-intensity for routine maintenance and transportation safety issues. One promising approach to resolving this issue is to predict structural damage before it occurs. However, structural damage prediction is a complex problem with many influential factors that no single index can precisely solve for in-service pavement. In this study, a new method is adopted that combines all implicit factors into a deep learning neural network (DNN) algorithm to achieve high accuracy for the prediction of pavement structural damage. The training and optimization procedures of the DNN model are introduced explicitly for model repeatability and development. The results show that the state-of-the-art (SOTA) model performed well, with an accuracy rate of 88.46%. The contribution of this research facilitates the development of pavement maintenance decision systems to provide a prediction method for on-site pavement structures. By employing this approach, structural damage can be identified early to prevent further expansion, which will reduce costs and save lives and natural resources.

Forest Gap Extraction Based on Convolutional Neural Networks and Sentinel-2 Images

As a type of small-scale disturbance, forest gap and its accurate extraction are of great significance to monitor forest long-term dynamics, to choose forest recovery mode and to predict forest recovery pace. Currently, airborne LiDAR and high-resolution multi-spectral data are commonly used to accurately classify forest gaps, but they are costly to acquire and have limited time and space availability. In contrast, the Sentinel-2 multi-spectral data with a 10 m spatial resolution overcomes these drawbacks in forest gap extraction. In this work, an integrated framework that combines multi-source remote sensing, machine learning and deep learning to extract forest gap in wide regions was proposed and tested in three sites. First, LiDAR, Sentinel series and random forest (RF) algorithm were synergized to produce a canopy height product in model training site. On this basis, samples for forest canopy, forest gap and non-such were identified from LiDAR-derived canopy height model (CHM) and Sentinel-based canopy height inversion (HI) data to train forest gap extraction models by applying the Deep Forest (DF) and Convolutional Neural Networks (CNN) algorithms, followed by a comparison of the accuracy and the transferability among the four models (DF-CHM, DF-HI, CNN-CHM and CNN-HI). The results indicated that the R2 and RMSE of Sentinel-based canopy height retrievals were estimated at 0.63, and 7.85 m respectively, the difference in the mean height and standard deviation between HI and CHM was 0.03 m and 4.7 m respectively. And there was a spatial agreement of about 98.60% between the HI-identified samples and the CHM-identified samples, with an agreement of 54.89% for the forest gap class. The CNN-HI model had the highest accuracy in both transfer learning test sites, with an overall accuracy (OA) of 0.85 and 0.87, Kappa coefficient at 0.78 and 0.81, respectively, proving that it has good transferability. Conversely, the DF-based models generally gave poorer accuracy and transferability. This study demonstrates that combining Sentinel-2 multi-spectral data and CNN algorithm is feasible and effective in forest gap extraction applications over wide regions.

An optimized facial emotion recognition architecture based on a deep convolutional neural network and genetic algorithm

An optimized facial emotion recognition architecture based on a deep convolutional neural network and genetic algorithm

Deep learning visual field global index prediction with optical coherence tomography parameters in glaucoma patients

The aim of this study was to predict three visual filed (VF) global indexes, mean deviation (MD), pattern standard deviation (PSD), and visual field index (VFI), from optical coherence tomography (OCT) parameters including Bruch's Membrane Opening-Minimum Rim Width (BMO-MRW) and retinal nerve fiber layer (RNFL) based on a deep-learning model. Subjects consisted of 224 eyes with Glaucoma suspects (GS), 245 eyes with early NTG, 58 eyes with moderate stage of NTG, 36 eyes with PACG, 57 eyes with PEXG, and 99 eyes with POAG. A deep neural network (DNN) algorithm was developed to predict values of VF global indexes such as MD, VFI, and PSD. To evaluate performance of the model, mean absolute error (MAE) was determined. The MAE range of the DNN model on cross validation was 1.9–2.9 (dB) for MD, 1.6–2.0 (dB) for PSD, and 5.0 to 7.0 (%) for VFI. Ranges of Pearson’s correlation coefficients were 0.76–0.85, 0.74–0.82, and 0.70–0.81 for MD, PSD, and VFI, respectively. Our deep-learning model might be useful in the management of glaucoma for diagnosis and follow-up, especially in situations when immediate VF results are not available because VF test requires time and space with a subjective nature.

A Study of Deep Learning Neural Network Algorithms and Genetic Algorithms for FJSP

Flexible job-shop scheduling problem (FJSP) is a new research hotspot in the field of production scheduling. To solve the multiobjective FJSP problem, the production of flexible job shop can run normally and quickly. This research takes into account various characteristics of FJSP problems, such as the need to ensure the continuity and stability of processing, the existence of multiple objectives in the whole process, and the constant complexity of changes. It starts with deep learning neural networks and genetic algorithms. Long short-term memory (LSTM) and convolutional neural networks (CNN) are combined in deep learning neural networks. The new improved algorithm is based on the combination of deep learning neural networks LSTM and CNN with genetic algorithm (GA), namely, CNN-LSTM-GA algorithm. Simulation results showed that the accuracy of the CNN-LSTM-GA algorithm was between 85.2% and 95.3% in the test set. In the verification set, the minimum accuracy of the CNN-LSTM-GA algorithm was 84.6%, both of which were higher than the maximum accuracy of the other two algorithms. In the FJSP simulation experiment, the AUC value of the CNN-LSTM-GA algorithm was 0.92. After 40 iterations, the F1 value of the CNN-LSTM-GA algorithm remained above 0.8, which was significantly higher than the other two algorithms. CNN-LSTM-GA is superior to the other two algorithms in terms of prediction accuracy and overall performance of FJSP. It is more suitable for solving the discrete manufacturing job scheduling problem with FJSP characteristics. This study significantly raises the utilisation rate of the assembly shop’s equipment, optimises the scheduling of FJSP, and fully utilises each processing device’s versatile characteristics, which are quite useful for the production processes of domestic vehicle manufacturing companies.

Deep Neural Networks Training by Stochastic Quasi-Newton Trust-Region Methods

While first-order methods are popular for solving optimization problems arising in deep learning, they come with some acute deficiencies. To overcome these shortcomings, there has been recent interest in introducing second-order information through quasi-Newton methods that are able to construct Hessian approximations using only gradient information. In this work, we study the performance of stochastic quasi-Newton algorithms for training deep neural networks. We consider two well-known quasi-Newton updates, the limited-memory Broyden–Fletcher–Goldfarb–Shanno (BFGS) and the symmetric rank one (SR1). This study fills a gap concerning the real performance of both updates in the minibatch setting and analyzes whether more efficient training can be obtained when using the more robust BFGS update or the cheaper SR1 formula, which—allowing for indefinite Hessian approximations—can potentially help to better navigate the pathological saddle points present in the non-convex loss functions found in deep learning. We present and discuss the results of an extensive experimental study that includes many aspects affecting performance, like batch normalization, the network architecture, the limited memory parameter or the batch size. Our results show that stochastic quasi-Newton algorithms are efficient and, in some instances, able to outperform the well-known first-order Adam optimizer, run with the optimal combination of its numerous hyperparameters, and the stochastic second-order trust-region STORM algorithm.

EEG-based epileptic seizure detection using binary dragonfly algorithm and deep neural network

Electroencephalogram (EEG) is one of the most common methods used for seizure detection as it records the electrical activity of the brain. Symmetry and asymmetry of EEG signals can be used as indicators of epileptic seizures. Normally, EEG signals are symmetrical in nature, with similar patterns on both sides of the brain. However, during a seizure, there may be a sudden increase in the electrical activity in one hemisphere of the brain, causing asymmetry in the EEG signal. In patients with epilepsy, interictal EEG may show asymmetric spikes or sharp waves, indicating the presence of epileptic activity. Therefore, the detection of symmetry/asymmetry in EEG signals can be used as a useful tool in the diagnosis and management of epilepsy. However, it should be noted that EEG findings should always be interpreted in conjunction with the patient's clinical history and other diagnostic tests. In this paper, we propose an EEG-based improved automatic seizure detection system using a Deep neural network (DNN) and Binary dragonfly algorithm (BDFA). The DNN model learns the characteristics of the EEG signals through nine different statistical and Hjorth parameters extracted from various levels of decomposed signals obtained by using the Stationary Wavelet Transform. Next, the extracted features were reduced using the BDFA which helps to train DNN faster and improve its performance. The results show that the extracted features help to differentiate the normal, interictal, and ictal signals effectively with 100% accuracy, sensitivity, specificity, and F1 score with a 13% selected feature subset when compared to the existing approaches.

A Hybrid DNN Multilayered LSTM Model for Energy Consumption Prediction

The ability to predict energy consumption in a world in which energy needs are ever-increasing is important for future growth and development. In recent years, deep learning models have made significant advancements in energy forecasting. In this study, a hybrid deep neural network (DNN) multilayered long short-term memory (LSTM) model was used to predict energy consumption in households. When evaluating the model, the individual household electric power consumption dataset was used to train, validate, and test the model. Preprocessing was applied to the data to minimize any prediction errors. Afterward, the DNN algorithm extracted the spatial features, and the multilayered LSTM model was used for sequential learning. The model showed a highly accurate predictive performance, as the actual consumption trends matched the predictive trends. The coefficient of determination, root-mean-square error, mean absolute error, and mean absolute percentage error were found to be 0.99911, 0.02410, 0.01565, and 0.01826, respectively. A DNN model and LSTM model were also trained to study how much improvement the proposed model would provide. The proposed model showed better performance than the DNN and LSTM models. Moreover, similar to other deep learning models, the proposed model’s performance was superior and provided accurate and reliable energy consumption predictions.

Long-term hydrological behavior of an Alpine glacier

Alpine glaciers are shrinking at a relentless pace, as an effect of global warming. The impact of these changes in the European Alps is relevant, given the importance of this territory from both ecological and economic viewpoints. While the ubiquitous reduction of glaciers’ mass through the Alps has been reported in several studies, its effect on streamflow is less studied. In the present work, we analyzed the long streamflow time series, available since 1976, of the Careser Stream, a headwater stream emerging from the Careser Glacier, in northeastern Italy. A large amount of missing data characterizes the streamflow time series, which we filled with a high level of confidence by using a Feed Forward Deep Neural Network algorithm. We explored the influence of climatic drivers on streamflow at monthly and annual time scales. Our analysis highlighted significant changes in the timing of streamflow due to the combined effect of early snow melting and a progressive reduction of the glacier’s area. The hydrological regime changed significantly with glacier water contributing to streamflow proportionally less after a tipping point identified in 1996. Projections performed by using eight bias-corrected climate models of the EURO-CORDEX collections confirmed this tendency with the complete transition of the catchment from the glacial to the nival regime by 2045 at the latest. The Deep Neural Network algorithm was very effective in filling the missing streamflow data and this offers an encouraging premise for applications in other glaciers of the Alps suffering from the same transformation.

Optimal design of one-dimensional elastic metamaterials through deep convolutional neural network and genetic algorithm

Deep learning (DL) has found extensive application in elastic metamaterial design. However, these methods are often limited to fixed design parameters and single structural configuration, lacking versatility. To overcome these, we propose a novel data-driven design framework combining deep convolutional neural network (DCNN) and genetic algorithm (GA) to design vibration-isolating metamaterial structures in foundations. This framework is highly adaptable, accommodating diverse design parameters by identical models. We discretize different structural configurations into a uniform tensor format, and generate multiple datasets. Two DCNN-based forward prediction models are then developed to accurately capture bandgap distributions for all body waves. Additionally, we incorporate multiple populations to enhance the parallel search capacity of GA, and integrate it with the trained DCNN models to simultaneously determine multiple optimal structures. The optimal design result for different parameters shows that the designed structures can achieve the target bandgap, and proves the effectiveness and generality of our method. Finally, through the design of two structural configurations, we find that four-layer structure exhibits wider low-frequency bandgaps and superior vibration reduction performance compared to the three-layer structure, under the same material composition and usage.

Development of machine learning models for estimating wheat biophysical variables using satellite-based vegetation indices

Monitoring wheat biophysical variables during the growing season is essential for achieving precision agriculture's important goals. This study estimated plant height, leaf development stage, leaf area index, dry matter, and plant nitrogen content in wheat from Landsat-7 images in North Iran using machine learning algorithms. Forty points were selected from ten wheat fields (270 ha), and the agronomic parameters were recorded during the 2011–2012 growing season. The machine learning models applied in this study were Artificial Neural Network (ANN), Support Vector Machine (SVM), and Deep Neural Network (DNN). Results suggest that the DNN model was the best for predicting all the parameters, except for the nitrogen percentage, in which SVM and DNN have the same accuracy. The DNN model R2 was 0.82, 0.95, 0.95, and 0.90 RMSE (Root Mean Square Error) 9.61, 0.46, 0.47 and 1.2 and MAE (Mean Absolute Error) 26.4, 0.34, 0.28, and 0.84 for estimating plant height, leaf area index, leaf development stage and dry matter biomass, respectively. The DNN and SVM algorithms can effectively predict plant nitrogen percentage with identical R2, RMSE, and MAE values of 0.83–0.84, 0.44–0.46, and 0.33–0.36, respectively. These results show that machine learning models have great potential to provide timely and accurate data for better wheat management under field conditions from satellite imagery.

THU676 Real-time Assessment Of The Beneficial Role Of Computer-Aided Diagnosis In The Diagnosis Of Thyroid Nodules On Ultrasound

Abstract Disclosure: Y. Kim: None. S. Shin: None. E. Lee: None. D. Kim: None. J. Kwak: None. Objective: Computer-aided Diagnosis (CAD) has been applied in various medical fields. SERA (SEveRance Artificial intelligence program) is a deep convolutional neural network algorithm. We conducted this study to evaluate whether CAD-SERA helps diagnose the malignancy when the ultrasound was used by physicians with low levels of experience and to compare with the experienced radiologist. Methods: In real-time, an inexperienced physician and a radiologist simultaneously assessed the same thyroid nodules to determine malignancy and performed CAD. The diagnosis was changed if it was necessary to correct it by referring to the CAD results, and this was compared with the surgical pathology or biopsy pathology. Results: We evaluated 191 thyroid nodules in 179 patients. Area Under the Receiver Operating Characteristics (AUROC) was analyzed with the diagnoses with/without CAD assistance and pathology results. The AUROC of the inexperienced physician was 0.716, and when assisted by CAD that increased to 0.740 (Accuracy 73.82 -> 76.44%). In the case of the radiologist, AUROC without CAD assist was 0.868 and 0.880 with the assist of CAD (Accuracy 84.29 -> 85.86%). Conclusion: CAD improves diagnostic performance in the diagnosis of thyroid nodules in various experienced physicians in ultrasound. In particular, the extent of this is greater in physicians with less experience in ultrasound. Presentation: Thursday, June 15, 2023