Neural Network Learning Methods Research Articles

FairScene: Learning unbiased object interactions for indoor scene synthesis

In this paper, we propose an unbiased graph neural network learning method called FairScene for indoor scene synthesis. Conventional methods directly apply graphical models to represent the correlation of objects for subsequent furniture insertion. However, due to the object category imbalance in dataset collection and complex object entanglement with implicit confounders, these methods usually generate significantly biased scenes. Moreover, the performance of these methods varies greatly for different indoor scenes. To address this, we propose a framework named FairScene which can fully exploit unbiased object interactions through causal reasoning, so that fair scene synthesis is achieved by calibrating the long-tailed category distribution and mitigating the confounder effects. Specifically, we remove the long-tailed object priors subtract the counterfactual prediction obtained from default input, and intervene in the input feature by cutting off the causal link to confounders based on the causal graph. Extensive experiments on the 3D-FRONT dataset show that our proposed method outperforms the state-of-the-art indoor scene generation methods and enhances vanilla models on a wide variety of vision tasks including scene completion and object recognition.

CardioDPi: An explainable deep-learning model for identifying cardiotoxic chemicals targeting hERG, Cav1.2, and Nav1.5 channels

The cardiotoxic effects of various pollutants have been a growing concern in environmental and material science. These effects encompass arrhythmias, myocardial injury, cardiac insufficiency, and pericardial inflammation. Compounds such as organic solvents and air pollutants disrupt the potassium, sodium, and calcium ion channels cardiac cell membranes, leading to the dysregulation of cardiac function. However, current cardiotoxicity models have disadvantages of incomplete data, ion channels, interpretability issues, and inability of toxic structure visualization. Herein, an interpretable deep-learning model known as CardioDPi was developed, which is capable of discriminating cardiotoxicity induced by the human Ether-à-go-go-related gene (hERG) channel, sodium channel (Na_v1.5), and calcium channel (Ca_v1.5) blockade. External validation yielded promising area under the ROC curve (AUC) values of 0.89, 0.89, and 0.94 for the hERG, Na_v1.5, and Ca_v1.5 channels, respectively. The CardioDPi can be freely accessed on the web server CardioDPipredictor (http://cardiodpi.sapredictor.cn/). Furthermore, the structural characteristics of cardiotoxic compounds were analyzed and structural alerts (SAs) can be extracted using the user-friendly CardioDPi-SAdetector web service (http://cardiosa.sapredictor.cn/). CardioDPi is a valuable tool for identifying cardiotoxic chemicals that are environmental and health risks. Moreover, the SA system provides essential insights for mode-of-action studies concerning cardiotoxic compounds.

Survey on graph neural network-based community detection and its applications

Detecting communities within a network is a critical component of network analysis. The process involves identifying clusters of nodes that exhibit greater similarity to each other compared to other nodes in the network. In the context of Complex networks (CN), community detection becomes even more important as these clusters provide relevant information of interest. Traditional mathematical and clustering methods have limitations in terms of data visualization and high-dimensional information extraction. To address these challenges, graph neural network learning methods have gained popularity in community detection, as they are capable of handling complex structures and multi-dimensional data. Developing a framework for community detection in complex networks using graph neural network learning is a challenging and ongoing research objective. Therefore, it is essential for researchers to conduct a thorough review of community detection techniques that utilize cutting-edge graph neural network learning methods [102], in order to analyze and construct effective detection models. This paper provides a brief overview of graph neural network learning methods based on community detection methods and summarizes datasets, evaluation metrics, applications, and challenges of community detection in complex networks.

Detection of diseases in rice leaf using convolutional neural network with transfer learning based on ResNeXt

Rice is the most cultivated crop in the agricultural field and is a major food source for more than 60% of the population in India. Occurrence of disease in rice leaves, majorly affects the rice quality, production and also the farmers’ income. Nowadays, new variety of diseases in rice leaves are identified and detected periodically throughout the world. Manual monitoring and detection of plant diseases proves to be time consuming for the farmer and also a costly affair for using chemicals in the disease treatment. In this paper, a deep learning method of convolutional neural networks (CNN) with a transfer learning technique is proposed for the detection of a variety of diseases in rice leaves. This method uses the ResNeXt network model for classifying the images of disease-affected plants. The proposed model’s performance is evaluated using accuracy, precision, recall, F1-score and specificity. The experimental results of ResNeXt model measured for accuracy, precision, recall, F1-score, and specificity, are respectively 99.22%, 92.87%, 91.97%, 90.95%, and 99.05%, which proves greater accuracy improvement than the existing methods of SG2-ADA, YOLOV5, InceptionResNetV2 and Raspberry Pi.

A sub-grid scale model for Burgers turbulence based on the artificial neural network method

The present study proposes a sub-grid scale (SGS) model for the one-dimensional Burgers turbulence based on the neural network and deep learning method. The filtered data of the direct numerical simulation is used to establish the training data set, the validation data set, and the test data set. The artificial neural network (ANN) method and Back Propagation method are employed to train parameters in the ANN. The developed ANN is applied to construct the sub-grid scale model for the large eddy simulation (LES) of the Burgers turbulence in the one-dimensional space. The proposed model well predicts the time correlation and the space correlation of the Burgers turbulence.

Effect of calcium content on the performance of CPVC cable protective tube

The influence mechanism of different calcium content on the mechanical properties of cable protection pipe was studied, and the feasibility of feedback verification on the using performance of cable protection pipe through neural network learning method was also studied. Methods Shore hardness tester, universal electronic testing machine and densitometer were used to test the performance of cable protection tubes with different calcium content, and the quantitative relationship curve between cable protection tubes and performance was obtained. The database of hardness, density and composition was established through BP neural network, and the feasibility of feedback quality prediction was verified through test data. Results The hardness and density of the cable protective tube can be effectively improved by increasing the calcium element in the cable protective tube. Through the feedback prediction of performance by BP neural network, its qualification rate can be accurately predicted. Through this study, the content of calcium element can be adjusted appropriately according to the use conditions to make its performance meet the use conditions. Through Shore hardness tester and spectrometer, the quality of cable protection tube can be determined, and the nondestructive testing effect can be achieved.

Identification and Classification of Coix seed Storage Years Based on Hyperspectral Imaging Technology Combined with Deep Learning

Developing a fast and non-destructive methodology to identify the storage years of Coix seed is important in safeguarding consumer well-being. This study employed the utilization of hyperspectral imaging (HSI) in conjunction with conventional machine learning techniques such as support vector machines (SVM), k-nearest neighbors (KNN), random forest (RF), extreme gradient boosting (XGBoost), as well as the deep learning method of residual neural network (ResNet), to establish identification models for Coix seed samples from different storage years. Under the fusion-based modeling approach, the model’s classification accuracy surpasses that of visible to near infrared (VNIR) and short-wave infrared (SWIR) spectral modeling individually. The classification accuracy of the ResNet model and SVM exceeds that of other conventional machine learning models (KNN, RF, and XGBoost). Redundant variables were further diminished through competitive adaptive reweighted sampling feature wavelength screening, which had less impact on the model’s accuracy. Upon validating the model’s performance using an external validation set, the ResNet model yielded more satisfactory outcomes, exhibiting recognition accuracy exceeding 85%. In conclusion, the comprehensive results demonstrate that the integration of deep learning with HSI techniques effectively distinguishes Coix seed samples from different storage years.

Expert System for Fourier Transform Infrared Spectra Recognition Based on a Convolutional Neural Network With Multiclass Classification.

Fourier transform infrared spectroscopy (FT-IR) is a widely used spectroscopic method for routine analysis of substances and compounds. Spectral interpretation of spectra is a labor-intensive process that provides important information about functional groups or bonds present in compounds and complex substances. In this paper, based on deep learning methods of convolutional neural networks, models were developed to determine the presence of 17 classes of functional groups or 72 classes of coupling oscillations in the FT-IR spectra. Using web scanning, the spectra of 14 361 FT-IR spectra of organic molecules were obtained. Several different variants of model architectures with different sizes of feature maps have been tested. Based on the Shapley additive explanations (SHAP) and gradient-weighted class activation mapping (GradCAM) methods, visualization tools have been developed for visualizing and highlighting the areas of absorption bands manifestation for corresponding functional groups or bonds in the spectrum. To determine 17 and 72 classes, the F1-weighted metric, which is the harmonic mean of the class' precision and class' recall weighted by class' fraction, reached 93 and 88%, respectively, when using data on the position of absorption maxima in the spectrum as an additional source layer. The resulting model can be used to facilitate the routine analysis of spectra for all areas such as organic chemistry, materials science, and biology, as well as to facilitate the preparation of the obtained experimental data for publication.

Multi-objective optimization analysis of combined heating and cooling transcritical CO2 system integrated with mechanical subcooling utilizing hydrocarbon mixture based on machine learning

To meet the simultaneous needs of high temperature disinfection and freezing in the field of food processing, a new concept of combined heating and cooling transcritical CO2 system integrated with dedicated mechanical subcooling utilizing hydrocarbon mixture is proposed. The system performance in terms of thermodynamics, economy and environment is studied and compared with the baseline combined heating and cooling transcritical CO2 system and four traditional combined heating and cooling solutions, considering the influence of temperature glide and the heat transfer deterioration. The new proposed system is further optimized by using the machine learning method of artificial neural network and non-dominated sorting genetic algorithm. Multi-objective optimization is conducted considering the objective function of energy efficiency, initial capital cost and life cycle carbon emissions of the new system, to obtain the optimum components and concentration ratio of the hydrocarbon mixture. The results indicate the thermodynamic performance and environmental benefits of subcooling subsystem with hydrocarbon mixture are better than those of the pure system. In contrast to that using pure R290 and R601, the coefficient of performance is enhanced by 8.20 % and 8.13 % and the life cycle carbon emission is reduced by 8.54 % and 9.31 %, respectively, when R290/R601 (50/50) is used. However, the initial capital cost is 9.25 % and 10.23 % higher than that of pure R290 and R601, respectively. Finally, the hydrocarbon mixture corresponding to the optimal design point is R1270/R601a (53/47), the corresponding discharge pressure is 12.86 MPa, and the subcooling degree is 37.50 °C. This study can provide a theoretical reference for the application of CO2 refrigeration and heat pump technology.

Hybrid ARMA-GARCH-Neural Networks for intraday strategy exploration in high-frequency trading

The frequency of armed conflicts increased during the last 20 years. The problems of the emergence of military disputes, not only concern social parameters, but also economic and financial dimensions. This study examines the potential impact of global geopolitical events on the stock market prices of the Dow Jones U.S. Aerospace & Defense Index and Foreign Exchange (FOREX) markets movements. We analyse whether defence stocks and exchange rate perform similarly during military incidents or geopolitical crises. We built an Autoregressive Moving Average Model with a Generalized Autoregressive Conditional Heteroskedasticity process (ARMA-GARCH) with the machine learning methods of Neural Networks, Deep Recurrent Convolutional Neural Networks, Deep Neural Decision Trees, Quantum Neural Networks, and Quantum Recurrent Neural Networks, aimed at detecting intraday patterns for forecasting defence stock market and FOREX markets disturbances in a market microstructure framework. The empirical results provide preliminary findings on the foreseeability of market disturbances and small differences are observed before and during geopolitical events. Additionally, we confirm the effectiveness of the hybrid model ARMA-GARCH with the machine learning approaches, being ARMA-GARCH-Quantum Recurrent Neural Network the technique that achieves the best accuracy results. Our work has a large potential impact on investment market agents and portfolio managers, as shocks from geopolitical events could provide a new methodology to support the decision-making process for trading in High-Frequency Trading.

CLASSIFICATION OF POTATO LEAF DISEASES USING CONVOLUTIONAL NEURAL NETWORK

Potatoes are an agricultural product that has the fourth highest content of wheat flour after corn, wheat, and rice. Although potatoes play a critical role in agriculture, this crop is susceptible to various diseases and pests. There are several potato leaf diseases that are not yet known to farmers. Dry spot potato leaf disease (late blight) and late blight. If not treated, this disease on potato leaves will spread to the stem and reduce crop yields, causing crop failure. By using technology in the form of digital image processing, this problem can be overcome. This research proposes an appropriate method for detecting disease in potato leaves. Classification will be carried out in three classes, namel, Early Blight, Healthy and Late Blight using the Deep Learning method of Convolutional Neural Network (CNN). The data used comes from an online dataset via the kaggle.com page with the file name Potato Disease Leaf Dataset (PLD) totaling 3251 training datasets which are then divided into training, testing, and validation. The processes carried out are image pre-processing, image augmentation, then image processing using a Convolutional Neural Network (CNN). In the classification process using the CNN method with RMSprop optimizer, the accuracy was 97.53% with a loss value of 0.1096.

A dataset of paddy rice planting areas along the Yangtze River (Jiangsu section) in 2020

As a staple food crop in China, paddy rice is one of the most widely grown crops in the country. Thanks to its abundant water resources, the Yangtze River has superiority in paddy rice planting. The prompt and accurate monitoring of paddy rice planting information plays an important role in providing essential technical support to safeguard China's food security. However, there is a scarcity of studies on the extraction of paddy rice planting areas along the Yangtze River using the remote sensing technology. In this paper, guided by paddy rice growth phenology information, we selected the Sentinel-2 satellite imagery that covered the paddy rice growing period (from May to September) along the Yangtze River in 2020. On the basis of high-spatial resolution images combined with field investigation, we obtained the real distribution map of the paddy rice planting area in Menghe Town and selected samples and validation samples from the map. Based on the maximum likelihood, neural network, SVM (Support Vector Machine) and other machine learning methods, we carried out the paddy rice extraction research, compared and analyzed the accuracy of extraction results of NDVI (Normalized Difference Vegetation Index) and REP (Red Edge Position Index) spectral index sets. The results show that the REP index sets of the three time phases in May, August and September are most effectively extracted using the neural network method. The overall accuracy of the results is 92.73%, with a corresponding Kappa coefficient of 0.72. We achieved global applicability by optimizing key parameters such as timing, spectral bands, and classifier methods for paddy rice extraction. Additionally, we conducted post-processing to refine the results. Finally, we obtained a vector dataset of paddy rice growing areas along the Yangtze River (Jiangsu section) in 2020, with an overall classification accuracy of 87.15%. This dataset is expected to provide a reference for further monitoring the paddy rice planting areas and enhancing the accuracy of grain production estimates throughout the Yangtze River region.

Automatically Predicting Material Properties with Microscopic Images: Polymer Miscibility as an Example.

Many material properties are manifested in the morphological appearance and characterized using microscopic images, such as scanning electron microscopy (SEM). Polymer miscibility is a key physical quantity of polymer materials and is commonly and intuitively judged using SEM images. However, human observation and judgment of the images is time-consuming, labor-intensive, and hard to be quantified. Computer image recognition with machine learning methods can make up for the defects of artificial judging, giving accurate and quantitative judgment. We achieve automatic miscibility recognition utilizing a convolutional neural network and transfer learning methods, and the model obtains up to 94% accuracy. We also put forward a quantitative criterion for polymer miscibility with this model. The proposed method can be widely applied to the quantitative characterization of the microstructure and properties of various materials.

Introduction to Citrus Fruit Ripens Using the Deep Learning Convolutional Neural Network (CNN) Learning Method

The export value of Indonesian fruits in 2023 will increase compared to 2021. For this reason, a program is needed to introduce fruit maturity, in this case, citrus fruits. Currently, the fruit maturity recognition system is still done manually which takes a long time and requires a lot of human resources. Thus, the purpose of this research is to use Machine Learning and the Convolution Neural Network (CNN) model in the classification of citrus fruit maturity. The computer image recognition method used is CNN, which has advantages in computer vision applications, face recognition, object detection, image recognition, and visual recognition. Datasets in the form of orange images are collected to be applied to the Machine Learning method. The test results showed that training accuracy reached 100% and validation accuracy reached 86.59% after 40 epochs using the CNN method on local varieties of orange images. Training loss reaches 0.7 and validation loss reaches 0.69 after 40 epochs.

Exploring buckling and post-buckling behavior of incompressible hyperelastic beams through innovative experimental and computational approaches

The objective of this paper is to conduct a comprehensive investigation into the buckling and post-buckling behavior of hyperelastic beams through both computational and experimental means. Natural rubber is used in the construction of a beam with a square cross-section. To determine the mechanical properties of natural rubber, a uniaxial tensile test is performed in accordance with ASTM D412. In finite element modeling (FEM), the nonlinear behavior of rubber is modeled using hyperelastic theory and the Yeoh strain energy function. The Static-Riks method is also implemented using Abaqus for the analysis of nonlinear buckling. To validate the present investigation results with FEM, an experimental test of digital image correlation (DIC) is conducted. The critical buckling force obtained via numerical methods exhibits an error of nearly 5% when compared to the corresponding results obtained from experimental testing. In order to ascertain the impact of various design parameters on the buckling behavior of the system, a comprehensive parametric analysis has been conducted. The parameters studied include the cross-sectional thickness, length of the structure, eccentric loads, as well as the mechanical properties of the materials used in the system. Consistent with the FEM outcomes, the critical buckling force exhibited by the hyperelastic beam demonstrates a positive correlation with increasing levels of hardness, cross-sectional thickness, and eccentric loads. The buckling behavior of the system is adversely affected by increasing its length. To ultimately validate the precision and reliability of the model, a supervised neural network (NN) learning method is employed.

FedHEONN: Federated and homomorphically encrypted learning method for one-layer neural networks

Federated learning (FL) is a distributed approach to developing collaborative learning models from decentralized data. This is relevant to many real applications, such as in the field of the Internet of Things, since the models can be used in edge computing devices. FL approaches are motivated by and designed to protect privacy, a highly relevant issue given current data protection regulations. Although FL methods are privacy-preserving by design, recently published papers show that privacy leaks do occur, caused by attacks designed to extract private data from information interchanged during learning. In this work, we present an FL method based on a neural network without hidden layers that incorporates homomorphic encryption (HE) to enhance robustness against the above-mentioned attacks. Unlike traditional FL methods that require multiple rounds of training for convergence, our method obtains the collaborative global model in a single training round, yielding an effective and efficient model that simplifies management of the FL training process. In addition, since our method includes HE, it is also robust against model inversion attacks. In experiments with big data sets and a large number of clients in a federated scenario, we demonstrate that use of HE does not affect the accuracy of the model, whose results are competitive with state-of-the-art machine learning models. We also show that behavior in terms of accuracy is the same for identically and non-identically distributed data scenarios.

Automated Detection of Breast Cancer Histopathology Image Using Convolutional Neural Network and Transfer Learning

cancer caused 2.3 million cases and 685,000 deaths in 2020. Histopathology analysis is one of the tests used to determine a patient’s prognosis. However, histopathology analysis is a time-consuming and stressful process. With advances in deep learning methods, computer vision science can be used to detect cancer in medical images, which is expected to improve the accuracy of prognosis. This study aimed to apply Convolutional Neural Network (CNN) and Transfer Learning methods to classify breast cancer histopathology images to diagnose breast tumors. This method used CNN, Transfer Learning ((Visual Geometry Group (VGG16), and Residual Network (ResNet50)). These models undergo data augmentation and balancing techniques applied to undersampling techniques. The dataset used for this study was ”The BreakHis Database of microscopic biopsy images of breast tumors (benign and malignant),” with 1693 data classified into two categories: Benign and Malignant. The results of this study were based on recall, precision, and accuracy values. CNN accuracy was 94%, VGG16 accuracy was 88%, and ResNet50 accuracy was 72%. The conclusion was that the CNN method is recommended in detecting breast cancer to diagnose breast cancer.

An accurate interatomic potential for the TiAlNb ternary alloy developed by deep neural network learning method.

The complex phase diagram and bonding nature of the TiAl system make it difficult to accurately describe its various properties and phases by traditional atomistic force fields. Here, we develop a machine learning interatomic potential with a deep neural network method for the TiAlNb ternary alloy based on a dataset built by first-principles calculations. The training set includes bulk elementary metals and intermetallic structures with slab and amorphous configurations. This potential is validated by comparing bulk properties-including lattice constant and elastic constants, surface energies, vacancy formation energies, and stacking fault energies-with their respective density functional theory values. Moreover, our potential could accurately predict the average formation energy and stacking fault energy of γ-TiAl doped with Nb. The tensile properties of γ-TiAl are simulated by our potential and verified by experiments. These results support the applicability of our potential under more practical conditions.

Prediction of Regional Forest Biomass Using Machine Learning: A Case Study of Beijing, China

Dynamic changes in forest biomass are closely related to the carbon cycle, climate change, forest productivity and biodiversity. However, most previous studies mainly focused on the calculation of current forest biomass, and only a few studies attempted to predict future dynamic changes in forest biomass which obtained uncertain results. Therefore, this study comprehensively considered the effects of multi-stage continuous survey data of forest permanent sample plots, site condition factors and corresponding meteorological factors using Beijing as an example. The geographic detector method was used to screen the key interfering factors that affect the growth of forest biomass. Then, based on the back-propagation artificial neural network (BP-ANN) and support vector machine (SVM) learning methods, 80% of the sample data were extracted to train the model, and thereby verify the prediction accuracy of different modeling methods using different training samples. The results showed that the forest biomass prediction models based on both the machine learning algorithms had good fitting accuracy, and there was no significant difference in the prediction results between the two models. However, the SVM model was better than the BP-ANN. While the BP-ANN model provided more volatile predictions, and the accuracy was above 80%, the prediction results of the SVM model were relatively stable, and the accuracy was above 90%. This study not only provides good technical support for the scientific estimation of regional forest biomass in the future, but also offers reliable basic data for sustainable forest management, planning decisions, forest carbon sequestration and sustainable development.

Deep learning methods for noisy sperm image classification from convolutional neural network to visual transformer: a comprehensive comparative study

Background With the gradual increase of infertility in the world, among which male sperm problems are the main factor for infertility, more and more couples are using computer-assisted sperm analysis (CASA) to assist in the analysis and treatment of infertility. Meanwhile, the rapid development of deep learning (DL) has led to strong results in image classification tasks. However, the classification of sperm images has not been well studied in current deep learning methods, and the sperm images are often affected by noise in practical CASA applications. The purpose of this article is to investigate the anti-noise robustness of deep learning classification methods applied on sperm images.Methods The SVIA dataset is a publicly available large-scale sperm dataset containing three subsets. In this work, we used subset-C, which provides more than 125,000 independent images of sperms and impurities, including 121,401 sperm images and 4,479 impurity images. To investigate the anti-noise robustness of deep learning classification methods applied on sperm images, we conducted a comprehensive comparative study of sperm images using many convolutional neural network (CNN) and visual transformer (VT) deep learning methods to find the deep learning model with the most stable anti-noise robustness.Results This study proved that VT had strong robustness for the classification of tiny object (sperm and impurity) image datasets under some types of conventional noise and some adversarial attacks. In particular, under the influence of Poisson noise, accuracy changed from 91.45% to 91.08%, impurity precison changed from 92.7% to 91.3%, impurity recall changed from 88.8% to 89.5%, and impurity F1-score changed 90.7% to 90.4%. Meanwhile, sperm precision changed from 90.9% to 90.5%, sperm recall changed from 92.5% to 93.8%, and sperm F1-score changed from 92.1% to 90.4%.Conclusion Sperm image classification may be strongly affected by noise in current deep learning methods; the robustness with regard to noise of VT methods based on global information is greater than that of CNN methods based on local information, indicating that the robustness with regard to noise is reflected mainly in global information.