Non-neural Network Research Articles

Deep Forest-Based Intelligent Yield Predicting of Buchwald-Hartwig Coupling Reaction

Buchwald-Hartwig coupling reaction is widely used in organic chemical synthesis, yield prediction is particularly important. In 2018, Science reported a yield prediction method based on random forest, but this method lacks feature learning. Therefore, an intelligent prediction and analysis method of coupling reaction yield based on deep forest is proposed. Combined with the advantages of deep learning and ensemble learning, the new deep model in the form of non-neural network is explored, which has good characterization learning ability and low difficulty in adjusting parameters, realizes the efficient prediction of chemical reaction, and analyzes the factors that have a significant impact on the prediction of reaction yield.

Android APK Identification using Non Neural Network and Neural Network Classifier

Currently adoption of mobile phones and mobile applications based on Android operating system is increasing rapidly. Many companies and emerging startups are carrying out digital transformation by using mobile applications to provide disruptive digital services to replace existing old styled services. This transformation prompted the attackers to create malicious software (malware) using sophisticate methods to target victims of Android mobile phone users. The purpose of this study is to identify Android APK files by classifying them using Artificial Neural Network (ANN) and Non Neural Network (NNN). The ANN is Multi-Layer Perceptron Classifier (MLPC), while the NNN are KNN, SVM, Decision Tree, Logistic Regression and Naïve Bayes methods. The results show that the performance using NNN has decreasing accuracy when training using larger datasets. The use of the K-Nearest Neighbor algorithm with a dataset of 600 APKs achieves an accuracy of 91.2% and dataset of 14170 APKs achieves an accuracy of 88%. The using of the Support Vector Machine algorithm with the 600 APK dataset has an accuracy of 99.1% and the 14170 APK dataset has an accuracy of 90.5%. The using of the Decision Tree algorithm with the 600 APK dataset has an accuracy of 99.2%, the 14170 APK dataset has an accuracy of 90.8%. The experiment using the Multi-Layer Perceptron Classifier has increasing with the 600 APK dataset reaching 99%, the 7000 APK dataset reaching 100% and the 14170 APK dataset reaching 100%.

Determination of adulteration in wheat flour using multi-grained cascade forest-related models coupled with the fusion information of hyperspectral imaging

Wheat flour (WF) is a common ingredient in staple foods. However, the presence of intentional or unintentional adulterants makes it difficult to guarantee WF quality. Multi-grained cascade forest (gcForest) model, a non-neural network deep learning structure, fused with image-spectra features from hyperspectral imaging (HSI) was employed for detecting adulterant type (peanut, walnut, or benzoyl peroxide) and the corresponding concentration (0.03%, 0.05%, 0.1%, 0.5%, 1%, and 2%). Based on the spectra of full wavelength and effective wavelength (EW) from hyperspectral images of WF samples, the gcForest-related models exhibited high performance (lowest ACCP = 92.45%) and stability (lowest area under the curve = 0.9986). Furthermore, the fusion of the EW and the image features extracted by the symmetric all convolutional neural network (SACNN) was used to establish the gcForest-related models. The maximum accuracy improvement of the fusion feature model relative to the single spectral model and the image model was 2.45% and 44.37%, respectively. The results indicate that the gcForest-related model, combined with the image-spectra fusion feature of HSI, provides an effective tool for detection in food and agriculture.

ON COMPARING MULTI-LAYER PERCEPTRON AND LOGISTIC REGRESSION FOR CLASSIFICATION OF DIABETIC PATIENTS IN FEDERAL MEDICAL CENTER YOLA, ADAMAWA STATE

The logistic regression (LR) and Multi-Layer (MLP) are used to handle regression analysis when the dependent response variable is categorical. Therefore, this study assesses the performance of LR and MLP in terms of classification of object/observations into identified component/groups. A data set consists of 553 cases of diabetes were collected at Federal Medical Center, . The variables measured: Age(years), Mass of a patient(kg/meters), glucose level (plasma glucose concentration, a 2-hour in an oral glucose tolerance test), pressure (Diastolic blood pressure ), insulin (2-hour serum insulin mu U/ml) and class variable (0 or 1) treating 0 as false or negative and 1 treated as true or positive test for diabetes. The method used in the study is Logistic regression analysis and the multi-Layer , a type of Artificial Neural Network, confusion matrix, classification, network algorithm and SPSS version 21 for Windows 10.1. The result of the study showed that LP classifies diabetic patients correctly with 91.8% accuracy. it classifies non-diabetic patients with 89.1% accuracy. MLP classifies diabetic patients with 88.6% accuracy while it classifies non-diabetic patients with 93.2% classification accuracy. Overall, MLP classifies better with 91% accuracy while LR classifies with 90.6% accuracy. This study complements other where MLP, a type Artificial neural network classifies and predicts better than other non-neural network classifiers.

Capsule Network Algorithm for Performance Optimization of Text Classification

In regions of visual inference, optimized performance is demonstrated by capsule networks on structured data. Classification of hierarchical multi-label text is performed with a simple capsule network algorithm in this paper. It is further compared to support vector machine (SVM), Long Short Term Memory (LSTM), artificial neural network (ANN), convolutional Neural Network (CNN) and other neural and non-neural network architectures to demonstrate its superior performance. The Blurb Genre Collection (BGC) and Web of Science (WOS) datasets are used for experimental purpose. The encoded latent data is combined with the algorithm while handling structurally diverse categories and rare events in hierarchical multi-label text applications.

Prediction of arsenic removal in aqueous solutions with non‐neural network algorithms

AbstractDespite some well‐known limitations of artificial neural network (ANN), the basic ANN and its derivatives are frequently used to predict the removal efficiency of different heavy metals like arsenic (As) using various adsorbents, including bio‐adsorbents. Notable examples of applying non‐neural network (NN) approaches, such as support vector regression (SVR) and random forest (RF), are almost non‐existent in theliterature. In the current study, the suitability of these modules to predict As removal efficiency was investigated based on seven independent experimental data sets. The SVR and RF experiments with the merged datasets of experimental and interpolated data points demonstrated their effectivity for the prediction. Specifically, the RF method achieved 97.3% Spearman's rank correlation coefficient (SRCC) and 95.9% R2 on average, whereas its SVR counterpart exhibited average performances of 96.3% SRCC and 93.9% R2 on a hold‐out test set. A general method based on the natural cubic spline technique was introduced to interpolate the experimental data points. Compared to the ANN methodology, the non‐NN approaches involved tuning fewer hyperparameters and easier training processes in the R open‐source framework. Furthermore, this kind of economical application of the machine learning algorithms allowed ranking the experimental parameters based on their relative contribution in the As removal efficiency. The study showed that the following parameters are the most influential ones, in decreasing order: pH, adsorbent dose, temperature, agitation time, and initial As concentration.

Deep forest regression based on cross-layer full connection

Key process parameters such as production qualities and environmental pollution indices are difficult to be measured online in complex industrial processes. High time and economic costs make only limited small sample data be obtained to build process models, while the deep neural network model requires massive training samples. Although the deep forest algorithm is based on nonneural network structure, it mainly is utilized to effectively address classification problems. Owing to the above problems, a new deep forest regression algorithm based on cross-layer full connection is proposed. First of all, sub-forest prediction values of the input layer forest module are processed to obtain the layer regression vector, which is combined with the raw feature vector as the input of the middle layer forest model. And then, a cross-layer full connection way connecting the former layer regression vector contributes to an augmented layer regression vector. Meanwhile, the deep layer’s number is adaptively adjusted via verifying the validation error. In the end, the output layer forest model is trained by using the augmented layer regression vector originated from the middle layer forest model and the raw feature vector. Sequentially, the maximum information flow is effectively ensured by information sharing. Moreover, the proposed method has the advantages of simple hyper-parameter setting criterion. Simulation results based on benchmark and industrial data show that the proposed method has equal or better performance than several state-of-art methods.

Question Answering Over Knowledge Base: A Scheme for Integrating Subject and the Identified Relation to Answer Simple Questions

Answering natural language question over a knowledge base is an important and challenging task with a wide range of application in natural language processing and information retrieval. Several existing knowledge-based question answering systems exploit complex end-to-end neural network approaches that are computationally expensive and take long to execute when training the neural network. More importantly, such an end-to-end approach makes it difficult to examine the process of query processing. In this study, we decompose the question answering problem in a three-step pipeline of entity detection, entity linking, and relation prediction, and solve each component separately. We explore basic neural network and non-neural network methods for entity detection and relation prediction plus a few heuristics for entity linking. We also introduce a method to identify ambiguity in the data and show that ambiguity in the data bounds the performance of the question answering system. The experiment on the SimpleQuestions benchmark data set shows that a combination of basic LSTMs, GRUs, and non-neural network techniques achieve reasonable performance while providing an opportunity to understand the question answering problem structure.

Recurrent disease progression networks for modelling risk trajectory of heart failure.

Recurrent neural networks (RNN) are powerful frameworks to model medical time series records. Recent studies showed improved accuracy of predicting future medical events (e.g., readmission, mortality) by leveraging large amount of high-dimensional data. However, very few studies have explored the ability of RNN in predicting long-term trajectories of recurrent events, which is more informative than predicting one single event in directing medical intervention. In this study, we focus on heart failure (HF) which is the leading cause of death among cardiovascular diseases. We present a novel RNN framework named Deep Heart-failure Trajectory Model (DHTM) for modelling the long-term trajectories of recurrent HF. DHTM auto-regressively predicts the future HF onsets of each patient and uses the predicted HF as input to predict the HF event at the next time point. Furthermore, we propose an augmented DHTM named DHTM+C (where "C" stands for co-morbidities), which jointly predicts both the HF and a set of acute co-morbidities diagnoses. To efficiently train the DHTM+C model, we devised a novel RNN architecture to model disease progression implicated in the co-morbidities. Our deep learning models confers higher prediction accuracy for both the next-step HF prediction and the HF trajectory prediction compared to the baseline non-neural network models and the baseline RNN model. Compared to DHTM, DHTM+C is able to output higher probability of HF for high-risk patients, even in cases where it is only given less than 2 years of data to predict over 5 years of trajectory. We illustrated multiple non-trivial real patient examples of complex HF trajectories, indicating a promising path for creating highly accurate and scalable longitudinal deep learning models for modeling the chronic disease.

Exploitation of Time Series Sentinel-2 Data and Different Machine Learning Algorithms for Detailed Tree Species Classification

The classification of tree species through remote sensing data is of great significance to monitoring forest disturbances, biodiversity assessment, and carbon estimation. The dense time series and a wide swath of Sentinel-2 data provided the opportunity to map tree species accurately and in a timely manner over a large area. Many current studies have applied machine learning (ML) algorithms combined with Sentinel-2 images to classify tree species, but it is still unclear, which algorithm is more effective in the automotive extraction of tree species. In this study, five ML algorithms were compared to identify the composition of tree species with multitemporal Sentinel-2 images in the JianShe forest farm, Northeast China. Three major types of deep neural networks [Conv1D, AlexNet, and long short-term memory (LSTM)] were tested to classify Sentinel-2 time series, which represent three disparate but effective strategies to apply sequential data. The other two models are support vector machine (SVM) and random forest (RF), which are renowned for extensive adoption and high performance for various remote sensing applications. The results show that the overall accuracy of neural network models is better than that of SVM and RF. The Conv1D model had the highest classification accuracy (84.19%), followed by the LSTM model (81.52%), and the AlexNet model (76.02%). For non-neural network models, RF's classification accuracy (79.04%) is higher than that of SVM (72.79%), but lower than that of Conv1D and LSTM. Therefore, the deep neural networks combined with multitemporal Sentinel-2 images can efficiently improve the accuracy of tree species classification.

A Survey of Deep Network Techniques All Classifiers Can Adopt.

Deep neural networks (DNNs) have introduced novel and useful tools to the machine learning community. Other types of classifiers can potentially make use of these tools as well to improve their performance and generality. This paper reviews the current state of the art for deep learning classifier technologies that are being used outside of deep neural networks. Non-neural network classifiers can employ many components found in DNN architectures. In this paper, we review the feature learning, optimization, and regularization methods that form a core of deep network technologies. We then survey non-neural network learning algorithms that make innovative use of these methods to improve classification performance. Because many opportunities and challenges still exist, we discuss directions that can be pursued to expand the area of deep learning for a variety of classification algorithms.

A new three-dimensional computerized ionospheric tomography model based on a neural network

Computerized ionospheric tomography (CIT) is an ill-posed inverse problem owing to insufficient data acquisition. Therefore, the ionospheric electron density (IED) distributions cannot be reconstructed accurately. Although many attempts have been made to deal with this issue, there is still a long way to go before it can be completely overcome. Specifically, the inverted IEDs of voxels without observational information show a strong dependence on initial values, which affects the overall accuracy of CIT. Taking this into account, a new three-dimensional CIT model is developed, based on a backpropagation neural network. The neural network model is trained using the characteristics and inverted IEDs of voxels with observational information, and then, the IEDs of voxels without observational information are predicted again. Careful validation of the proposed model is performed by conducting numerical experiments with GPS simulation and real data under both quiet and disturbed ionospheric conditions. Compared with the traditional non-neural network method in the simulation experiment, the proposed method offers improvements of 62.0 and 56.89% in root mean square error and the mean absolute error for those voxels without observational information, respectively, while it offers improvements of 30.98 and 26.67% for all voxels of the whole region. In the real data experiment, the IEDs of the control groups obtained by the proposed method are compared with the target IEDs for all periods. The result presents correlation coefficient greater than 0.96 between this predicted IEDs and the target IEDs for all periods, and this further certifies the feasibility of the proposed method. Additionally, the latitude–longitude maps and profiles of the ionospheric electron density also show that the ill-posedness problem has a significantly weaker effect for those voxels without observational information.

Neural resampler for Monte Carlo reweighting with preserved uncertainties

Monte Carlo event generators are an essential tool for data analysis in collider physics. To include subleading quantum corrections, these generators often need to produce negative weight events, which leads to statistical dilution of the datasets and downstream computational costs for detector simulation. Building on the recent proposal of a positive resampler method to rebalance weights within histogram bins, we introduce neural resampling: an unbinned approach to Monte Carlo reweighting based on neural networks that scales well to high-dimensional and variable-dimensional phase space. We pay particular attention to preserving the statistical properties of the event sample, such that neural resampling not only maintains the mean value of any observable but also its Monte Carlo uncertainty. This uncertainty preservation scheme is general and can also be applied to binned (non-neural network) resampling. To illustrate our neural resampling approach, we present a case study from the Large Hadron Collider of top quark pair production at next-to-leading order matched to a parton shower.

A hybrid deep forest approach for outlier detection and fault diagnosis of variable refrigerant flow system

Abstract This paper presents a hybrid deep forest approach for outlier detection and fault diagnosis. Isolation forest algorithm is combined with Pearson's correlation coefficient for outlier detection. The physical significance of outliers detected by the proposed algorithm is explained by origin analysis, which is rarely mentioned in existing studies. In addition, a novel non-neural network deep learning model-cascade forest model is proposed to fault diagnosis of HVAC system for the first time to achieve high precision accuracy in low-dimensional features. The proposed approach is validated with the refrigerant charge fault of VRF system. The results show that the isolation forest algorithm can improve the performance of fault diagnosis model and the mainly outliers of VRF system are defrosting data. The IF-CF model has short operation time, and high accuracy in low-dimensional features. When the dimension drops to 6, the accuracy of the IF-CF model is 94.16%, which is 5.26%, 10.02%, 5.87% and 3.34% higher than the IF-MLP, IF-BPNN, IF-SVM and IF-LSTM models, respectively. Moreover, IF-CF model does not require complex hyper-parameter optimization strategy because its maximum accuracy difference in different hyper-parameters is 2.04%. This study is enlightening which may inspire the potential of outlier detection technology and deep learning in HVAC field.

Analyzing Age-Related Macular Degeneration Progression in Patients with Geographic Atrophy Using Joint Autoencoders for Unsupervised Change Detection.

Age-Related Macular Degeneration (ARMD) is a progressive eye disease that slowly causes patients to go blind. For several years now, it has been an important research field to try to understand how the disease progresses and find effective medical treatments. Researchers have been mostly interested in studying the evolution of the lesions using different techniques ranging from manual annotation to mathematical models of the disease. However, artificial intelligence for ARMD image analysis has become one of the main research focuses to study the progression of the disease, as accurate manual annotation of its evolution has proved difficult using traditional methods even for experienced practicians. In this paper, we propose a deep learning architecture that can detect changes in the eye fundus images and assess the progression of the disease. Our method is based on joint autoencoders and is fully unsupervised. Our algorithm has been applied to pairs of images from different eye fundus images time series of 24 ARMD patients. Our method has been shown to be quite effective when compared with other methods from the literature, including non-neural network based algorithms that still are the current standard to follow the disease progression and change detection methods from other fields.

Multi-criteria comprehensive study on predictive algorithm of heating energy consumption of district heating station based on timeseries processing

Refined control of district heating station relies on reasonable and accurate prediction of heating energy consumption. Due to the influence of building thermal inertia and time-delay of the district heating system, certain research work is necessary to thoroughly illustrate and analyze the impact of data timeseries processing on various prediction models. Four kinds of prediction algorithms were investigated and compared in this paper. Results showed that all of three timeseries processing methods, namely time feature construction, sliding window and building thermal inertia coefficient (CBTI), can improve the prediction accuracy of all four models and CBTI had the greatest impact on model accuracy improvement. Moreover, timeseries processing method has no limitation on the types of prediction model and it is a general method to improve the model accuracy. Time feature construction and sliding window had greater influence on the non-neural network models while CBTI was the opposite. In terms of model robustness, the robustness had been significantly improved after introducing timeseries processing except random forest, and the comprehensive robustness coefficient for the other three models had been reduced by about 95%. Recurrent neural network had extremely excellent robustness under different temporal granularity.

Broad Forest: A Non-Neural Network Style Broad Model for Streaming Video QoE Evaluation

Currently, video streaming services put more emphasis on user feeling or satisfaction than before. How to design suitable model and algorithm to effectively and efficiently evaluate user quality of experience (QoE) has become a significant technical challenge. To get over this dilemma, this paper proposes the broad forest, a non-neural network style broad model for streaming video QoE evaluation. The design target of broad forest is to take advantage of representation potential of forest and low complexity characteristic of broad learning system. Specifically, we first give the construction of broad forest. Then, the associated incremental learning algorithm for efficiently supporting the added structure and inputs is designed. Finally, we apply the broad forest to streaming video QoE evaluation. Experimental results show that the broad forest can not only guarantee accuracy, but also decrease training time. When subjective features are considered, it can further promote performance of QoE evaluation.

Deep-Like Hashing-in-Hash for Visual Retrieval: An Embarrassingly Simple Method.

Existing hashing methods have yielded significant performance in image and multimedia retrieval, which can be categorized into two groups: shallow hashing and deep hashing. However, there still exist some intrinsic limitations among them. The former generally adopts a one-step strategy to learn the hashing codes for discovering the discriminative binary feature, but the latent discriminative information in the learned hashing codes is not well exploited. The latter, as deep neural network based hashing models, can learn highly discriminative and compact features, but relies on large-scale data and computation resources for numerous network parameters tuning with back-propagation optimization. Straightforward training of deep hashing models from scratch on small-scale data is almost impossible. Therefore, in order to develop efficient but effective learning to hash algorithm that depends only on small-scale data, we propose a novel non-neural network based deep-like learning framework, i.e. multi-level cascaded hashing (MCH) approach with hierarchical learning strategy, for image retrieval. The contributions are threefold. First, a hashing-in-hash architecture is designed in MCH, which inherits the excellent traits of traditional neural networks based deep learning, such that discriminative binary features that are beneficial to image retrieval can be effectively captured. Second, in each level the binary features of all preceding levels and the visual appearance feature are simultaneously cascaded as inputs of all subsequent levels to retrain, which fully exploits the implicated discriminative information. Third, a basic learning to hash (BLH) model with label constraint is proposed for hierarchical learning. Without loss of generality, the existing hashing models can be easily integrated into our MCH framework. We show experimentally on small- and large-scale visual retrieval tasks that our method outperforms several state-of-the-arts.

Lightweight Multilayer Random Forests for Monitoring Driver Emotional Status

This study proposes a lightweight multilayer random forest (LMRF) model, which is a non-neural network style deep model consisting of layer-by-layer random forests. Although a deep neural network (DNN) is a powerful algorithm for facial expression recognition (FER), the requirement of too many parameters, careful parameter tuning, a huge amount of training data, black-box models, and a pre-trained architecture remain significant burdens for a current DNN, particularly for real-time processing. To overcome the limitations of a DNN, our FER system uses LMRF consisting of a two-layer structure and a small number of trees per layer for fast FER. The proposed LMRF achieves a similar performance as a DNN even with a small number of hyper-parameters, and a faster processing time using a CPU. We conducted experiments using a benchmark database captured indoors and a real driving database captured using a near-infrared (NIR) camera. Based on a performance evaluation against a few other state-of-the-art FER methods, the proposed method showed a more uniform performance than DNN-based methods, and required a reduced number of parameters and operations without a loss of accuracy when compared to DNN model compression algorithms. As a replacement for deeper and wider networks, the proposed model can be embedded in low-power and low-memory in vehicle systems for the monitoring of a driver's emotion.

A Systematic Literature Review of Android Malware Detection Using Static Analysis

Android malware has been in an increasing trend in recent years due to the pervasiveness of Android operating system. Android malware is installed and run on the smartphones without explicitly prompting the users or without the user's permission, and it poses great threats to users such as the leakage of personal information and advanced fraud. To address these threats, various techniques are proposed by researchers and practitioners. Static analysis is one of these techniques, which is widely applied to Android malware detection and can detect malware quickly and prohibit malware before installation. To provide a clarified overview of the latest work in Android malware detection using static analysis, we perform a systematic literature review by identifying 98 studies from January 2014 to March 2020. Based on the features of applications, we first divide static analysis in Android malware detection into four categories, which include Android characteristic-based method, opcode-based method, program graph-based method, and symbolic execution-based method. Then we assess the malware detection capability of static analysis, and we compare the performance of different models in Android malware detection by analyzing the results of empirical evidence. Finally, it is concluded that static analysis is effective to detect Android malware. Moreover, there is a preliminary result that neural network model outperforms the non-neural network model in Android malware detection. However, static analysis still faces many challenges. Thus, it is necessary to derive some novel techniques for improving Android malware detection based on the current research community. Moreover, it is essential to establish a unified platform that is used to evaluate the performance of a series of techniques in Android malware detection fairly.