Kernel Model Research Articles

Synergistic Modeling of Liquid Properties: Integrating Neural Network-Derived Molecular Features with Modified Kernel Models.

A significant challenge in applying machine learning to computational chemistry, particularly considering the growing complexity of contemporary machine learning models, is the scarcity of available experimental data. To address this issue, we introduce an approach that derives molecular features from an intricate neural network-based model and applies them to a simpler conventional machine learning model that is robust to overfitting. This method can be applied to predict various properties of a liquid system, including viscosity or surface tension, based on molecular features drawn from the ab initio calculated free energy of solvation. Furthermore, we propose a modified kernel model that includes Arrhenius temperature dependence to incorporate theoretical principles and diminish extreme nonlinearity in the model. The modified kernel model demonstrated significant improvements in certain scenarios and possible extensions to various theoretical concepts of molecular systems.

NIMG-02. DEVELOPING A RADIOMIC HIERARCHICAL GAUSSIAN PROCESS BOOSTING MODEL TO PREDICT PRIMARY TUMOR ORIGIN FROM MULTICENTRIC LONGITUDINAL MRI DATA OF BRAIN METASTASES

Abstract Focal neurologic deficits due to brain metastases might be the initial presentation in patients with cancer. Common approaches to investigate the occult primary malignancy include a whole-body PET-CT, whole-body MRI, studying tumor markers, or histopathological examination. Most of these modalities are expensive, time-consuming, invasive, and delay treatment. The field of radiomics leverages quantitative features extracted from images to improve the decision-making process in clinical practices. Although not easily understandable, these features are proxies for the tumor’s shape, contour, texture, and intensity patterns. They can help understand tumor heterogeneity, predict tumor behavior, and assess treatment response. We propose a robust radiomic workflow using longitudinal MRI data from 914 high-resolution imaging studies from 12 different centers (Ocaña-Tienda et al. (2023), Ramakrishnan et al. (2023), Wang et al. (2023)) to predict the origin of brain metastases from brain MRI scans. After using ensemble feature selection methods, we trained three fixed-effects and one mixed-effects multiclass classification model to account for longitudinal data. Fixed-effects logistic regression and support vector kernel models performed the worst (AUC 0.74 and 0.75, respectively), while the random forest performed better (AUC 0.97). The mixed-effects hierarchical Gaussian process boosting (GPBoost) model performed the best with an accuracy of 85.8%, precision of 85.4%, sensitivity of 85.8%, specificity of 95.7%, and an AUC of 0.98. In one-versus-rest classification, the GPBoost model identified the primary tumor as melanoma with the highest accuracy (96.1%) and non-small cell lung cancer with the lowest accuracy (89.0%). Developing such models holds great promise in reducing patient and institutional costs, improving time to effective treatment, and enhancing overall survival rates through personalized therapy approaches. In the future, we hope that incorporating newer patient data and refining the model to improve external validity would enable more accurate and generalizable predictions, leading to better clinical outcomes across diverse populations.

Optimizing neural network models for predicting nuclear reactor channel temperature: A study on hyperparameter tuning and performance analysis

This study emphasizes how important accurate prediction of channel temperatures in nuclear reactors is for safety and operational efficiency. While traditional methods require long and complex processes such as kernel modeling and mathematical simulations, artificial neural networks (ANN) provide more efficient predictions by accelerating this process. The superior ability of ANNs to process large data sets is intended to demonstrate that this study will provide a valuable alternative compared to conventional methods and increase the accuracy of reactor temperature predictions. In this study, the training performances of Artificial Neural Network (ANN) developed to determine the nuclear reactor channel temperature with different hyperparameter combinations were analysed. It was conducted several experimental studies to assess the influence of hyperparameters on our model for nuclear reactor parameter data prediction. The training and validation results indicates that learning rate, hidden layer sizes and number have critical effects for the more precisive prediction. It was observed that models with a learning rate of 0.05 and 0.5 achieved successful learning with less fluctuation in training and validation errors. When looking at hidden layer sizes, networks with 32 and 64 neurons performed better than networks with 16 neurons. For the test phase our model can successfully predict data with slight error margin. As a result, we demonstrated that neural networks are a powerful tool in nuclear reactor channel temperature prediction through our proposed model.

Expressibility-induced concentration of quantum neural tangent kernels

Quantum tangent kernel methods provide an efficient approach to analyzing the performance of quantum machine learning models in the infinite-width limit, which is of crucial importance in designing appropriate circuit architectures for certain learning tasks. Recently, they have been adapted to describe the convergence rate of training errors in quantum neural networks in an analytical manner. Here, we study the connections between the expressibility and value concentration of quantum tangent kernel models. In particular, for global loss functions, we rigorously prove that high expressibility of both the global and local quantum encodings can lead to exponential concentration of quantum tangent kernel values to zero. Whereas for local loss functions, such issue of exponential concentration persists owing to the high expressibility, but can be partially mitigated. We further carry out extensive numerical simulations to support our analytical theories. Our discoveries unveil a fundamental feature of quantum neural tangent kernels, indicating that the issue of their concentration cannot be bypassed merely by transitioning to a local encoding scheme while maintaining high expressibility. This offers valuable insights for the design of wide quantum variational circuit models in practical applications.

Dynamic evolution and influencing factors of internal and external synergy in China's energy industry: From the perspective of industrial chain

The distortion of factors between the upstream and downstream of the energy industry, as well as the lack of collaborative models between it and the energy equipment manufacturing industry, impede the smooth achievement of the "dual-carbon" goal, and industrial synergy can better solve this problem. From the perspective of the industrial chain, this study measures for the first time the level of internal and external synergies in the energy industry based on industrial data from 30 provinces in China from 2013-2022, and analyzes in depth the dynamic evolution of internal and external synergies in the energy industry and the factors influencing them. Using ArcMap 10.2 and the variance decomposition model, it is found that the upstream and downstream sectors of the energy industry have non-equilibrium in time and space, and their factor mismatch in resource-based regions is still more serious. Therefore, a state of synergy needs to be created between upstream and downstream sectors, between the energy industry and the energy equipment manufacturing industry. Using each of the status quo maps as well as the Kernel model, it was concluded that external synergies decrease and then increase, with a gradual decrease in the gap between regions. However, internal synergies show an upward and then downward trend. Meanwhile, the results of the GTWR model show that the effects of various influences on internal and external synergies are not only positively or negatively different, but also spatially more unbalanced. It should accelerate the promotion of policy guidance, actively cultivate the external environment, rationalize the layout of regional industries, and promote the synergistic development of the energy industry internally and externally.

Extraordinarily Time- and Memory-Efficient Large-Scale Canonical Correlation Analysis in Fourier Domain: From Shallow to Deep.

Canonical correlation analysis (CCA) is a correlation analysis technique that is widely used in statistics and the machine-learning community. However, the high complexity involved in the training process lays a heavy burden on the processing units and memory system, making CCA nearly impractical in large-scale data. To overcome this issue, a novel CCA method that tries to carry out analysis on the dataset in the Fourier domain is developed in this article. Appling Fourier transform on the data, we can convert the traditional eigenvector computation of CCA into finding some predefined discriminative Fourier bases that can be learned with only element-wise dot product and sum operations, without complex time-consuming calculations. As the eigenvalues come from the sum of individual sample products, they can be estimated in parallel. Besides, thanks to the data characteristic of pattern repeatability, the eigenvalues can be well estimated with partial samples. Accordingly, a progressive estimate scheme is proposed, in which the eigenvalues are estimated through feeding data batch by batch until the eigenvalues sequence is stable in order. As a result, the proposed method shows its characteristics of extraordinarily fast and memory efficiencies. Furthermore, we extend this idea to the nonlinear kernel and deep models and obtained satisfactory accuracy and extremely fast training time consumption as expected. An extensive discussion on the fast Fourier transform (FFT)-CCA is made in terms of time and memory efficiencies. Experimental results on several large-scale correlation datasets, such as MNIST8M, X-RAY MICROBEAM SPEECH, and Twitter Users Data, demonstrate the superiority of the proposed algorithm over state-of-the-art (SOTA) large-scale CCA methods, as our proposed method achieves almost same accuracy with the training time of our proposed method being 1000 times faster. This makes our proposed models best practice models for dealing with large-scale correlation datasets. The source code is available at https://github.com/Mrxuzhao/FFTCCA.

Fuzzy Neural Tangent Kernel Model for Identifying DNA N4-Methylcytosine Sites

Fuzzy Neural Tangent Kernel Model for Identifying DNA N4-Methylcytosine Sites

A novel hybrid random convolutional kernels model for price volatlity forecasting of precious metals

ABSTRACTPrecious metals are rare metals with high economic value. Forecasting the price volatility of precious metals is essential for investment purposes. In this work, we propose a novel hybrid model of random convolutional kernels‐based neural network model (RCK) and generalized autoregressive conditional heteroscedasticity (GARCH) model for forecasting the metal price volatilities of gold, silver, and platinum. Realized volatility calculated on logarithmic returns is used as an estimate for the volatility of prices, and data standardization is performed before feeding the price volatility data to the RCK model. RCK model applies multiple carefully designed random convolution kernels on the time series input to extract robust features for forecasting. The proportion of positive values (PPV) is extracted as features from the output of convolving convolutional kernels with time‐series inputs, which are then passed through a regressor to forecast volatility. Compared to the existing methods, the proposed method has the advantage that the weights of the random convolutional kernels need not be trained, unlike other neural network models. Further, no other work has made use of random convolutional kernels for precious metal forecasting, to the best of our knowledge. We incorporated novel learning and data augmentation strategies to achieve better performance. In particular, we used the cosine annealing learning rate strategy and Mixup data augmentation technique to improve the proposed model's performance. We have used MSE (Mean Squared Error), RMSE (Root Mean Squared Error), MAE (Mean Absolute Error), and MAPE (Mean Absolute Percentage Error) as metrics to compare the proposed models' performance. The proposed model decreases the MSE by 53% compared to the GARCH‐LSTM model, which is the current state‐of‐the‐art hybrid model for volatility forecasting.

Physics-constrained adaptive kernel interpolation for region-to-region acoustic transfer function: a Bayesian approach

A kernel interpolation method for the acoustic transfer function (ATF) between regions constrained by the physics of sound while being adaptive to the data is proposed. Most ATF interpolation methods aim to model the ATF for fixed source by using techniques that fit the estimation to the measurements while not taking the physics of the problem into consideration. We aim to interpolate the ATF for a region-to-region estimation, meaning we account for variation of both source and receiver positions. By using a very general formulation for the reproducing kernel function, we have created a kernel function that considers both directed and residual fields as two separate kernel functions. The directed field kernel considers a sparse selection of reflective field components with large amplitudes and is formulated as a combination of directional kernels. The residual field is composed of the remaining densely distributed components with lower amplitudes. Its kernel weight is represented by a universal approximator, a neural network, in order to learn patterns from the data freely. These kernel parameters are learned using Bayesian inference both under the assumption of Gaussian priors and by using a Markov chain Monte Carlo simulation method to perform inference in a more directed manner. We compare all established kernel formulations with each other in numerical simulations, showing that the proposed kernel model is capable of properly representing the complexities of the ATF.

Optimizing Lung Cancer Diagnosis with Machine Learning and Feature Selection Methods

Lung cancer is a prevalent disease, with nearly 238,000 new cases diagnosed in 2023. This study utilizes clinical predictors from a Kaggle dataset containing 309 observations across 15 variables to aid in lung cancer diagnosis. The variables include swallowing difficulty, peer pressure, gender, allergy, yellow fingers, anxiety, wheezing, alcohol consumption, chronic disease, chest pain, coughing, fatigue, smoking, age, and shortness of breath. The research aims to develop and compare various supervised machine learning models for classifying and predicting lung cancer, while also identifying key clinical tests and parameters using unsupervised statistical models. The dataset was divided into training and test sets, balanced, and preprocessed for unbiased training. Feature selection and machine learning models were applied to identify crucial predictors. The study explored tree models, logistic regression, Naïve Bayes, support vector machine (SVM), ensemble, neural network, and kernel models. Among these, the linear SVM achieved the highest accuracy of 93.75% with 5-fold cross-validation. However, it showed overfitting, with a lower test accuracy of 82.55%. The Gaussian Naïve Bayes model emerged as the optimal choice, providing consistent performance between validation and test cases. It achieved the highest cross-validation classification accuracy of 82.81% using only 9 variables: swallowing difficulty, peer pressure, gender, allergy, yellow fingers, anxiety, wheezing, alcohol consumption, and chronic disease. This model allows for effective training with fewer predictors without compromising classification

The fast committor machine: Interpretable prediction with kernels.

In the study of stochastic systems, the committor function describes the probability that a system starting from an initial configuration x will reach a set B before a set A. This paper introduces an efficient and interpretable algorithm for approximating the committor, called the "fast committor machine" (FCM). The FCM uses simulated trajectory data to build a kernel-based model of the committor. The kernel function is constructed to emphasize low-dimensional subspaces that optimally describe the A to B transitions. The coefficients in the kernel model are determined using randomized linear algebra, leading to a runtime that scales linearly with the number of data points. In numerical experiments involving a triple-well potential and alanine dipeptide, the FCM yields higher accuracy and trains more quickly than a neural network with the same number of parameters. The FCM is also more interpretable than the neural net.

A Formal Verification Approach for Linux Kernel Designing

Although the Linux kernel is widely used, its complexity makes errors common and potentially serious. Traditional formal verification methods often have high overhead and rely heavily on manual coding. They typically verify only specific functionalities of the kernel or target microkernels and do not support continuous verification of the entire kernel. To address these limitations, we introduce LMVM (Linux Kernel Modeling and Verification Method), a formal method based on type theory that ensures the correct design of the Linux architecture. In the model, the kernel is treated as a top-level type, subdivided into the following sublevels: subsystem, dentry, file, struct, function, and base. These types are defined in the structure and relationships. The verification process includes checking the design specifications for both type relationships and the presence of each type. Our contribution lies primarily in the following two points: 1. This is a lightweight verification. As long as the modeling is complete, architectural errors in the design phase can be identified promptly. 2. The designed “model refactor” module supports kernel updating, and the kernel can be continuously verified by extending the kernel model. To test its usefulness, we develop a set of security communication mechanisms in the kernel, which are verified using our method.

Detection of network false information based on artificial intelligence models

There is often some false information in social platforms to mislead public opinion. Due to the rapid development of the Internet, the spread of false information on the Internet has become easier, which has brought many losses to people's economy and life. In this paper, the relevant research on false information based on bidirectional convolutional networks is analyzed, and the method is divided into four stages: data preprocessing, model architecture, training process and prediction process. Then, the relevant research on rumor detection by propagation tree kernel model are analyzed. Finally, this paper delineates a comprehensive framework that amalgamates enhanced Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, and a hybridized Black Widow Optimization (BWO) with Moth Optimization Algorithm (MOA) (referred to as HM-BWO) for the accurate detection of network-based false information. This paper analyzes and discusses the challenges of poor universality and insufficient detection speed encountered by the current rumor detection research and puts forward the idea of introducing migration model to solve the problem of poor universality and Spark to solve the problem of insufficient detection speed. This article provides a good overview of the field of online disinformation.

GIS-based G × E modeling of maize hybrids through enviromic markers engineering.

Through enviromics, precision breeding leverages innovative geotechnologies to customize crop varieties to specific environments, potentially improving both crop yield and genetic selection gains. In Brazil's four southernmost states, data from 183 distinct geographic field trials (also accounting for 2017-2021) covered information on 164 genotypes: 79 phenotyped maize hybrid genotypes for grain yield and their 85 nonphenotyped parents. Additionally, 1342 envirotypic covariates from weather, soil, sensor-based, and satellite sources were collected to engineer 10 K synthetic enviromic markers via machine learning. Soil, radiation light, and surface temperature variations remarkably affect differential genotype yield, hinting at ecophysiological adjustments including evapotranspiration and photosynthesis. The enviromic ensemble-based random regression model showcases superior predictive performance and efficiency compared to the baseline and kernel models, matching the best genotypes to specific geographic coordinates. Clustering analysis has identified regions that minimize genotype-environment (G × E) interactions. These findings underscore the potential of enviromics in crafting specific parental combinations to breed new, higher-yielding hybrid crops. The adequate use of envirotypic information can enhance the precision and efficiency of maize breeding by providing important inputs about the environmental factors that affect the average crop performance. Generating enviromic markers associated with grain yield can enable a better selection of hybrids for specific environments.

Measuring Cybercrime in Calls for Police Service

Conventional police databases contain much information on cybercrime, but extracting it remains a practical challenge. This is because these databases rarely contain labels that could be used to automatically retrieve all cybercrime incidents. In this article, we present a supervised machine learning method for extracting cybercrime incidents in calls for police service datasets. Data from the Korean National Police (2020, 9 months, N = 15 million call logs) is used for the demonstration. We combined methods of keyword query selection, minority oversampling, and majority voting techniques to develop a classifier. Three classification techniques, including Naïve Bayes, linear SVM, and kernel SVM, were tested, and the kernel model was chosen to build the final model (accuracy, 93.4%; F1-score, 92.4). We estimate that cybercrime only represents 4.6% of the cases in the selected dataset (excluding traffic-related incidents), but that it can be prevalent with some crime types. We found, for example, that about three quarters (76%) of all fraud incidents have a cyber dimension. We conclude that the cybercrime classification method proposed in this study can support further research on cybercrime and that it offers considerable advantages over manual or keyword-based approaches.

Exponential concentration in quantum kernel methods

Kernel methods in Quantum Machine Learning (QML) have recently gained significant attention as a potential candidate for achieving a quantum advantage in data analysis. Among other attractive properties, when training a kernel-based model one is guaranteed to find the optimal model’s parameters due to the convexity of the training landscape. However, this is based on the assumption that the quantum kernel can be efficiently obtained from quantum hardware. In this work we study the performance of quantum kernel models from the perspective of the resources needed to accurately estimate kernel values. We show that, under certain conditions, values of quantum kernels over different input data can be exponentially concentrated (in the number of qubits) towards some fixed value. Thus on training with a polynomial number of measurements, one ends up with a trivial model where the predictions on unseen inputs are independent of the input data. We identify four sources that can lead to concentration including expressivity of data embedding, global measurements, entanglement and noise. For each source, an associated concentration bound of quantum kernels is analytically derived. Lastly, we show that when dealing with classical data, training a parametrized data embedding with a kernel alignment method is also susceptible to exponential concentration. Our results are verified through numerical simulations for several QML tasks. Altogether, we provide guidelines indicating that certain features should be avoided to ensure the efficient evaluation of quantum kernels and so the performance of quantum kernel methods.

Performance of logistic regression and support vector machine conjunction with the GIS and RS in the landslide susceptibility assessment: Case study in Nakhon Si Thammarat, southern Thailand

The occurrence of landslides has risen in the past few decades, particularly in mountainous regions worldwide, including Nakhon Si Thammarat, southern Thailand. Despite various methods being employed for the initial management of landslide disasters, none have proven universally effective. The goal of this research is to create and assess landslide susceptibility maps (LSMs) within this area by employing support vector machine (SVM) and logistic regression, together with Geographic Information System (GIS) and Remote Sensing (RS) techniques. Eleven factors contributing to landslides were identified as topographic, environmental, and geological influences. The 365 landslides in the past were aimlessly selected into training (70%) and testing (30%) datasets. The four LSMs indicated that approximately 13%–20% of this study area exhibit a high susceptibility to landslides corresponding to the regions of high elevation with relatively steep slope angles. To evaluate and compare LSM models, the AUC value for training dataset were 0.977, 0.975, 0.958, and 0.967 and testing dataset were 0.973, 0.969, 0.956, and 0.964 for SVM with the radial basis function (rbf) kernel, SVM with polynomial deg 2, SVM with linear kernel and logistic regression models, respectively. Among these models, SVMs with rbf demonstrated the highest prediction rate. However, it requires a significant amount of time to choose the best parameters for achieving the highest accuracy prediction. In summary, these maps are applicable at the regional level to enhance the management of landslide hazards.

Predicting the PCM-incorporated building's performance using optimized linear kernel and tree-based machine learning methods

The devising of a precise machine learning-based energy consumption prediction model holds significant importance for taking effective decisions during the early stages of phase change material (PCM) incorporated building design to reduce energy consumption. Few researchers have developed prediction models for estimating the consumption of energy in PCM incorporated buildings. However, there is a need to develop the best-performing machine learning-based prediction model, which is less complex, involve a lower number of hyperparameters, and has a greater ability to generalize hidden patterns among the dependent and independent variables. To address the above shortcomings, the linear kernel and tree model-based prediction models were formulated for PCM-incorporated buildings located in eight cities of subtropical highland climate, considering the variations in their hyperparameters. The evaluation and validation of the formulated models for prediction were performed utilizing several statistical metrics. Test results substantiated that the ensemble-boosted tree-based prediction model (EBT20) exhibited superior efficacy in estimating the energy consumption for PCM-integrated building, having an average R2 > 0.93, NMBE <4 %, and CV-RMSE <8 %. The developed model showed less complexity and better generalization ability at the optimized values of ensemble boosted tree hyperparameters, which are as follows: minimum leaf size = 32, number of learners = 99, and learning rate = 0.1. From comprehensive evaluation conducted to evaluate the interpretability of the prediction model, it was found that the building orientation and PCMs melting temperature are the most influential parameters for achieving energy savings of up to 33 % in the selected climate zone.

Combination of static and dynamic neural imaging features to distinguish sensorineural hearing loss: a machine learning study.

Sensorineural hearing loss (SNHL) is the most common form of sensory deprivation and is often unrecognized by patients, inducing not only auditory but also nonauditory symptoms. Data-driven classifier modeling with the combination of neural static and dynamic imaging features could be effectively used to classify SNHL individuals and healthy controls (HCs). We conducted hearing evaluation, neurological scale tests and resting-state MRI on 110 SNHL patients and 106 HCs. A total of 1,267 static and dynamic imaging characteristics were extracted from MRI data, and three methods of feature selection were computed, including the Spearman rank correlation test, least absolute shrinkage and selection operator (LASSO) and t test as well as LASSO. Linear, polynomial, radial basis functional kernel (RBF) and sigmoid support vector machine (SVM) models were chosen as the classifiers with fivefold cross-validation. The receiver operating characteristic curve, area under the curve (AUC), sensitivity, specificity and accuracy were calculated for each model. SNHL subjects had higher hearing thresholds in each frequency, as well as worse performance in cognitive and emotional evaluations, than HCs. After comparison, the selected brain regions using LASSO based on static and dynamic features were consistent with the between-group analysis, including auditory and nonauditory areas. The subsequent AUCs of the four SVM models (linear, polynomial, RBF and sigmoid) were as follows: 0.8075, 0.7340, 0.8462 and 0.8562. The RBF and sigmoid SVM had relatively higher accuracy, sensitivity and specificity. Our research raised attention to static and dynamic alterations underlying hearing deprivation. Machine learning-based models may provide several useful biomarkers for the classification and diagnosis of SNHL.

Flavor release from walnut kernels in an in-vitro mastication model with decoupled oral parameters

Consumer preferences for walnut products are largely determined by the flavors released during mastication. In this study, a peeled walnut kernel (PWK) model was established with oral parameters decoupled using a Hutchings 3D model. The model explored in vitro variations using head-space solid-phase microextraction-gas chromatography-mass spectrometry and intelligent sensory techniques. The fracture strength, hardness, particle size, adhesiveness, springiness, gumminess, and chewiness were significantly reduced during mastication. We identified 61 volatile compounds and found that 2,5-dimethyl-3-ethylpyrazine is a key component, releasing predominantly baking and milky notes. Glutamic acid, alanine, arginine, and sucrose were identified as the key compounds in taste perception. The method can help establish a mastication model for nuts and facilitate breakthroughs in the development of walnut products and processing methods.