Traditional Grid Search Research Articles

Optimized Random Forest Method for 3D Evaluation of Coalbed Methane Content Using Geophysical Logging Data.

Accurate evaluation of coalbed methane (CBM) content is crucial for effective exploration and development. Traditional gas content measurement methods based on laboratory analysis of drill core samples are costly, whereas geophysical logging methods offer a cost-effective alternative by providing continuous high-resolution profiles of rock layer physical properties. However, the relationship between CBM content and geophysical logging data is complex and nonlinear, necessitating an advanced prediction method. This study focuses on the No. 3 coal seam in the Shizhuang South Block of the Qinshui Basin, utilizing geophysical logging data and 148 sets of laboratory core samples. We employed the Random Forest (RF) method optimized with a simulated annealing-genetic algorithm (SA-GA) to develop the SA-GA-RF model for evaluating CBM content. The model's performance was validated using test data and new CBM well data, and it was applied to calculate the vertical gas content profiles of No. 3 coal seam across 128 wells. The SA-GA-RF model demonstrated an average relative error of 13.13% in the test data set, outperforming Backpropagation Neural Network (BPNN), Least Squares Support Vector Machine (LSSVM), Extreme Learning Machine (ELM), and multivariate regression (MR) methods. The model also exhibited strong generalizability in new wells and improved model-building efficiency compared to traditional cross-validation grid search methods. The construction of a three-dimensional CBM content model, incorporating well coordinates and elevation data, allowed for detailed identification of high gas content areas and layers. This three-dimensional model offers a more precise characterization than traditional two-dimensional isopleth maps, providing valuable insights for CBM exploration, reserve evaluation, and production optimization.

Research on stock trend prediction based on grey wolf optimized RF and SVM models

The volatility of the stock market has a profound impact on the stability of the financial market, and then has a significant effect on the overall economic situation, so the accurate prediction of stock trend has always been the focus of research in the financial field. In this paper, the raw minute-level stock data of SSE 50 and CSI 300 in 2021 are normalized, and the historical training and test sets of daily stocks for model training and testing are established based on the datasets. The RF-SVM model based on the gray wolf optimization algorithm is constructed to compare the traditional support vector machine and random forest model, and the relevant performance indexes are calculated. In this study, the RF-SVM model based on gray wolf optimization is highly correlated with the characteristics of stock data, and the prediction accuracy of stock index data is more than 85%. The cross-validation score is over 0.77 and the overall weighted score is as high as 0.98. Compared with the SVM and random forest models under the default parameters, the RF-SVM model based on gray wolf optimization has an average advantage of 25% in performance indicators. The generalization ability is increased by about 26%; The fitting error was reduced by about 51.15%; The overall average score advantage is about 34%. These results prove that the RF-SVM model of gray wolf optimization has superior performance and significant optimization effect in stock prediction, and compared with the traditional grid search optimization algorithm, gray wolf optimization has excellent adaptability and can gradually obtain the global optimal parameter configuration.

TPTM-HANN-GA: A Novel Hyperparameter Optimization Framework Integrating the Taguchi Method, an Artificial Neural Network, and a Genetic Algorithm for the Precise Prediction of Cardiovascular Disease Risk

The timely and precise prediction of cardiovascular disease (CVD) risk is essential for effective prevention and intervention. This study proposes a novel framework that integrates the two-phase Taguchi method (TPTM), the hyperparameter artificial neural network (HANN), and a genetic algorithm (GA) called TPTM-HANN-GA. This framework efficiently optimizes hyperparameters for an artificial neural network (ANN) model during the training stage, significantly enhancing prediction accuracy for cardiovascular disease (CVD) risk. The proposed TPTM-HANN-GA framework requires far fewer experiments than a traditional grid search, making it highly suitable for application in resource-constrained, low-power computers, and edge artificial intelligence (edge AI) devices. Furthermore, the proposed TPTM-HANN-GA framework successfully identified the optimal configurations for the ANN model’s hyperparameters, resulting in a hidden layer of 4 nodes, a tanh activation function, an SGD optimizer, a learning rate of 0.23425849, a momentum rate of 0.75462782, and seven hidden nodes. This optimized ANN model achieves 74.25% accuracy in predicting the risk of cardiovascular disease, which exceeds the existing state-of-the-art GA-ANN and TSTO-ANN models. The proposed TPTM-HANN-GA framework enables personalized CVD prediction to be efficiently conducted on low-power computers and edge-AI devices, achieving the goal of point-of-care testing (POCT) and empowering individuals to manage their heart health effectively.

Enhanced osteoporotic fracture prediction in postmenopausal women using Bayesian optimization of machine learning models with genetic risk score.

This study aimed to enhance the fracture risk prediction accuracy in major osteoporotic fractures (MOFs) and hip fractures (HFs) by integrating genetic profiles, machine learning (ML) techniques, and Bayesian optimization. The genetic risk score (GRS), derived from 1,103 risk single nucleotide polymorphisms (SNPs) from genome-wide association studies (GWAS), was formulated for 25,772 postmenopausal women from the Women's Health Initiative dataset. We developed four ML models: Support Vector Machine (SVM), Random Forest, XGBoost, and Artificial Neural Network (ANN) for binary fracture outcome and 10-year fracture risk prediction. GRS and FRAX clinical risk factors (CRFs) were used as predictors. Death as a competing risk was accounted for in ML models for time-to-fracture data. ML models were subsequently fine-tuned through Bayesian optimization, which displayed marked superiority over traditional grid search. Evaluation of the models' performance considered an array of metrics such as accuracy, weighted F1 Score, the area under the precision-recall curve (PRAUC), and the area under the receiver operating characteristic curve (AUC) for binary fracture predictions, and the C-index, Brier score, and dynamic mean AUC over a 10-year follow-up period for fracture risk predictions. We found that GRS-integrated XGBoost with Bayesian optimization is the most effective model, with an accuracy of 91.2% (95% CI: 90.4-92.0%) and an AUC of 0.739 (95% CI: 0.731-0.746) in MOF binary predictions. For 10-year fracture risk modeling, the XGBoost model attained a C-index of 0.795 (95% CI: 0.783-0.806) and a mean dynamic AUC of 0.799 (95% CI: 0.788-0.809). Compared to FRAX, the XGBoost model exhibited a categorical net reclassification improvement (NRI) of 22.6% (P= .004). A sensitivity analysis, which included BMD but lacked GRS, reaffirmed these findings. Furthermore, portability tests in diverse non-European groups, including Asians and African Americans, underscored the model's robustness and adaptability. This study accentuates the potential of combining genetic insights and optimized ML in strengthening fracture predictions, heralding new preventive strategies for postmenopausal women.

Investigation of ash fusion characteristics on co-combustion of coal and biomass (straw, sludge, and herb residue) based on experimental and machine learning method.

Co-combustion of coal and biomass has the potential to reduce the cost of power generation in plants. However, because of the high content of the alkali metal of biomass ash, co-combustion of these two fuels leads to unpredictable ash fusion temperature (AFT). This study conducted experiments to measure the AFT of straw, sludge, and herb residue when they were blended with coal at different ratios. Additionally, a machine learning algorithm called tuna swarm optimization (TSO) was employed to optimize the support vector regression (SVR) model to predict the softening temperature (ST) of samples. The results indicate that straw and sludge were found to be suitable for blending in small proportions, while herb residue was suitable for blending in larger proportions. In comparison to the traditional grid search optimization model, the TSO algorithm significantly enhances the prediction accuracy of both training and test sets, and improves the generalization ability of SVR.

PSO‐GA‐SVRmodel for S‐parameters of radio‐frequency power amplifier under different temperatures

AbstractSupport vector machine (SVR) has been introduced into the modeling of S‐parameters in radio‐frequency (RF) power amplifiers (PA). The modeling accuracy and speed of SVR are primarily affected by the penalty parametercand the kernel function coefficientγ. Using the traditional grid search technique to determine these two parameters is time‐consuming and labor‐intensive, and ensuring the model's accuracy is not easy. This article proposes an S‐parameters modeling method based on PSO‐GA‐SVR to improve the SVR's modeling accuracy and speed. The model mainly focuses on particle swarm optimization (PSO) and combines selection, crossover, and mutation operations in genetic algorithms (GA). The fitness values are arranged from small to large in each iteration process, and the first 1/3 are selected for crossover and mutation. Then, the resulting new particle swarms are introduced into the original particle swarm population for searching. On the one hand, PSO‐GA extends the population size and reduces the possibility of falling into local optimization. On the other hand, due to population size expansion, the number of iteration rounds is reduced, and the modeling speed is also increased. The experimental results show that compared to SVR, GA‐SVR, and PSO‐SVR, the proposed PSO‐GA‐SVR can improve the modeling accuracy by more than one magnitude or more while also increasing modeling speed by one magnitude or more. Furthermore, compared with the classical machine learning algorithms such as gradient boosting, random forest, and gcForset, the proposed PSO‐GA‐SVR improves the modeling accuracy by one order of magnitude and the modeling speed by two orders of magnitude more than gradient boosting, random forest, and improves the modeling speed by one order of magnitude more than gcForset.

Utilizing a Two-Stage Taguchi Method and Artificial Neural Network for the Precise Forecasting of Cardiovascular Disease Risk.

The complexity of cardiovascular disease onset emphasizes the vital role of early detection in prevention. This study aims to enhance disease prediction accuracy using personal devices, aligning with point-of-care testing (POCT) objectives. This study introduces a two-stage Taguchi optimization (TSTO) method to boost predictive accuracy in an artificial neural network (ANN) model while minimizing computational costs. In the first stage, optimal hyperparameter levels and trends were identified. The second stage determined the best settings for the ANN model's hyperparameters. In this study, we applied the proposed TSTO method with a personal computer to the Kaggle Cardiovascular Disease dataset. Subsequently, we identified the best setting for the hyperparameters of the ANN model, setting the hidden layer to 4, activation function to tanh, optimizer to SGD, learning rate to 0.25, momentum rate to 0.85, and hidden nodes to 10. This setting led to a state-of-the-art accuracy of 74.14% in predicting the risk of cardiovascular disease. Moreover, the proposed TSTO method significantly reduced the number of experiments by a factor of 40.5 compared to the traditional grid search method. The TSTO method accurately predicts cardiovascular risk and conserves computational resources. It is adaptable for low-power devices, aiding the goal of POCT.

Automation in solid state NMR

Automation in solid state NMR (ssNMR) requires appropriate hardware, from rotor loading mechanisms over highly stable rf-transmitters and probe circuitry to automatic tuning and matching capabilities including automatic magic angle adjustment for ssNMR probes. While these hardware capabilities are highly desirable and are, to various degrees, provided by manufacturers, we focus herein on automating experiment setup using radio frequency (rf) fields, which are key parameters in solid state NMR experiments. Specifically, these include spinlock fields during cross polarization (CP), or rf-fields for homo- or heteronuclear spin recoupling or decoupling. Often, these fields have specific relationships to the magic angle spinning (MAS) frequency. Relying on a well-maintained spectrometer, the experiment setup shifts from traditionally required optimization of rf-power values for each element of an experiment sequence to automatically setting all parameters correctly without any need for optimization. The proposed approach allows executing an experiment by reading its rf-amplitude requirements based on the actual MAS rotation frequency just before starting data acquisition, while all other hardware-related parameters are automatically provided through global tables and scripts. Under modest MAS frequencies, no further rf-power optimization is required while providing optimal sensitivity of better than 90% of the optimal signal to noise. Any optional parameter optimization relates only to adjusting rf-nutation frequencies to the requirements of the sample and the sample rotation frequency rather than the spectrometer hardware. Fast MAS CP experiments with MAS frequencies above 40 kHz require a semi-automated setup by optimizing Hartmann-Hahn (HH) matched rf-fields that are synchronously varied relative to the MAS-frequency. This allows for a significant reduction of setup steps by up to one order of magnitude for such experiments, avoiding the traditional grid search for optimal CPMAS conditions. The approach presented here can also be applied to decoupling or recoupling sequences, requiring rotor synchronized rf-fields, reducing the setup to a few steps addressing the spin system’s properties rather than the spectrometer hardware. Our approach permits automating all basic solid state NMR experiments for high throughput analytical tasks.

Particle swarm optimization‐XGBoost‐based modeling of radio‐frequency power amplifier under different temperatures

AbstractXGBoost is the optimization of gradient boosting with the best overall performance among machine learning algorithms. By introducing a regularization term into the loss function of gradient boosting, XGBoost can effectively limit the complexity of the model, improve the generalization ability, and solve the overfitting problem. In this paper, XGBoost is first introduced into modeling radio‐frequency (RF) power amplifiers (PA) under different temperatures. Furthermore, the modeling effect of XGBoost is mainly dependent on hyperparameters. As traditional grid search is time‐consuming and labor‐intensive, this paper combines particle swarm optimization (PSO) with XGBoost for searching hyperparameters. The experimental results show that XGBoost can effectively suppress the overfitting problem in gradient boosting while modeling RF PAs in different ambient temperatures. In addition, compared to classic machine learning algorithms, including support vector regression (SVR), gradient boosting, and XGBoost, the proposed PSO‐XGBoost can increase the modeling accuracy by one order of magnitude or more while also increasing the modeling speed by more than one magnitude or more. The PSO‐XGBoost model proposed in this paper can be introduced into modeling other microwave/RF devices and circuits to improve modeling accuracy and reduce modeling time.

CS-GA-XGBoost-Based Model for a Radio-Frequency Power Amplifier under Different Temperatures.

Machine learning methods, such as support vector regression (SVR) and gradient boosting, have been introduced into the modeling of power amplifiers and achieved good results. Among various machine learning algorithms, XGBoost has been proven to obtain high-precision models faster with specific parameters. Hyperparameters have a significant impact on the model performance. A traditional grid search for hyperparameters is time-consuming and labor-intensive and may not find the optimal parameters. To solve the problem of parameter searching, improve modeling accuracy, and accelerate modeling speed, this paper proposes a PA modeling method based on CS-GA-XGBoost. The cuckoo search (CS)-genetic algorithm (GA) integrates GA's crossover operator into CS, making full use of the strong global search ability of CS and the fast rate of convergence of GA so that the improved CS-GA can expand the size of the bird nest population and reduce the scope of the search, with a better optimization ability and faster rate of convergence. This paper validates the effectiveness of the proposed modeling method by using measured input and output data of 2.5-GHz-GaN class-E PA under different temperatures (-40 °C, 25 °C, and 125 °C) as examples. The experimental results show that compared to XGBoost, GA-XGBoost, and CS-XGBoost, the proposed CS-GA-XGBoost can improve the modeling accuracy by one order of magnitude or more and shorten the modeling time by one order of magnitude or more. In addition, compared with classic machine learning algorithms, including gradient boosting, random forest, and SVR, the proposed CS-GA-XGBoost can improve modeling accuracy by three orders of magnitude or more and shorten modeling time by two orders of magnitude, demonstrating the superiority of the algorithm in terms of modeling accuracy and speed. The CS-GA-XGBoost modeling method is expected to be introduced into the modeling of other devices/circuits in the radio-frequency/microwave field and achieve good results.

Selective Query Processing: A Risk-Sensitive Selection of Search Configurations

In information retrieval systems, search parameters are optimized to ensure high effectiveness based on a set of past searches, and these optimized parameters are then used as the search configuration for all subsequent queries. A better approach, however, would be to adapt the parameters to fit the query at hand. Selective query expansion is one such an approach, in which the system decides automatically whether or not to expand the query, resulting in two possible search configurations. This approach was extended recently to include many other parameters, leading to many possible search configurations where the system automatically selects the best configuration on a per-query basis. One problem with this approach is the system training, which requires evaluation of each training query with every possible configuration. In real-world systems, so many parameters and possible values must be evaluated that this approach is impractical, especially when the system must be updated frequently, as is the case for commercial search engines. In general, the more configurations, the greater the effectiveness when configuration selection is appropriate but also the greater the risk of decreasing effectiveness in the case of an inappropriate configuration selection. To determine the ideal configurations to be used for each query in real-world systems, we have developed a method in which a limited number of possible configurations are pre-selected, then used in a meta-search engine that decides the best search configuration for each query. We define a risk-sensitive approach for configuration pre-selection that considers the risk-reward tradeoff between the number of configurations kept and system effectiveness. We define two alternative risk functions to apply to different goals. For final configuration selection, the decision is based on query feature similarities. We compare two alternative risk functions on two query types (ad hoc and diversity) and compare these to more sophisticated machine learning based methods. We find that a relatively small number of configurations (20) selected by our risk-sensitive model is sufficient to obtain results close to the best achievable results for each query. Effectiveness is increased by about 15% according to the P@10 and nDCG@10 evaluation metrics when compared to traditional grid search using a single configuration and by about 20% when compared to learning to rank documents. Our risk-sensitive approach works for both diversity- and ad hoc oriented searches. Moreover, the similarity-based selection method outperforms the more sophisticated approaches. Thus, we demonstrate the feasibility of developing per-query information retrieval systems, which will guide future research in this direction.

I-Cubid: a nonlinear cubic graph-based approach to visualize and in-depth browse Flickr image results.

The existing image search engines allow web users to explore images from the grids. The traditional interaction is linear and lookup-based. Notably, scanning web search results is horizontal-vertical and cannot support in-depth browsing. This research emphasizes the significance of a multidimensional exploration scheme over traditional grid layouts in visually exploring web image search results. This research aims to antecedent the implications of visualization and related in-depth browsing via a multidimensional cubic graph representation over a search engine result page (SERP). Furthermore, this research uncovers usability issues in the traditional grid and 3-dimensional web image search space. We provide multidimensional cubic visualization and nonlinear in-depth browsing of web image search results. The proposed approach employs textual annotations and descriptions to represent results in cubic graphs that further support in-depth browsing via a search user interface (SUI) design. It allows nonlinear navigation in web image search results and enables exploration, browsing, visualization, previewing/viewing, and accessing images in a nonlinear, interactive, and usable way. The usability tests and detailed statistical significance analysis confirm the efficacy of cubic presentation over grid layouts. The investigation reveals improvement in overall user satisfaction, screen design, information & terminology, and system capability in exploring web image search results.

Shear-Wave Splitting Analysis Using Optimization Algorithms

Abstract Shear-wave splitting (SWS) analysis is used to predict fractures in subsurface media. Specifically, two parameters relevant to SWS analysis (the azimuth of the fast shear wave and the time delay between the fast and slow shear waves) are used to quantify the main azimuth and degree of the fracture development, respectively. However, the algorithms of SWS analysis using a grid search have relatively low computational efficiency, as they need to calculate the objective function values of all grid points. To improve the efficiency of SWS analysis, we proposed new algorithms using the gradient descent, Newton, and advance-retreat methods. The new methods use the direction of the fastest gradient descent, the intersection points of the tangent plane of the first-order objective function with the zero plane, and narrowing the range of extremum points to determine the search path. Therefore, this removes the necessity to compare all grid points in the value region. We compared the three methods and the rotation-correlation method, and both synthetic and field data tests indicated that all three methods had higher computational efficiency than the traditional grid search method. Among the proposed methods, the gradient-descent method obtained the most accurate results for both synthetic and field data. Our study shows that SWS analysis combined with the gradient-descent method can accurately and efficiently obtain SWS parameters for fracture prediction.

A meta-learning configuration framework for graph-based similarity search indexes

Similarity searches retrieve elements in a dataset with similar characteristics to the input query element. Recent works show that graph-based methods have outperformed others in the literature, such as tree-based and hash-based methods. However, graphs are highly parameter-sensitive for indexing and searching, which usually demands extra time for finding a suitable trade-off for specific user requirements. Current approaches to select parameters rely on observing published experimental results or Grid Search procedures. While the former has no guarantees that good settings for a dataset will also perform well on a different one, the latter is computationally expensive and limited to a small range of values. In this work, we propose a meta-learning-based recommender framework capable of providing a suitable graph configuration according to the characteristics of the input dataset. We present two instantiations of the framework: a global instantiation that uses the whole meta-database to train meta-models and a dataset-similarity-based instantiation that relies on clustering to generate meta-models tailored to datasets with similar characteristics. We also developed generic and tuned versions of the instantiations. The generic versions can satisfy user requirements in orders of magnitude faster than the traditional Grid Search. The tuned versions provide more accurate predictions at a higher cost. Our results show that the tuned methods outperform the Grid Search for most cases, providing recommendations close to the optimal one and being a suitable alternative, particularly for more challenging datasets.

Development of Intelligent Fault Diagnosis Technique of Rotary Machine Element Bearing: A Machine Learning Approach.

The bearing is an essential component of a rotating machine. Sudden failure of the bearing may cause an unwanted breakdown of the manufacturing plant. In this paper, an intelligent fault diagnosis technique was developed to diagnose various faults that occur in a deep groove ball bearing. An experimental setup was designed and developed to generate faulty data in various conditions, such as inner race fault, outer race fault, and cage fault, along with the healthy condition. The time waveform of raw vibration data generated from the system was transformed into a frequency spectrum using the fast Fourier transform (FFT) method. These FFT signals were analyzed to detect the defective bearing. Another significant contribution of this paper is the application of a machine learning (ML) algorithm to diagnose bearing faults. The support vector machine (SVM) was used as the primary algorithm. As the efficiency of SVM heavily depends on hyperparameter tuning and optimum feature selection, the particle swarm optimization (PSO) technique was used to improve the model performance. The classification accuracy obtained using SVM with a traditional grid search cross-validation (CV) optimizer was 92%, whereas the improved accuracy using the PSO-based SVM was found to be 93.9%. The developed model was also compared with other traditional ML techniques such as k-nearest neighbor (KNN), decision tree (DT), and linear discriminant analysis (LDA). In every case, the proposed model outperformed the existing algorithms.

Study on the High Accuracy and Fast Acquisition of Satellite Signals Based on the Blind Source Separation Technique

To address the problems of slow acquisition speed and low accuracy faced by existing grid search-based satellite acquisition methods in complex scenarios, this study proposes a high accuracy and fast satellite signal acquisition method based on blind source separation. The proposed method first adopts wavelet threshold denoising to reduce the noise in the overlapped satellite signal received by the receiver so as to improve the signal-to-noise ratio of the satellite signal. On this basis, a subspace estimation method is introduced to construct a blind acquisition algorithm for satellite signals, which achieves high accuracy and fast solution for the Doppler frequency shift and code phase shift of satellite carrier signals and is able to recover the unobservable individual source signals. The effectiveness of the proposed method is verified through experiments and compared against the traditional grid search type acquisition method, which confirmed that the method is suitable for the acquisition requirements of weak signals and has a good engineering application value.

Fast selection of nonlinear mixed effect models using penalized likelihood

Nonlinear Mixed effects models are hidden variables models that are widely used in many fields such as pharmacometrics. In such models, the distribution characteristics of hidden variables can be specified by including several parameters such as covariates or correlations which must be selected. Recent development of pharmacogenomics has brought averaged/high dimensional problems to the field of nonlinear mixed effects modeling for which standard covariates selection techniques like stepwise methods are not well suited. The selection of covariates and correlation parameters using a penalized likelihood approach is proposed. The penalized likelihood problem is solved using a stochastic proximal gradient algorithm to avoid inner-outer iterations. Speed of convergence of the proximal gradient algorithm is improved using component-wise adaptive gradient step sizes. The practical implementation and tuning of the proximal gradient algorithm are explored using simulations. Calibration of regularization parameters is performed by minimizing the Bayesian Information Criterion using particle swarm optimization, a zero-order optimization procedure. The use of warm restart and parallelization allowed computing time to be reduced significantly. The performance of the proposed method compared to the traditional grid search strategy is explored using simulated data. Finally, an application to real data from two pharmacokinetics studies is provided, one studying an antifibrinolytic and the other studying an antibiotic.

Analysis of Cloud Resource Optimal Allocation Strategy Based on Bayesian Algorithm

With the increasing popularity of big data technology, the scale of computer network system is larger and more complex, especially the rise of cloud technology, to a large extent, subverts many concepts of traditional computer network, which also means that the traditional modeling technology has been difficult to meet the accurate modeling of modern computer network. In order to manage and schedule cloud resources more efficiently, people have proposed various technical schemes, among which Bayesian optimization is one of the most concerned technologies. Bayesian optimization is a global optimization method, which can quickly find the optimal solution even in the calculation process without closed form and high complexity, and the difficulty of optimal allocation of cloud resources is just here. The experimental results show that the proposed Bayesian optimization method can find the optimal configuration in a finite number of times, which is obviously better than the traditional grid search and random search methods.

Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics

An approach to the time-accurate prediction of chaotic solutions is by learning temporal patterns from data. Echo State Networks (ESNs), which are a class of Reservoir Computing, can accurately predict the chaotic dynamics well beyond the predictability time. Existing studies, however, also showed that small changes in the hyperparameters may markedly affect the network’s performance. The overarching aim of this paper is to improve the robustness in the selection of hyperparameters in Echo State Networks for the time-accurate prediction of chaotic solutions. We define the robustness of a validation strategy as its ability to select hyperparameters that perform consistently between validation and test sets. The goal is three-fold. First, we investigate routinely used validation strategies. Second, we propose the Recycle Validation, and the chaotic versions of existing validation strategies, to specifically tackle the forecasting of chaotic systems. Third, we compare Bayesian optimization with the traditional grid search for optimal hyperparameter selection. Numerical tests are performed on prototypical nonlinear systems that have chaotic and quasiperiodic solutions, such as the Lorenz and Lorenz-96 systems, and the Kuznetsov oscillator. Both model-free and model-informed Echo State Networks are analysed. By comparing the networks’ performance in learning chaotic (unpredictable) versus quasiperiodic (predictable) solutions, we highlight fundamental challenges in learning chaotic solutions. The proposed validation strategies, which are based on the dynamical systems properties of chaotic time series, are shown to outperform the state-of-the-art validation strategies. Because the strategies are principled – they are based on chaos theory such as the Lyapunov time – they can be applied to other Recurrent Neural Networks architectures with little modification. This work opens up new possibilities for the robust design and application of Echo State Networks, and Recurrent Neural Networks, to the time-accurate prediction of chaotic systems.

Computer vision AC-STEM automated image analysis for 2D nanopore applications

Transmission electron microscopy (TEM) has led to important discoveries in atomic imaging and as an atom-by-atom fabrication tool. Using electron beams, atomic structures can be patterned, annealed and crystallized, and nanopores can be drilled in thin membranes. We review current progress in TEM analysis and implement a computer vision nanopore-detection algorithm that achieves a 96% pixelwise precision in TEM images of nanopores in 2D membranes (WS2), and discuss parameter optimization including a variation on the traditional grid search and gradient ascent. Such nanopores have applications in ion detection, water filtration, and DNA sequencing, where ionic conductance through the pore should be concordant with its TEM-measured size. Standard computer vision methods have their advantages as they are intuitive and do not require extensive training data. For completeness, we briefly comment on related machine learning for 2D materials analysis and discuss relevant progress in these fields. Image analysis alongside TEM allows correlated fabrication and analysis done simultaneously in situ to engineer devices at the atomic scale.