Traditional Ensemble Methods Research Articles

A novel ensemble machine learning exposure model system for ground-level ozone at the national scale: A case of mainland China from 2013 to 2020

In epidemiological research, accurate estimation of historical ground-level ozone (O3) concentrations with enhanced spatiotemporal resolution is crucial for effective exposure assessment. The current state-of-the-art for estimating air pollutant concentrations is a two-stage ensemble method that integrates outputs from multiple machine learning algorithms. Despite its effectiveness, opportunities exist to refine this approach for more precise O3 estimation. In this study, we propose an enhanced ensemble method that incorporates four key strategies. First, we employ high-resolution spatiotemporal predictors derived from prior machine learning studies for refined secondary learning. Second, we use sophisticated algorithms, including categorical gradient boosting, deep neural network, random forest, stochastic variable Gaussian process, transformer, and a combination of convolutional neural network and long short-term memory neural network, as sublearners to enhance learning capabilities. Third, we spatiotemporally split the sample set and then train submodels separately on each subset to eliminate the unobserved spatiotemporal heterogeneity. Finally, we apply a complex machine learning algorithm, rather than the generalized additive model, for integrating sublearner predictions, enabling the capture of intricate nonlinear relationships beyond basic spatiotemporal linear weights. To validate these improvements, we estimated daily maximum 8-h moving average O3 concentrations ([O3]MDA8) across Chinese mainland from 2013 to 2020 at a 1 km spatial resolution. The proposed method demonstrated notable accuracy, achieving an out-of-station determination coefficient (R2) of 0.943 and a root-mean-square error (RMSE) of 10.197 μg/m3. This performance marks a nearly 15% improvement over the best existing Chinese O3 exposure model based on a single algorithm and also surpasses previous studies utilizing traditional ensemble methods for other air pollutants. Our enhanced ensemble approach significantly bolsters the reliability and robustness of future environmental epidemiological studies by further mitigating “misclassification” errors.

BENN: Balanced Ensemble Neural Network for Handling Class Imbalance in Big Data

ABSTRACTClass imbalance is a critical challenge in big data analytics, often leading to biased predictive models. This imbalance can lead to biased models that perform well on the majority class but poorly on the minority class. Many machine learning models tend to be biased towards the majority class because they aim to minimise overall error, often leading to poor performance on the minority class. This paper presents the balanced ensemble neural network, a novel solution to effectively address class imbalance in big data. Balanced ensemble neural network combines the robust capabilities of neural networks with the power of ensemble learning, incorporating class balancing strategies to ensure fair representation of minority classes. The methodology involves integrating multiple neural networks, each trained on balanced subsets of data using techniques like Synthetic Minority Over‐sampling Technique and Random Undersampling. This integration aims to leverage the strengths of individual networks while reducing their inherent biases. Our extensive experiments across various datasets reveal that BENN achieves an AUC‐ROC score of 0.94, surpassing other models such as random forest (0.88), support vector (0.84) and single neural net (0.80). It was also observed that BENN's performance is better compared to traditional neural network models and standard ensemble methods in key metrics like accuracy, precision, recall, F1‐score and AUC‐ROC. The results specifically highlight BENN's effectiveness in accurately classifying instances of minority classes, a notable challenge in many existing models. These findings underscore BENN's potential as a substantial advancement in handling class imbalance within big data environments, offering a promising direction for future research and application in machine learning.

Which classifiers are connected to others? An optimal connection framework for multi-layer ensemble systems

Ensemble learning is a powerful machine learning strategy that combines multiple models e.g. classifiers to improve predictions beyond what any single model can achieve. Until recently, traditional ensemble methods typically use only one layer of models which limits the exploration of different aspects in the classifiers’ predictions. On the other hand, the rise of deep learning has introduced multi-layer architectures that can learn complex functions by transforming data into multiple levels of representation. This characteristic of deep learning suggests that multi-layer ensembles may potentially provide better performance compared to single-layer ensembles. However, a problem which might arise is that in the subsequent layers, not all the inputs to a classifier are desirable, leading to lower performance. In this paper, we introduce a novel multi-layer ensemble of classifiers named COME in which each classifier at a specific layer is connected to multiple classifiers in the previous layer. These connections signify the use of the previous-layer-classifiers’ outputs as inputs for training the current layer's classifier. Each classifier can be connected to different classifiers in the previous layer, which allows inputs in each layer to be optimally selected. We propose a binary encoding scheme to encode the topology of the proposed multi-layer ensemble with defined connections between layers. Differential Evolution, a popular evolutionary computation method, is used as the optimisation algorithm to search for the optimal set of connections. Experimental results on 30 datasets from the UCI Machine Learning Repository and OpenML demonstrate that our proposed ensemble outperforms many state-of-the-art ensemble learning algorithms.

Ensemble Vision Transformer for Dementia Diagnosis.

In recent years, deep learning has gained momentum in computer-aided Alzheimer's Disease (AD) diagnosis. This study introduces a novel approach, Monte Carlo Ensemble Vision Transformer (MC-ViT), which develops an ensemble approach with Vision transformer (ViT). Instead of using traditional ensemble methods that deploy multiple learners, our approach employs a single vision transformer learner. By harnessing Monte Carlo sampling, this method produces a broad spectrum of classification decisions, enhancing the MC-ViT performance. This novel technique adeptly overcomes the limitation of 3D patch convolutional neural networks that only characterize partial of the whole brain anatomy, paving the way for a neural network adept at discerning 3D inter-feature correlations. Evaluations using the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset with 7199 scans and Open Access Series of Imaging Studies-3 (OASIS-3) with 1992 scans showcased its performance. With minimal preprocessing, our approach achieved an impressive 90% accuracy in AD classification, surpassing both 2D-slice CNNs and 3D CNNs.

Screening depression among university students utilizing GHQ-12 and machine learning

The escalating incidence of depression has brought attention to the increasing concern regarding the mental well-being of university students in the current academic environment. Given the increasing mental health challenges faced by students, there is a critical need for efficient, scalable, and accurate screening methods. This study aims to address the issue by using the General Health Questionnaire-12 (GHQ-12), a well recognized tool for evaluating psychological discomfort, in combination with machine learning (ML) techniques. Firstly, for effective screening of depression, a comprehensive questionnaire has been created with the help of an expert psychiatrist. The questionnaire includes the GHQ-12, socio-demographic, and job and career-related inquiries. A total of 804 responses has been collected from various public and private universities across Bangladesh. The data has been then analyzed and preprocessed. It has been found that around 60% of the study population are suffering from depression. Lastly, 16 different ML models, including both traditional algorithms and ensemble methods has been applied to examine the data to identify trends and predictors of depression in this demographic. The models' performance has been rigorously evaluated in order to ascertain their effectiveness in precisely identifying individuals who are at risk. Among the ML models, Extremely Randomized Tree (ET) has achieved the highest accuracy of 90.26%, showcasing its classification effectiveness. A thorough investigation of the performance of the models compared, therefore clarifying their possible relevance in the early detection of depression among university students, has been presented in this paper. The findings shed light on the complex interplay among socio-demographic variables, stressors associated with one's profession, and mental well-being, which offer an original viewpoint on utilizing ML in psychological research.

An intelligent solvent selection approach in carbon capturing process: A comparative study of machine learning multi-class classification models

Carbon capture is crucial for mitigating climate change and achieving global emissions reduction targets. Among various technologies, absorption-based methods using aqueous solvents are among the most common and efficient, offering high capture rates and retrofit capabilities for existing industrial facilities. However, selecting the optimal solvent is challenging due to its direct impact on the performance, efficiency, and cost-effectiveness of the carbon capture process. This research introduces a novel multi-class classification methodology for solvent selection in carbon capture. Leveraging a dataset of 656 data points, including features such as temperature, pressure, molar ratio, and CO2 solubility, the study employs nine machine learning algorithms—traditional models (Naïve Bayes, k-Nearest Neighbors, Decision Tree) and ensemble methods (Gradient Boosted Tree, Random Forest, Bagging, AdaBoost, Stacking, Voting)—to develop robust classification models. The results highlight the superior performance of the Stacking ensemble classifier, achieving 99.24 % accuracy, 0.76 % error rate, 99.04 % weighted mean recall, 98.08 % weighted mean precision, a 98.56 % F1 score, and a 0.9911 Cohen's Kappa coefficient on the test set. The effectiveness of ensemble techniques over traditional models underscores their ability to capture the complex relationships inherent in solvent selection. This approach streamlines the decision-making process for designers and engineers, enabling rapid identification of the most suitable solvent class based on operational conditions. Integrating this methodology into chemical engineering software could offer practical benefits, facilitating real-time solvent selection and optimization in industrial carbon capture, ultimately contributing to more sustainable and efficient processes.

Ensemble learning based on relative accuracy approach and diversity teams

Ensemble learning, which involves combining the opinions of multiple experts to arrive at a better result, has been used for centuries. In this work, a review of the major voting methods in ensemble learning is explored. This work will focus on a new method for combining the results of individual learners. The method depends on the relative accuracy and diversity of teams. Instead of trying to assign weight to each different trainer, the concept of diversity teams is presented. Each team will vote as one player; however, the individual accuracies of each learner still be implemented. The concept of relaxing parameters that deal with each team as one player is presented. Our experiments demonstrate that traditional ensemble voting methods outperform individual learners. There is a limit to the superiority of the ensemble learner that any ensemble learner cannot go beyond. The proposed voting method gives the same results as the traditional ensemble voting methods, however, a different diversity of the proposed method from the traditional voting method or for different values of the relaxing parameter can be achieved.

Verification of Long‐Term Ensemble Evapotranspiration Hindcast Using a Conditional Nonlinear Optimal Parameter Perturbation Ensemble Prediction Method on the Tibetan Plateau

AbstractIn this research, a conditional nonlinear optimal perturbation related to parameters (CNOP‐P) method is employed to propose an ensemble prediction method titled as the conditional nonlinear optimal perturbation related to parameters ensemble prediction (CNOP‐PEP) method. Within the CNOP‐PEP method, all ensemble members are generated by the CNOP‐P method. To explore the operability and validity of the CNOP‐PEP method, long‐term evapotranspiration (ET) hindcast skill is evaluated at 13 sites on the Tibetan Plateau (TP) during the period of 2001–2018 with a land surface model. Two traditional ensemble prediction methods (the stochastically perturbed parameters (SPP) method and the one‐at‐a‐time (OAT) method) are also applied to assess the ET hindcast skills. The numerical results indicate that the CNOP‐PEP method shows the best hindcast skill for estimated ET at 13 stations on the TP among the three methods (CNOP‐PEP, OAT, and SPP methods) during the period of 2001–2018. This suggests that the CNOP‐PEP method is helpful and effective for long‐term and interannual hydrological predictions, such as long‐term and interannual ET predictions. The numerical results also indicate that ensemble members with certain special properties of fast‐growing types should be selected to implement ensemble prediction compared to these ordinary and traditional samples. The CNOP‐PEP method could supply special samples to improve the ensemble prediction and hindcast skill of hydrological cycle.

CGO-ensemble: Chaos game optimization algorithm-based fusion of deep neural networks for accurate Mpox detection

The rising global incidence of human Mpox cases necessitates prompt and accurate identification for effective disease control. Previous studies have predominantly delved into traditional ensemble methods for detection, we introduce a novel approach by leveraging a metaheuristic-based ensemble framework. In this research, we present an innovative CGO-Ensemble framework designed to elevate the accuracy of detecting Mpox infection in patients. Initially, we employ five transfer learning base models that integrate feature integration layers and residual blocks. These components play a crucial role in capturing significant features from the skin images, thereby enhancing the models' efficacy. In the next step, we employ a weighted averaging scheme to consolidate predictions generated by distinct models. To achieve the optimal allocation of weights for each base model in the ensemble process, we leverage the Chaos Game Optimization (CGO) algorithm. This strategic weight assignment enhances classification outcomes considerably, surpassing the performance of randomly assigned weights. Implementing this approach yields notably enhanced prediction accuracy compared to using individual models. We evaluate the effectiveness of our proposed approach through comprehensive experiments conducted on two widely recognized benchmark datasets: the Mpox Skin Lesion Dataset (MSLD) and the Mpox Skin Image Dataset (MSID). To gain insights into the decision-making process of the base models, we have performed Gradient Class Activation Mapping (Grad-CAM) analysis. The experimental results showcase the outstanding performance of the CGO-ensemble, achieving an impressive accuracy of 100% on MSLD and 94.16% on MSID. Our approach significantly outperforms other state-of-the-art optimization algorithms, traditional ensemble methods, and existing techniques in the context of Mpox detection on these datasets. These findings underscore the effectiveness and superiority of the CGO-Ensemble in accurately identifying Mpox cases, highlighting its potential in disease detection and classification.

A fusion method for capacity estimation of fast-charging lithium-ion battery under varied degradation mode

Recently, lithium-ion batteries with fast-charging capability have been gradually equipped in electric products. Capacity estimation specially developed for fast-charging batteries is still an open question, and most of the existing works are proposed for batteries charged under the 1C rate. However, incorrect fast-charging strategies may result in Li plating leading to battery capacity aging rapidly and degradation mode variation. Moreover, unprecise sampling by BMS increases the difficulty of accurately estimating battery capacity in such scenarios. Thus, a fusion prognostic method based on ensemble learning is proposed for the above issues in fast-charging batteries. Firstly, measurement-based health features are extracted from the portion charging phase. Then, the accuracy of the validation dataset-based strategy is proposed to achieve time-varying weight allocation. Finally, a fusion prognostic model is constructed based on the above weight allocation strategy and health features. The effectiveness of the proposed fusion method is verified by a self-design fast-charging battery dataset that maintains stable and reliable performance under sparse sampling and degradation mode variation conditions. In addition, the robustness and adaptability are validated by the comparison experiments with traditional ensemble learning-based methods, in which accuracy is improved by 27.5% minimally.

Soft sensor method for endpoint carbon content and temperature of BOF based on multi-cluster dynamic adaptive selection ensemble learning

Abstract The accurate control of the endpoint in converter steelmaking is of great significance and value for energy saving, emission reduction, and steel quality improvement. The key to endpoint control lies in accurately predicting the carbon content and temperature. Converter steelmaking is a dynamic process with a large fluctuation of samples, and traditional ensemble learning methods ignore the differences among the query samples and use all the sub-models to predict. The different performances of each sub-model lead to the performance degradation of ensemble learning. To address this issue, we propose a soft sensor method based on multi-cluster dynamic adaptive selection (MC-DAS) ensemble learning for converter steelmaking endpoint carbon content and temperature prediction. First, to ensure the diversity of the ensemble learning base model, we propose a clustering algorithm with different data partition characteristics to construct a pool of diverse base models. Second, a model adaptive selection strategy is proposed, which involves constructing diverse similarity regions for individual query samples and assessing the model’s performance in these regions to identify the most suitable model and weight combination for each respective query sample. Compared with the traditional ensemble learning method, the simulation results of actual converter steelmaking process data show that the prediction accuracy of carbon content within ±0.02% error range reaches 92.8%, and temperature within ±10°C error range reaches 91.6%.

: A Semantics-Guided Safety Enhancement Framework for AI-Enabled Cyber-Physical Systems

Cyber-Physical Systems (CPSs) have been widely adopted in various industry domains to support many important tasks that impact our daily lives, such as automotive vehicles, robotics manufacturing, and energy systems. As Artificial Intelligence (AI) has demonstrated its promising abilities in diverse tasks like decision-making, prediction, and optimization, a growing number of CPSs adopt AI components in the loop to further extend their efficiency and performance. However, these modern AI-enabled CPSs have to tackle pivotal problems that the AI-enabled control systems might need to compensate the balance across <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multiple operation requirements</i> and avoid possible defections in advance to safeguard human lives and properties. Modular redundancy and ensemble method are two widely adopted solutions in the traditional CPSs and AI communities to enhance the functionality and flexibility of a system. Nevertheless, there is a lack of deep understanding of the effectiveness of such ensemble design on AI-CPSs across diverse industrial applications. Considering the complexity of AI-CPSs, existing ensemble methods fall short of handling such huge state space and sophisticated system dynamics. Furthermore, an ideal control solution should consider the multiple system specifications in real-time and avoid erroneous behaviors beforehand. Such that, a new specification-oriented ensemble control system is of urgent need for AI-CPSs. In this paper, we propose <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathtt {SIEGE}$</tex-math></inline-formula> , a semantics-guided ensemble control framework to initiate an early exploratory study of ensemble methods on AI-CPSs and aim to construct an efficient, robust, and reliable control solution for multi-tasks AI-CPSs. We first utilize a semantic-based abstraction to decompose the large state space, capture the ongoing system status and predict future conditions in terms of the satisfaction of specifications. We propose a series of new semantics-aware ensemble strategies and an end-to-end Deep Reinforcement Learning (DRL) hierarchical ensemble method to improve the flexibility and reliability of the control systems. Our large-scale, comprehensive evaluations over five subject CPSs show that 1) the semantics abstraction can efficiently narrow the large state space and predict the semantics of incoming states, 2) our semantics-guided methods outperform state-of-the-art individual controllers and traditional ensemble methods, and 3) the DRL hierarchical ensemble approach shows promising capabilities to deliver a more robust, efficient, and safety-assured control system. To enable further research along this direction to build better AI-enabled CPS, we made all of the code and experimental results data publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://sites.google.com/view/ai-cps-siege/home</uri> .

Methods to investigate nucleosome structure and dynamics with single-molecule FRET.

The nucleosome is the fundamental building block of chromatin. Changes taking place at the nucleosome level are the molecular basis of chromatin transactions with various enzymes and factors. These changes are directly and indirectly regulated by chromatin modifications such as DNA methylation and histone post-translational modifications including acetylation, methylation, and ubiquitylation. Nucleosomal changes are often stochastic, unsynchronized, and heterogeneous, making it very difficult to monitor with traditional ensemble averaging methods. Diverse single-molecule fluorescence approaches have been employed to investigate the structure and structural changes of the nucleosome in the context of its interactions with various enzymes such as RNA Polymerase II, histone chaperones, transcription factors, and chromatin remodelers. We utilize diverse single-molecule fluorescence methods to study the nucleosomal changes accompanying these processes, elucidate the kinetics of these processes, and eventually learn the implications of various chromatin modifications in directly regulating these processes. The methods include two- and three-color single-molecule fluorescence resonance energy transfer (FRET), single-molecule fluorescence correlation spectroscopy, and fluorescence (co-)localization. Here we report the details of the two- and three-color single-molecule FRET methods we currently use. This report will help researchers design their single-molecule FRET approaches to investigating chromatin regulation at the nucleosome level.

Recent Advances in Single-Molecule Tracking and Imaging Techniques.

Since the early 1990s, single-molecule detection in solution at room temperature has enabled direct observation of single biomolecules at work in real time and under physiological conditions, providing insights into complex biological systems that the traditional ensemble methods cannot offer. In particular, recent advances in single-molecule tracking techniques allow researchers to follow individual biomolecules in their native environments for a timescale of seconds to minutes, revealing not only the distinct pathways these biomolecules take for downstream signaling but also their roles in supporting life. In this review, we discuss various single-molecule tracking and imaging techniques developed to date, with an emphasis on advanced three-dimensional (3D) tracking systems that not only achieve ultrahigh spatiotemporal resolution but also provide sufficient working depths suitable for tracking single molecules in 3D tissue models. We then summarize the observables that can be extracted from the trajectory data. Methods to perform single-molecule clustering analysis and future directions are also discussed.

Semi-supervised interlayer intelligent recognition method

The development and distribution of interlayers in sandstones can lead to increased heterogeneity in the formation within the reservoir, which further affects the movement of fluids in the formation. Therefore, it is essential to accurately evaluate the interlayers in sandstones, as the precise evaluation of interlayers in sandstones is of great importance for identifying the distribution of underground fluid systems. The logging data of interlayers are inadequate for traditional Machine Learning training due to their low measurement proportion compared to the conventional layers. In logging data, the amount of data in interlayers is significantly smaller than that of conventional reservoirs. Traditional machine learning models are mostly based on samples with balanced distribution. By contrast, semi-supervised learning requires a small number of labeled samples for learning, and then combines a large number of unlabeled samples for modeling. To verify the feasibility of semi-supervised learning in the identification of interlayers, the Donghe sandstone section of the H oilfield was taken as an example. First, the core analysis results were used to label the logging data; then, to uncover more response information that can characterize the interlayers on the logging curve, multiple features were extracted to construct cross features. Finally, an improved model based on autoencoders—probabilistic autoencoder (PAE)—is proposed to address the issue of interlayer recognition for imbalanced samples. The PAE model can calculate the probability of belonging to a different class for unlabeled samples, and classify new samples according to the maximum probability. Experimental results show that, compared with traditional machine learning methods and ensemble learning methods, PAE achieves higher recognition accuracy and better generalization performance by updating the algorithm, and can be used as a simple and fast method for interlayer recognition. The results of the algorithm demonstrate that the semi-supervised method is of great significance for exploring and developing complex heterogeneous oil reservoirs. The research results are of great importance for exploring and developing complex heterogeneous oil reservoirs.

SWSEL: Sliding Window-based Selective Ensemble Learning for class-imbalance problems

For class-imbalance problems, traditional supervised learning algorithms tend to favor majority instances (also called negative instances). Therefore, it is difficult for them to accurately identify the minority instances (also called positive instances). Ensemble learning is a common method to solve the class-imbalance problem. They build multiple classifier systems on the training dataset to improve the recognition accuracy of minority instances. Sliding window is a commonly used method for processing data stream. Few researchers have used sliding windows to select majority instances and construct ensemble learning models. Traditional ensemble learning methods use some or all of the majority instances for modeling by oversampling or undersampling. However, they also inherit the drawbacks of the preprocessing methods. Therefore, in this paper, we try to use similarity mapping to construct pseudo-sequences of majority instances. Then, according to the sliding window idea, we fully use all existing majority instances, and a novel sliding window-based selective ensemble learning method (SWSEL) is proposed to deal with the class-imbalance problem. This method uses the idea of distance alignment in multi-view alignment to align the centers of the minority instances with the majority instances, and slide to select the majority instances on the sequence of pseudo-majority instances. In addition, to prevent too many classifiers from leading to long running times, we use distance metric to select a certain number of base classifiers to build the final ensemble learning model. Extensive experimental results on various real-world datasets show that using SVM, MLP and RF as the base classifier, SWSEL achieves a statistically significant performance improvement on two evaluation metrics, AUC and G-mean, compared to state-of-the-art methods.

Operando Imaging of Crystallinity-Dependent Multicolor Thermochromic Processes for Single Hydrated Hybrid Perovskite Particles.

The thermochromic properties of hydrated metal halide perovskites (MHPs) are promising for applications in smart windows, solar cells, optical sensors, and information storage. Traditional ensemble characterization methods always study the averaged thermochromic activity, lacking the accurate structure-activity correlation. Here we utilize dark-field microscopy (DFM) to in situ image the thermochromic processes of single isolated hydrated hybrid perovskite (CH3NH3)4PbI6-xClx·2H2O (MA4PbI6-xClx·2H2O) microparticles. The thermal-induced dehydration transition is demonstrated to alter the color of single MA4PbI6-xClx·2H2O particles. Operando single-particle mapping results reveal the significant intra- and interparticle variations of thermochromic behaviors, yielding unexpected single or multistep multicolor thermochromic processes. These phenomena are confirmed to be governed by the crystallinity of single MA4PbI6-xClx·2H2O particles that results in distinct composition-dependent bandgaps and thermal decomposition pathways. The present work highlights the important role of single-particle imaging for resolving the intrinsic thermochromic characteristic of hydrated MHPs, therefore opening a way for rational design of stimuli-responsive materials.

A Use Case of Patent Classification Using Deep Learning with Transfer Learning

Abstract Purpose Patent classification is one of the areas in Intellectual Property Analytics (IPA), and a growing use case since the number of patent applications has been increasing worldwide. We propose using machine learning algorithms to classify Portuguese patents and evaluate the performance of transfer learning methodologies to solve this task. Design/methodology/approach We applied three different approaches in this paper. First, we used a dataset available by INPI to explore traditional machine learning algorithms and ensemble methods. After preprocessing data by applying TF-IDF, FastText and Doc2Vec, the models were evaluated by cross-validation in 5 folds. In a second approach, we used two different Neural Networks architectures, a Convolutional Neural Network (CNN) and a bi-directional Long Short-Term Memory (BiLSTM). Finally, we used pre-trained BERT, DistilBERT, and ULMFiT models in the third approach. Findings BERTTimbau, a BERT architecture model pre-trained on a large Portuguese corpus, presented the best results for the task, even though with a performance of only 4% superior to a LinearSVC model using TF-IDF feature engineering. Research limitations The dataset was highly imbalanced, as usual in patent applications, so the classes with the lowest samples were expected to present the worst performance. That result happened in some cases, especially in classes with less than 60 training samples. Practical implications Patent classification is challenging because of the hierarchical classification system, the context overlap, and the underrepresentation of the classes. However, the final model presented an acceptable performance given the size of the dataset and the task complexity. This model can support the decision and improve the time by proposing a category in the second level of ICP, which is one of the critical phases of the grant patent process. Originality/value To our knowledge, the proposed models were never implemented for Portuguese patent classification.

Direct Observation of Emulsion Morphology, Dynamics, and Demulsification.

Herein, we present the direct observation and quantification of a water-in-oil (w/o) emulsion, its destabilization, and the effect of additives on such processes at the nanoscale. This is achieved via liquid phase transmission electron microscopy (LPTEM), wherein a small volume of emulsion is encapsulated against vacuum in its liquid state to allow observation of its initial morphology and its evolution over time at excellent spatial and temporal resolution. Emulsions of this class are useful for delivering payloads of materials insoluble in their delivery medium and are currently widely used across food science, pharmaceuticals, and environmental applications. However, their utility is inherently limited by their thermodynamic tendency to demulsify, eventually leading to bulk phase separation. This occurs via several degradation mechanisms, operating at times collectively, and which are difficult to differentiate via traditional ensemble methods (e.g., light scattering), obscuring mechanistic nuances. LPTEM as a characterization technique has the potential to augment our understanding of emulsion behavior and improve performance and formulations. In this work, we also emphasize the importance of the included videographic Supporting Information data in demonstrating the behavior of the studied materials.

A new ensemble deep graph reinforcement learning network for spatio-temporal traffic volume forecasting in a freeway network

Spatio-temporal traffic volume forecasting technologies can effectively improve freeway traffic efficiency and the travel comfort of humans. To construct a high-precision traffic volume forecasting model, this study proposed a new ensemble deep graph reinforcement learning network. The modeling process of the spatio-temporal prediction model mainly included three steps. In step I, raw spatiotemporal traffic network datasets (traffic volumes, traffic speeds, weather, and holidays) were preprocessed and the adjacency matrix was constructed. In step II, a graph attention network (GAT) and graph convolution network (GCN) were used as the main predictors to build the spatio-temporal traffic volume forecasting model and obtain the forecasting results, respectively. In step III, deep reinforcement learning was used to effectively analyze the correlations between the forecasting results from these two neural networks and the final results, so as to optimize the weight coefficient. The final result of the proposed model was obtained by combining the forecasting results from the GAT and GCN with the weight coefficient. Based on summarizing and analyzing the experimental results, it can be concluded that: (1) deep reinforcement learning can effectively integrate the two different graph neural networks and achieve better results than traditional ensemble methods; and (2) the presented ensemble model performs better than twenty-one models proposed by other researchers for all studied cases.