Individual Learning Methods Research Articles

Knowledge graph-enhanced multi-agent reinforcement learning for adaptive scheduling in smart manufacturing

AbstractSelf-organizing manufacturing network has emerged as a viable solution for adaptive manufacturing control within the mass personalization paradigm. This approach involves three critical elements: system modeling and control architecture, interoperable communication, and adaptive manufacturing control. However, current research often separates interoperable communication from adaptive manufacturing control as isolated areas of study. To address this gap, this paper introduces Knowledge Graph-enhanced Multi-Agent Reinforcement Learning (MARL) method that integrates interoperable communication via Knowledge Graphs with adaptive manufacturing control through Reinforcement Learning. We hypothesize that implicit domain knowledge obtained from historical production job allocation records can guide each agent to learn more effective scheduling policies with accelerated learning rates. This is based on the premise that machine assignment preferences effectively could reduce the Reinforcement Learning search space. Specifically, we redesign machine agents with new observation, action, reward, and cooperation mechanisms considering the preference of machines, building upon our previous MARL base model. The scheduling policies are trained under extensive simulation experiments that consider manufacturing requirements. During the training process, our approach demonstrates improved training speed compared with individual Reinforcement Learning methods under the same training hyperparameters. The obtained scheduling policies generated by our Knowledge Graph-enhanced MARL also outperform both individual Reinforcement Learning methods and heuristic rules under dynamic manufacturing settings.

Pooling and winsorizing machine learning forecasts to predict stock returns with high-dimensional data

We evaluate US market return predictability using a novel data set of several hundred ag- gregated firm-level characteristics. We apply LASSO, Elastic Net, Random Forest, Neural Net, Extreme Gradient Boosting, and Light Gradient Boosting Machine methods and find these models experience large prediction errors that lead to forecast failures. However, winsorizing and pooling machine learning model forecasts provides consistent out-of-sample predictability. To assess robustness, we apply machine learning methods to high-dimensional data for Canada, China, Germany and the UK as well as the Goyal–Welch data. All machine learning models we consider, except for the ensemble pooled methods, fail to significantly predict returns across our samples, highlighting the importance of pooling, evaluating additional economies, and the fragility of individual machine learning methods. Our results shed light on the sparsity versus density debate as the degree of sparsity and variable importance evolves over time.

Enhancing Translation Competence: Investigating Summarization Impact on English/French to Arabic Translation.

The training of oral and written summarization is embedded in Cognitive Apprenticeship Theory (CAT). This study examines how this form of intervention can improve the long-term reading comprehension skills of students at the Intermediate level; under a program to train translators in English/French into Arabic. It assesses the effectiveness of individual learning and cooperative study methods. One group was given personalized learning, another collaborative; both were split further into spoken and written groups. The 120 students in this cohort all came from the translation studies department at Al-Noor University College (all under age 23). During the six sessions, a delayed post-test used summarization and cooperation scales. The results obtained from one-way ANOVA clarify the impact of summarization training, with verbal groups performing better than written and control learning. In the delayed post-test, groups engaged in collaborative learning notably improved. According to survey results, questions about teacher and student feedback offer mainly positive evaluations of summarization strategies and collaborative learning in translation studies. This study stresses the advanced reading skills in translation studies students gained through summarization training or collaborative learning. This study investigates if there are significant differences in the delayed improvement of reading skills among Translation Department students due to different patterns of summarization training, individually and collaboratively. It also investigates whether there is a significant difference in the delayed effects on reading skills between collaborative and individual approaches, as well as summarizing strategies perception among teachers and students working in Translation Departments

A blending ensemble learning model for crude oil price forecasting

AbstractTo efficiently capture diverse fluctuation profiles in forecasting crude oil prices, we here propose to combine heterogenous predictors for forecasting the prices of crude oil. Specifically, a forecasting model is developed using blended ensemble learning that combines various machine learning methods, including k-nearest neighbor regression, regression trees, linear regression, ridge regression, and support vector regression. Data for Brent and WTI crude oil prices at various time series frequencies are used to validate the proposed blending ensemble learning approach. To show the validity of the proposed model, its performance is further benchmarked against existing individual and ensemble learning methods used for predicting crude oil price, such as lasso regression, bagging lasso regression, boosting, random forest, and support vector regression. We demonstrate that our proposed blending-based model dominates the existing forecasting models in terms of forecasting errors for both short- and medium-term horizons.

Predicting Patent Quality Based on Machine Learning Approach

The investment budget allocated by companies in R&D activities has increased due to increased competition in the market. Applications for industrial property rights by countries, investors, companies, and universities to protect inventions obtained as an outcome of investments have also increased. The selection of the patent to be invested becomes more difficult with the increasing number of applications. Therefore, predicting patent quality is quite significant for companies to be successful in the future. The level to which a patent meets the expectations of decision makers is referred to as patent quality. Patent indices represent decision makers' expectations. In this study, an approach is proposed to predict patent quality in practice. The proposed approach uses supervised learning algorithms and analytic hierarchy process (AHP) method. The proposed approach is applied to patents related to personal digital assistant technologies. The performances of individual and ensemble machine learning methods have been also analyzed to establish the prediction model. In addition, 75% split ratio and the five-fold cross-validation methods have been used to verify the prediction model. The multilayer perceptron algorithm has 76% accuracy value. The proposed prediction model is essential in directing R&D studies to the right technology areas and transferring the incentives to patent applications with a high quality rate.

Ч. АЙТМАТОВДУН “КЫЛЫМ КАРЫТАР БИР КҮН” РОМАНЫНДА- ГЫ НАЙМАН ЭНЕ ЖАНА МАҢКУРТ ЖӨНҮНДӨГҮ УЛАМЫШТЫ САХНАЛАШТЫРУУ АРКЫЛУУ КЕП ӨСТҮРҮҮ (окутуу орус тилинде жүргүзүлгөн мектептер үчүн)

This article presents speech development methods and means of increasing the motivation of students of an intermediate (continuing) group of higher educational institutions through staging on the example of "The Legend of the Mankurt" by Chingiz Aitmatov. This form of learning through the means of a literary text has the greatest incentive to study, to speak Kyrgyz language, to increase practical orientation and to gain deep knowledge. Learning through staging is based on the method of communicative learning, active learning methods, game methods, individual learning methods and art pedagogy, where the main technologies are play and improvisation. In the scientific article, it was emphasized that the teacher should take a creative approach to his work, go out of the framework of the lesson that has been done for a long time, and use the power of free, lively, artistic speech will undoubtedly increase the student's activity. It was also described our desire to increase the practical value of language development by relying on the method of teaching the Kyrgyz language through dramatization in Russian-language schools.

The Effect of Bioclimatic Covariates on Ensemble Machine Learning Prediction of Total Soil Carbon in the Pannonian Biogeoregion

This study employed an ensemble machine learning approach to evaluate the effect of bioclimatic covariates on the prediction accuracy of soil total carbon (TC) in the Pannonian biogeoregion. The analysis involved two main segments: (1) evaluation of base environmental covariates, including surface reflectance, phenology, and derived covariates, compared to the addition of bioclimatic covariates; and (2) assessment of three individual machine learning methods, including random forest (RF), extreme gradient boosting (XGB), and support vector machine (SVM), as well as their ensemble for soil TC prediction. Among the evaluated machine learning methods, the ensemble approach resulted in the highest prediction accuracy overall, outperforming the individual models. The ensemble method with bioclimatic covariates achieved an R2 of 0.580 and an RMSE of 10.392, demonstrating its effectiveness in capturing complex relationships among environmental covariates. The results of this study suggest that the ensemble model consistently outperforms individual machine learning methods (RF, XGB, and SVM), and adding bioclimatic covariates improves the predictive performance of all methods. The study highlights the importance of integrating bioclimatic covariates when modeling environmental covariates and demonstrates the benefits of ensemble machine learning for the geospatial prediction of soil TC.

Improved prediction of shale gas productivity in the Marcellus shale using geostatistically generated well-log data and ensemble machine learning

This study proposes the application of geostatistically generated well-log data to predict well productivity in Marcellus shale reservoirs using ensemble machine learning (ESM). ESM was voting weighted by four individual machine learning methods: random forest (RF), gradient boosting, extreme gradient boosting, and light gradient boosting. The ESM model was trained based on 9921 wells that are qualified as training data. The well-log data had to be predicted using ordinary kriging for all 9921 wells because there were only 18 wells with well-log data for gamma-ray, resistivity, and density. Three cases were compared to analyze the influence of well-log data on machine learning performance. Case 1 comprised only basic well information, such as well location and fracturing-related parameters. Case 2 included the data from Case 1, and the well-log data of the closest well among the 18 wells were assigned to 9921 wells. In Case 3, to data of Case 1, the kriging-generated well-log data were added. In conclusion, Case 3 yielded the best performance among the machine learning algorithms, the highest coefficients of determination, and the lowest mean square errors. According to the RF models, the feature importance of the well-log data in Case 3 was double that in Case 2. The geostatistical method allowed us to make good use of the well-log data despite the restricted number of wells with the well-log compared to the enormous scale of the Marcellus shale. Moreover, basic well information is affordable compared to costly data such as permeability and porosity, even before the production start point. Thus, it can be applied to guide decision-makers when determining a drilling target. Furthermore, owing to its versatility in data affordability, the proposed method could be expanded to other shale basins, such as Barnett, Eagle Ford, and Scoop.

SOFTWARE DEFECT PREDICTION USING THE MACHINE LEARNING METHODS

Reliability of software systems is one of the main indicators of quality. Defects occurring when developing software systems have a direct effect on reliability. Precise prediction of defects in software systems helps software engineers to ensure the reliability of software systems and to properly allocate resources for the trial process. The development of an ensemble method by combining several classification methods occupies one of the main places in research conducted in the field of error prediction in software modules. This paper proposes a method based on the application of ensemble training for defect detection. Here, a database obtained from PROMISE and GITHUB software engineering registry is used to detect defects. Experiments are conducted using Weka software. The prediction efficiency is evaluated based on F-measure and ROC-area. As a result of experiments, the defect detection accuracy of the proposed method is proven to be higher than that of individual machine learning methods.

Evaluation of MBKM Program Implementation in Elementary Schools

The two most important things in education in Indonesia today are 'Freedom to Learn' and driving teachers. This study aims to provide an overview of the implementation of the 'Freedom to Learn' program at the elementary school level, as well as to explain the various innovations, obstacles encountered, and solutions made by the school and teachers as teaching boards. This study uses descriptive quantitative research methods, with 166 respondents from 21 districts/cities in East Java. The results of this study show that; (1) the successful implementation of the 'Freedom to Learn' program can be seen from the ability of schools and teachers to develop teaching creativity during the learning process. (2) Most of the obstacles encountered during program implementation stem from the inability of the teacher to provide learning methods that are right on target, skills that are not sufficient and the limited ability of students to accept the 'transfers of knowledge' provided by the teacher. This study concludes that schools and teachers can optimize learning innovations by using various methods adapted to the student's environment, such as; applying student center learning learning methods, exposition discovery learning methods, individual group learning methods, brainstorming methods, blended learning, and other methods that can improve students' skills and creativity, by implementing fun teaching and learning programs.

Machine Learning-Based House Rent Prediction Using Stacking Integration Method

With the advancement of urbanization and the gradual increase of the rental population, the housing rental market is growing rapidly. It is important to achieve accurate housing rent prediction in order to stabilize the rental housing market. The influence of spatial and temporal factors has led to the complexity of house rent prediction, so it has always been difficult to find an appropriate method. In recent years, machine learning models have been widely studied and applied in various fields, which may provide a promising solution to it. In this paper, a stacking-based ensemble learning model is proposed to solve the problem of house rent prediction. First, the raw data are preprocessed, including decomposing hybrid features, removing rent outliers using scatterplot, removing uncorrelated features, and applying one-hot encoding to transform categorical features into numerical features. Second, the pre-processed data is normalized to unify the magnitudes. Then, the competent base predictive models are selected from all the trained base predictive models and integrated into a comprehensive ensemble model using the stacking integration method to make the final prediction. Finally, the various models are evaluated by some metrics. The experimental results show that the proposed stacking integration-based machine learning method outperforms the individual machine learning methods in solving the house rent prediction problem.

Predicting the Performance of Retail Market Firms: Regression and Machine Learning Methods

The problem of predicting profitability is exceptionally relevant for investors and company owners. This paper examines the factors affecting firm performance and tests and compares various methods based on linear and non-linear dependencies between variables for predicting firm performance. In this study, the methods include random effects regression, individual machine learning algorithms with optimizers (DNN, LSTM, and Random Forest), and advanced machine learning methods consisting of sets of algorithms (portfolios and ensembles). The training sample includes 551 retail-oriented companies and data for 2017–2019 (panel data, 1653 observations). The test sample contains data for these companies for 2020. This study combines two approaches (stages): an econometric analysis of the influence of factors on the company’s profitability and machine learning methods to predict the company’s profitability. To compare forecasting methods, we used parametric and non-parametric predictive measures and ANOVA. The paper shows that previous profitability has a strong positive impact on a firm’s performance. We also find a non-linear positive effect of sales growth and web traffic on firm profitability. These variables significantly improve the prediction accuracy. Regression is inferior in forecast accuracy to machine learning methods. Advanced methods (portfolios and ensembles) demonstrate better and more steady results compared with individual machine learning methods.

Euclidpreparation

TheEuclidSpace Telescope will provide deep imaging at optical and near-infrared wavelengths, along with slitless near-infrared spectroscopy, across ~15 000deg2of the sky.Euclidis expected to detect ~12 billion astronomical sources, facilitating new insights into cosmology, galaxy evolution, and various other topics. In order to optimally exploit the expected very large dataset, appropriate methods and software tools need to be developed. Here we present a novel machine-learning-based methodology for the selection of quiescent galaxies using broadbandEuclid IE,YE,JE, andHEphotometry, in combination with multi-wavelength photometry from other large surveys (e.g. theRubinLSST). The ARIADNE pipeline uses meta-learning to fuse decision-tree ensembles, nearest-neighbours, and deep-learning methods into a single classifier that yields significantly higher accuracy than any of the individual learning methods separately. The pipeline has been designed to have 'sparsity awareness', such that missing photometry values are informative for the classification. In addition, our pipeline is able to derive photometric redshifts for galaxies selected as quiescent, aided by the 'pseudo-labelling' semi-supervised method, and using an outlier detection algorithm to identify and reject likely catastrophic outliers. After the application of the outlier filter, our pipeline achieves a normalised mean absolute deviation of ≲0.03 and a fraction of catastrophic outliers of ≲0.02 when measured against the COSMOS2015 photometric redshifts. We apply our classification pipeline to mock galaxy photometry catalogues corresponding to three main scenarios: (i)EuclidDeep Survey photometry with ancillaryugriz,WISE, and radio data; (ii)EuclidWide Survey photometry with ancillaryugriz,WISE, and radio data; and (iii)EuclidWide Survey photometry only, with no foreknowledge of galaxy redshifts. In a like-for-like comparison, our classification pipeline outperformsUVJselection, in addition to theEuclid IE–YE,JE–HEandu–IE, IE–JEcolour-colour methods, with improvements in completeness and theF1-score (the harmonic mean of precision and recall) of up to a factor of 2.

Productivity prediction in the Wolfcamp A and B using weighted voting ensemble machine learning method

This study investigates the applicability of ensemble machine learning to predict cumulative gas production for the first 36 months of shale gas reservoirs in the Wolfcamp A and B, Delaware basin. The machine learning application was conducted from three notable perspectives: (1) Well log data generation based on stratigraphy interpretation, manual well correlations, and pre-process of category type data to utilize it for machine learning training. (2) General applicability of machine learning using well data without production rate information. (3) Ensemble methods (ESM) with voting weight composed of four individual machine learning methods based on bagging and boosting: random forest (RF), gradient boosting (GB), extreme gradient boosting (XGB), and light gradient boosting machine (LGBM) providing reliable shale gas productivity prediction performance and consistent results from different training and test data compositions. Coefficient of determination (R2) and mean square error (MSE) are notably improved in comparison between the primary level data and proposed data package with well log data (7% increase and 19% decrease, respectively). Furthermore, the ESM with voting weight produces the best predictive performance on average for the ten different training and test data pools among the six cases (RF, GB, XGB, LGBM, and ESM with even/voting weight). In the ESM with voting weight, the average weights of the RF, GB, XGB, and LGBM are 0.259, 0.243, 0.281, and 0.216, respectively. Although the RF is the best predictive model among the four individual prediction results, XGB produces the highest weight. The voting weight system of the ESM lets the four individual machine learning methods complement one another, balancing the final ESM model. Besides, the final ESM model brought around 12% uncertainty reduction MSE-wise compared to the RF model. Therefore, enhanced universality and reliability are obtained for the given learning data, permitting the successful productivity prediction of shale gas reservoirs.

The Practical of Interactive Online Learning in the Japanese Language by Using LoiLoNote School Application

This paper introduces a practical example of applying it to language learning based on the author’s experience in the Japanese curriculum at Bina Nusantara University, Indonesia. In 2020, due to the pandemic, it was decided to conduct all classes, and faculty collaboration would be conducted online from Japan. This study proposes a three-year interactive Japanese language learning class, including a face-to- face comparison from February 2021 to February 2023. The approach to classroom practice is only from Japan application presented in this paper. This application, demonstrated in this study, was used in each semester’s classes, and the author found the optimal individual learning method. It should be added that the application was used in grammar and reading classes. The survey results of this study were obtained from two types of surveys, a quantitative survey and a qualitative survey of the students. Students did not find the new learning method as “new” and could use it smoothly. The effectiveness of this application was discussed based on the difference in learning ability and students’ comments after using it. Therefore, this research shows how practical this application is in ensuring interactivity in students’ language learning.

Adaptive risk-aware sharable and individual subspace learning for cancer survival analysis with multi-modality data.

Biomedical multi-modality data (also named multi-omics data) refer to data that span different types and derive from multiple sources in clinical practices (e.g. gene sequences, proteomics and histopathological images), which can provide comprehensive perspectives for cancers and generally improve the performance of survival models. However, the performance improvement of multi-modality survival models may be hindered by two key issues as follows: (1) how to learn and fuse modality-sharable and modality-individual representations from multi-modality data; (2) how to explore the potential risk-aware characteristics in each risk subgroup, which is beneficial to risk stratification and prognosis evaluation. Additionally, learning-based survival models generally refer to numerous hyper-parameters, which requires time-consuming parameter setting and might result in a suboptimal solution. In this paper, we propose an adaptive risk-aware sharable and individual subspace learning method for cancer survival analysis. The proposed method jointly learns sharable and individual subspaces from multi-modality data, whereas two auxiliary terms (i.e. intra-modality complementarity and inter-modality incoherence) are developed to preserve the complementary and distinctive properties of each modality. Moreover, it equips with a grouping co-expression constraint for obtaining risk-aware representation and preserving local consistency. Furthermore, an adaptive-weighted strategy is employed to efficiently estimate crucial parameters during the training stage. Experimental results on three public datasets demonstrate the superiority of our proposed model.

Potential genetic biomarkers predict adverse pregnancy outcome during early and mid-pregnancy in women with systemic lupus erythematosus.

Effectively predicting the risk of adverse pregnancy outcome (APO) in women with systemic lupus erythematosus (SLE) during early and mid-pregnancy is a challenge. This study was aimed to identify potential markers for early prediction of APO risk in women with SLE. The GSE108497 gene expression dataset containing 120 samples (36 patients, 84 controls) was downloaded from the Gene Expression Omnibus database. Weighted gene co-expression network analysis (WGCNA) was performed, and differentially expressed genes (DEGs) were screened to define candidate APO marker genes. Next, three individual machine learning methods, random forest, support vector machine-recursive feature elimination, and least absolute shrinkage and selection operator, were combined to identify feature genes from the APO candidate set. The predictive performance of feature genes for APO risk was assessed using area under the receiver operating characteristic curve (AUC) and calibration curves. The potential functions of these feature genes were finally analyzed by conventional gene set enrichment analysis and CIBERSORT algorithm analysis. We identified 321 significantly up-regulated genes and 307 down-regulated genes between patients and controls, along with 181 potential functionally associated genes in the WGCNA analysis. By integrating these results, we revealed 70 APO candidate genes. Three feature genes, SEZ6, NRAD1, and LPAR4, were identified by machine learning methods. Of these, SEZ6 (AUC = 0.753) showed the highest in-sample predictive performance for APO risk in pregnant women with SLE, followed by NRAD1 (AUC = 0.694) and LPAR4 (AUC = 0.654). After performing leave-one-out cross validation, corresponding AUCs for SEZ6, NRAD1, and LPAR4 were 0.731, 0.668, and 0.626, respectively. Moreover, CIBERSORT analysis showed a positive correlation between regulatory T cell levels and SEZ6 expression (P < 0.01), along with a negative correlation between M2 macrophages levels and LPAR4 expression (P < 0.01). Our preliminary findings suggested that SEZ6, NRAD1, and LPAR4 might represent the useful genetic biomarkers for predicting APO risk during early and mid-pregnancy in women with SLE, and enhanced our understanding of the origins of pregnancy complications in pregnant women with SLE. However, further validation was required.

A Novel Ensemble Earthquake Prediction Method (EEPM) by Combining Parameters and Precursors

A leading cause of death from natural disasters over the last 50years is witnessed by none other than earthquake occurrences which have a negative economic impact on the world and claimed thousands of lives over the years, causing devastation to properties. In this paper, a novel Ensemble Earthquake Prediction Method (EEPM) is proposed and implemented to produce a strong learner (ensemble method) having better accuracy in prediction, less variance, and less errors. Data (parameters) which is continuous in nature is collected from two countries, India and Nepal, for five years, and surveyor’s data (precursor) which is categorical in nature is collected from three countries India, Nepal, and Kenya for five years on the specific earthquake-prone regions. The preprocessed data is generated by combining parameters and precursor data. EEPM focuses on detecting the accurate and better early signs of an earthquake and finding the probability of occurrence of an earthquake in the specified region, i.e., better prediction and robustness. The results of EEPM produced better R 2 and less variance and less error in comparison to individual machine learning methods as well as better accuracy 87.8%, compared to state-of-the-art ensemble methods. The prediction of earthquake will alarm not only the people of the society but also the different organizations to explain the appropriate range of magnitude and dynamics of occurrence of earthquake.

The effects of model-based cooperative and individual learning methods on pre-service science teachers’ conceptual understanding of gases

"The aim of this study was to determine the effect of model-based cooperative (Reading Writing Application) and individual learning methods on conceptual understandings of pre-service science teachers and to eliminate their misconceptions related to gases. For this reason, a pretest/post-test non-equivalent comparison group design was applied across two experimental groups. The sample consisted of 42 pre-service science teachers; one of the research groups was a Reading Writing Application- Model Group (n=22) and the other group was an Individual Learning- Model Group (n=20). The Gases Concept Test (GCT) was used for pre- and post-test as the data collection instrument. For analysing data, descriptive statistics were determined, and the Mann-Whitney U test was performed. There was not a significant difference between groups with respect to their development of their conceptual understanding of gases. In addition, some preservice science teachers from both groups had various misconceptions about the topic after the application."

Evolutionary bagging for ensemble learning

Ensemble learning has gained success in machine learning with major advantages over other learning methods. Bagging is a prominent ensemble learning method that creates subgroups of data, known as bags, that are trained by individual machine learning methods such as decision trees. Random forest is a prominent example of bagging with additional features in the learning process. Evolutionary algorithms have been prominent for optimisation problems and also been used for machine learning. Evolutionary algorithms are gradient-free methods that work with a population of candidate solutions that maintain diversity for creating new solutions. In conventional bagged ensemble learning, the bags are created once and the content, in terms of the training examples, are fixed over the learning process. In our paper, we propose evolutionary bagged ensemble learning, where we utilise evolutionary algorithms to evolve the content of the bags in order to iteratively enhance the ensemble by providing diversity in the bags. The results show that our evolutionary ensemble bagging method outperforms conventional ensemble methods (bagging and random forests) for several benchmark datasets under certain constraints. We find that evolutionary bagging can inherently sustain a diverse set of bags without reduction in performance accuracy.