Data Sampling Methods Research Articles

An objective absence data sampling method for landslide susceptibility mapping

The accuracy and quality of the landslide susceptibility map depend on the available landslide locations and the sampling strategy for absence data (non-landslide locations). In this study, we propose an objective method to determine the critical value for sampling absence data based on Mahalanobis distances (MD). We demonstrate this method on landslide susceptibility mapping of three subdistricts (Upazilas) of the Rangamati district, Bangladesh, and compare the results with the landslide susceptibility map produced based on the slope-based absence data sampling method. Using the 15 landslide causal factors, including slope, aspect, and plan curvature, we first determine the critical value of 23.69 based on the Chi-square distribution with 14 degrees of freedom. This critical value was then used to determine the sampling space for 261 random absence data. In comparison, we chose another set of the absence data based on a slope threshold of < 3°. The landslide susceptibility maps were then generated using the random forest model. The Receiver Operating Characteristic (ROC) curves and the Kappa index were used for accuracy assessment, while the Seed Cell Area Index (SCAI) was used for consistency assessment. The landslide susceptibility map produced using our proposed method has relatively high model fitting (0.87), prediction (0.85), and Kappa values (0.77). Even though the landslide susceptibility map produced by the slope-based sampling also has relatively high accuracy, the SCAI values suggest lower consistency. Furthermore, slope-based sampling is highly subjective; therefore, we recommend using MD -based absence data sampling for landslide susceptibility mapping.

Robust Data Sampling in Machine Learning: A Game-Theoretic Framework for Training and Validation Data Selection

How to sample training/validation data is an important question for machine learning models, especially when the dataset is heterogeneous and skewed. In this paper, we propose a data sampling method that robustly selects training/validation data. We formulate the training/validation data sampling process as a two-player game: a trainer aims to sample training data so as to minimize the test error, while a validator adversarially samples validation data that can increase the test error. Robust sampling is achieved at the game equilibrium. To accelerate the searching process, we adopt reinforcement learning aided Monte Carlo trees search (MCTS). We apply our method to a car-following modeling problem, a complicated scenario with heterogeneous and random human driving behavior. Real-world data, the Next Generation SIMulation (NGSIM), is used to validate this method, and experiment results demonstrate the sampling robustness and thereby the model out-of-sample performance.

A multi stage approach to handle class imbalance:An ensemble method

An imbalanced learning problem is a challenge that has grabbed the interest of academics and industry. Class imbalance is a widespread problem in machine learning due to the unavailability of data in a specific category. Several under-sampling and oversampling methods handle class imbalance; however, finding the correct pattern heavily depends on the sub-sampling. This paper proposes a two-stage approach with a data sampling method where Mutually Disjoint Data Sets(MDS) created from majority samples and binded with minority samples. A set of heterogeneous machine learning algorithms were applied on these binded sub-samples to create ensemble models. MDS ensembles give importance to the minority samples with a double voting method. The proposed method has an improvement of 3.78, 13.78 percent of average recall on the train and test data compared to best-base model results

Intraday forecasting of OEE through sensor data and machine learning

Key performance indicators are crucial for measuring and managing the performance of industrial processes. They are employed to detect deviations in production, enabling opportunities to improve manufacturing processes within the three dimensions of time, quality, and cost. In this context, the timeliness of information plays a decisive role in the success of measures since delayed information availability can leave decision-makers with no time to react. Usually, key performance indicators are updated on a shift or daily basis and thus only allow a delayed reaction to deviations that have occurred. To tackle this problem, different mixed data sampling and machine learning methods for the intraday monitoring of key performance indicators based on high-frequency sensor and machine data are investigated. By benchmarking the approaches on real-world data from the process industry, the authors show how to obtain models for real-time detection of performance deviations. To enable application for practitioners and academia in the future, a method for implementation and deployment in real-world scenarios is given. The developed methodology offers the construction of high-frequency forecasts for key performance metrics, and therefore accelerates managers’ reactions to associated problems.

Uncertainty sources affecting operational efficiency of ML algorithms in UAV-based precision agriculture: A 2013–2020 systematic review

&lt;abstract&gt; &lt;p&gt;Conventional methods of data sampling in agriculture are time consuming, labor intensive, destructive, subject to human error and affected by field conditions. Thus, remote sensing technologies such as unmanned aerial vehicles (UAVs) became widely used as an alternative for data collection. Nevertheless, the big data captured by the UAVs is challenging to interpret. Therefore, machine learning algorithms (MLs) are used to interpret this data. However, the operational efficiency of those MLs is yet to be improved due to different sources affecting their modeling certainty. Therefore, this study aims to review different sources affecting the accuracy of MLs regression and classification interventions in precision agriculture. In this regard, 109 articles were identified in the Scopus database. The search was restricted to articles written in English, published during 2013–2020, and used UAVs as in-field data collection tools and ML algorithms for data analysis and interpretation. This systematic review will be the point of review for researchers to recognize the possible sources affecting the certainty of regression and classification results associated with MLs use. The recognition of those sources points out areas for improvement of MLs performance in precision agriculture. In this review, the performance of MLs is still evaluated in general, which opens the road for further detailed research.&lt;/p&gt; &lt;/abstract&gt;

A transzformatív turizmus lehetőségei az Európa Kulturális Fővárosa projektben résztvevő városok számára – Kassa példája

The study aims to explore an avenue that former European Capitals of Culture can choose to develop long-term sustainable tourism. The topicality lies in the fact that Veszprém-Balaton obtained the title for the year of 2023, as well as in the fact that the coronavirus and the war have shown the vulnerability of tourism. Furthermore, after the year-long festivities many former European Capitals of Culture cannot keep up the momentum and new facilities suffer from overcapacity. This study builds on three pillars. The first pillar is to explore the theoretical background of the topic in a multidisciplinary way, the second introduces the European Union project involving six former European Capitals of Culture that try to find a long-term successful tourism model using the principles of transformative tourism. The third pillar is an exploratory case study of Kosice. The research uses document analysis, media analysis and interview data sampling methods.

A geographical similarity-based sampling method of non-fire point data for spatial prediction of forest fires

Understanding the mechanisms and risks of forest fires by building a spatial prediction model is an important means of controlling forest fires. Non-fire point data are important training data for constructing a model, and their quality significantly impacts the prediction performance of the model. However, non-fire point data obtained using existing sampling methods generally suffer from low representativeness. Therefore, this study proposes a non-fire point data sampling method based on geographical similarity to improve the quality of non-fire point samples. The method is based on the idea that the less similar the geographical environment between a sample point and an already occurred fire point, the greater the confidence in being a non-fire point sample. Yunnan Province, China, with a high frequency of forest fires, was used as the study area. We compared the prediction performance of traditional sampling methods and the proposed method using three commonly used forest fire risk prediction models: logistic regression (LR), support vector machine (SVM), and random forest (RF). The results show that the modeling and prediction accuracies of the forest fire prediction models established based on the proposed sampling method are significantly improved compared with those of the traditional sampling method. Specifically, in 2010, the modeling and prediction accuracies improved by 19.1% and 32.8%, respectively, and in 2020, they improved by 13.1% and 24.3%, respectively. Therefore, we believe that collecting non-fire point samples based on the principle of geographical similarity is an effective way to improve the quality of forest fire samples, and thus enhance the prediction of forest fire risk.

공공하수처리시설 단위공정별 고장예지 알고리즘 개발

In order to develop an automatic control system for public sewage treatment facilities, it is necessary to conduct a technical analysis of each water quality measurement item to discover new trends or patterns. In this study, a failure prediction algorithm for each instrument or unit process was developed for the automatic control of public sewage treatment facilities, with the optimal data sampling method and abnormality detection criteria for each water quality item. In order to establish the confidence intervals for the five telemonitoring systems (TMS) parameters (chemical oxygen demand, suspended solids, total nitrogen, total phosphorus, and pH) in the sedimentation tank and bioreactor, that during a set period, 48, 72, or 168 samples were taken and the error rate was reviewed. Based on this, 48 samples were established as the optimal level with an error rate of 1.46%. In addition, the abnormality rate for the TMS data at a confidence interval of μ±3σ averaged 1.73%, confirming that any abnormalities that were recorded were not solely the result of simple data errors but also included genuine problems with an instrument or unit process.

Detection of Malicious Websites Using Symbolic Classifier

Malicious websites are web locations that attempt to install malware, which is the general term for anything that will cause problems in computer operation, gather confidential information, or gain total control over the computer. In this paper, a novel approach is proposed which consists of the implementation of the genetic programming symbolic classifier (GPSC) algorithm on a publicly available dataset to obtain a simple symbolic expression (mathematical equation) which could detect malicious websites with high classification accuracy. Due to a large imbalance of classes in the initial dataset, several data sampling methods (random undersampling/oversampling, ADASYN, SMOTE, BorderlineSMOTE, and KmeansSMOTE) were used to balance the dataset classes. For this investigation, the hyperparameter search method was developed to find the combination of GPSC hyperparameters with which high classification accuracy could be achieved. The first investigation was conducted using GPSC with a random hyperparameter search method and each dataset variation was divided on a train and test dataset in a ratio of 70:30. To evaluate each symbolic expression, the performance of each symbolic expression was measured on the train and test dataset and the mean and standard deviation values of accuracy (ACC), AUC, precision, recall and f1-score were obtained. The second investigation was also conducted using GPSC with the random hyperparameter search method; however, 70%, i.e., the train dataset, was used to perform 5-fold cross-validation. If the mean accuracy, AUC, precision, recall, and f1-score values were above 0.97 then final training and testing (train/test 70:30) were performed with GPSC with the same randomly chosen hyperparameters used in a 5-fold cross-validation process and the final mean and standard deviation values of the aforementioned evaluation methods were obtained. In both investigations, the best symbolic expression was obtained in the case where the dataset balanced with the KMeansSMOTE method was used for training and testing. The best symbolic expression obtained using GPSC with the random hyperparameter search method and classic train–test procedure (70:30) on a dataset balanced with the KMeansSMOTE method achieved values of ACC¯, AUC¯, Precsion¯, Recall¯ and F1-score¯ (with standard deviation) 0.9992±2.249×10−5, 0.9995±9.945×10−6, 0.9995±1.09×10−5, 0.999±5.17×10−5, 0.9992±5.17×10−6, respectively. The best symbolic expression obtained using GPSC with a random hyperparameter search method and 5-fold cross-validation on a dataset balanced with the KMeansSMOTE method achieved values of ACC¯, AUC¯, Precsion¯, Recall¯ and F1-score¯ (with standard deviation) 0.9994±1.13×10−5, 0.9994±1.2×10−5, 1.0±0, 0.9988±2.4×10−5, and 0.9994±1.2×10−5, respectively.

Improved off‐policy reinforcement learning algorithm for robust control of unmodeled nonlinear system with asymmetric state constraints

AbstractIn this article, an improved data‐based off‐policy reinforcement learning algorithm is proposed for the robust control of unmodeled nonlinear systems with asymmetric state constraints. An improved nonlinear mapping is defined for the asymmetric state constraint problem, which can ensure that the mapping state has better response speed and amplitude than the original state. Then, an auxiliary mapping error system is constructed for the off‐policy robust controller design. At the same time, an innovative network dimensionality reduction method based on principal component analysis is proposed to simplify the useless activation function of action network in off‐policy algorithm, which can effectively reduce the computational burden of data episodes. Considering the uncertain data caused by disturbances, a dominant data sampling method is designed to extract samples that are beneficial to algorithm convergence. On this basis, the improved off‐policy robust control algorithm is constructed. Based on an industrial manipulator system, the effectiveness of the dominant data sampling method and the improved off‐policy robust control algorithm is verified by comparative simulation.

Influence of data sampling methods on the representation of neural spiking activity in vivo

Invivo single-unit recordings distinguish the basal spiking properties of neurons in different experimental settings and disease states. Here, we examined over 300 spike trains recorded from Purkinje cells and cerebellar nuclei neurons to test whether data sampling approaches influence the extraction of rich descriptors of firing properties. Our analyses included neurons recorded in awake and anesthetized control mice, and disease models of ataxia, dystonia, and tremor. We find that recording duration circumscribes overall representations of firing rate and pattern. Notably, shorter recording durations skew estimates for global firing rate variability toward lower values. We also find that only some populations of neurons in the same mouse are more similar to each other than to neurons recorded in different mice. These data reveal that recording duration and approach are primary considerations when interpreting task-independent single neuron firing properties. If not accounted for, group differences may be concealed or exaggerated.

State-of-the-art magnetic resonance imaging sequences for pediatric body imaging.

Longer examination time, need for anesthesia in smaller children and the inability of most children to hold their breath are major limitations of MRI in pediatric body imaging. Fortunately, with technical advances, many new and upcoming MRI sequences are overcoming these limitations. Advances in data acquisition and k-space sampling methods have enabled sequences with improved temporal and spatial resolution, and minimal artifacts. Sequences to minimize movement artifacts mainly utilize radial k-space filling, and examples include the stack-of-stars method for T1-weighted imaging and the periodically rotated overlapping parallel lines with enhanced reconstruction (PROPELLER)/BLADE method for T2-weighted imaging. Similarly, the sequences with improved temporal resolution and the ability to obtain multiple phases in a single breath-hold in dynamic imaging mainly use some form of partial k-space filling method. New sequences use a variable combination of data sampling methods like compressed sensing, golden-angle radial k-space filling, parallel imaging and partial k-space filling to achieve free-breathing, faster sequences that could be useful for pediatric abdominal and thoracic imaging. Simultaneous multi-slice method has improved diffusion-weighted imaging (DWI) with reduction in scan time and artifacts. In this review, we provide an overview of data sampling methods like parallel imaging, compressed sensing, radial k-space sampling, partial k-space sampling and simultaneous multi-slice. This is followed by newer available and upcoming sequences for T1-, T2- and DWI based on these other advances. We also discuss the Dixon method and newer approaches to reducing metal artifacts.

Bayesian Approach for Optimizing Forest Inventory Survey Sampling with Remote Sensing Data

In large-area forest inventories, a trade-off between the amount of data to be sampled and the corresponding collection costs is necessary. It is not always possible to have a very large data sample when dealing with sampling-based inventories. It is therefore important to optimize the sampling design with the limited resources. Whereas this sort of inventories are subject to these constraints, the availability of remote sensing (RS) data correlated with the forest inventory variables is usually much higher. For this reason, the RS and sampled field measurement data are often used in combination for improving the forest inventory estimation. In this study, we propose a model-based data sampling method founded on Bayesian optimization and machine learning algorithms which utilizes RS data to guide forest inventory sample selection. We evaluate our method in empirical experiments using real-world volume of growing stock data from the Aland region in Finland. The proposed method is compared against two baseline methods: simple random sampling and the local pivotal method. When a suitable model link is selected, the empirical experiments show on best case on average up to 22% and 79% improvement in population mean and variance estimation respectively over baselines. However, the results also illustrate the importance of model selection which has a clear effect on the results. The novelty of the study is in the application of Bayesian optimization in national forest inventory survey sampling.

Critical analysis of Lean Six Sigma black belt certification courses offered in Brazil

PurposeThis study aims to present an overview and analyze the Lean Six Sigma Black Belt (LSSBB) certifications offered by institutions operating in Brazil.Design/methodology/approachThis research analyzed LSSBB certification courses offered by 48 institutions in Brazil by comparing the syllabi of the classes to the reference model proposed by the American Society for Quality (ASQ) in the Six Sigma Black Belt Body of Knowledge. This study employed the content analysis technique and hierarchical cluster analysis to analyze the data.FindingsThe results revealed a lack of standardization in the content of Lean Six Sigma (LSS) training in Brazil. 100% of the LSSBB courses analyzed covered four of the 108 techniques recommended by the ASQ Body of Knowledge (i.e. data types, measurement scales, sampling, and data collection plans and methods). In contrast, more than 75% of the courses covered all techniques related to the macro areas of organization-wide planning and deployment, organizational process management and measures, measure, and improve. The major shortcoming of LSS training is related to the macro area Design for Six Sigma framework and methodologies. LSS training is offered in a highly concentrated area in Brazil, the wealthiest region, where universities play a crucial role in disseminating LSS.Originality/valueThe literature lacks studies that critically examine LSS certification courses. There is little research on LSS in Brazil and there are no studies on LSS training in this country.

Ultrasound-based radiomics analysis for differentiating benign and malignant breast lesions: From static images to CEUS video analysis.

BackgroundContinuous contrast-enhanced ultrasound (CEUS) video is a challenging direction for radiomics research. We aimed to evaluate machine learning (ML) approaches with radiomics combined with the XGBoost model and a convolutional neural network (CNN) for discriminating between benign and malignant lesions in CEUS videos with a duration of more than 1 min.MethodsWe gathered breast CEUS videos of 109 benign and 81 malignant tumors from two centers. Radiomics combined with the XGBoost model and a CNN was used to classify the breast lesions on the CEUS videos. The lesions were manually segmented by one radiologist. Radiomics combined with the XGBoost model was conducted with a variety of data sampling methods. The CNN used pretrained 3D residual network (ResNet) models with 18, 34, 50, and 101 layers. The machine interpretations were compared with prospective interpretations by two radiologists. Breast biopsies or pathological examinations were used as the reference standard. Areas under the receiver operating curves (AUCs) were used to compare the diagnostic performance of the models.ResultsThe CNN model achieved the best AUC of 0.84 on the test cohort with the 3D-ResNet-50 model. The radiomics model obtained AUCs between 0.65 and 0.75. Radiologists 1 and 2 had AUCs of 0.75 and 0.70, respectively.ConclusionsThe 3D-ResNet-50 model was superior to the radiomics combined with the XGBoost model in classifying enhanced lesions as benign or malignant on CEUS videos. The CNN model was superior to the radiologists, and the radiomics model performance was close to the performance of the radiologists.

Antiviral Treatment and One-Year Follow-Up of Chronic Hepatitis B Patients in Bandung: An Observational Study

Background: Hepatitis B Virus (HBV) is a health problem that has chronically infected 257 million globally. Appropriate therapy is immensely needed to prevent disease progression to cirrhosis and hepatocellular carcinoma (HCC). Moreover, routine monitoring is required to assess the efficacy of the given treatment. This study aims to describe the treatment and its follow-up outcomes among chronic HBV patients after one year of therapy in Bandung.Method: This was a cross-sectional descriptive study with a data sampling method obtained from assessing the medical records of treated chronic HBV outpatients at the Gastrohepatoenterology Clinic of Hasan Sadikin Hospital Bandung from 2018 to 2020. Laboratory findings before and after one year of therapy were further assessed.Results: Out of 107 patients treated, the proportion of tenofovir was 72.90%, telbivudine 16.80%, lamivudine 8.40%, and interferon group 1.90%. There were 52.30% of patients who did follow-up visits after one year of therapy. The therapeutic outcome rate was evaluated among total patients with the criteria of decreasing AST (91.18%) and ALT levels (93.10%) levels, reduction of HBV-DNA values (100%), and HBeAg seroconversion (14.29%).Conclusion: The most given therapy among chronic HBV outpatients was tenofovir. The follow-up rate of patients after one year of treatment was 52.30%. Overall, antiviral therapies produced decreases in liver laboratory parameters, HBV-DNA, and HBeAg.

Benchmarks in antimicrobial peptide prediction are biased due to the selection of negative data.

Antimicrobial peptides (AMPs) are a heterogeneous group of short polypeptides that target not only microorganisms but also viruses and cancer cells. Due to their lower selection for resistance compared with traditional antibiotics, AMPs have been attracting the ever-growing attention from researchers, including bioinformaticians. Machine learning represents the most cost-effective method for novel AMP discovery and consequently many computational tools for AMP prediction have been recently developed. In this article, we investigate the impact of negative data sampling on model performance and benchmarking. We generated 660 predictive models using 12 machine learning architectures, a single positive data set and 11 negative data sampling methods; the architectures and methods were defined on the basis of published AMP prediction software. Our results clearly indicate that similar training and benchmark data set, i.e. produced by the same or a similar negative data sampling method, positively affect model performance. Consequently, all the benchmark analyses that have been performed for AMP prediction models are significantly biased and, moreover, we do not know which model is the most accurate. To provide researchers with reliable information about the performance of AMP predictors, we also created a web server AMPBenchmark for fair model benchmarking. AMPBenchmark is available at http://BioGenies.info/AMPBenchmark.

A Novel Double Ensemble Algorithm for the Classification of Multi-Class Imbalanced Hyperspectral Data

The class imbalance problem has been reported to exist in remote sensing and hinders the classification performance of many machine learning algorithms. Several technologies, such as data sampling methods, feature selection-based methods, and ensemble-based methods, have been proposed to solve the class imbalance problem. However, these methods suffer from the loss of useful information or from artificial noise, or result in overfitting. A novel double ensemble algorithm is proposed to deal with the multi-class imbalance problem of the hyperspectral image in this paper. This method first computes the feature importance values of the hyperspectral data via an ensemble model, then produces several balanced data sets based on oversampling and builds a number of classifiers. Finally, the classification results of these diversity classifiers are combined according to a specific ensemble rule. In the experiment, different data-handling methods and classification methods including random undersampling (RUS), random oversampling (ROS), Adaboost, Bagging, and random forest are compared with the proposed double random forest method. The experimental results on three imbalanced hyperspectral data sets demonstrate the effectiveness of the proposed algorithm.

Research on Prediction Model of Rock and Soil Layer Information Based on Adjacent Boreholes

Aiming at the problem that the drilling core cannot be drilled and core is unable to be drilled to find the stratum due to the construction measures or large equipment occupation in the survey site, based on the thickness information of the adjacent hole position of the borehole to be measured, the BP neural network model for stratum prediction was constructed, and the training data sampling method and comprehensive analysis results were put forward, which were based on the training data sampling method and comprehensive analysis results in the front, back, left, right and circle positions of the borehole, combined with the actual investigation Engineering. The influence of soil layer training data collected by different methods on the accuracy of soil layer thickness prediction was analyzed. The results showed that: (1) By comparing the prediction results of 10 groups of actual engineering holes to be measured, it was revealed that the prediction accuracy of soil thickness had great correlation with the orientation selection. (2) In azimuth prediction, each group collects 21 adjacent boreholes as training data samples by continuous cycle of three longitudinal and seven horizontal directions. The prediction results of neural network were stable. (3) A selection strategy was proposed to analyze the preliminary prediction results of neural network. Based on the circular prediction results, the data with the difference within ±0.2m in azimuth prediction were compared and screened. The comprehensive prediction value obtained by test was very close to the true value, and the data error is within 9%, and the prediction effect was good. The research results provide a way of thinking for the prediction of small sample soil sequence, and can provide reference for geotechnical engineering investigation and design application.

A Method to Study Group Relationship of College Students Based on Predictive Social Network

With the rise of social network platforms such as WeChat, Weibo, and TikTok, social networks have developed from simple social networks to complex social networks. Researchers have gradually found that the traditional data sampling methods can no longer meet the development needs of the complex social network structure. In order to save network resources, various methods of social relationship prediction have been proposed. In this paper, we propose a BTCS algorithm based on low sampling rate under cognitive model and conduct several sets of comparison experiments under different networks and different sampling rates, and the results show that the BTCS algorithm improves the prediction accuracy and reduces the prediction time under low sampling rate. To address the problems of poor stability and slow prediction speed of random sampling prediction methods, this paper proposes a CCS algorithm in colleges and universities using the characteristics of high awareness among nodes within the same college. It can effectively combine the cognitive characteristics of the nodes with the college attributes and apply them to the relationship prediction to realize the college-oriented relationship prediction. The simulation results show that the CCS algorithm is more stable than other random sampling prediction methods. The results make full use of the cognitive characteristics and college attributes of nodes in social networks; reduce the influence of multiple factors such as response time, data packet loss, and individual behavior on relationship prediction; and improve the efficiency of college student group relationship prediction, which has certain theoretical significance and application prospects.