Data Preparation Stage Research Articles

A new platform for machine-learning-based network traffic classification

This study provides a new platform for classifying encrypted network traffic based on machine learning (ML) techniques. The architecture of the platform is designed for real-world network traffic classification problems with performance-oriented, practical, and up-to-date software technologies. In addition, this study introduces a new feature extraction method to the literature. The proposed platform applies ML techniques with flow-based statistical features of encrypted network traffic and new feature extraction. It takes network traffic packets as input and passes them through feature extraction, data preparation, and ML stages. In the feature extraction stage, network flows are extracted from the network traffic data by calculating their features with the NFStream tool. During the data preparation stage, the dataset is transformed into a processable state for the ML algorithm with the Apache Spark framework. This stage also includes the feature selection operation. The ML stage runs GBTree, LightGBM, and XGBoost algorithms. Moreover, we use the MLflow framework in the proposed process management to observe the ML lifecycle, including experimentation, reproducibility, and deployment. The experimental results show that the XGBoost algorithm achieves the best result with an F1 score of above 99%.

Data optimization for the knowledge bases in the oil and gas Monitoring-While-Drilling (MWD) Systems

The ability of a knowledge-oriented system to learn requires the selection and organization of data. The article is devoted to the disclosure of the stages of data preparation for the application of case-based procedures to support decision-making by the operator of the technological process in the field of oil and gas wells drilling. The presented concept allows concentrating expert experience in the subject area and satisfies the criteria of the main sources of knowledge required for the functioning of case-based considerations. The main potential of artificial intelligence to learn is based on semantic dictionaries of the subject area, case databases in the form of knowledge bases, metrics of similarity, adaptation and configuration containers, which are identified as the sources of knowledge. The effectiveness of using computer simulators as a platform for experimental research of new models of complex technological processes is estimated.

Analisis Sentimen Wisatawan terhadap Kualitas Layanan Hotel dan Resort di Lombok Menggunakan SERVQUAL dan CRISP-DM

The era of digital transformation has sparked innovations in product and service marketing strategies in various sectors, one of which is the tourism sector. In the hospitality industry context, product marketing using website-based digital media allows consumers as hotel guests to review the products and services received. The Tripadvisor website is a digital marketing platform that provides review features for app users, especially consumers, to give ratings and reviews. This study aims to analyze the quality of hotel services using the Service Quality (SERVQUAL) framework based on the results of the classification of hotel guest sentiment data using the Naïve Bayes Classifier (NBC) and Support Vector Machine (SVM) algorithm by the stages of the Cross-Industry Standard Process for Data Mining (CRISP-DM). The CRISP-DM framework consists of six stages, namely: business understanding stage, data understanding stage, data preparation stage, modeling stage, evaluation stage, and deployment stage. The SERVQUAL framework consists of several dimensions: reliability dimension; responsiveness; assurance; empathy; tangibles. The review data that will be processed is the consumer review data of The Oberoi Beach Resort Lombok; Sheraton Senggigi Beach Resort; Sudamala Resort Sengiggi; Holiday Resort Lombok; Aston Sunset Beach Resort. The results of this study show that the SVM algorithm performs better than NBC, where the accuracy value is 98.57%, the precision value is 100%, the recall value is 97.14%, and the f-measure value is 98.54%. The AUC value is 100%, and the t-Test value is 98.6%. Unlike the case with the results of SVM's algorithm performance evaluation without using the SMOTE Upgrading Operator, where the accuracy value is 95.71%, the precision value is 95.71%, the recall value is 100%, and the f-measure value is 97.81%. In addition, the AUC value is 91.1%, and the t-Test value is 95.7%.

Human intelligence-based metaverse for co-learning of students and smart machines.

This paper proposes a Human Intelligence (HI)-based Computational Intelligence (CI) and Artificial Intelligence (AI) Fuzzy Markup Language (CI&AI-FML) Metaverse as an educational environment for co-learning of students and machines. The HI-based CI&AI-FML Metaverse is based on the spirit of the Heart Sutra that equips the environment with teaching principles and cognitive intelligence of ancient words of wisdom. There are four stages of the Metaverse: preparation and collection of learning data, data preprocessing, data analysis, and data evaluation. During the data preparation stage, the domain experts construct a learning dictionary with fuzzy concept sets describing different terms and concepts related to the course domains. Then, the students and teachers use the developed CI&AI-FML learning tools to interact with machines and learn together. Once the teachers prepare relevant material, students provide their inputs/texts representing their levels of understanding of the learned concepts. A Natural Language Processing (NLP) tool, Chinese Knowledge Information Processing (CKIP), is used to process data/text generated by students. A focus is put on speech tagging, word sense disambiguation, and named entity recognition. Following that, the quantitative and qualitative data analysis is performed. Finally, the students' learning progress, measured using progress metrics, is evaluated and analyzed. The experimental results reveal that the proposed HI-based CI&AI-FML Metaverse can foster students' motivation to learn and improve their performance. It has been shown in the case of young students studying Software Engineering and learning English.

Coupled VEM–BEM Approach for Isotropic Damage Modelling in Composite Materials

Numerical prediction of composite damage behaviour at the microscopic level is still a challenging engineering issue for the analysis and design of modern materials. In this work, we document the application of a recently developed numerical technique based on the coupling between the virtual element method (VEM) and the boundary element method (BEM) within the framework of continuum damage mechanics (CDM) to model the in-plane damage evolution characteristics of composite materials. BEM is a widely adopted and efficient numerical technique that reduces the problem dimensionality due to its underlying formulation. It substantially simplifies the pre-processing stage and decreases the computational effort without affecting the solution’s accuracy. VEM is a recent generalization to general polygonal mesh elements of the finite element method that ensures noticeable simplification in the data preparation stage of the analysis, notably for computational micro-mechanics problems, whose analysis domain often features complex geometries. The numerical technique has been applied to artificial microstructures, representing the transverse section of composite material with stiffer circular-shaped inclusions embedded in a softer matrix. BEM is used to model the inclusions that are supposed to behave within the linear elastic range, while VEM is used to model the surrounding matrix material, developing nonlinear behaviours. Numerical results are reported and discussed to validate the proposed method.

Automatic Voice Disorder Detection Using Self-Supervised Representations

Many speech features and models, including Deep Neural Networks (DNN), are used for classification tasks between healthy and pathological speech with the Saarbruecken Voice Database (SVD). However, accuracy values of 80.71% for phrases or 82.8% for vowels /aiu/ are the highest reported for audio samples in SVD when the evaluation includes the wide amount of pathologies in the database, instead of a selection of some pathologies. This paper targets this top performance in the state-of-the-art Automatic Voice Disorder Detection (AVDD) systems. In the framework of a DNN-based AVDD system we study the capability of Self-Supervised (SS) representation learning for describing discriminative cues between healthy and pathological speech. The system processes the SS temporal sequence of features with a single feed-forward layer and Class-Token (CT) Transformer for obtaining the classification between healthy and pathological speech. Furthermore, there is evaluated a suitable data extension of the training set with out-of-domain data is also evaluated to deal with the low availability of data for using DNN-based models in voice pathology detection. Experimental results using audio samples corresponding to phrases in the SVD dataset, including all pathologies available, show classification accuracy values until 93.36%. This means that the proposed AVDD system achieved accuracy improvements of 4.1% without the training data extension, and 15.62% after the training data extension compared to the baseline system. Beyond the novelty of using SS representations for AVDD, the fact of obtaining accuracies over 90% in these conditions and using the whole set of pathologies in the SVD is a milestone for voice disorder-related research. Furthermore, the study on the amount of in-domain data in the training set related to the system performance show guidance for the data preparation stage. Lessons learned in this work suggest guidelines for taking advantage of DNN, to boost the performance in developing automatic systems for diagnosis, treatment, and monitoring of voice pathologies.

Klasifikasi Mutu Pembelajaran Hybrid berdasarkan Algoritma C.45, Random Forest dan Naïve Bayes dengan Optimasi Bootsrap Areggating (Bagging) pada masa COVID-19

This study aims to find the best model from the 3 (three) models chosen, namely Random Forest, Naïve Bayes and C4.5 against the classification of hybrid learning quality held at SMK Cendekia Cirebon City during the COVID-19 period. The research method is carried out by referring to the approach or technique of machine learning with the stages of data collection, data preparation, data transformation, modeling and evaluation. The final results of this study are measured by the level of accuracy of hybrid learning quality aspects, namely teachers, subject matter, integrity, student motivation, and technology resources with bad, sufficient, good and very good categories.. Based on the experiments that have been carried out, the accuracy results for the Random Forest algorithm were obtained by 93.92%, Naïve Bayes by 93.22% and C4.5 by 77.16%. Thus this study recommends the best classification algorithm model based on dataset processing to measure this hybrid learning quality model is Random Forest

A Method to Classify Steel Plate Faults Based on Ensemble Learning

With the industrial revolution 4.0, machine learning methods are widely used in all aspects of manufacturing to perform quality prediction, fault diagnosis, or maintenance. In the steel industry, it is important to precisely detect faults/defects in order to produce high-quality steel plates. However, determining the exact first-principal model between process parameters and mechanical properties is a challenging process. In addition, steel plate defects are detected through manual, costly, and less productive offline inspection in the traditional manufacturing process of steel. Therefore, it is a great necessity to enable the automatic detection of steel plate faults. To this end, this study explores the capabilities of the following three machine learning models Adaboost, Bagging, and Random Forest in detecting steel plate faults. The well-known steel plate failure dataset provided by Communication Sciences Research Centre Semeion was used in this study. The aim of many studies using this dataset is to correctly classify defects in steel plates using traditional machine learning models, ignoring the applicability of the developed models to real-world problems. Manufacturing is a dynamic process with constant adjustments and improvements. For this reason, it is necessary to establish a learning process that determines the best model based on the arrival of new information. Contrary to previous studies on the steel plate failure dataset, this article presents a systematic modelling approach that includes the normalization step in the data preparation stage to reduce the effects of outliers, the feature selection step in the dimension reduction stage to develop a machine learning model with fewer inputs, and hyperparameter optimization step in the model development stage to increase the accuracy of the machine learning model. The performances of the developed machine learning models were compared according to statistical metrics in terms of precision, recall, sensitivity, and accuracy. The results revealed that AdaBoost performed well on this dataset, achieving accuracy scores of 93.15% and 91.90% for the training and test datasets, respectively.

Introducing Two Parsimonious Standard Power Mixture Models for Bimodal Proportional Data with Application to Loss Given Default

The need to model proportional data is common in a range of disciplines however, due to its bimodal nature, U- or J-shaped data present a particular challenge. In this study, two parsimonious mixture models are proposed to accurately characterise this proportional U- and J-shaped data. The proposed models are applied to loss given default data, an application area where specific importance is attached to the accuracy with which the mean is estimated, due to its linear relationship with a bank’s regulatory capital. In addition to using standard information criteria, the degree to which bias reduction in the estimation of the distributional mean can be achieved is used as a measure of model performance. The proposed models outperform the benchmark model with reference to the information criteria and yield a reduction in the distance between the empirical and distributional means. Given the special characteristics of the dataset, where a high proportion of observations are close to zero, a methodology for choosing a rounding threshold in an objective manner is developed as part of the data preparation stage. It is shown how the application of this rounding threshold can reduce bias in moment estimation regardless of the model choice.

Analysis of UMAP, the method for reducing the dimensionality of initial data in machine learning for the purpose of failure prediction in a motive power service

Aim.Feature transformation is one of the stages of machine learning applicationthat has a significant effect on the quality of regression models. The paper aims to developcriteria for evaluating the quality of data dimensionality reduction at the stage of featuretransformation and adaptation of the UMAP method to the problem of prediction of the numberof days to failure in the locomotives of JSC RZD.Methods.The data transformation methodsare divided into two groups, those that attempt to preserve the global data structure, andthose that attempt to preserve the distances between points. The paper examines in detail theUMAP no-linear method of dimensionality reduction, whose low-dimensional data presentationis based on a transformation of a nearest neighbour graph retaining the data structure. Thestructure of the initial data manifold is examined using topological data analysis and simplifiedfuzzy set construction methods.Results.The analysis of UMAP theory conducted in theRussian language for the first time enabled a substantiated identification of the three primaryparameters of the method, whose variation significantly affects the type of data obtained asthe result of a transformation. In particular, that pertains to the quality of class separationover a two-dimensional space. Additionally, the characteristics of the input set of parameterswere identified that affect the UMAP results. Practical results of UMAP application weredemonstrated. Intermediate results included a list of nearest neighbours, a weighted graph of nearest neighbours. The fundamental result is a low-dimensional data representation (outof 44 initial measurements) over a two-dimensional space with class separation, which isconfirmed both by calculations, and visually.Conclusions.It was identified that UMAP is anefficient and substantiated method of dimensionality reduction that allows – through parametervariation – transforming data in such a way as to improve the quality of data submitted tomachine learning models by the criterion of “evident class separation”. The transformation is anintermediate stage of data preparation for regression model application, and class separationwas performed for the purpose of eliminating the probability of gross regression errors.

Fuzzy c-Means Clustering untuk Pengenalan Pola Studi kasus Data Saham

Fuzzy Clustering is one of the five roles used by data mining experts to transform large amounts of data into useful information, and one method that is often and widely used is Fuzzy c-Means (FCM) Clustering. FCM is a data clustering technique where the existence of each data point in the cluster is based on the degree of membership. This study aims to see the pattern of data samples or data categories using FCM clustering. The analyzed data is stock data on Jakarta Stock Exchange (BEJ) in the Property and Real Estate sector (issuer group). The data mining processes comply Cross Industry Standard Process Model for Data mining Process (Crisp-DM), with several stages, starting with the stage of getting to know the business process (Business Understanding) then studying the data (Data Understanding), continuing with the Data Preparation stage, Modeling stage, Evaluation stage and finally the Deployment stage. In the modeling stage, the FCM model is used. FCM clustering model data mining can analyze data in large databases with many variables and complicated, especially to get patterns from the data. Then a Fuzzy Inference System (FIS) was built based on a known pattern for simulating input data into output data based on fuzzy logic. Keywords: Fuzzy c-Means Clustering, Pattern Recognition

PEMEROLEHAN FONOLOGI BAHASA PERTAMA MADURA PADA ANAK USIA TIGA TAHUN: KAJIAN PSIKOLINGISTIK

This study aims to describe the acquisition of the first language of children aged three years based on phonological aspects including, the acquisition of vowels, acquisition of consonants, and factors that influence the acquisition of phonology in children. This research uses a descriptive qualitative approach. The research data is in the form of children's speech. The data source was obtained from a three-year-old child named AMS who grew up in an environment with Madurese first language. At the age of three years, AMS is going through a golden period or a critical period. During this golden period, children begin to be able to carry out the language acquisition process. Data collection was carried out using observation and interview methods. After that, the data obtained were then analyzed through three stages, namely the data reduction stage, the data preparation stage, and the conclusion drawing stage. The findings of the study indicate that at the age of three, children are already fluent in the Madurese language, but there is little influence of Indonesian on the child's Madurese language. At this age, children get a phonemic inventory of the Madurese language, including Indonesian. The child has mastered the vowels [a], [i], [u], [e], and [o] along with their allophones. In addition, children also master several consonant sounds, including [b], [p], [m], [n], [w], [j], [k], [g], [p], [ l], [ŋ], [ɲ], [h], [tʃ],, [ʃ] and [ʤ]. Meanwhile, the sounds [r], [s], and [f] cannot be pronounced properly even though they are located at the beginning, in the middle, or at the end of the word. Based on the results of observations, the limitations of children in pronouncing some of these consonants are caused by the child's articulator factor which has not grown perfectly. Even though the researchers provoked consonants that had not yet appeared in the child's pronunciation, the child was still not able to pronounce the sound of the consonant phoneme. In this regard, it can be seen that the phonological acquisition of a child's first language is not determined by the year, but is strongly influenced by neurobiological factors. Therefore, it is not impossible if the acquisition of the sounds of children of the same age in the same year can be different in number, because what is universal is not the year, but the appearance of these sounds. Another factor that affects the acquisition of AMS phonology is the stimulus provided by the family and the environment around the child. Based on some of the data found, certain sounds are the result of imitation. Therefore, the acquisition of a child's first language is also greatly influenced by the environment around the child.

Data Preparation and Visualization of Electricity Consumption for Load Profiling

The construction of daily electricity consumption profiles is a common practice for user characterization and segmentation tasks. As in any data analysis project, to obtain these load profiles, a stage of data preparation is necessary. This article explores to what extent does the selection of the data preparation technique impacts load profiling. The techniques discussed are used in the following tasks: standardization, construction of data, dimensionality reduction and data enrichment. The analysis reveals a great incidence of the data preparation on the result. The need to make the data preparation process explicit in each report is identified. In particular, it is highlighted that the most usual default standardization process, column standardization, is not adequate in the preparation of energy consumption profiles.

The Development of Audio Visual Learning Media in Improving Children’s Religious and Moral Values

Morality and religion are the main fortresses for the formation of children's character in responding to the flow of globalization in the digital era. The form of concrete efforts is carried out through a quality learning process for children. This study aims to develop audio-visual learning media in improving the religious and moral values of early childhood. The author uses the type of research R&D (Research and Development). The use of this research method is intended to produce new products in learning and to test the effectiveness of these products. In this context, the product that will be produced is in the form of media in the form of the film "Aku Anak Salih" to improve religious and moral values for early childhood with the Borg and Gall model. The procedural stages include; data collection and preparation stage, product development stage, product validity stage, revision stage, product trial stage, dissimilation and implementation. The design of the media for the film "Aku Anak Salih" was carried out through determining the theme, making a logline, developing a film synopsis, making storyboards, and implementing the production system. The result is that the media is considered effective and valid by experts to be applied to students of RA Nurul Aflah, Medan Labuhan District. These media have advantages in terms of convenience that are presented to increase moral and religious values in early childhood, attract children's learning attention, and are in accordance with audio-visual learning styles. Likewise, there are weaknesses from this media in the form of a relatively long time in designing the media, demanding teacher creativity in designing technology-based media, and demanding adequate facilities in schools (projectors, sound systems, classrooms).

Principle, Methodology and Application for Data Cleaning techniques

Con-temporarily, information data has become the cornerstone of every company’s decision-making. In a vast flow of information, choosing the right data is the first step in developing successful predictions. After the determinations of the requirements, analysis purpose and prediction direction, outlier processing, missing value processing and repeated value processing are usually encountered. This paper introduces the limitations, advantages and disadvantages of different methods in application in detail. At the same time, this paper introduces some interpolation methods based on mathematical statistics, such as thermal interpolation, Lagrange interpolation and Newton interpolation. At the same time, it also provides the normal distribution processing method which is better in dealing with outlier problems, and the popular K-nearest neighbor algorithm. Finally, it illustrates the logic diagram of data cleaning in the data preparation stage. Overall, these results offer a guideline for selecting the appropriate treatment in the corresponding situation during data cleaning process.

A multiple frequency bands parallel spatial–temporal 3D deep residual learning framework for EEG-based emotion recognition

Electroencephalography (EEG) based emotion recognition has become a hot research issue in the field of cognitive interaction and brain-computer interface (BCI). How to build a deep learning model which can fully learn frequency-spatial–temporal representation from complex emotional EEG data and has good neurological interpretability is still challenging. In this paper, a novel multiple frequency bands parallel spatial–temporal 3D deep residual learning framework (MFBPST-3D-DRLF) is proposed for EEG-based emotion recognition. Firstly, a new optimal frequency bands selection method based on group sparse regression is designed for characteristic analysis on frequency domain. Secondly, spatial–temporal 3D feature representations of multiple frequency bands are generated in the data preparation stage for fully expressing the discriminative local patterns among brain responses of different emotional states. Finally, a novel parallel 3D deep residual networks architecture is elaborately constructed to simultaneously extract high level abstract features and achieve accurate classification. Emotional EEG recognition performance of the proposed method has been evaluated on two benchmark datasets, namely SEED and SEED-IV. The proposed MFBPST-3D-DRLF achieves 96.67% and 88.21% on both datasets, outperforming several state-of-the-art algorithms. In addition, investigations on the intermediate results and model parameters reveal that neural signatures associated with different emotional states are traceable and gamma band is most suitable for EEG based emotion recognition.

Algoritma Random Forest Untuk Prediksi Kelangsungan Hidup Pasien Gagal Jantung Menggunakan Seleksi Fitur Bestfirst

Heart failure is a global health problem that not only causes physical problems, other impacts such as psychological, social, and economic, as well as depression, which affects treatment, worsens functional status, and increases hospitalization rates to death. According to the World Health Organization (WHO), nearly 17.5 million people die from cardiovascular disease, which represents 31% of deaths in the world. Using machine learning to predict the survival of patients with heart failure so that they can take precautions from the start. The stages of the research carried out include the business understanding stage, the data understanding stage, the data preparation stage, the modeling stage, and the evaluation stage. In this study, using feature selection using best-first resulted in 4 very influential features, namely age, injection_fraction, serum_creatinene and time, and handling imbalance class using the class balancer model. Random forest algorithm with 80% percentage split method which produces 91.45% accuracy, mean absolute error 0.1874, incorrectly classified instances 8.55%, precision 0.915, recall 0.914, AUC 0.953.

Towards automation of in-season crop type mapping using spatiotemporal crop information and remote sensing data

CONTEXTMapping crop types from satellite images is a promising application in agricultural systems. However, it is a challenge to automate in-season crop type mapping over a large area because of the insufficiency of ground truth and issues of scalability, reusability, and accessibility of the classification model. This study introduces a framework for automatic crop type mapping using spatiotemporal crop information and Sentinel-2 data based on Google Earth Engine (GEE). The main advantage of the framework is using the trusted pixels extracted from the historical Cropland Data Layer (CDL) to replace ground truth and label training samples in satellite images. OBJECTIVEThis paper will achieve three objectives: (1) assessing spatiotemporal crop information derived from the historical crop cover maps; (2) mapping crop cover, mainly crop fields without regular historical crop rotation patterns, from remote sensing data using supervised learning classification and validating mapping results; and (3) automating in-season crop mapping and exploring the scalability of the framework. METHODSThe proposed crop mapping workflow consists of four stages. The data preparation stage preprocesses CDL and Sentinel-2 data into the required structure. The spatiotemporal crop information sampling stage extracts trusted pixels from the historical CDL time series and labels Sentinel-2 data. Then a crop type classification model can be trained using the supervised learning classifier in the model training stage. In the mapping/validation stage, an in-season crop cover map over the full Sentinel-2 tile will be produced using the trained model and the classification performance will be validated using CDL or other ground truth data. RESULTS AND CONCLUSIONSWe systematically perform a group of experiments for in-season mapping of five major crop types (corn, cotton, rice, soybeans, and soybeans-wheat double cropping) over the Mississippi Delta region. The result indicates that the crop cover map of the study area is expected to reach 80%–90% agreement with CDL within the growing season. To further facilitate the use of the framework, we also develop a GEE-enabled online prototype, In-season Crop Mapping Kit, and explore its scalability over agricultural fields in various ecoregions including California, Idaho, Kansas, and Illinois. SIGNIFICANCEThe mapping-without-ground-truth approach described in this paper can significantly reduce ground truthing process and save substantial resource needs and labor costs, which is applicable to the production of in-season CDL-like data for the entire United States. The findings and outputs will benefit the agriculture community and other agricultural sectors ranging from government, academia, and companies.

Hyperspectral anomaly detection based on adaptive weighting method combined with autoencoder and convolutional neural network

ABSTRACT Deep learning has achieved excellent performance in the field of computer vision and gained attention in the field of hyperspectral anomaly detection (HAD). Convolutional neural networks (CNN) based on pixel pairing features have achieved a certain detection effect, which is limited by background complexity in practical applications. This study proposes an adaptive weighted HAD algorithm that combines an autoencoder (AE) and CNN. Labelled hyperspectral images (HSIs) are used to train a CNN as a binary classifier for similarity discrimination in the data preparation stage, where morphological attribute filtering (MAF) is performed on images in the spatial dimension of the HSIs to highlight anomaly. The spectral angle is used as a measure of reconstruction error to train the AE, and the reconstruction error obtained using the AE is used as an adaptive weight to calculate the anomaly score, which considerably eliminates the blurring of the boundary between the background and anomalies. Through comparisons with some state-of-the-art methods, the effectiveness of the proposed method in improving detection accuracy and increasing background and anomaly discrimination is verified on three real datasets.

Statistical analysis of the results of mechanical tests of metal and pipes of trunk pipelines

Modern approaches to processing large bulk of data accumulated as a result of mechanical testing, based on statistical analysis and machine learning methodology are discussed. The factors affecting the strength and durability of pipe products are considered. The stages of the initial data preparation are analyzed and the main steps of basic statistical processing, correlation and variance analyses, approaches to grouping and ranking of test data are discussed. The results of the developed approaches are demonstrated by the example of constructing a classification by steel grades and technical conditions of the pipe production according to the actual mechanical characteristics of the pipe metal. The real effect is shown on the example of the estimating the number of pipe defects that require priority to repair. The presented algorithms can be used as the elements of the formed information base and data banks in the field of pipeline transportation of oil and oil products. The developed approaches require a comprehensive testing and further development towards taking into account the actual loading of the pipe and assessing the failure probability.