Lazy Learning Approach Research Articles

Data-driven prediction of possible quality deterioration in injection molding processes

We propose an approach for the automated prediction of possible quality deteriorations at injection molding machines using data-driven models. This approach relies on data solely recorded during the regular production phase without the need to (i.) collect data from anomalous phases, which may be cost-intensive due to production waste, (ii.) perform time-intensive (manual) annotation cycles for assigning class labels, and (iii.) know specific fault modes in advance. Our approach embeds two main concepts: the first is the establishment of causal relations between process variables, which serve as online residual generators. The multi-variate residual signals are analyzed using an advanced independent component analysis, whose reconstruction characteristics serve as a control signal indicating a potential anomaly of any type/mode, upon violation of the internal dependence structure. No parameters are needed for the online analysis, which in turn avoids the use of (pre-collected) data from anomalous phases, which is typically required to tune the parameters properly. The second concept performs a direct prediction of quality criteria, which are permanently by-measured through the usage of an optical inspection system of the produced/molded parts. Time series-based trends of the process data are used to establish quality prediction models based on a specific time series transformation technique. It preserves the local structure of the data to ensure it can be effectively combined with predictive fuzzy systems training. To account for system dynamics, the prior trained prediction models can be automatically updated on the fly with new measurements. To further increase flexibility in the case of higher dynamics, as was the case in our application, a dynamic lazy learning (dLL) approach was developed in combination with partial least squares. This technique is able to permanently update the reference data base and establish a local model per new query sample with reduced input dimensionality. Our approach was successfully evaluated during the real injection molding processes of bottle caps: several real-occurring anomalies and significant changes in material and production parameters were successfully found over a horizon of 7162 production cycles. Furthermore, three essential quality criteria were predicted with a correlation of 0.7, 0.89 and 0.98 between predicted and observed trend lines, with mean absolute error rates below 5%. This makes the detection of quality deteriorations possible at an early stage.

Automatic Recognition of Beam Attachment for Massive MIMO System in Densely Distributed Renewable Energy Resources

Several large-scale and distributed systems such as renewable energy systems (RESs) require ubiquitous and reliable communication. RESs are designed to provide efficient power management and improve both energy production and consumption. Decision making in RESs heavily depends on real-time communication. Fifth and sixth-generation (5G, 6G) wireless networks promise to deliver significant communication advantages to RESs including ultra-low latency, high throughput and improved coverage. However, the communication behavior in RESs is categorized as unpredictable due to aspects such as system flexibility and equipment heterogeneity. This may affect the stability of the entire RES, which results in further issues such as signal reliability and degraded coverage. Therefore, precise identification of user equipment’s (UE) location greatly improves the sustainability of 5G and 6G wireless services. In this work, we propose a novel low-complexity technique to automatically recognize UE locations in an area of interest. The approach aims at providing precise identification of UE with minimum memory and feature requirements. We use the lazy learning approach to build a prediction model to construct beam-attachment maps. We then train the model to provide distributed intelligent models to automatically recognize beam-attachment indexes. We compare the proposed approach with instance-based techniques to measure its ability at predicting beam-attachment maps. The results show that the proposed model has the ability to provide an accurate prediction with respect to the beam-attachment index (around 100%) with minimal complexity.

Improving Large-Scale k-Nearest Neighbor Text Categorization with Label Autoencoders

In this paper, we introduce a multi-label lazy learning approach to deal with automatic semantic indexing in large document collections in the presence of complex and structured label vocabularies with high inter-label correlation. The proposed method is an evolution of the traditional k-Nearest Neighbors algorithm which uses a large autoencoder trained to map the large label space to a reduced size latent space and to regenerate the predicted labels from this latent space. We have evaluated our proposal in a large portion of the MEDLINE biomedical document collection which uses the Medical Subject Headings (MeSH) thesaurus as a controlled vocabulary. In our experiments we propose and evaluate several document representation approaches and different label autoencoder configurations.

Fast and Data Efficient Reinforcement Learning from Pixels via Non-parametric Value Approximation

We present Nonparametric Approximation of Inter-Trace returns (NAIT), a Reinforcement Learning algorithm for discrete action, pixel-based environments that is both highly sample and computation efficient. NAIT is a lazy-learning approach with an update that is equivalent to episodic Monte-Carlo on episode completion, but that allows the stable incorporation of rewards while an episode is ongoing. We make use of a fixed domain-agnostic representation, simple distance based exploration and a proximity graph-based lookup to facilitate extremely fast execution. We empirically evaluate NAIT on both the 26 and 57 game variants of ATARI100k where, despite its simplicity, it achieves competitive performance in the online setting with greater than 100x speedup in wall-time.

Classifications of Skull Fractures using CT Scan Images via CNN with Lazy Learning Approach

Classification of skull fracture is a challenging task for both radiologists and researchers. Skull fractures result in broken pieces of bone, which can cut into the brain and cause bleeding and other injury types. So it is vital to detect and classify the fracture very early. In real world, often fractures occur at multiple sites. This makes it harder to detect the fracture type where many fracture types might summarize a skull fracture. Unfortunately, manual detection of skull fracture and the classification process is time-consuming, threatening a patient's life. Because of the emergence of deep learning, this process could be automated. Convolutional Neural Networks (CNNs) are the most widely used deep learning models for image categorization because they deliver high accuracy and outstanding outcomes compared to other models. We propose a new model called SkullNetV1 comprising a novel CNN by taking advantage of CNN for feature extraction and lazy learning approach which acts as a classifier for classification of skull fractures from brain CT images to classify five fracture types. Our suggested model achieved a subset accuracy of 88%, an F1 score of 93%, the Area Under the Curve (AUC) of 0.89 to 0.98, a Hamming score of 92% and a Hamming loss of 0.04 for this seven-class multi-labeled classification.

Multi-class object detection in tunnels from 3D point clouds: An auto-optimized lazy learning approach

This research develops an auto-optimized lazy learning approach named BOKNN (Bayesian optimized K nearest neighbor) method to detect seepage and multi-classify various objects (e.g., segment, pipe, track, support, and cable) in operating tunnels from 3D point clouds. Firstly, the 3D laser scanning is employed to acquire raw point cloud data, and the equidistant pooling for down-sampling is conducted to improve class imbalance issues and enhance the efficiency. Then, the K-nearest neighbor (KNN) model is built on the trimmed dataset, where the Bayesian optimization is performed to obtain the optimal combination of hyper-parameters in the KNN model. A realistic cross-river tunnel section in China is used as a case study to demonstrate the applicability and effectiveness of the developed approach. Results indicate that (1) The established BOKNN model displays a high performance in multi-class detection, together with a total accuracy of 0.935, a macro F1 score of 0.896, and a weighted F1 score of 0.939. (2) It performs well even in minor class detection, and the detection of seepage is conservative, where only 4.1% of seepage points are misclassified as non-seepage points. (3) It displays better detection performance than the other representative machine learning models (i.e, Adaboost, Support Vector Machine, and Naive Bayes). The developed approach is nonparametric and training-free, which can be used as a decision tool to substitute the present manual detection and improve the detection efficiency.

Machine vision based fault diagnosis of photovoltaic modules using lazy learning approach

Machine Vision is an advanced and powerful imaging based technique that has been applied in various fields like robotics, inspection and process control. Machine vision, in industrial terms, is termed as a subcategory of computer vision. The primary aim of the proposed study is to distinguish various visual fault conditions that hinder the performance and life span of photovoltaic (PV) modules using computer vision and a machine learning approach. Literatures state that thermographic and electroluminescence images were used in deep learning for identifying faults in photovoltaic modules. However, the effectiveness of normal RGB images with the fusion of deep learning and machine learning techniques is less explored. This paper deals with the classification of normal RGB images of PV modules acquired using a fusion of deep learning and machine learning techniques. Visual faults like delamination, snail trail, burn marks, glass breakage and discoloration that occur in a photovoltaic module (PVM) were considered in the study. A machine learning approach was used to handle this problem which contains three stages: (i) feature extraction, (ii) feature selection and (iii) feature classification. Initially, the features from the aerial images of PVM (acquired from unmanned aerial vehicles (UAVs) equipped with digital cameras) were extracted using convolutional neural networks (CNN). Secondly, J48 decision tree algorithm was employed to select the features of utmost significance and dominance from the extracted image features. Finally, a set of lazy classifiers like locally weighted learning (LWL), K-star algorithm (KS), nearest neighbor (NN) and k-nearest neighbor (kNN) were adopted to execute the classification task on the selected image features. The classification accuracies of all the aforementioned classifiers were compared and it was found that the k-nearest neighbor classifier achieved a maximum accuracy of 98.95% with a lesser computational time of 0.04 s.

Monolingual and Cross-Lingual Intent Detection without Training Data in Target Languages

Due to recent DNN advancements, many NLP problems can be effectively solved using transformer-based models and supervised data. Unfortunately, such data is not available in some languages. This research is based on assumptions that (1) training data can be obtained by the machine translating it from another language; (2) there are cross-lingual solutions that work without the training data in the target language. Consequently, in this research, we use the English dataset and solve the intent detection problem for five target languages (German, French, Lithuanian, Latvian, and Portuguese). When seeking the most accurate solutions, we investigate BERT-based word and sentence transformers together with eager learning classifiers (CNN, BERT fine-tuning, FFNN) and lazy learning approach (Cosine similarity as the memory-based method). We offer and evaluate several strategies to overcome the data scarcity problem with machine translation, cross-lingual models, and a combination of the previous two. The experimental investigation revealed the robustness of sentence transformers under various cross-lingual conditions. The accuracy equal to ~0.842 is achieved with the English dataset with completely monolingual models is considered our top-line. However, cross-lingual approaches demonstrate similar accuracy levels reaching ~0.831, ~0.829, ~0.853, ~0.831, and ~0.813 on German, French, Lithuanian, Latvian, and Portuguese languages.

Decision branch joint venture ex-fold T-z re-validation

Comparing classifier performances may seem a banal affair but makes a side show in machine learning. Usually the paired t-test is used. It requires that two classifiers were run simultaneously or this was simulated. This is not always possible and then entails creating a superstructure only for that purpose. However, the utility of t-test in the given context is altogether doubted. The literature on alternatives is much involved. This does not measure up to the scale of the issue. In this paper the topics in connection with accuracy calculation are surveyed once more, emphasizing the result variation. The known technique of multifold cross-validation is exemplified. A simplified methodology for comparison of classifier performances is proposed. It is based on the accuracy mean and variance and calculating differences between objects defined in these terms. It is being applied to the naive Bayesian and decision tree classifiers implemented on different platforms. The lazy learning approach, applicable to decision trees in discrete domains, is closely followed with an imposition of how it can be improved. Examples are given from the field of health diagnostics.

Autonomic Workload Performance Modeling for Large-Scale Databases and Data Warehouses Through Deep Belief Network With Data Augmentation Using Conditional Generative Adversarial Networks

Databases and warehouses are experiencing workload of different types such as Decision Support System (DSS), Online Transaction Processing (OLTP) and Mixed workloads. Handling variety of workload in autonomic systems is a critical task. After self-configuring the workload, the next challenge is workload performance tuning that motives towards self-predictive systems. Existing studies provide performance modeling solutions on small-scale data repositories of Database Management System (DBMS) and Data Warehouse (DWH) using either classical eager or lazy learning approaches. However, in real-world problems, we normally have to deal with large-scale data repositories. Therefore, there is a need to develop performance models that provide data augmentation to solve large-scale data repositories that are not publicly available. In this study, deep learning approaches have been investigated for performance tuning of large-scale data repositories. We propose a performance prediction model called Optimized GAN-based Deep Learning (OGDL) model. For data augmentation, Conditional Generative Adversarial Networks (CGAN) is applied. For autonomic perspective, we incorporated MAPE-K model to manage the workload autonomically. Different deep learning models are applied, and it was observed that Deep Belief Network (DBN) performed better as compared to other deep learning models such as Deep Neural Network (DNN). We performed a number of experiments and from results it is observed that deep learning models performed the best in comparison with classical machine learning and lazy learning and a $6-8\%$ increase in accuracy is recorded in our experiments using DBN. The proposed OGDL model performed the best in workload performance predictions in an optimized way.

A new Feature Reduction Algorithm Based on Fuzzy Rough Relation for the Multi-label Classification

The paper aims to improve the multi-label classification performance using the feature reduction technique. According to the determination of the dependency among features based on fuzzy rough relation, features with the highest dependency score will be retained in the reduction set. The set is subsequently applied to enhance the performance of the multi-label classifier. We investigate the effectiveness of the proposed model againts the baseline via time complexity.
 Keywords: 
 Fuzzy rough relation, label-specific feature, feature reduction set
 References
 [1] Richard Jensen, Chris Cornelis, Fuzzy-Rough Nearest Neighbor Classification and Prediction. Proceedings of the 6th International Conference on Rough Sets and Current Trends in Computing, 2011, 310-319.
 [2] Y.H. Qian, Q. Wang, H.H. Cheng, J.Y. Liang, C.Y. Dang, Fuzzy-Rough feature selection accelerator, Fuzzy Sets Syst. 258 (2014) 61-78.
 [3] Quang-Thuy Ha, Thi-Ngan Pham, Van-Quang Nguyen, Minh-Chau Nguyen, Thanh-Huyen Pham, Tri-Thanh Nguyen, A New Text Semi-supervised Multi-label Learning Model Based on Using the Label-Feature Relations, International Conference on Computational Collective Intelligence, LNAI 11055, Springer, 2018, pp. 403-413.
 [4] Daniel Kostrzewa, Robert Brzeski, The data Dimensionality Reduction and Feature Weighting in the Classification Process Using Forest Optimization Algorithm, ACIIDS, 2019, pp. 97-108.
 [5] Nele Verbiest, Fuzzy Rough and Evolutionary Approaches to Instance Selection, PhD Thesis, Ghent University, 2014.
 [6] Y. Yu, W. Pedrycz, D.Q. Miao, Multi-label classification by exploiting label correlations, Expert syst, Appl. 41 (2014) 2989-3004.
 [7] M.L. Zhang, LIFT: Multi-label learning with label-specific features, IEEE Trans, Pattern Anal, Mach, Intell 37 (2015) 107-120.
 [8] Suping Xu, Xibei Yang, Hualong Yu, Dong-Jun Yu, Jingyu Yang, Eric CC Tsang, Multi-label learning with label-specific feature reduction, Knowledge-Based Systems 104 (2016) 52-61. https://doi.org/10.1080/24751839.2017.1364925.
 [9] Thi-Ngan Pham, Van-Quang Nguyen, Van-Hien Tran, Tri-Thanh Nguyen, Quang-Thuy Ha, A Semi-supervised multi-label classification framework with feature reduction and enrichment, Journal of Information and Telecommunication 1(4) (2017) 305-318.
 [10] M. Ghaemi, M.R. Feizi-Derakhshi, Feature selection using forest optimization algorithm, Pattern Recognition 60 (2016) 121-129.
 [11] M.L. Zhang, Z.H. Zhou, ML-KNN: A lazy learning approach to multi-label learning, Pattern Recognition 40 (2007) 2038-2048.
 [12] M.Z. Ahmad, M.K. Hasan, A New Approach for Computing Zadeh's Extension Principle, MATEMATIKA. 26(1) (2010) 71-81.
 [13] Richard Jensen, Neil Mac Parthaláin and Qiang Shen. Fuzzy-rough data mining (using the Weka data mining suite), A Tutorial, IEEE WCCI 2014, Beijing, China, July 6, 2014.
 [14] D. Dubois, H. Prade, Rough fuzzy sets and fuzzy rough sets, Int. J. Gen. Syst. 17 (1990) 191-209.

Autonomic performance prediction framework for data warehouse queries using lazy learning approach

Information is one of the most important assets of an organization. In recent years, the volume of data stored in organizations, varying user requirements, time constraints, and query management complexities have grown exponentially. Due to these problems, the performance modeling of queries in data warehouses (DWs) has assumed a key role in organizations. DWs make relevant information available to decision-makers; however, DW administration is becoming increasingly difficult and time-consuming. DW administrators spend too much time managing queries, which also affects data warehouse performance. To enhance the performance of overloaded data warehouses with varying queries, a prediction-based framework is required that forecasts the behavior of query performance metrics in a DW. In this study, we propose a cluster-based autonomic performance prediction framework using a case-based reasoning approach that determines the performance metrics of the data warehouse in advance by incorporating autonomic computing characteristics. This prediction is helpful for query monitoring and management. For evaluation, we used metrics for precision, recall, accuracy, and relative error rate. The proposed approach is also compared with existing lazy learning techniques. We used the standard TPC-H dataset. Experiments show that our proposed approach produce better results compared to existing techniques.

A lazy learning approach for condition monitoring of wind turbine blade using vibration signals and histogram features

The main objective of the proposed research study is to discriminate different blade fault conditions which affect the wind turbine blades under operating condition through machine learning approach. A three bladed wind turbine was chosen and the faults like blade bend, blade cracks, blade erosion, hub-blade loose connection and pitch angle twist were considered in the study. This problem is formulated as a machine learning problem which consists of three phases, namely feature extraction, feature selection and feature classification. Histogram features were extracted from vibration signals and feature selection was carried out using J48 decision tree algorithm. Feature classification was performed using lazy classifiers like nearest neighbour, k-nearest neighbours, locally weighted learning and K-star classifier. The results of these classifiers were compared with respect to their correctly classified instances (accuracy percentage) and found that, locally weighted learning yielded a maximum accuracy of 93.83% with a computational time of 0.07 s.

Enhanced time-aware QoS prediction in multi-cloud: a hybrid k-medoids and lazy learning approach (QoPC)

Cloud service providers should be able to predict the future states of their infrastructure in order to avoid any violation of Service Level Agreement. This becomes more complex when vendors have to deal with services from various providers in multi-clouds. As a result, QoS prediction can significantly support service providers in a better understanding of their resources future states. Users should also be very well aware of their resource needs, as well as the Quality of Service relative values. This paper proposes a hybrid approach to the prediction of the future value of the QoS features. The hybrid approach uses a modified version of k-medoids algorithm for the clustering of large time-series datasets, as well as a proposed algorithm inspired from the lazy learning and lower bound Dynamic Time Warping (LB-Keogh) for pruned DTW computations. The proposed method in this manuscript is a shape-based QoS prediction with a novel pre-processing method, which fulfills the missing data with statistically semi-real data. In order to solve the cold start problem, we proposed new algorithm based on the DTW Barycenter Averaging (DBA) algorithm. The results showed that our predicted values are very close to real values and achieve only 0.35 of the normalized mean absolute error rate, on average, for the WSDream dataset and 0.07 for the Alibaba dataset.

A hybrid of whale optimization and late acceptance hill climbing based imputation to enhance classification performance in electronic health records.

Electronic health records (EHR) are a major source of information in biomedical informatics. Yet, missing values are prominent characteristics of EHR. Prediction on dataset with missing values results in inaccurate inferences. Nearest neighbour imputation based on lazy learning approach is a proven technique for missing data imputation and is recognized as one among the top ten data mining algorithms due to its simplicity and understandability. But its performance is deteriorated due to the curse of dimensionality as unimportant features are likely to dominate. We address this problem by proposing a novel approach for feature weighting based on a hybrid of metaheuristic whale optimization algorithm (WOA) and local search late acceptance hill climbing algorithm (LAHCA) on nearest neighbour imputation method. Our proposed approach Metaheuristic and Local Search based Feature Weighted Nearest Neighbour Imputation (kNN+LAHCAWOA) also learns different k values for different test points. Our approach is tested on benchmark EHR datasets with three proven classifiers Support Vector Machines(SVM), Random forest(RF) and Deep neural networks(DNN). The results prove that kNN+LAHCAWOA is an effective imputation strategy and aids in improving the classification performance when compared with its competitor methods.

A Semieager Classifier for Open Relation Extraction

A variety of open relation extraction systems have been developed in the last decade. And deep learning, especially with attention model, has gained much success in the task of relation classification. Nevertheless, there is, yet, no research reported on classifying open relation tuples to our knowledge. In this paper, we propose a novel semieager learning algorithm (SemiE) to tackle the problem of open relation classification. Different from the eager learning approaches (e.g., ANNs) and the lazy learning approaches (e.g., kNN), the SemiE offers the benefits of both categories of learning scheme, with its significantly lower computational cost (O(n)). This algorithm can also be employed in other classification tasks. Additionally, this paper presents an adapted attention model to transform relation phrases into vectors by using word embedding. The experimental results on two benchmark datasets show that our method outperforms the state-of-the-art methods in the task of open relation classification.

A Comparative Study for Condition Monitoring on Wind Turbine Blade using Vibration Signals through Statistical Features: a Lazy Learning Approach

This study is to identify whether the wind turbine blades are in good or faulty conditions. If faulty, then the objective to find which fault condition are the blades subjected to. The problem identification is carried out by machine learning approach using vibration signals through statistical features. In this study, a three bladed wind turbine was chosen and faults like blade cracks, hub-blade loose connection, blade bend, pitch angle twist and blade erosion were considered. Here, the study is carried out in three phases namely, feature extraction, feature selection and feature classification. In phase 1, the required statistical features are extracted from the vibration signals which obtained from the wind turbine through accelerometer. In phase 2, the most dominating or the relevant feature is selected from the extracted features using J48 decision tree algorithm. In phase 3, the selected features are classified using machine learning classifiers namely, K-star (KS), locally weighted learning (LWL), nearest neighbour (NN), k-nearest neighbours (kNN), instance based K-nearest using log and Gaussian weight kernels (IBKLG) and lazy Bayesian rules classifier (LBRC). The results were compared with respect to the classification accuracy and the computational time of the classifier.

Recognition of Multiple Power Quality Disturbances Using KNN-Bayesian

This paper presents a novel approach using multi-label classification to predict the classes of multiple disturbances for a power quality (PQ) event in power systems. A novel multi-label lazy learning approach named KNN-Bayesian is designed to recognize the multiple power quality disturbances based on k-nearest neighbor and Bayesian methods. In terms of the especial multi-label evaluation metrics, the proposed approach is tested on powerline signals that contain single and multiple disturbances. Experimental results indicate that KNN-Bayesian can effectively classify multiple PQ disturbances under different noise conditions

Lazy attribute selection: Choosing attributes at classification time

Attribute selection is a data preprocessing step which aims at identifying relevant attributes for the target machine learning task --namely classification in this paper. In this paper, we propose a new attribute selection strategy --based on a lazy learning approach --which postpones the identification of relevant attributes until an instance is submitted for classification. Our strategy relies on the hypothesis that taking into account the attribute values of an instance to be classified may contribute to identifying the best attributes for the correct classification of that particular instance. Experimental results using the k-NN and Naive Bayes classifiers, over 40 different data sets from the UCI Machine Learning Repository and five large data sets from the NIPS 2003 feature selection challenge, show the effectiveness of delaying attribute selection to classification time. The proposed lazy technique in most cases improves the accuracy of classification, when compared with the analogous attribute selection approach performed as a data preprocessing step. We also propose a metric to estimate when a specific data set can benefit from the lazy attribute selection approach.

Tag Tagging: Towards More Descriptive Keywords of Image Content

Tags have been demonstrated to be effective and efficient for organizing and searching social image content. However, these human-provided keywords are far from a comprehensive description of the image content, which limits their effectiveness in tag-based image search. In this paper, we propose an automatic scheme called tag tagging to supplement semantic image descriptions by associating a group of property tags with each existing tag. For example, an initial tag “tiger” may be further tagged with “white”, “stripes”, and “bottom-right” along three tag properties: color, texture, and location, respectively. In this way, the descriptive ability of the existing tags can be greatly enhanced. In the proposed scheme, a lazy learning approach is first applied to estimate the corresponding image regions of each initial tag, and then a set of property tags that correspond to six properties, including location, color, texture, size, shape, and dominance, are derived for each initial tag. These tag properties enable much more precise image search especially when certain tag properties are included in the query. The results of the empirical evaluation show that tag properties remarkably boost the performance of social image retrieval.