Inductive Learning Techniques Research Articles

Automatic characterization of human embryos at day 4 post-insemination from time-lapse imaging using supervised contrastive learning and inductive transfer learning techniques

Background: Embryo morphology is a predictive marker for implantation success and ultimately live births. Viability evaluation and quality grading are commonly used to select the embryo with the highest implantation potential. However, the traditional method of manual embryo assessment is time-consuming and highly susceptible to inter- and intra-observer variability. Automation of this process results in more objective and accurate predictions.Method: In this paper, we propose a novel methodology based on deep learning to automatically evaluate the morphological appearance of human embryos from time-lapse imaging. A supervised contrastive learning framework is implemented to predict embryo viability at day 4 and day 5, and an inductive transfer approach is applied to classify embryo quality at both times.Results: Results showed that both methods outperformed conventional approaches and improved state-of-the-art embryology results for an independent test set. The viability result achieved an accuracy of 0.8103 and 0.9330 and the quality results reached values of 0.7500 and 0.8001 for day 4 and day 5, respectively. Furthermore, qualitative results kept consistency with the clinical interpretation.Conclusions: The proposed methods are up to date with the artificial intelligence literature and have been proven to be promising. Furthermore, our findings represent a breakthrough in the field of embryology in that they study the possibilities of embryo selection at day 4. Moreover, the grad-CAMs findings are directly in line with embryologists’ decisions. Finally, our results demonstrated excellent potential for the inclusion of the models in clinical practice.

Inductive Graph Representation Learning for fraud detection

Graphs can be seen as a universal language to describe and model a diverse set of complex systems and data structures. However, efficiently extracting topological information from dynamic graphs is not a straightforward task. Previous works have explored a variety of inductive graph representation learning frameworks, but despite the surge in development, little research deployed these techniques for real-life applications. Most earlier studies are restricted to a set of benchmark experiments, rendering their practical generalisability questionable. This paper evaluates the proclaimed predictive performance of state-of-the-art inductive graph representation learning algorithms on highly imbalanced credit card transaction networks. More specifically, we assess the inductive capability of GraphSAGE and Fast Inductive Graph Representation Learning in a fraud detection setting. Credit card transaction fraud networks pose two crucial challenges for graph representation learners: First, these networks are highly dynamic, continuously encountering new transactions. Second, they are heavily imbalanced, with only a small fraction of transactions labelled as fraudulent. This paper contributes to the literature by (i) proving how inductive graph representation learning techniques can be leveraged to enhance predictive performance for fraud detection and (ii) demonstrating the benefit of graph-level undersampling for representation learning in imbalanced networks.

Utilizing Indonesian Universal Language Model Fine-tuning for Text Classification

Inductive transfer learning technique has made a huge impact on the computer vision field. Particularly, computer vision applications including object detection, classification, and segmentation, are rarely trained from scratch; instead, they are fine-tuned from pretrained models, which are products of learning from huge datasets. In contrast to computer vision, state-of-the-art natural language processing models are still generally trained from the ground up. Accordingly, this research attempts to investigate an adoption of the transfer learning technique for natural language processing. Specifically, we utilize a transfer learning technique called Universal Language Model Fine-tuning (ULMFiT) for doing an Indonesian news text classification task. The dataset for constructing the language model is collected from several news providers from January to December 2017 whereas the dataset employed for text classification task comes from news articles provided by the Agency for the Assessment and Application of Technology (BPPT). To examine the impact of ULMFiT, we provide a baseline that is a vanilla neural network with two hidden layers. Although the performance of ULMFiT on validation set is lower than the one of our baseline, we find that the benefits of ULMFiT for the classification task significantly reduce the overfitting, that is the difference between train and validation accuracies from 4% to nearly zero.

Welding Process Quality Improvement with Machine Learning Techniques

This paper presents a methodology for selecting design parameters with influence in quality product, using data analysis and machine learning techniques applied to a manufacturing problem where execution results can provide data for design improvement. Techniques are developed and used to extract the maximum information from existing process and quality data, and therefore to get feedback at design stages regarding how parameters variations affect the product performance when certain quality tests are applied. An algorithm is developed to sample multidimensional points in order to get the most representative subset, and various techniques are applied for data cleaning and outliers detection, such as Mahalanobis distance, Cook distance and HDOutliers method. Finally, inductive learning techniques, such as decision trees and association rules, are implemented to get conclusions about the relationships of the different variables. This paper finally shows this methodology in an automotive welding scenario.

Transfer Learning Study of Gas Adsorption in Metal-Organic Frameworks.

Metal-organic frameworks (MOFs) are a class of materials promising for gas adsorption due to their highly tunable nanoporous structures and host-guest interactions. While machine learning (ML) has been leveraged to aid the design or screen of MOFs for different purposes, the needs of big data are not always met, limiting the applicability of ML models trained against small data sets. In this work, we introduce an inductive transfer learning technique to improve the accuracy and applicability of ML models trained with a small amount of MOF adsorption data. This technique leverages potentially shareable knowledge from a source task to improve the models on the target tasks. As demonstrations, a deep neural networks (DNNs) trained on H2 adsorption data with 13 506 MOF structures at 100 bar and 243 K is used as the source task. When transferring knowledge from the source task to H2 adsorption at 100 bar and 130 K (one target task), the average predictive accuracy on target tasks was improved from 0.960 (direct training) to 0.991 (transfer learning), and transfer learning works in 89.3% of the cases. We also tested transfer learning across different gas species (i.e., from H2 to CH4), with an average predictive accuracy of CH4 adsorption being improved from 0.935 (direct training) to 0.980 (transfer learning), and transfer learning works in 82.0% of the cases. More importantly, transfer learning is shown to effectively improve the models on the target tasks with low accuracy from direct training. However, when transferring the knowledge from the source task to Xe/Kr adsorption, the transfer learning does not improve the predictive accuracy significantly and even makes it worse in ∼50.0% of the cases, which is attributed to the lack of common descriptors that is key to the underlying knowledge.

Using bipartite heterogeneous networks to speed up inductive semi-supervised learning and improve automatic text categorization

Due to the volume of texts available in digital form, the organization, management and knowledge extraction are laborious and frequently impossible to be handled. To automatically cope with these tasks, usually classification models are generated through supervised learning techniques. Unfortunately, this type of learning usually demands a huge human effort to label large volume of texts to build accurate classification models. Since collecting unlabeled texts is easy and inexpensive in several domains, the generation of classification models through inductive semi-supervised learning has been highlighted in recent years. Inductive semi-supervised learning allows to build a classification model using labeled and unlabeled texts. In this scenario, the goal is to augment the set of labeled documents with unlabeled documents to better discriminate class patterns. Hence, fewer texts must be previously labeled. However, semi-supervised learning algorithms that consider texts represented in a vector space model usually obtain unsatisfactory classification performances and are surpassed by semi-supervised learning algorithms that consider texts represented in a network. Nevertheless, despite the classification performances, effective approaches based on networks are generated through the similarities among documents and the classification of a new document are also based on the computation of similarities. This implies to set parameters and compute similarities to both generation the networks and classification of new documents. This approach is not feasible to generate fast responses and consequently to classify a huge volume of texts. In this article, we propose an approach to induce a classification model through semi-supervised learning considering text collections represented by bipartite heterogeneous networks. Bipartite networks are easily and quickly generated, leading to classification performance equivalent or better than other approaches based on network or vector space model and allows a fast classification of new documents. The results presented in this article demonstrate that the proposed approach is able to (i) speed up semi-supervised learning, (ii) speed up the classification of new documents and (iii) surpass classification performance of other existing inductive semi-supervised learning techniques.

Reconstruction of solid models from sketches including curves with inductive learning technique

ABSTRACTSince decades, many methods have been proposed to automatically reconstruct solid models from their sketches. However, it was difficult for the methods to handle sketches including curved lines such as cylinders. Recent years, the authors have proposed a method for the reconstruction of sketches including curves. In the method, the features of sketches of cuboids, cylinders, etc. were defined beforehand. When a sketch was input to the method, each feature was detected and extracted repeatedly from that, and a solid model was obtained by combining 3D features. However, after a feature was extracted from a sketch, some restoration process was needed because of the next feature extraction. Since the restoration process was ad hoc, its programming tended to be difficult. In this paper, a new method is proposed. This method would formalize and simplify the restoration process by applying our inductive learning technique. Although the learning technique had an issue that properties of geometric elements...

Learning Planning Domain Descriptions in RDDL

Recently, there is increasing interest in action model learning. However, most previous studies focused on learning effect-based action models. On the other hand, a rule-based planning domain description language was proposed in the latest planning competition. That is the Relational Dynamic Influence Diagram Language (RDDL). It uses rules to describe transitions instead of action models. In this paper, we build a system to learn planning domain descriptions in the RDDL. There are three major parts of an RDDL domain description: constraints, transitions and rewards. We first take advantage of the finite state machine analysis to identify constraints. Then, we employ the inductive learning technique to learn transitions. At last, we use regression to fix rewards. The evaluation was performed on benchmarks from planning competitions. It showed that our system can learn domain descriptions in the RDDL with low error rates. Moreover, our system is developed based on classical approaches. It implicates that the RDDL roots in previous planning languages. Therefore, more classical approaches could be useful in the RDDL domains.

Technology as knowledge – towards a new perspective on knowledge management in electronics

This paper gives practical examples of the concept of knowledge management with reference to an SME in the electronics sector. The study covers collaboration with the authors over a six year period. Whilst recognizing the relatively immature stage of this emerging subject's development, three practical demonstrations are given of its potential for enhancing the new product introduction process by analysing circuit designs in the light of subsequent manufacturing data. In particular, the power of inductive learning techniques is applied to thick film component design. One particular example highlights its capacity to uncover new knowledge directly from such data. The implications of this work are discussed, firstly for the management of the technology of design and manufacture and secondly for the improvement of business performance. Significant improvement in defect rates, process yields, stockturns per annum and turnover per head are reported.

An Overview of Inductive Learning Algorithms

learning enables the system to recognize patterns and regularities in previous knowledge or training data and extract the general rules from them. In literature there are proposed two main categories of inductive learning methods and techniques. Divide-and-Conquer algorithms also called decision Tree algorithms and Separate-and-Conquer algorithms known as covering algorithms. This paper first briefly describe the concept of decision trees followed by a review of the well known existing decision tree algorithms including description of ID3, C4.5 and CART algorithms. A well known example of covering algorithms is RULe Extraction System (RULES) family. An up to date overview of RULES algorithms, and Rule Extractor-1 algorithm, their solidity as well as shortage are explained and discussed. Finally few application domains of inductive learning are presented.

Analysis of Textual Data Based on Inductive Learning Techniques

This paper introduces knowledge discovery methods based on inductive learning techniques from textual data. The author argues three methods extracting features of the textual data. First one activates a key concept dictionary, second one does a key phrase pattern dictionary, and third one does a named entity extractor. These features are used in order to generate rules representing relationships between the features and text classes. The rules are described in the format of a fuzzy decision tree. Also, these features are used in order to acquire a classification model based on SVM (Support Vector Machine). The model can classify new textual data into the text classes with high classification accuracy. Lastly, this paper introduces two application tasks based on these methods and verifies the effect of the methods.

Neuro-fuzzy inverse model control structure of robotic manipulators utilized for physiotherapy applications

This paper presents a new neuro-fuzzy controller for robot manipulators. First, an inductive learning technique is applied to generate the required inverse modeling rules from input/output data recorded in the off-line structure learning phase. Second, a fully differentiable fuzzy neural network is developed to construct the inverse dynamics part of the controller for the online parameter learning phase. Finally, a fuzzy-PID-like incremental controller was employed as Feedback servo controller. The proposed control system was tested using dynamic model of a six-axis industrial robot. The control system showed good results compared to the conventional PID individual joint controller.

Using Naïve Bayes and Bayesian Networkfor Prediction of Potential Problematic Cases in Tuberculosis

Both Data Mining techniques and Machine Learning algorithms are tools that can be used to provide beneficial support in constructing models that could effectively assist medical practitioners in making comprehensive decisions regarding potential problematic cases in Tuberculosis (TB). This study introduces two machine learning techniques which are Naive Bayes inductive learning technique and the state of the art Bayesian Networks. These two techniques can be used towards constructing a model that can be used for predicting potential problematic cases in Tuberculosis. To construct a model, this study made have use of data collected from an Epidemiology laboratory. The volume of data was collated and divided into two data sets which are the training dataset and the investigation dataset. The model constructed by this study has shown a high predictive capability strength compared to other models presented on similar studies. DOI: http://dx.doi.org/10.11591/ij-ict.v1i2.1424

Automated Event Recognition for Football Commentary Generation

The enjoyment of many games can be enhanced by in-game commentaries. In this paper, the authors focus on the automatic generation of commentaries for football games, using Championship Manager as a case study. The basis of this approach is a real-time mapping of game states to commentary concepts, such as “dangerous situation for team A”. While in some cases it is feasible to provide such a mapping by hand-coding, in some cases it is not straight-forward because the meaning of the concepts cannot be easily formalized. In these cases, the authors propose to use inductive learning techniques that learn such a mapping from annotated game traces.

Varieties of Justification in Machine Learning

Forms of justification for inductive machine learning techniques are discussed and classified into four types. This is done with a view to introduce some of these techniques and their justificatory guarantees to the attention of philosophers, and to initiate a discussion as to whether they must be treated separately or rather can be viewed consistently from within a single framework.

Inductive learning for the Semantic Web: What does it buy?

Nowadays, building ontologies is a time consuming task since they are mainly manually built. This makes hard the full realization of the Semantic Web view. In order to overcome this issue, machine learning techniques, and specifically inductive learning methods, could be fruitfully exploited for learning models from existing Web data. In this paper we survey methods for (semi-)automatically building and enriching ontologies from existing sources of information such as Linked Data, tagged data, social networks, ontologies. In this way, a large amount of ontologies could be quickly available and possibly only refined by the knowledge engineers. Furthermore, inductive incremental learning techniques could be adopted to perform reasoning at large scale, for which the deductive approach has showed its limitations. Indeed, incremental methods allow to learn models from samples of data and then to refine/enrich the model when new (samples of) data are available. If on one hand this means to abandon sound and complete reasoning procedures for the advantage of uncertain conclusions, on the other hand this could allow to reason on the entire Web. Besides, the adoption of inductive learning methods could make also possible to dial with the intrinsic uncertainty characterizing the Web, that, for its nature, could have incomplete and/or contradictory information.

Forecasting of thunderstorms in pre-monsoon season over northwest India

Early detection and forecasting of thunderstorms is important in safeguarding and prevention of damages resulting from violent thunderstorms. An Expert System for Thunderstorm Forecasting (ESTF) during pre-monsoon season over Delhi the representative location over northwest India has been developed for the first time in India by using technique of approaching the problem “bottom up” by using inductive machine learning techniques to automatically acquire the knowledge about thunderstorm forecasting from the weather development data set consisting of TEMP data of Delhi for the months of May and June for the years from 1995-1999. Only input required is the entry of 0000 UTC TEMP data of Delhi at surface, 850, 700, 500 and 300 hPa levels. The rules are based on stability indices and other thermodynamic parameters evaluated from the said sounding. The system also provides climatological information about thunderstorms over Delhi. To compare the ESTF with the objective techniques, Dynamical-statistical methods for yes or no type thunderstorm occurrence forecast over Delhi during pre-monsoon months of May and June have been developed by using graphical discrimination method and multiple regression method and by using the same development data set i.e., TEMP data of Delhi for the months of May and June for the years from 1995-1999 and by using the same potential predictors as used in development of ESTF. In multiple regression method the parameters were found to be significant by stepwise screening procedure. The three methods developed were tested with independent data sets of May and June for the years from 2000-2001. Comparison of verification parameters of the forecast issued by Graphical Discrimination method, Multiple Regression Technique and by ESTF indicates that results of multiple regression method are better than those of graphical discrimination method. The results obtained by using ESTF were better than those obtained by using dynamical statistical models.

Application of rough set and decision tree for characterization of premonitory factors of low seismic activity

This paper presents a machine learning approach to characterizing premonitory factors of earthquake. The characteristic asymmetric distribution of seismic events and sampling limitations make it difficult to apply the conventional statistical predictive techniques. The paper shows that inductive machine learning techniques such as rough set theory and decision tree (C4.5 algorithm) allows developing knowledge representation structure of seismic activity in term of meaningful decision rules involving premonitory descriptors such as space-time distribution of radon concentration and environmental variables. The both techniques identify significant premonitory variables and rank attributes using information theoretic measures, e.g., entropy and frequency of occurrence in reducts. The cross-validation based on ''leave-one-out'' method shows that although the overall predictive and discriminatory performance of decision tree is to some extent better than rough set, the difference is not statistically significant.

Developing a continental-scale measure of gross primary production by combining MODIS and AmeriFlux data through Support Vector Machine approach

Remote sensing is a potentially powerful technology with which to extrapolate eddy covariance-based gross primary production (GPP) to continental scales. In support of this concept, we used meteorological and flux data from the AmeriFlux network and Support Vector Machine (SVM), an inductive machine learning technique, to develop and apply a predictive GPP model for the conterminous U.S. In the following four-step process, we first trained the SVM to predict flux-based GPP from 33 AmeriFlux sites between 2000 and 2003 using three remotely-sensed variables (land surface temperature, enhanced vegetation index (EVI), and land cover) and one ground-measured variable (incident shortwave radiation). Second, we evaluated model performance by predicting GPP for 24 available AmeriFlux sites in 2004. In this independent evaluation, the SVM predicted GPP with a root mean squared error (RMSE) of 1.87 gC/m 2/day and an R 2 of 0.71. Based on annual total GPP at 15 AmeriFlux sites for which the number of 8-day averages in 2004 was no less than 67% (30 out of a possible 45), annual SVM GPP prediction error was 32.1% for non-forest ecosystems and 22.2% for forest ecosystems, while the standard Moderate Resolution Imaging Spectroradiometer GPP product (MOD17) had an error of 50.3% for non-forest ecosystems and 21.5% for forest ecosystems, suggesting that the regionally tuned SVM performed better than the standard global MOD17 GPP for non-forest ecosystems but had similar performance for forest ecosystems. The most important explanatory factor for GPP prediction was EVI, removal of which increased GPP RMSE by 0.85 gC/m 2/day in a cross-validation experiment. Third, using the SVM driven by remote sensing data including incident shortwave radiation, we predicted 2004 conterminous U.S. GPP and found that results were consistent with expected spatial and temporal patterns. Finally, as an illustration of SVM GPP for ecological applications, we estimated maximum light use efficiency ( e max), one of the most important factors for standard light use efficiency models, for the conterminous U.S. by integrating the 2004 SVM GPP with the MOD17 GPP algorithm. We found that e max varied from ∼ 0.86 gC/MJ in grasslands to ∼ 1.56 gC/MJ in deciduous forests, while MOD17 e max was 0.68 gC/MJ for grasslands and 1.16 gC/MJ for deciduous forests, suggesting that refinements of MOD17 e max may be beneficial.

Prediction of Continental-Scale Evapotranspiration by Combining MODIS and AmeriFlux Data Through Support Vector Machine

Application of remote sensing data to extrapolate evapotranspiration (ET) measured at eddy covariance flux towers is a potentially powerful method to estimate continental-scale ET. In support of this concept, we used meteorological and flux data from the AmeriFlux network and an inductive machine learning technique called support vector machine (SVM) to develop a predictive ET model. The model was then applied to the conterminous U.S. In this process, we first trained the SVM to predict 2000-2002 ET measurements from 25 AmeriFlux sites using three remotely sensed variables [land surface temperature, enhanced vegetation index (EVI), and land cover] and one ground-measured variable (surface shortwave radiation). Second, we evaluated the model performance by predicting ET for 19 flux sites in 2003. In this independent evaluation, the SVM predicted ET with a root-mean-square error (rmse) of 0.62 mm/day (approximately 23% of the mean observed values) and an R2 of 0.75. The rmse from SVM was significantly smaller than that from neural network and multiple-regression approaches in a cross-validation experiment. Among the explanatory variables, EVI was the most important factor. Indeed, removing this variable induced an rmse increase from 0.54 to 0.77 mm/day. Third, with forcings from remote sensing data alone, we used the SVM model to predict the spatial and temporal distributions of ET for the conterminous U.S. for 2004. The SVM model captured the spatial and temporal variations of ET at a continental scale