Relational Learning Techniques Research Articles

Comparison of machine learning models predicting the pull-off strength of modified epoxy resin floors

Abstract It is becoming popular to replace destructive laboratory testing with related nondestructive testing (NDT) and/or machine learning (ML) techniques. Such an approach is becoming particularly desirable in operating facilities, where failing components result not only in the need for repair but also in the suspension of facility use for up to several months. Supporting construction work with artificial intelligence (AI) offers the potential for breakthroughs in this area. Commonly, this approach is already being used in the construction industry to determine compressive strength using, for example, information about the composition of a composite. Determination of pull-off strength can be approached in a similar way. In this paper, the ML model presented can be used to predict the pull-off strength of resin coatings containing granite powder and linen fibers. To obtain satisfactory results, the selected ML algorithms were analyzed on a database consisting of 140 sets of parameter values containing information about the composition of the resin coating. Indices indicating high performance (R = 0.885; RMSE = 0.138; MAPE = 3.72%) were obtained by a model based on the random forest (RF) algorithm containing 160 trees with a depth of 10 nodes. A comparison of the predicted fb pull-off strength with the strength determined by in-situ tests was developed. The results suggest that using artificial intelligence to determine the fb of resin coatings is a promising alternative.

Discovery and learning of feature geometry in underwater acoustic sensing: Case studies and challenges in active sonar applications

Interpretable artificial intelligence (AI) and related machine learning (ML) techniques are gaining popularity for underwater acoustic sensing applications. However, interpretability of machine learnt features poses a fundamental challenge to successful application of these powerful data science techniques to underwater acoustics. Autonomous sonar target recognition is especially interesting from the data science perspective as beyond target-specific features, a sonar ping response typically includes significant interference from the environment, e.g., clutter, multipath scattering, etc. Such environmental interference, resulting from complex, dynamic, unpredictable, and often unknown factors, manifest as identifiable structures in acoustic color or alternate multi-dimensional feature representations that can lead to machine learning and classification errors. In this talk, we will discuss these challenges commonly encountered in underwater acoustics using case studies from field experiments in active sonar target recognition. Some results will also include physics-driven simulations in this domain to provide robust ground truths. In particular, we will posit how braided feature geometry and its representation, as well geometry of overlap between features can render a sonar target feature informative, explainable, and discoverable. [Work funded by the ONR Grant Nos. N000142112420 and N000142312503 and DoD Navy (NEEC) Grant No. N001742010016.]

An Unsupervised Machine Learning Approach for Monitoring Data Fusion and Health Indicator Construction.

The prediction of system degradation is very important as it serves as an important basis for the formulation of condition-based maintenance strategies. An effective health indicator (HI) plays a key role in the prediction of system degradation as it enables vital information for critical tasks ranging from fault diagnosis to remaining useful life prediction. To address this issue, a method for monitoring data fusion and health indicator construction based on an autoencoder (AE) and a long short-term memory (LSTM) network is proposed in this study to improve the predictability and effectiveness of health indicators. Firstly, an unsupervised method and overall framework for HI construction is built based on a deep autoencoder and an LSTM neural network. The neural network is trained fully based on the normal operating monitoring data and then the construction error of the AE model is adopted as the health indicator of the system. Secondly, we propose related machine learning techniques for monitoring data processing to overcome the issue of data fusion, such as mutual information for sensor selection and t-distributed stochastic neighbor embedding (T-SNE) for operating condition identification. Thirdly, in order to verify the performance of the proposed method, experiments are conducted based on the CMAPSS dataset and results are compared with algorithms of principal component analysis (PCA) and a vanilla autoencoder model. Result shows that the LSTM-AE model outperforms the PCA and Vanilla-AE model in the metrics of monotonicity, trendability, prognosability, and fitness. Fourthly, in order to analyze the impact of the time step of the LSMT-AE model on HI construction, we construct and analyze the system HI curve under different time steps of 5, 10, 15, 20, and 25 cycles. Finally, the results demonstrate that the proposed method for HI construction can effectively characterize the health state of a system, which is helpful for the development of further failure prognostics and converting the scheduled maintenance into condition-based maintenance.

Detection of unknown strawberry diseases based on OpenMatch and two-head network for continual learning.

For continual learning in the process of plant disease recognition it is necessary to first distinguish between unknown diseases from those of known diseases. This paper deals with two different but related deep learning techniques for the detection of unknown plant diseases; Open Set Recognition (OSR) and Out-of-Distribution (OoD) detection. Despite the significant progress in OSR, it is still premature to apply it to fine-grained recognition tasks without outlier exposure that a certain part of OoD data (also called known unknowns) are prepared for training. On the other hand, OoD detection requires intentionally prepared outlier data during training. This paper analyzes two-head network included in OoD detection models, and semi-supervised OpenMatch associated with OSR technology, which explicitly and implicitly assume outlier exposure, respectively. For the experiment, we built an image dataset of eight strawberry diseases. In general, a two-head network and OpenMatch cannot be compared due to different training settings. In our experiment, we changed their training procedures to make them similar for comparison and show that modified training procedures resulted in reasonable performance, including more than 90% accuracy for strawberry disease classification as well as detection of unknown diseases. Accurate detection of unknown diseases is an important prerequisite for continued learning.

Surgical Tool Datasets for Machine Learning Research: A Survey

This paper is a comprehensive survey of datasets for surgical tool detection and related surgical data science and machine learning techniques and algorithms. The survey offers a high level perspective of current research in this area, analyses the taxonomy of approaches adopted by researchers using surgical tool datasets, and addresses key areas of research, such as the datasets used, evaluation metrics applied and deep learning techniques utilised. Our presentation and taxonomy provides a framework that facilitates greater understanding of current work, and highlights the challenges and opportunities for further innovative and useful research.

Artificial Intelligence for Glaucoma: Creating and Implementing Artificial Intelligence for Disease Detection and Progression

On September 3, 2020, the Collaborative Community on Ophthalmic Imaging conducted its first 2-day virtual workshop on the role of artificial intelligence (AI) and related machine learning techniques in the diagnosis and treatment of various ophthalmic conditions. In a session entitled "Artificial Intelligence for Glaucoma," a panel of glaucoma specialists, researchers, industry experts, and patients convened to share current research on the application of AI to commonly used diagnostic modalities, including fundus photography, OCT imaging, standard automated perimetry, and gonioscopy. The conference participants focused on the use of AI as a tool for disease prediction, highlighted its ability to address inequalities, and presented the limitations of and challenges to its clinical application. The panelists' discussion addressed AI and health equities from clinical, societal, and regulatory perspectives.

Banana Plant Disease Classification Using Hybrid Convolutional Neural Network.

Banana cultivation is one of the main agricultural elements in India, while the common problem of cultivation is that the crop has been influenced by several diseases, while the pest indications have been needed for discovering the infections initially for avoiding the financial loss to the farmers. This problem will affect the entire banana productivity and directly affects the economy of the country. A hybrid convolution neural network (CNN) enabled banana disease detection, and the classification is proposed to overcome these issues guide the farmers through enabling fertilizers that have to be utilized for avoiding the disease in the initial stages, and the proposed technique shows 99% of accuracy that is compared with the related deep learning techniques.

Satellite-Based Computing Networks with Federated Learning

Driven by the ever increasing penetration and proliferation of data-driven applications, a new generation of wireless communication, the sixth generation (6G) mobile system enhanced by artificial intelligence, has attracted substantial research interests. Among various candidate technologies of 6G, low Earth orbit (LEO) satellites have appealing characteristics of ubiquitous wireless access. However, the costs of satellite communication (SatCom) are still high, relative to their counterparts of ground mobile networks. To support massively interconnected devices with intelligent adaptive learning and reduce expensive traffic in SatCom, we propose federated learning (FL) in LEO-based satellite communication networks. We first review the state-of-the-art LEO-based SatCom and related machine learning (ML) techniques, and then analyze four possible ways of combining ML with satellite networks. The learning performance of the proposed strategies is evaluated by simulation and results reveal that FL-based computing networks improve the performance of communication overheads and latency. Finally, we discuss future research topics along this research direction.

Performance enrichment through parameter tuning of random forest classification for imbalanced data applications

One of the foremost application domains in today’s real-life scenario is unequal data distribution within datasets, related classifiers, and its Performance Enrichment Techniques (PET). Random Forest Classification (RFC) is one of the most efficient techniques that can function speedily over binary or multiclass imbalanced characteristics datasets. With its built-in ensemble capacity, building a generalized model on any Binary Imbalanced Datasets (BID) and Multiclass Imbalanced Datasets (MID) gets much easier. Related work carried out here implies that the attention of researchers is inclined towards ID applications and related machine learning techniques. RFC gives improvised performance due to Ensemble Approach (EA). EA generates several classifiers and segregates the results as PET. The performance of the single classifier is lower than the performance of the set of multiple classifiers. The associated study in this research paper provides insight into three PET, which are the count of folds in cross-validation, variable count of trees and the feature selection method through Regression Analysis (RA) and it’s tuning with RFC Classifier. Time and Cost required for classification is saved with the help of RA. Extensive experimental analysis carried out with a predictive modeler Salford System with proposed PET and tested on 32 benchmark datasets, which justifies the enhanced performance of RFC.

Deep heading estimation for pedestrian dead reckoning

Currently, the heading estimation could be easily achieved by many built-in direction sensors, such as the smartphone. However, the obtained heading angle is only suitable for situations where an equipped pedestrian’s movement and orientation keep the same, such as normal forward walking and turning. When the pedestrian faces a different direction from his movement, the heading angle remains the pedestrian orientation but not the movement and thus causes a heading estimation error. In this paper, we introduce several related deep learning techniques to explore their respective abilities in heading estimation of the waist-mounted Miniature Inertial Measurement Unit (MIMU). Specifically, this paper adopts two kinds of methods to analyze the data collected from MIMU, include acceleration, angular velocity, or their combination, to predict the heading angle. Firstly, considering the heading estimation is a time series prediction problem actually, this paper introduces the powerful Long Short-Time Memory (LSTM) model. On the other hand, this paper uses the Graph Convolutional Network (GCN) model to consider the relationship between the orientation and the direction of motion at different times. In experiments, we show that the accuracy of the proposed LSTM model in the test set achieved a promising 99.12%. To test our method in the real scenes, this paper designs simulation experiments and mobile terminal tests based on Tensorflow Lite. The experimental results show that the movement heading can be effectively judged based on the waist-mounted sensor data, and has a very significant accuracy.

Review of visible watermark’s influence on private information-related machine learning techniques

Under the increasing needs of sharing datasets and risks of leaking privacy during data sharing, there is growing attention of the owners of datasets to claim ownership and protect their privacy from being misused. To achieve this secure purpose, the visible watermark may be an effective way. However, due to the loss of visual information, it is still unclear the influence of these visible watermark methods on related machine learning techniques. To investigate these, this paper provides a review of visible watermark’s influence on private information-related machine learning techniques. This paper compared the performance of different CNN methods with or without visible watermarking. According to the results of the experiments, the accuracy of watermarked images’ identification decreased significantly. By average, the accuracy would decrease by 4.75%. And the performance changed when using different CNN structures, watermarked methods and datasets. After that, there are some experiments to explore the feasibility of using none-watermarked images trained network to identify the watermarked images. The results show that the accuracy of identification at least decreased by 25% when only one of the train set and test set are watermarked.

A Deep Learning Strategy For On-Orbit Servicing Via Space Robotic Manipulator

Autonomous robotic systems are currently being addressed as a critical element in the development of present and future on-orbit operations. Modern missions are calling for systems capable of reproducing human’s decision-making process, thus enhancing their performance. Generally, space manipulators are mounted on a floating spacecraft in a microgravity environment, consequently leading to a mutual influence between the robotic arms and the platform dynamics, thus making the motion planning and control design more challenging than those of terrestrial robots. Another aspect to be considered is that space robots are designed as lightweight systems, resulting in a significant dynamic coupling between their rigid motion and structural elasticity. These effects involve critical issues in modelling their dynamics and designing a suitable controller. In this context, Deep Neural Network (DNN) architectures and the related Deep Learning (DL) techniques have widely proved to have powerful capability in solving data-driven nonlinear modelling problems and they can hence represent a viable solution for space activities. The present paper deals with the design of a DNN controller for a space manipulator system, which has to follow a specific path for a typical on-orbit servicing mission. The goal is to provide proper control inputs autonomously adapting to the given desired trajectory. Structural flexibility and joint friction features are implemented in the dynamic model as well.

Soft governance by hard fact? The OECD as a knowledge broker in education policy

As the policy field of education has become increasingly internationalized over the last two decades, international organizations like the Organisation for Economic Co-operation and Development (OECD) play an increasingly decisive role in the diffusion of knowledge, monitoring, and research in global education policy. Although the OECD lacks any binding or coercive governance instruments in relation to states, or ability to provide material incentives for compliance, it has nevertheless successively expanded its influence regarding education. From a perspective of social constructivism, we argue that the transmission of ideas and information generated through ratings and rankings can be viewed as a crucial governance tool for the influence of an international organization (IO). Our article seeks to analyze how the OECD uses large-scale education assessments to promote the economically based idea of human capital and related learning techniques in education policy – thus influencing national education systems. Furthermore, the OECD and its distinctive approach of soft governance through putative hard fact may become a role model for other IOs, both in the field of education and beyond.

Interactive Visual Graph Mining and Learning

This article presents a platform for interactive graph mining and relational machine learning called GraphVis. The platform combines interactive visual representations with state-of-the-art graph mining and relational machine learning techniques to aid in revealing important insights quickly as well as learning an appropriate and highly predictive model for a particular task (e.g., classification, link prediction, discovering the roles of nodes, and finding influential nodes). Visual representations and interaction techniques and tools are developed for simple, fast, and intuitive real-time interactive exploration, mining, and modeling of graph data. In particular, we propose techniques for interactive relational learning (e.g., node/link classification), interactive link prediction and weighting, role discovery and community detection, higher-order network analysis (via graphlets, network motifs), among others. GraphVis also allows for the refinement and tuning of graph mining and relational learning methods for specific application domains and constraints via an end-to-end interactive visual analytic pipeline that learns, infers, and provides rapid interactive visualization with immediate feedback at each change/prediction in real-time. Other key aspects include interactive filtering, querying, ranking, manipulating, exporting, as well as tools for dynamic network analysis and visualization, interactive graph generators (including new block model approaches), and a variety of multi-level network analysis techniques.

Semi-supervised local multi-manifold Isomap by linear embedding for feature extraction

In this paper, we mainly propose a semi-supervised local multi-manifold Isomap learning framework by linear embedding, termed SSMM-Isomap, that can apply the labeled and unlabeled training samples to perform the joint learning of neighborhood preserving local nonlinear manifold features and a linear feature extractor. The formulation of SSMM-Isomap aims at minimizing pairwise distances of intra-class points in the same manifold and maximizing the distances over different manifolds. To enhance the performance of nonlinear manifold feature learning, we also incorporate the neighborhood reconstruction error to preserve local topology structures between both labeled and unlabeled samples. To enable our SSMM-Isomap to extract local manifold features from the outside new data, we also add a feature approximation error that correlates manifold features with embedded features by the jointly learnt feature extractor. Thus, the learnt linear extractor can extract the local manifold features from the new data efficiently by direct embedding. To optimize the proposed objective function, two effective schemes are presented, i.e., Scaling by MAjorizing a Complicated Function and Eigen-decomposition. Notice that the comparison of the proposed two solvers is also described. We mainly evaluate SSMM-Isomap for manifold feature learning, data clustering and classification. Extensive simulation results verify the effectiveness of our SSMM-Isomap algorithm, compared with other related feature learning techniques.

Learning Maliciousness in Cybersecurity Graphs

Systems and methods for utilizing statistical relational learning techniques in order to predict factors for nodes of a node graph, such as a node graph that represents attacks and incidents to a computing system, are described. In some embodiments, the systems and methods identify certain nodes (of a node graph) as representing malicious attributes of an email or other threat artifact received by a computing system or network and utilize relational learning to predict the maliciousness of attributes represented by other nodes (of the node graph).

Learning Relational Event Models from Video

Event models obtained automatically from video can be used in applications ranging from abnormal event detection to content based video retrieval. When multiple agents are involved in the events, characterizing events naturally suggests encoding interactions as relations. Learning event models from this kind of relational spatio-temporal data using relational learning techniques such as Inductive Logic Programming (ILP) hold promise, but have not been successfully applied to very large datasets which result from video data. In this paper, we present a novel framework REMIND (Relational Event Model INDuction) for supervised relational learning of event models from large video datasets using ILP. Efficiency is achieved through the learning from interpretations setting and using a typing system that exploits the type hierarchy of objects in a domain. The use of types also helps prevent over generalization. Furthermore, we also present a type-refining operator and prove that it is optimal. The learned models can be used for recognizing events from previously unseen videos. We also present an extension to the framework by integrating an abduction step that improves the learning performance when there is noise in the input data. The experimental results on several hours of video data from two challenging real world domains (an airport domain and a physical action verbs domain) suggest that the techniques are suitable to real world scenarios.

Ontology Based Concept Hierarchy Extraction of Web Data

This paper proposes the method of Ontology Based Concept Hierarchy Extraction of Web Data. This helps to extract Concept Hierarchy efficient way for ontology construction. It is very useful for learning the ontology from the text in more efficient way. In General, Natural Language is Complexity and Uncertainty. The existing system used either Statistical based learning or logic based learning Techniques. Statistical based learning techniques gives solution only for complexity and Logic based techniques gives solution for uncertainty alone. But the Statistical Relational Learning Techniques give solution for both Complexity and Uncertainty. So, our proposed system uses Statistical Relational Learning Technique, named Markov Logic Network. Markov Logic Network is a technique in which identify the concept in the domain and order the candidate terms in hierarchical way. An experimental result provides the best concept hierarchy extractions compared to the state-of-art methods. Keywords: Concept Hierarchy Extraction, Hearst Pattern, Markov Logic Network, Ontology, Semantic Web.

Prediction of protein-protein interaction sites from weakly homologous template structures using meta-threading and machine learning.

The identification of protein-protein interactions is vital for understanding protein function, elucidating interaction mechanisms, and for practical applications in drug discovery. With the exponentially growing protein sequence data, fully automated computational methods that predict interactions between proteins are becoming essential components of system-level function inference. A thorough analysis of protein complex structures demonstrated that binding site locations as well as the interfacial geometry are highly conserved across evolutionarily related proteins. Because the conformational space of protein-protein interactions is highly covered by experimental structures, sensitive protein threading techniques can be used to identify suitable templates for the accurate prediction of interfacial residues. Toward this goal, we developed eFindSite(PPI) , an algorithm that uses the three-dimensional structure of a target protein, evolutionarily remotely related templates and machine learning techniques to predict binding residues. Using crystal structures, the average sensitivity (specificity) of eFindSite(PPI) in interfacial residue prediction is 0.46 (0.92). For weakly homologous protein models, these values only slightly decrease to 0.40-0.43 (0.91-0.92) demonstrating that eFindSite(PPI) performs well not only using experimental data but also tolerates structural imperfections in computer-generated structures. In addition, eFindSite(PPI) detects specific molecular interactions at the interface; for instance, it correctly predicts approximately one half of hydrogen bonds and aromatic interactions, as well as one third of salt bridges and hydrophobic contacts. Comparative benchmarks against several dimer datasets show that eFindSite(PPI) outperforms other methods for protein-binding residue prediction. It also features a carefully tuned confidence estimation system, which is particularly useful in large-scale applications using raw genomic data. eFindSite(PPI) is freely available to the academic community at http://www.brylinski.org/efindsiteppi.

Refinement-based disintegration: An approach to re-representation in relational learning

We present a new approach lo learn from relational data based on re-representation of the examples. This approach, called property-based re-representation is based on a new analysis of the structure of refinement graphs used in ILP and relational learning in general. This analysis allows the characterization of relational examples by a set of multi-relational patterns called properties. Using them, we perform a property-based re-representation of relational examples that facilitates the development of relational learning techniques. Additionally, we show the usefulness of re-representation with a collection of experiments in the context of nearest neighbor classification.