Learning-based Paradigm Research Articles

PP-GNN: Pretraining Position-aware Graph Neural Networks with the NP-hard metric dimension problem

On a graph G=(V,E), we call S⊂V resolving if ∀u,v∈V with u≠v, ∃w∈V such that d(u,w)≠d(v,w). The smallest possible cardinality of S is called the metric dimension, computing which is known to be NP-hard. Solving the metric dimension problem (MDP) and the associated minimal resolving set has many important applications across science and engineering. In this paper, we introduce MaskGNN, a method using a graph neural network (GNN) model to learn the minimal resolving set in a self-supervised manner by optimizing a novel surrogate objective. We provide a construction showing the global minimum of this objective coincides with the solution to the MDP. MaskGNN attains 51%–72% improvement over the best baseline and up to 98% the reward of integer programming in 0.72% of the running time. On this foundation, we introduce Pretraining Position-aware GNNs (PP-GNN) and evaluate on popular benchmark position-based tasks on graphs. PP-GNN’s strong results challenge the currently popular paradigm – heuristic-driven anchor-selection – with a new learning-based paradigm — simultaneously learning the metric basis of the graph and pretraining position-aware representations for transferring to downstream position-based tasks.

Learning-Based Near-Infrared Band Simulation with Applications on Large-Scale Landcover Classification.

Multispectral sensors are important instruments for Earth observation. In remote sensing applications, the near-infrared (NIR) band, together with the visible spectrum (RGB), provide abundant information about ground objects. However, the NIR band is typically not available on low-cost camera systems, which presents challenges for the vegetation extraction. To this end, this paper presents a conditional generative adversarial network (cGAN) method to simulate the NIR band from RGB bands of Sentinel-2 multispectral data. We adapt a robust loss function and a structural similarity index loss (SSIM) in addition to the GAN loss to improve the model performance. With 45,529 multi-seasonal test images across the globe, the simulated NIR band had a mean absolute error of 0.02378 and an SSIM of 89.98%. A rule-based landcover classification using the simulated normalized difference vegetation index (NDVI) achieved a Jaccard score of 89.50%. The evaluation metrics demonstrated the versatility of the learning-based paradigm in remote sensing applications. Our simulation approach is flexible and can be easily adapted to other spectral bands.

Supervised deep learning-based paradigm to screen the enhanced oil recovery scenarios

High oil prices and concern about limited oil reserves lead to increase interest in enhanced oil recovery (EOR). Selecting the most efficient development plan is of high interest to optimize economic cost. Hence, the main objective of this study is to construct a novel deep-learning classifier to select the best EOR method based on the reservoir’s rock and fluid properties (depth, porosity, permeability, gravity, viscosity), and temperature. Our deep learning-based classifier consists of a one-dimensional (1D) convolutional neural network, long short-term memory (LSTM), and densely connected neural network layers. The genetic algorithm has been applied to tune the hyperparameters of this hybrid classifier. The proposed classifier is developed and tested using 735 EOR projects on sandstone, unconsolidated sandstone, carbonate, and conglomerate reservoirs in more than 17 countries. Both the numerical and graphical investigations approve that the structure-tuned deep learning classifier is a reliable tool to screen the EOR scenarios and select the best one. The designed model correctly classifies training, validation, and testing examples with an accuracy of 96.82%, 84.31%, and 82.61%, respectively. It means that only 30 out of 735 available EOR projects are incorrectly identified by the proposed deep learning classifier. The model also demonstrates a small categorical cross-entropy of 0.1548 for the classification of the involved enhanced oil recovery techniques. Such a powerful classifier is required to select the most suitable EOR candidate for a given oil reservoir with limited field information.

Knowledge-Driven Accurate Opponent Trajectory Prediction for Gun-Dominated Autonomous Air Combat

The autonomous air combat (AAC) technique has been a lasting topic for decades. Accurate opponent trajectory prediction provides a fundamental basis for decision making in AAC. In this work, we propose a knowledge-driven scheme for the opponent trajectory prediction problem in one-versus-one gun-dominated within-visual-range (WVR) air combat. A dedicated air combat engagement database is first constructed via skilled human pilots flying WVR air combat. Two baseline algorithms following rule-based and learning-based paradigms are developed and optimized. The knowledge-driven scheme begins with defining handcrafted features extracted from the opponent history movements, and principal components analysis is adopted to compress/refine the features. A generalized regression neural network is then developed to compensate for the residual of the rule-based trajectory prediction method, wherein the refined features are used as inputs, and the residual compensation are used as outputs. Via extensive simulation tests, the proposed scheme shows a more accurate performance as compared with the two other baseline algorithms. To demonstrate the applicability of the proposed scheme, an automatic gun-firing strategy for commencing gun attack in AAC is also illustrated, which justifies the proposed scheme.

Learning-based padding: From connectivity on data borders to data padding

Padding is employed vastly in convolutional neural networks (CNNs) to maintain the output size. In current padding schemes, the input feature maps are padded using quite simple strategies, e.g., constant zero values in zero padding, which is the most common padding choice. In this work, we propose to use learning-based paradigms to calculate the padding data from the connectivity of the input images on the data borders. Two different modules, i.e., learning-based padding by convolution (LPC) and learning-based padding by attention (LPA), are designed to obtain the padding data from the interdependencies of the image border data along the channel and spatial dimensions, respectively. The designed LPC or LPA is formulated as a generic, plug-and-play unit, which can be a direct replacement for the conventional padding technique. Extensive experiments on image classification and semantic segmentation tasks show that the proposed padding schemes can consistently obtain higher accuracy than standard padding schemes, in various deep network backbones. The codes and trained modules are available at https://github.com/ICSResearch/LP.

Reinforcement learning based flow and energy management in resource-constrained wireless networks

This paper aims at developing a learning-based framework for MAC sleep–listen–transmit scheduling in wireless networks. The Reinforcement Learning-based paradigm is shown to work in the absence of network time-synchronization and other complex hardware features, such as carrier-sensing, thus making it suitable for low-cost transceivers for IoT and wireless sensor nodes. The framework allows wireless nodes to learn policies that can support throughput-sustainable flows while minimizing node energy expenditure and sleep-induced packet drops and delays. Each node independently learns a scheduling policy without explicit communication with other network nodes. The trade-off between packet drops and energy efficiency is analyzed, and an application-specific solution is proposed for handling the trade-off. It is shown how this model allows users to prioritize between energy efficiency and packet drops depending on specific application requirements. An analytical model is developed for understanding the underlying system dynamics, which is also validated using extensive simulation experiments. Moreover, the developed mechanism is experimented with for heterogeneous network topologies and traffic patterns

Machine Learning-based Seismic Fragility Curves for RC Bridge Piers

A significant part of ongoing studies in the field of earthquake engineering is directed toward the seismic risk assessment of buildings and infrastructures at a territorial scale. This task is usually accomplished by grouping the structures into homogenous classes in terms of typology, for which seismic fragility curves are then obtained for different limit states via numerical simulations or from the statistical analysis of observational data when available. Particularly, the development of typological fragility curves for bridges under earthquake is useful for assessing the reliability and resilience of transportation networks in seismic areas and can be also effective decision-making support. Within this framework, the proposed study establishes a machine learning-based paradigm for the closed-form prediction of the main statistical parameters required to obtain relevant seismic fragility curves for reinforced concrete bridge piers. Initially, a huge training dataset has been obtained by Monte Carlo simulations and displacement-based bridge pier assessments by assuming data representative of the Italian highway transportation network. Next, symbolic nonlinear regression formulae for estimating the main statistical parameters of seismic fragility curves have been generated. With the aid of those formulae, the effort of calculating the seismic fragility curves is greatly reduced since the corresponding main statistical parameters can be directly calculated from a set of commonly available attributes. Therefore, the proposed study provides a helpful tool for the rapid preliminary assessment of damage and risk level of existing highway transportation networks exposed to seismic hazards.

Automatic Detection of Outliers in Multi-Channel EMG Signals Using MFCC and SVM

The automatic detection of noisy channels in surface Electromyogram (sEMG) signals, at the time of recording, is very critical in making a noise-free EMG dataset. If an EMG signal contaminated by high-level noise is recorded, then it will be useless and can’t be used for any healthcare application. In this research work, a new machine learning-based paradigm is proposed to automate the detection of low-level and high-level noises occurring in different channels of high density and multi-channel sEMG signals. A modified version of mel frequency cepstral coefficients (mMFCC) is proposed for the extraction of features from sEMG channels along with other statistical parameters i-e complexity coefficient, hurst exponent, and root mean square. Several state-of-the-art classifiers such as Support Vector Machine (SVM), Ensemble Bagged Trees, Ensemble Subspace Discriminant, and Logistic Regression are used to automatically identify an EMG channel either bad or good based on these extracted features. Comparison-based analyses of these classifiers have also been considered based on total classification accuracy, prediction speed (observations/sec), and processing time. The proposed method is tested on 320 simulated EMG channels as well as 640 experimental EMG channels. SVM is used as our main classifier for the detection of noisy channels which gives a total classification accuracy of 99.4% for simulated EMG channels whereas accuracy of 98.9% is achieved for experimental EMG channels.

Learning-Based Resource Allocation in Heterogeneous Ultradense Network

Learning-based resource allocation (LRA) is envisioned as an integral element of 6G. This article proposes a novel learning-based paradigm to address resource allocation problems in heterogeneous ultradense networks (HUDNs). Our paradigm is a highly efficient realization for utilizing the inherence properties in HUDN, which comprise the local validity and correlation attenuation. Concretely, we formulate the HUDNs as a heterogeneous bipartite graph model and propose the corresponding heterogeneous bipartite graph neural network (HBGNN). The local sampling characteristic of HBGNN matches the inherence properties. Meanwhile, our paradigm combines data-driven and model-driven learnings and employs online and offline trainings. Hence, two of LRA’s obstacles: 1) the overreliance on the perfect data set and 2) the low calculation efficiency are mitigated, and the realizability of our paradigm is improved. Besides, entropy regularization is utilized to guarantee the effectiveness of exploration in the configuration space. We apply our approach to a representative resource allocation problem, the jointly user association (UA) and power allocation (JUAPA) problem. We formulate JUAPA as a combination classification and regression problem and adopt a dynamical hyperparameter output layer to address the discrete variable of UA. Simulation results demonstrate that the proposed method has better performance and higher computational efficiency than traditional optimization algorithms.

NeoAI 1.0: Machine learning-based paradigm for prediction of neonatal and infant risk of death

BackgroundThe Neonatal mortality rate in the United States is 3.8 deaths per 1000 live births, which is comparably higher than other nations. PurposeThe aim of the proposed study is to design and develop Artificial Intelligence (AI) models (NeoAI 1.0, Global Biomedical Technologies, Inc., Roseville, CA, USA) on risk variables extracted from the National Center for Health Statistics (NCHS) data from 2014 to 2017 duration, consisting of birth-death infant files to predict neonatal and infant deaths. MethodologyThe NCHS data consisted of 15.8 million live birth records, including 91,773 infant deaths, out of which 61,222 were neonatal (life <28 days) and the rest were non-deaths. We designed and developed two different kinds of systems, labelled as neonatal and infant death systems. The data preparation consisted of balancing the two classes using the Adaptive Synthetic oversampling technique (ADASYN) paradigm. The best features were extracted using mutual information followed by 5-fold cross-validation using four different models, namely AdaBoost, XGBoost, Random Forest, and Logistic Regression based on balanced and unbalanced paradigms. ResultsXGBoost gave the best results for the neonatal system with AUC of 0.97 and 0.99 (p < 0.0001), while for the infant system, the scores were 0.91 and 0.99, both systems, without/with ADASYN integration, respectively. Further, there was a 60% increase in F1-score and sensitivity with ADASYN integration. The most important risk factors for classifier models along with feature extraction were maternal age and maternal race by Hispanic classification. Further, gestational age, labour aid and newborn condition were also part of the top five risk factors for these models. ConclusionsNoeAI showed two independent powerful machine learning (ML) systems and selected the best risk predictors combined with classification models for neonatal and infant deaths. The response time of the online platform was less than a second.

Towards Development of Machine Learning Framework for Enhancing Security in Internet of Things

An IoT system is a smart network that connects all items to the Internet and exchanges data using Internet Engineering Task Force established protocols. As a consequence, everything is instantly accessible from any place and at any time. The Internet of Things (IoT) network is built on the backbone of tiny sensors embedded in common objects. There is no need for human intervention in the interactions of IoT devices. The Internet of Things (IoT) security risk cannot be ignored. Untrusted networks, such as the Internet, are utilized to provide remote access to IoT devices. As a result, IoT systems are susceptible to a broad range of harmful activities, including cyberattacks. If security problems are not addressed, critical information may be hacked at any time. This article describes a feature selection and machine learning-based paradigm for improving security in the Internet of Things. Because network data are inherently abundant, it must be reduced in size before processing. Dimension reduction is the process of constructing a subset of an original data collection that removes superfluous content from the essential data set. Dimension reduction is a data mining approach. To minimize the number of dimensions in a dataset, linear discriminant analysis (LDA) is used. Following that, the data set with fewer dimensions is put into machine learning predictors as a training set. The effectiveness of machine learning approaches has been assessed using a range of criteria.

FL-NoiseMap: A Federated Learning-based privacy-preserving Urban Noise-Pollution Measurement System

AbstractIncreasing levels of noise pollution in urban environments are a primary cause of various physical and psychological health issues. There is an urgent requirement to manage environmental noise by assessing the current levels of noise pollution by gathering real-world data and building a fine-granularity real-time noise map. Traditionally, simulation-based, small-scale sensor-network-based, and participatory sensing-based approaches have been used to estimate noise levels in urban areas. These techniques are inadequate to gauge the prevalence of noise pollution in urban areas and have been shown to leak private user data. This paper proposes a novel federated learning-based urban noise mapping system, FL-NoiseMap, that significantly enhances the privacy of participating users without adversely affecting the application performance. We list several state-of-the-art urban noise monitoring systems that can be seamlessly ported to the federated learning-based paradigm and show that the existing privacy-preserving approaches can be used as an add-on to enhance participants’ privacy. Moreover, we design an “m-hop” application model modification approach for privacy preservation, unique to FL-NoiseMap. We also describe techniques to maintain data reliability for the proposed application. Numerical experiments on simulated datasets showcase the superiority of the proposed scheme in terms of users’ privacy preservation and noise map reliability. The proposed scheme achieves the lowest average normalized root mean square error in the range of 4% to 7% as the number of participants varies between 500 and 5000 while providing maximum coverage of over 95% among various competing algorithms. The proposed malicious contribution removal framework can decrease the average normalizedroot mean square error by more than 50% for simulations having up to 20% malicious users.

Automated deep learning-based paradigm for high-risk plaque detection in B-mode common carotid ultrasound scans: an asymptomatic Japanese cohort study.

The death due to stroke is caused by embolism of the arteries which is due to the rupture of the atherosclerotic lesions in carotid arteries. The lesion formation is over time, and thus, early screening is recommended for asymptomatic and moderate-risk patients. The previous techniques adopted conventional methods or semi-automated and, more recently, machine learning solutions. A handful of studies have emerged based on solo deep learning (SDL) models such as UNet architecture. The proposed research is the first to adopt hybrid deep learning (HDL) artificial intelligence models such as SegNet-UNet. This model is benchmarked against UNet and advanced conventional models using scale-space such as AtheroEdge 2.0 (AtheroPoint, CA, USA). All our resultant statistics of the three systems were in the order of UNet, SegNet-UNet, and AtheroEdge 2.0. Using the database of 379 ultrasound scans from a Japanese cohort of 190 patients having moderate risk and implementing the cross-validation deep learning framework, our system performance using area-under-the-curve (AUC) for UNet, SegNet-UNet, and AtheroEdge 2.0 were 0.93, 0.94, and 0.95 (P<0.001), respectively. The coefficient of correlation between the three systems and ground truth (GT) were: 0.82, 0.89, and 0.85 (P<0.001 for all three), respectively. The mean absolute area error for the three systems against manual GT was 4.07±4.70 mm2, 3.11±3.92 mm2, 3.72±4.76 mm2, respectively, proving the superior performance SegNet-UNet against UNet and AtheroEdge 2.0, respectively. Statistical tests were also conducted for their reliability and stability. The proposed study demonstrates a fast, accurate, and reliable solution for early detection and quantification of plaque lesions in common carotid artery ultrasound scans. The system runs on a test US image in <1 second, proving overall performance to be clinically reliable.

Toward Augmented Reality in Museums: Evaluation of Design Choices for 3D Object Pose Estimation

The solutions to many computer vision problems, including that of 6D object pose estimation, are dominated nowadays by the explosion of the learning-based paradigm. In this paper, we investigate 6D object pose estimation in a practical, real-word setting in which a mobile device (smartphone/tablet) needs to be localized in front of a museum exhibit, in support of an augmented-reality application scenario. In view of the constraints and the priorities set by this particular setting, we consider an appropriately tailored classical as well as a learning-based method. Moreover, we develop a hybrid method that consists of both classical and learning based components. All three methods are evaluated quantitatively on a standard, benchmark dataset, but also on a new dataset that is specific to the museum guidance scenario of interest.

A novel learning-based paradigm to investigate the visual-cognitive bases of lung nodule detection

A novel learning-based paradigm to investigate the visual-cognitive bases of lung nodule detection

Retrieval Oriented Deep Feature Learning With Complementary Supervision Mining.

Deep convolutional neural networks (CNNs) have been widely and successfully applied in many computer vision tasks, such as classification, detection, semantic segmentation, and so on. As for image retrieval, while off-the-shelf CNN features from models trained for classification task are demonstrated promising, it remains a challenge to learn specific features oriented for instance retrieval. Witnessing the great success of low-level SIFT feature in image retrieval and its complementary nature to the semantic-aware CNN feature, in this paper, we propose to embed the SIFT feature into the CNN feature with a Siamese structure in a learning-based paradigm. The learning objective consists of two kinds of loss, i.e., similarity loss and fidelity loss. The first loss embeds the image-level nearest neighborhood structure with the SIFT feature into CNN feature learning, while the second loss imposes that the CNN feature with the updated CNN model preserves the fidelity of that from the original CNN model solely trained for classification. After the learning, the generated CNN feature inherits the property of the SIFT feature, which is well oriented for image retrieval. We evaluate our approach on the public data sets, and comprehensive experiments demonstrate the effectiveness of the proposed method.

LEAPS2: Learning based Evolutionary Assistive Paradigm for Surrogate Selection

We propose a learning-based paradigm (LEAPS2) to recommend the best surrogate/ with minimal computational effort using the input-output data of a complex physico-numerical system. Emulating the knowledge pyramid, LEAPS2 uses several attributes to extract system information from the data, correlates them with surrogate performances, stores this attribute-surrogate knowledge in a regression tree ensemble, and uses the ensemble to recommend surrogates for unknown systems. We implement LEAPS2 using data from 66 diverse analytical functions, 18 attributes, and 25 surrogates. By progressively adding data, we demonstrate that LEAPS2 learns to improve computational efficiency and functional accuracy. Besides, the architecture of LEAPS2 enables its evolution via more attributes and surrogates. We employ LEAPS2 to recommend surrogates for estimating the bubble and dew point temperatures of LNG. Interestingly, our assistive tool suggests a different surrogate for each temperature, and hints that DPT may be harder to approximate than BPT.

Goal-directed affordance prediction at the subtask level

Purpose – This paper aims to present a hybrid control approach that combines learning-based reactive control and affordance-based deliberate control for autonomous mobile robot navigation. Unlike many current navigation approaches which only use learning-based paradigms, the authors focus on how to utilize the machine learning methods for reactive control together with the affordance knowledge that is simultaneously inherent in natural environments to gain advantages from both local and global optimization. Design/methodology/approach – The idea is to decompose the complex and large-scale robot navigation task into multiple sub-tasks and use the hierarchical reinforcement learning (HRL) algorithm, which is well-studied in the learning and control algorithm domains, to decompose the overall task into sub-tasks and learn a grid-topological map of the environment. An affordance-based deliberate controller is used to inspect the affordance knowledge of the obstacles in the environment. The hybrid control architecture is then designed to integrate the learning-based reactive control and affordance-based deliberate control based on the grid-topological and affordance knowledge. Findings – Experiments with computer simulation and an actual humanoid NAO robot have demonstrated the effectiveness of the proposed hybrid approach for mobile robot navigation. Originality/value – The main contributions of this paper are a new robot navigation framework that decomposes a complex navigation task into multiple sub-tasks using the HRL approach, and hybrid control architecture development that integrates learning-based and affordance-based paradigms for autonomous mobile robot navigation.

Machine learning-based augmented reality for improved surgical scene understanding

In orthopedic and trauma surgery, AR technology can support surgeons in the challenging task of understanding the spatial relationships between the anatomy, the implants and their tools. In this context, we propose a novel augmented visualization of the surgical scene that mixes intelligently the different sources of information provided by a mobile C-arm combined with a Kinect RGB-Depth sensor. Therefore, we introduce a learning-based paradigm that aims at (1) identifying the relevant objects or anatomy in both Kinect and X-ray data, and (2) creating an object-specific pixel-wise alpha map that permits relevance-based fusion of the video and the X-ray images within one single view. In 12 simulated surgeries, we show very promising results aiming at providing for surgeons a better surgical scene understanding as well as an improved depth perception.

Learning-based curriculum development

This article is written to inspire curriculum developers to centre their efforts on the learning processes of students. It presents a learning-based paradigm for higher education and demonstrates the close relationship between curriculum development and students’ learning processes. The article has three sections: Section “The role of higher education (HE) institutions” presents a discussion of the role of higher education in the knowledge society. Section “Contextual learning” presents the paradigm of contextual learning which we see as a useful foundation for curriculum development. Section “Curriculum development in practice—the BETA course” shows how a particular course in Business Economic Theory and Analysis has been developed using this paradigm. The article will be of interest to all academics interested in students’ learning processes but is especially relevant to those responsible for curriculum development.